Step 1: Problem Statement Analysis — WhatsApp

Problem Statement

"Design a Data Model for a Real-Time Messaging Platform like WhatsApp"

Understanding the Problem

What is WhatsApp?

WhatsApp is a cross-platform, end-to-end encrypted messaging application used by over 2 billion people globally. Users:

• Send text, image, video, audio, and document messages
• Make voice and video calls (1-on-1 and group)
• Create group chats with up to 1024 participants
• Share real-time location and live status updates (24-hour Stories)
• Broadcast messages to curated recipient lists
• View delivery and read receipts (double-tick, blue-tick)
• Manage disappearing messages with configurable expiry
• Use multi-device sync (phone, desktop, web, tablet independently)
• Share contacts and polls within chats
• React to messages with emojis

Core Requirements Analysis

1. User Identity & Authentication

What we need:

• Phone-number-based user identity (no email/password)
• Display name, profile photo, "About" bio text
• Last-seen and online/offline presence tracking
• Privacy settings (who can see last seen, profile photo, about, status)
• Multi-device session management (up to 4 linked devices)
• Account deactivation and deletion workflows

Why it matters:

• Phone number is the universal identifier — no usernames, no duplicates
• Privacy controls are a core differentiator vs. competitors
• Multi-device support requires session-level token management

2. One-on-One Private Chats

What we need:

• Create a conversation when the first message is sent
• End-to-end encrypted message payloads
• Message types: text, image, video, audio, document, contact, location, poll
• Delivery states: sent (single grey tick), delivered (double grey tick), read (double blue tick)
• Message editing (within 15-minute window) and deletion ("Delete for Everyone")
• Reply-to threading (quoting a specific earlier message)
• Forwarding messages (with "Forwarded" label after 5+ hops)
• Starring/bookmarking important messages

Why it matters:

• Private 1:1 messaging is the core product — must be fast and reliable
• Delivery receipts are a critical UX signal
• Edit/delete functionality requires careful state management

3. Group Chats

What we need:

• Group creation with name, description, and icon
• Group admin and super-admin roles
• Member management: add, remove, promote, demote
• Group settings: who can send messages (all members vs. admins only), who can edit group info
• Group invite links (with optional expiry and approval gates)
• Community structure: groups can be organized into Communities (meta-groups)
• Group participant limit: up to 1024 members

Why it matters:

• Groups are the viral growth engine of the platform
• Role-based access control is essential for large groups
• Communities (launched 2023) add a hierarchical layer above groups

4. Message Delivery & Status Tracking

What we need:

• Per-recipient delivery tracking in group chats (sent, delivered, read per participant)
• Offline message queuing (store-and-forward when recipient is offline)
• Message ordering guarantees (causal ordering within a chat)
• Disappearing messages (7 days, 24 hours, 90 days configurable per chat)
• View-once media (photo/video that auto-deletes after first viewing)

Why it matters:

• In a group of 500 people, each message generates up to 500 status rows
• Disappearing messages require TTL-based cleanup at scale
• View-once media needs server-side enforcement to prevent re-download

5. Voice & Video Calls

What we need:

• 1-on-1 voice and video calls
• Group calls (up to 32 participants)
• Call metadata: caller, callee(s), start time, end time, duration, call type
• Call status: ringing, answered, missed, rejected, busy
• Call quality metrics (optional: codec, bitrate, packet loss)

Why it matters:

• Calls are the second core product after messaging
• Call history is a frequently accessed feature
• Group calls need participant-level tracking (who joined, who left, when)

6. Status Updates (Stories)

What we need:

• Ephemeral content (auto-expires after 24 hours)
• Status types: text (with background color), image, video (up to 30 seconds)
• Privacy control: share with all contacts, selected contacts, or exclude specific contacts
• View tracking: who viewed each status and when
• Mute capability: hide specific contacts' statuses

Why it matters:

• Stories are a high-engagement feature borrowed from Instagram/Snapchat
• Viewer tracking creates a fan-out-on-read pattern (potentially millions of reads)
• 24-hour TTL requires scheduled background cleanup

7. Broadcast Lists

What we need:

• One-to-many messaging where recipients see it as a private 1:1 message
• Broadcast list management (add/remove contacts)
• Only contacts who have the sender's number saved will receive the broadcast
• Delivery tracking per recipient

Why it matters:

• Used heavily by businesses and influencers for announcements
• Unlike groups, recipients don't see each other — preserving privacy

8. Media & File Management

What we need:

• Image compression and thumbnail generation
• Video transcoding to multiple qualities
• Document storage (PDF, DOCX, XLSX, etc. up to 2GB)
• Audio message recording and playback
• Media auto-download settings per network type (WiFi vs. cellular)
• Storage usage tracking per chat

Why it matters:

• Media accounts for >70% of bandwidth usage
• Thumbnails are critical for fast chat rendering
• 2GB file limit requires chunked upload infrastructure

9. End-to-End Encryption (E2EE)

What we need:

• Signal Protocol implementation (Double Ratchet + X3DH key exchange)
• Per-device encryption keys (identity key, signed pre-key, one-time pre-keys)
• Key rotation and re-keying on device changes
• Security code verification between users
• Encrypted backup support (Google Drive / iCloud with user-managed password)

Why it matters:

• E2EE is a foundational privacy guarantee — the server never sees plaintext
• Key management directly affects the data model (key storage tables)
• Multi-device support requires key distribution across linked devices

10. Blocking & Reporting

What we need:

• Block a contact (prevents messages, calls, status views, last-seen visibility)
• Report a user or group (spam, harassment, illegal content)
• Report metadata: reporter, reported entity, reason, evidence (last 5 messages)
• Automated spam detection signals

Why it matters:

• Trust & Safety is a regulatory requirement (GDPR, DSA, IT Act)
• Block state must be enforced across messaging, calls, and status

Scope Definition

In Scope:

✅ User profiles and phone-based identity ✅ 1-on-1 private chats with E2EE ✅ Group chats with admin roles and settings ✅ Message types: text, image, video, audio, document, contact, location, poll ✅ Delivery receipts: sent, delivered, read (per-recipient in groups) ✅ Voice and video calls (1-on-1 and group) ✅ Status updates (24-hour Stories) with viewer tracking ✅ Broadcast lists ✅ Message reactions, replies, forwards, edits, deletions ✅ Blocking and reporting ✅ Multi-device session management ✅ Disappearing messages and view-once media

Out of Scope:

❌ WhatsApp Business API / Catalog / Shopping ❌ WhatsApp Pay (payments integration) ❌ Channels (newsletter-style one-way broadcasting) ❌ AI-powered chat features (Meta AI integration) ❌ Sticker packs marketplace ❌ Cloud backup encryption key escrow details ❌ Infrastructure-level message routing and queuing

Key Challenges to Address

1. Phone Number as Primary Key

Challenge: Users can change phone numbers, and phone numbers can be recycled by carriers Solution: Internal user_id (UUID) as PK, with phone_number as a unique indexed column that can be updated

2. Per-Recipient Delivery Tracking in Groups

Challenge: A single message to a 1000-member group generates 1000 delivery-status rows Solution: message_status junction table with composite key (message_id, user_id) and aggressive TTL cleanup

3. E2EE Key Storage

Challenge: Each user-device pair has multiple cryptographic keys that rotate Solution: Dedicated device_sessions and encryption_keys tables with versioning

4. Disappearing Messages

Challenge: Messages must be auto-deleted after a configurable TTL (24h, 7d, 90d) Solution: expires_at timestamp on messages + background sweeper job

5. Group Membership Changes

Challenge: When a user joins a group, they should NOT see messages from before they joined Solution: joined_at timestamp in group_members compared against message.sent_at

6. Broadcast vs. Group Semantics

Challenge: Broadcasts look like 1:1 chats to recipients but are managed as a list by the sender Solution: Separate broadcast_lists and broadcast_recipients tables, with messages delivered as individual chat messages

7. Multi-Device Message Sync

Challenge: A message sent from one device must appear on all linked devices Solution: device_sessions table tracks linked devices; message delivery fans out to all active sessions

8. Status (Stories) Fan-Out

Challenge: A user with 5000 contacts posts a status — do we write 5000 rows or read on demand? Solution: Fan-out-on-read: store the status once, compute viewers at read time using the contact graph

Business Rules to Implement

User Rules:

1. Phone number must be unique across the platform
2. Display name max 25 characters
3. About/bio max 139 characters
4. Profile photo visible based on privacy setting (Everyone / My Contacts / Nobody)
5. Users can link up to 4 companion devices

Chat Rules:

1. A 1:1 chat is created lazily on first message
2. Messages are stored as encrypted blobs — server cannot read content
3. Deleted messages show "This message was deleted" to the recipient
4. Edited messages show "(edited)" label and preserve edit history
5. Forwarded messages show "Forwarded" label; frequently forwarded shows double-arrow

Group Rules:

1. Only admins can add/remove members (configurable)
2. Only admins can change group name, icon, description (configurable)
3. Super-admin (group creator) cannot be removed by other admins
4. Group invite links can be reset by any admin
5. "Send messages" can be restricted to admins only

Call Rules:

1. Missed calls show as a notification in the chat
2. Group calls can have up to 32 participants
3. Call history is stored with full participant list and durations

Status Rules:

1. Statuses auto-expire after 24 hours
2. View-once media cannot be re-opened after first view
3. Muted statuses do not appear at the top of the status feed

Data Model Goals

1. Normalization

• Eliminate redundancy (e.g., user info stored once, referenced everywhere)
• Use junction tables for M:N relationships (group members, broadcast recipients, call participants)
• Follow 3NF with strategic denormalization for read-heavy paths

2. Performance

• Sub-100ms message delivery
• Instant chat list rendering (most recent message per chat)
• Fast status feed generation (contacts' statuses, sorted by recency)
• Efficient unread count computation

3. Scalability

• Handle 100 billion messages per day (real WhatsApp volume)
• Support 2 billion registered users
• Groups with up to 1024 participants
• Status updates with 5000+ contact fan-out

4. Privacy & Security

• E2EE key storage with device-level granularity
• Privacy settings enforced at the query level
• Soft deletes with cryptographic erasure for GDPR compliance
• Block state enforced across all features

Success Metrics

Our data model should support queries for:

1. Chat List: Get all chats for a user, ordered by most recent message, with unread counts
2. Message History: Get paginated messages for a chat (newest first)
3. Group Info: Get group details, members, admins, and settings
4. Delivery Receipts: Get delivery and read status per recipient for a group message
5. Call History: Get all calls for a user with participant details and durations
6. Status Feed: Get all unexpired statuses from a user's contacts, sorted by recency
7. Status Viewers: Get who viewed a specific status update and when
8. Search: Full-text search across message content (client-side, since E2EE)
9. Blocked Users: Check if user A has blocked user B (for enforcement)
10. Online Presence: Get last-seen timestamp for a contact (respecting privacy settings)

This content is reserved for Premium Members.