Here is solutions for Design Collaborative Document System - Follow-up questions.

Main topic:

• Handle conflict: concurrency write, OT(Operational Transformation) and CRDTs (Conflict-Free Replicated Data Types)

• Fan-out websocket real-time update per collaborator.

1. Patterns for Design collaborative Document System

1. Split 2 tables: Document table, Document Operation table -> Merge by LSM Tree (Log-based operations) later.

2. Split 2 services: documentsvc (to source of truth data) and collaborationsvc (to interact with user).

3. Websocket in collaborationsvc: for Real-time Updates
   • WebSocket sessions are sticky to a Collaboration Service instance. Subscriptions per list are tracked in Redis sets. Broadcasts are done in two phases: optimistic echo on produce, then commit update on DB apply.

   Client -> WebSocket -> CollaborationSvc -> Kafka -> TodoTaskSvc -> DB -> Kafka(commit event) -> CollaborationSvc -> WebSocket clients
   

   Clients
   └─ WebSocket / HTTP
      └─ CollaborationSvc
           ├─ produce -> todo.ops
           └─ consume  <- todo.commits
      └─ TodoTaskSvc
           ├─ consume  <- todo.ops
           ├─ write DB
           └─ produce -> todo.commits
      └─ NotificationSvc
           └─ consume  <- todo.commits
      └─ Analytics / ActivitySvc
           └─ consume  <- todo.commits
   

4. Versioning documents: using versionings or vector clocks for handle race condition.

5. Tasks: each update the document, have version

6. Handle edit race condition - concurrency and ordering
   • Clients send op {op_id, list_id, type, payload, client_version}.
   • Collaboration produces to Kafka and echoes.
   • Task Service consumes in partition order, writes with CAS semantics to ensure idempotency and monotonic server_version.

   • It stores op_id on the affected row(s) so retries are no‑ops.

   • Text descriptions that truly need co‑editing can use a CRDT (RGA or JSON CRDT) stored as a blob and updated via ops. For most to‑do apps, field‑level updates with ordered ops are sufficient.

7. Partition list_id in kafka parititon of a topic -> We can have: 1 parititon = 1 list_id -> 1 consumer in consumer group.

8. Operational Transformation (OT)
   • Step 1: Alice sends first

     • Server receives

       Insert(5, " World")
       

     • Document becomes

       Hello World
       

     • Current positions:

       Hello World
       012345678901
       

   • Step 2: Bob’s operation arrives
     • Bob originally created

     Insert(5, "!!!")
     

     • But position 5 is no longer correct because Alice already inserted text.

     • The server transforms Bob’s operation.

     • Before transformation

       Insert(5, "!!!")
       

     • After transformation

       Insert(11, "!!!")
       

     • because “ World” has length 6.

   • Step 3: Apply transformed operation

     Hello World!!!
     

   • Everyone receives

     Insert(5, " World")
     Insert(11, "!!!")
     

   • Every client ends with: Hello World!!!

9. CRDT Data Structures

   H → e → l → l → o
                  │
        ┌────────┴────────┐
        ▼                 ▼
        W                 !
        │                 │
        ▼                 ▼
        o                 !
        │                 │
        ▼                 ▼
        r                 !
        │
        ▼
        l
        │
        ▼
        d
   

   • Output:

   Hello World!!!
   

   • Or:

   Hello!!! World
   

   • If

   Replica A's insert
   comes before
   Replica B's insert