Debug This Architecture

This architecture contains several critical flaws that will lead to data corruption, poor user experience, and database failure under load. Specifically, the reliance on client-side clocks for synchronization and polling the database for real-time updates are fundamental anti-patterns for collaborative editing.

Here is the breakdown of failure modes, race conditions, and bottlenecks, ordered by severity, with proposed solutions and trade-offs.

1. Critical Failure: Last-Write-Wins (LWW) with Client Clocks

The Issue: The system relies on "Last-write-wins with timestamps from client clocks." This is a fatal flaw for collaborative editing.

• Clock Skew: Client clocks are rarely synchronized. If User A’s computer is 5 minutes behind User B’s, User A’s valid work will be silently overwritten by User B, even if User A typed first.
• Race Conditions: If two users edit the same paragraph simultaneously, the "last" writer to hit the server wipes out the "first" writer's changes. This results in the "lost update" problem.
• Data Corruption: Since the storage unit is the full HTML snapshot, merging changes at the paragraph level is destructive.

Solution: Implement Operational Transformation (OT) or Conflict-free Replicated Data Types (CRDTs).

• Instead of sending the full HTML or a paragraph, send individual operations (e.g., "Insert 'a' at index 5").
• Use a central sequencing server or a deterministic algorithm to merge these operations so that concurrent edits result in the same document state on all clients.

Trade-offs:

• Complexity: OT and CRDTs are mathematically complex to implement correctly.
• Latency: CRDTs can grow the document state size significantly (requiring periodic compaction/garbage collection), whereas OT requires careful state management.

2. Performance Bottleneck: Database Polling

The Issue: "Other servers poll PostgreSQL every 2 seconds for changes."

• Database Overload: PostgreSQL is optimized for transactional processing, not for acting as a pub/sub notification system. Frequent polling (even every 2s) from multiple API servers will consume significant I/O and CPU, preventing the DB from handling actual writes.
• Latency: A 2-second delay is unacceptable for a "real-time" editor. Users will see typing lag or "jumping" text, breaking the collaborative illusion.

Solution: Introduce a Pub/Sub Message Broker (e.g., Redis Pub/Sub, Kafka, or RabbitMQ).

• When an API server receives a change, it writes to PostgreSQL and publishes a message to the broker.
• All other API servers subscribe to the broker and push updates to connected clients immediately via WebSocket.

Trade-offs:

• Infrastructure Complexity: Adds another moving part (Redis/Kafka) to manage.
• Message Ordering: You must ensure messages are processed in order (FIFO) per document, which requires careful partitioning logic in the broker.

3. Storage Failure: Full HTML Snapshots

The Issue: "Documents saved as full HTML snapshots every 30 seconds."

• Performance Degradation: As documents grow large (e.g., 50 pages), saving the full HTML every 30 seconds creates massive write spikes on the database.
• Storage Bloat: You lose version history granularity. If a user makes a mistake, they can only revert to 30-second ago, losing recent work.
• Merge Conflicts: Saving full HTML makes it impossible to merge non-conflicting edits (e.g., User A edits the header while User B edits the footer) without complex string diffing, which is error-prone.

Solution: Append-Only Operation Log + Periodic Snapshotting.

• Save every individual operation (insert/delete) to an operations table.
• Periodically (e.g., every 10 minutes or every 100 operations), compact the log into a new document snapshot.

Trade-offs:

• Read Performance: Loading a document requires reading the latest snapshot and replaying all subsequent operations. This can be slow if the log is long.
• Compaction Logic: You must write background jobs to handle the snapshotting and log truncation.

4. Scalability Bottleneck: Organization Partitioning

The Issue: "Document partitioning by organization ID."

• The "Hot" Shard Problem: This creates a massive bottleneck. If a large organization (e.g., a Fortune 500 company) uses the app, all their traffic hits a single database shard or a subset of API servers. This limits the effective scalability of the system to the size of your largest single customer.

Solution: Partition by Document ID (Hashing).

• Hash the Document ID to determine the shard. This spreads the load of a large organization across multiple database shards and servers.

Trade-offs:

• Cross-Document Queries: Queries like "Find all documents for User X" become slower because you must query every shard (scatter-gather) rather than just one.

5. Networking Failure: Load Balancer Strategy

The Issue: "Load balancer (round-robin)" + "Each API server maintains its own WebSocket connections."

• Broken Handshakes: WebSockets require a persistent connection. If the Load Balancer (LB) uses round-robin, the initial HTTP handshake might go to Server A, but subsequent WebSocket frames might go to Server B, which has no context for that connection. The connection will fail or drop immediately.
• Inconsistent State: Even if sticky sessions are used, the architecture implies users on Server A won't see updates from users on Server B for 2 seconds (due to polling).

Solution: Session Affinity (Sticky Sessions) at the Load Balancer level.

• Configure the LB to route a specific client IP (or cookie) to the same API server for the duration of the session.
• Combine this with the Redis Pub/Sub solution mentioned above to ensure that when Server A receives an update, it publishes it so Server B can immediately forward it to its own clients.

Trade-offs:

• Uneven Load: If one server has many long-lived connections with idle users, and another has few active users, the load becomes unbalanced. "Least Connections" routing is better than Round Robin here.

6. Security Risk: JWT in LocalStorage

The Issue: "JWT tokens... stored in localStorage."

• XSS Vulnerability: Any malicious JavaScript injected into the page (e.g., via a 3rd party script or a vulnerability in the rich text editor) can read the localStorage and steal the user's identity token.

Solution: Store JWTs in HttpOnly, Secure, SameSite Cookies.

• These cookies cannot be accessed by JavaScript, mitigating XSS token theft.

Trade-offs:

• CSRF Vulnerability: HttpOnly cookies are susceptible to Cross-Site Request Forgery (CSRF). You must implement CSRF tokens (double-submit cookie pattern) or SameSite attributes to mitigate this.

7. Caching Logic Error: Caching API Responses

The Issue: "CDN... caches API responses for 5 minutes."

• Stale Data: In a collaborative editor, data changes every second. Caching the API response (the document content) for 5 minutes means a user opening a document will see a version that is 5 minutes old. They will type, send an update based on old state, and likely cause corruption or overwrite others.

Solution: Do not cache dynamic document content at the CDN.

• Only cache static assets (JS bundles, CSS, images).
• For API responses, use short-lived Redis caching (milliseconds) only for highly repetitive metadata reads, but never for the document body itself.

Trade-offs:

• Increased Latency: Initial document load will hit the backend servers directly rather than the CDN edge. However, this is necessary for data integrity.

Summary of Architecture Changes Required

To make this system viable, you must:

1. Replace LWW with OT or CRDTs.
2. Replace DB Polling with Redis Pub/Sub.
3. Replace Full HTML Snapshots with an Append-Only Log.
4. Switch Sticky Sessions on the Load Balancer.
5. Move Auth to HttpOnly Cookies.
6. Remove API Caching from the CDN.