Debug This Architecture

Identified Issues and Proposed Solutions:

1. Cross-Server Sync Delay (Polling Bottleneck)

   • Issue: 2-second polling introduces latency for cross-server updates, breaking real-time sync.
   • Solution: Replace polling with Redis Pub/Sub. Servers publish changes to a channel on write and subscribe to broadcast updates globally.
   • Trade-off: Adds Redis dependency and pub/sub complexity. Requires message deduplication.

2. Unreliable Conflict Resolution (Client Timestamps)

   • Issue: Client clocks may skew, causing incorrect "last-write-wins."
   • Solution: Use server-generated timestamps (synchronized via NTP) or vector clocks for causal ordering.
   • Trade-off: Server clocks require synchronization; vector clocks increase metadata overhead.

3. Data Loss Risk with Snapshots

   • Issue: 30-second snapshots risk losing intermediate changes during crashes.
   • Solution: Use Operational Transform (OT) or CRDTs to track incremental changes, with periodic snapshots.
   • Trade-off: Increased complexity in handling merge operations and history tracking.

4. JWT Security in localStorage

   • Issue: localStorage is vulnerable to XSS attacks;
   • Solution: Store JWTs in httpOnly cookies with CSRF tokens or use short-lived tokens + refresh tokens.
   • Trade-off: Cookies require CSRF mitigation; refresh tokens need secure server-side storage.

5. Stale CDN Caching

   • Issue: 5-minute API caching serves outdated document versions.
   • Solution: Invalidate CDN cache on document update via webhooks or versioned URLs (e.g., /api/doc?id=1&v=123).
   • Trade-off: Higher CDN costs and complexity in cache invalidation logic.

6. Stateless Load Balancing with WebSockets

   • Issue: Round-robin disrupts WebSocket statefulness, fragmenting user sessions.
   • Solution: Implement sticky sessions using the Sec-WebSocket-Key header or IP hashing.
   • Trade-off: Reduced load-balancing efficiency; potential uneven traffic distribution.

7. Read Replica Lag

   • Issue: Replicas may serve stale data during heavy write loads.
   • Solution: Use a write-through Redis cache for frequently accessed documents, or route critical reads to the primary DB.
   • Trade-off: Cache invalidation complexity; primary DB read load increases.

8. Hotspots in Document Partitioning

   • Issue: Sharding by organization ID may create hotspots for large orgs.
   • Solution: Composite shard keys (e.g., org_id + doc_id_hash) or dynamic resharding.
   • Trade-off: Shard management complexity; potential data downtime during resharding.

9. Race Conditions in Snapshot Saves

   • Issue: Concurrent snapshot writes may overwrite changes.
   • Solution: Use optimistic concurrency control (e.g., UPDATE ... WHERE version = X).
   • Trade-off: Increased DB contention and retry logic in application code.

10. High Database Load from Polling

    • Issue: Frequent polling by multiple servers strains PostgreSQL.
    • Solution: Replace polling with a PostgreSQL LISTEN/NOTIFY for real-time change events.
    • Trade-off: Limited scalability in high write volumes; requires connection pooling.

Summary of Trade-offs:

• Complexity vs. Performance: Introducing Redis Pub/Sub or CRDTs improves real-time sync but adds operational overhead.
• Security vs. Usability: httpOnly cookies enhance security but complicate token management.
• Consistency vs. Scalability: Sticky sessions ensure WebSocket consistency but reduce load-balancing flexibility.

Final Recommendations:

• Prioritize Redis Pub/Sub for real-time sync and server timestamps for conflict resolution.
• Migrate to incremental changes (CRDTs) and short-lived JWTs with refresh tokens.
• Implement sticky sessions and CDN cache invalidation to address fragmentation and staleness.