Debug This Architecture

Critical Issues Analysis

1. Split-Brain Real-time Synchronization

Problem: Users connected to different API servers experience 2+ second delays due to polling mechanism, creating divergent document states and poor collaboration experience.

Solution: Implement Redis Pub/Sub or dedicated message broker (RabbitMQ/Kafka)

• Implementation: Each server publishes changes to Redis channel, all servers subscribe
• Trade-offs:
  • ✅ Sub-100ms propagation between servers
  • ❌ Additional Redis load and potential SPOF
  • ❌ Need to handle Redis connection failures gracefully

2. Clock Skew in Conflict Resolution

Problem: Client clock differences cause incorrect last-write-wins resolution, potentially losing recent edits.

Solution: Hybrid Vector Clocks + Server Timestamps

{
  serverTimestamp: 1234567890,
  vectorClock: {userId1: 5, userId2: 3},
  operation: {...}
}

• Trade-offs:
  • ✅ Accurate causality tracking
  • ❌ Increased complexity and message size
  • ❌ Requires conflict resolution UI for true conflicts

3. Data Loss from 30-Second Snapshot Window

Problem: System crash or network partition can lose up to 30 seconds of work.

Solution: Implement Operation Log with Checkpointing

• Store individual operations in PostgreSQL with periodic snapshots
• Reconstruct documents from last snapshot + operations
• Trade-offs:
  • ✅ Minimal data loss, full audit trail
  • ❌ Increased storage (mitigate with compression)
  • ❌ Complex reconstruction logic

4. WebSocket Connection Storm on Server Failure

Problem: Server crash causes all clients to reconnect simultaneously, overwhelming remaining servers.

Solution: Exponential Backoff with Jitter + Connection Pooling

reconnectDelay = Math.min(baseDelay * Math.pow(2, attempt) + random(0, 1000), maxDelay)

• Implement WebSocket connection limits per server
• Trade-offs:
  • ✅ Prevents cascade failures
  • ❌ Slower recovery for some users
  • ❌ Need sticky sessions or state migration

5. Operational Transform Inconsistency

Problem: Concurrent edits to same paragraph with last-write-wins causes character-level corruption.

Solution: Implement Proper OT or CRDT (Yjs/Automerge)

• Use operation transformation for character-level merging
• Trade-offs:
  • ✅ Preserves all users' intentions
  • ❌ Significant implementation complexity
  • ❌ Larger operation payload size

6. PostgreSQL Write Bottleneck

Problem: All changes funnel through single master database, limiting throughput.

Solution: Write-Through Cache + Async Batch Writes

// Buffer operations in Redis, batch write to PostgreSQL
await redis.lpush(`doc:${docId}:ops`, operation);
// Background job processes queue every 100ms or 100 operations

• Trade-offs:
  • ✅ 10x+ write throughput improvement
  • ❌ Potential data loss if Redis fails
  • ❌ Increased system complexity

7. JWT Token Security Vulnerabilities

Problem: localStorage tokens vulnerable to XSS, no refresh mechanism, 24-hour window too long.

Solution: httpOnly Cookies + Refresh Token Pattern

• Access token: 15 minutes, in memory
• Refresh token: 7 days, httpOnly cookie
• Implement token rotation on refresh
• Trade-offs:
  • ✅ Significantly improved security
  • ❌ More complex auth flow
  • ❌ CSRF protection needed

8. CDN Cache Poisoning

Problem: 5-minute API response caching can serve stale document data.

Solution: Cache-Control Headers + Versioned Endpoints

Cache-Control: private, max-age=0, must-revalidate
ETag: "document-version-hash"

• Use CDN only for truly static assets
• Trade-offs:
  • ✅ Always fresh data
  • ❌ Higher origin server load
  • ❌ Increased bandwidth costs

9. Database Partition by Organization Creates Hot Shards

Problem: Large organizations create hotspots, limiting scalability.

Solution: Consistent Hashing with Virtual Nodes

• Hash(orgId + docId) for distribution
• Implement document sharding within large orgs
• Trade-offs:
  • ✅ Even load distribution
  • ❌ Cross-shard queries become complex
  • ❌ Need distributed transaction support

10. No Presence Awareness

Problem: Users can't see who else is editing, leading to conflicts.

Solution: Cursor Position Broadcasting + Presence Service

• Track active users per document in Redis
• Broadcast cursor positions via WebSocket
• Trade-offs:
  • ✅ Better collaboration UX
  • ❌ Increased message volume
  • ❌ Privacy considerations

Priority Fixes

1. Immediate: Replace polling with Pub/Sub (fixes collaboration lag)
2. Critical: Implement proper OT/CRDT (prevents data corruption)
3. Important: Add operation log (prevents data loss)
4. Important: Fix auth security (prevents breaches)
5. Nice-to-have: Add presence awareness (improves UX)

Each fix should be implemented incrementally with feature flags for safe rollout and rollback capability.