Debug This Architecture

I'll analyze this architecture systematically, identifying critical issues and proposing solutions.

Critical Issues & Solutions

1. Split-Brain Problem: WebSocket Broadcast Isolation

Issue: Changes are only broadcast to clients connected to the same server. Users on different servers experience 2+ second delays. Solution: Implement Redis Pub/Sub or message queue (RabbitMQ/Kafka)

// Current problem
server1.broadcast(change); // Only server1 clients see this immediately

// Solution
redis.publish('doc:123:changes', change); // All servers receive instantly

Trade-offs:

• ✅ Real-time sync across all servers
• ❌ Additional infrastructure complexity
• ❌ ~10-50ms latency added

2. Clock Skew in Conflict Resolution

Issue: Client clocks can differ by seconds/minutes, making last-write-wins unreliable Solution: Use Hybrid Logical Clocks (HLC) or server-assigned timestamps

// Problem
{ text: "Hello", timestamp: clientTime } // Client A: 14:30:00
{ text: "World", timestamp: clientTime } // Client B: 14:29:55 (clock behind)

// Solution with HLC
{ text: "Hello", hlc: { time: serverTime, counter: 1, clientId: "A" }}

Trade-offs:

• ✅ Guarantees causal ordering
• ❌ Requires server timestamp synchronization
• ❌ More complex implementation

3. Data Loss Window

Issue: 30-second save intervals risk losing up to 30 seconds of work Solution: Implement write-ahead log (WAL) with operation-based storage

// Store operations immediately
operations: [
  { op: 'insert', pos: 45, text: 'Hello', timestamp: ... },
  { op: 'delete', pos: 50, length: 5, timestamp: ... }
]
// Snapshot periodically for performance

Trade-offs:

• ✅ No data loss
• ✅ Enables undo/redo
• ❌ More complex storage model
• ❌ Increased storage requirements

4. PostgreSQL Polling Bottleneck

Issue: N servers polling every 2 seconds = N queries/2 seconds Solution: Replace with PostgreSQL LISTEN/NOTIFY or change data capture (CDC)

-- Real-time notifications
NOTIFY document_changes, '{"docId": 123, "change": {...}}';

-- Or use logical replication/CDC tools like Debezium

Trade-offs:

• ✅ Real-time updates
• ✅ Reduced database load
• ❌ Requires persistent connections
• ❌ More complex error handling

5. Operational Transform (OT) Missing

Issue: Paragraph-level last-write-wins loses collaborative edits Solution: Implement OT or CRDTs (Conflict-free Replicated Data Types)

// Current: entire paragraph replaced
User A: "The quick brown fox"
User B: "The quick brown fox jumps" 
Result: One user's changes lost

// With OT: operations transformed
Op A: insert(4, "very ")
Op B: insert(19, " jumps")
Result: "The very quick brown fox jumps"

Trade-offs:

• ✅ True real-time collaboration
• ✅ No lost changes
• ❌ Complex implementation
• ❌ Requires operation history

6. WebSocket Connection Management

Issue: No reconnection strategy; load balancer may route reconnections to different servers Solution: Implement sticky sessions or connection state management

// Connection state in Redis
redis.hset(`user:${userId}:connections`, {
  serverId: 'server-1',
  docId: 123,
  lastSeenOperation: 456
});

// On reconnect, route to same server or sync state

Trade-offs:

• ✅ Seamless reconnections
• ✅ Maintains client state
• ❌ Complicates load balancing
• ❌ Requires session affinity

7. JWT Security Issues

Issue: 24-hour expiry in localStorage vulnerable to XSS; no refresh mechanism Solution: Shorter-lived tokens with refresh tokens in httpOnly cookies

// Access token: 15 minutes, memory only
// Refresh token: 7 days, httpOnly cookie
// Implement token rotation on refresh

Trade-offs:

• ✅ Improved security
• ✅ XSS mitigation
• ❌ More complex auth flow
• ❌ Additional server calls

8. CDN Cache Invalidation

Issue: 5-minute API cache can serve stale document data Solution: Implement cache tags and targeted invalidation

// Tag responses
res.headers['Cache-Tag'] = `doc-${docId}`;

// Invalidate on changes
cdn.purge({ tags: [`doc-${docId}`] });

Trade-offs:

• ✅ Fresh data when needed
• ✅ Maintains CDN benefits
• ❌ Requires CDN with tag support
• ❌ Purge propagation delay

9. Database Write Bottleneck

Issue: Every keystroke writes to PostgreSQL Solution: Buffer changes in Redis, batch write to PostgreSQL

// Buffer in Redis
redis.lpush(`doc:${docId}:pending`, change);

// Batch write every 100ms or 10 operations
const batch = await redis.lrange(`doc:${docId}:pending`, 0, -1);
await postgres.insertBatch(batch);

Trade-offs:

• ✅ Reduced database load
• ✅ Better write performance
• ❌ Slight increase in complexity
• ❌ Potential data loss if Redis fails

10. No Presence Awareness

Issue: Users can't see who else is editing Solution: Implement presence system with cursor positions

// Track active users and cursor positions
presence: {
  users: {
    'user1': { cursor: 145, selection: null, color: '#FF5733' },
    'user2': { cursor: 89, selection: [89, 94], color: '#33FF57' }
  }
}

Trade-offs:

• ✅ Better collaboration UX
• ✅ Prevents edit conflicts
• ❌ Additional state management
• ❌ More WebSocket traffic

Recommended Architecture Changes

1. Replace polling with Redis Pub/Sub for real-time server-to-server communication
2. Implement CRDT-based text editing (like Yjs) for better conflict resolution
3. Add connection state management with sticky sessions or state synchronization
4. Buffer writes in Redis with asynchronous PostgreSQL persistence
5. Implement proper auth token rotation with secure storage
6. Add operation log storage instead of HTML snapshots only
7. Implement presence awareness for better collaboration

This architecture would be suitable for small-scale use but needs these fundamental changes to scale reliably for a production Google Docs-like system.