- `LIVE_RELOAD=false npm run dev` keeps the renderer alive across a machine suspend; the reload client otherwise destroys the session under test. - `dev-peer.sh` plus a separate userdata dir runs a second local peer. - `tools/voice-probe.js` samples peer state and RTP counters from a live window, persisting to localStorage so a renderer reload cannot erase it. - e2e helpers for voice pairs, peer-role election, and a TURN relay. - Electron single-instance and dev-client-load decisions move into rules files with colocated specs.
4.7 KiB
07 — Data channel drops & recovery
User theme: “Connection drops / chat and files die / voice dies after a blip.”
Severity: P0/P1 — shared control plane for chat sync, attachments, emoji, screen control, and currently media (because recovery rebuilds the whole PC).
How the data channel is supposed to work (user view)
- Once you’re in a community with other people, messages, files, emoji, and many live updates “just sync.”
- Brief network glitches should self-heal.
- You should not need to restart the app to get chat syncing again.
- Voice should ideally survive control-plane blips (product docs claim this; code currently does not).
- Users should sync all but only load a portion into ram for viewing so the app doesn't crash of high ram usage.
How the data channel is supposed to work (system)
- Single ordered RTCDataChannel per peer pair (label typically chat/control).
- Carries: chat events, inventory sync, attachments, avatar/emoji chunks, voice/screen control, pings, plugin bus, game activity, etc.
- Back-pressure: high 4MB / low 1MB watermarks.
- Ping every 5s for RTT.
- On failure, recovery should restore control and ensure inventory/resync runs when channel reopens.
Documented recovery (README / voice-webrtc.md)
- Non-fatal error on open channel → request voice-state snapshot on same channel.
- Closed channel → initiator renegotiates new DC on existing PC (preserve AV); non-initiator waits then full rebuild if missing.
- Closing-but-not-closed → short grace (2.5s).
replaceDataChanneladopts the new channel.
Actual recovery (peer-recovery.ts)
- Closed →
repairUnavailableDataChannel→removePeer+attemptPeerReconnect/schedulePeerReconnect(full PC teardown). - Closing → wait
DATA_CHANNEL_RECOVERY_GRACE_MS(2.5s) → same full recreate. replaceDataChannelexists on the manager and is wired into handlers, but recovery path never calls it; specs assert it is not called in several cases.- After 12 reconnect attempts (~60s): abandon silently.
This is a documented lie — treat README paragraph as aspirational until code matches or docs are corrected in the same PR as a deliberate decision.
Failure modes
1 — Full rebuild drops audio/video on every DC close
User hears a “drop” even when ICE media might have survived. Cascades into voice bug reports.
2 — Silent abandon
No toast, no “Reconnect” CTA, no automatic retry on later user_joined. Mesh looks permanently broken until navigation/restart.
3 — Live chat limp vs history dead
While DC down, chat_message WS fallback may still deliver live text. Inventory sync is DC-only → late joiners / catch-up fail until P2P returns (08).
4 — Large payloads kill shared channel
Custom emoji / attachment floods can stress or close the shared ordered channel (lessons). One feature outage becomes total control-plane outage.
5 — Attachment announce vs message ordering
file-announce on DC can beat chat-message on WS → auto-download gives up unless re-queued on message bind (lesson; verify still present when touching attachments).
6 — Replacement channel race
If soft-replace is reintroduced, must close old channel to release SCTP (voice-webrtc changelog). Current full rebuild avoids that class but at higher cost.
Key files
peer-connection-manager/messaging/data-channel.tspeer-connection-manager/recovery/peer-recovery.ts(+ specs)peer-connection.manager.ts(replaceDataChannel)realtime.constants.ts- Consumers: chat sync effects, attachment transfer, custom emoji chunking, screen-share request
Fix directions (interview)
| Option | Idea | Tradeoff |
|---|---|---|
| A (align code to docs) | Implement true soft DC replace on connected PC; full rebuild only if PC not connected | Harder; matches user expectation for voice survival |
| B (align docs to code) | Keep full rebuild; fix reattach + force inventory on reopen; surface give-up UX | Faster; still interrupts voice |
| C | Separate unreliable channel for bulk (files/emoji) vs reliable small control | Larger design |
Recommend starting with instrumentation + B’s UX/resync, then A if voice drop rate stays high.
Proof of done
- Force-close DC in debug: control messages resume; inventory runs; user sees progress or success.
- If soft-replace chosen: audio continues through DC replace (automated or manual with metrics).
- After max attempts: visible error + manual retry works.
- Attachment + emoji transfer during recovery does not deadlock the mesh.
- Update
realtime/README.md+voice-webrtc.mdin the same change set so they match behavior.