Files

T

myxelium e49b3ec112 chore: dev-stack switches, shared e2e harness, and desktop shell rules

- `LIVE_RELOAD=false npm run dev` keeps the renderer alive across a machine
  suspend; the reload client otherwise destroys the session under test.
- `dev-peer.sh` plus a separate userdata dir runs a second local peer.
- `tools/voice-probe.js` samples peer state and RTP counters from a live
  window, persisting to localStorage so a renderer reload cannot erase it.
- e2e helpers for voice pairs, peer-role election, and a TURN relay.
- Electron single-instance and dev-client-load decisions move into rules
  files with colocated specs.

2026-08-14 03:19:29 +02:00

5.5 KiB

Raw Blame History

06 — Voice & WebRTC bugs

User theme: “Voice is broken / one-way / connecting forever / works until reconnect.”
Severity: P0.

How voice is supposed to work (user view)

Click a voice channel (or answer a call).
Mic turns on (unless muted); you appear in the channel roster for others.
You hear everyone in that channel; they hear you.
Mute / deafen / camera / screen share behave Discord-like.
Navigate away → floating controls; voice continues.
Joining another voice target auto-leaves the previous one.
Second device shows you’re in voice elsewhere; Join takes over.
After a short network blip, voice returns without restarting the app.

Voice is not supposed to depend on the data channel for audio itself — but DC carries control/state; broken DC recovery currently rebuilds the whole peer, which drops media too (07).

The users should always be able to hear each other and see each other in joined calls and voice channels. (Not 1 out of 6 cant be heard for 3 users!)

How voice is supposed to work (system)

Piece	Responsibility
`voice-session`	Session metadata, floating UI, settings, exclusivity, takeover rules
`voice-connection`	Facade, VAD, per-peer playback gain
`MediaManager`	getUserMedia, RNNoise, gain, same-channel track routing
`PeerConnectionManager`	RTCPeerConnection, negotiation, ICE
Signaling	`offer`/`answer`/`ice_candidate` relay; `voice_state`; `voice_client_takeover`
ICE	STUN defaults; TURN optional via settings — no bundled TURN

Initiator election: deterministic compare of local id vs peer oderId so only one side offers.

Audio routing: attach/detach mic based on matching voiceState.roomId + serverId. Playback similarly scoped.

Failure modes

1 — Cross-signal initiator / politeness uses home id

Symptom: Peers never connect or glare forever when users have different home servers / foreign actor ids in the room.

Mechanism

Presence peer ids = per-server actor oderId.
getLocalOderId / polite-peer path often uses getIdentifyCredentials() → home id (signaling-transport-handler.ts, realtime-session.service.ts, negotiation.ts).
Election localOderId < peerId inconsistent across clients → dual offer, dual wait, or stuck non-initiator.

Fix direction: elect and politeness using per-signal-url identify credentials (getIdentifyCredentialsForSignalUrl(peerSignalUrl) / room source URL), not home-only.

2 — Voice allow-list misses peer map key → one-way / silence

Symptom: Connected peer, speaking UI maybe wrong, no audio out or in.

Mechanism: syncOutgoingVoiceRouting / playback only recognizes certain aliases (id / oderId / peerId). If activePeerConnections key is another alias → track detached.

3 — Silent give-up after reconnect budget

Symptom: After ~60s of failures, voice never returns; no error toast.

Constants: PEER_RECONNECT_MAX_ATTEMPTS = 12, interval 5s. Tracker deleted; no UI.

4 — Transient signaling drops offers during WS reconnect

Offers/ICE classified transient may be deferred/dropped while socket reconnecting → half-open peers; depends on fallback offer timers (USER_JOINED_FALLBACK, non-initiator give-up 5s).

5 — DC recovery tears down media

Closed control channel → removePeer + full reconnect (peer-recovery.ts). Docs claim soft replaceDataChannel preserving AV — code does not do that (07, 10).

6 — Auth / presence missing

If identify/join failed (04/05), no RTC relay eligibility / no peer discovery → empty voice.

7 — Same-channel filter false negatives

Remote voice_state missing/stale channel ids → locally mute peer while UI still lists them in channel.

8 — No TURN by default

Symmetric NAT / strict firewalls fail ICE with STUN-only. User-configurable TURN exists; many installs never set it. Product decision needed: ship defaults vs document limitation.

9 — Multi-device ownership races

Takeover / heartbeat / voiceActive routing wrong → offers hit passive device; active device silent.

Code vs docs

Doc claim	Code
Soft DC renegotiation preserves media	Always full peer recreate on closed DC
Deterministic initiator from logical peer ids	Uses home identify credentials in several paths
TURN supported	Configurable only; not default

Key files

domains/voice-session/, domains/voice-connection/
infrastructure/realtime/media/media.manager.ts
peer-connection-manager/**, negotiation.ts, peer-recovery.ts
signaling-message-handler.ts (server_users / user_joined offers)
ice-server-settings.service.ts, realtime.constants.ts
Feature: agents-docs/features/voice-webrtc.md

Proof of done

Two users same home signal, same voice channel: bidirectional audio < 5s after both join.
Two users different home signals, same foreign-hosted room (both provisioned): bidirectional audio.
Toggle mute/deafen; camera; screen share request path.
Kill Wi‑Fi 15s: recovers or shows actionable error (not silent forever).
Second client takeover: first stops transmitting; second owns mic.
Regression test for initiator election with mismatched home vs actor ids.

Interview prompts (when implementing)

Soft DC replace vs keep full rebuild but fix media reattach + UX error?
Ship public TURN defaults or document “requires open NAT / user TURN”?
Instrument-only first week vs behavior fix first?

5.5 KiB Raw Blame History Unescape Escape