Voice & WebRTC

Status: Active Last updated: 2026-07-05

Overview

Voice channels, camera, and screen-share use direct WebRTC peer connections between clients. The signaling server relays SDP offers/answers/ICE and voice presence (voice_state); media never flows through the server.

Responsibilities

Layer	Owns
`infrastructure/realtime/`	WebRTC sessions, negotiation, data channels, RNNoise worklet
`voice-connection` / `voice-session` domains	Facades, workspace UI, settings, multi-device ownership
`screen-share` domain	Source picker, quality presets (Electron)
Signaling server	RTC relay + `voice_state` / `voice_client_takeover` broadcast

WebSocket signaling types

type	Purpose
`offer`, `answer`, `ice_candidate`	WebRTC negotiation relay to `targetUserId`
`voice_state`	Voice roster (mute/deafen/speaking, channel id); sets `voiceActive` on one connection per user
`voice_client_takeover`	Notify sibling tabs to yield voice ownership

Relay rules: RTC messages require shared server membership (except DM-specific types). See signaling.md.

Multi-device voice

Only one connection per oderId may be voiceActive; RTC offers route to that connection (fallback: any open connection).
Other tabs show passive UI and may send voice_client_takeover.
clientInstanceId in identify and voice payloads distinguishes tabs.

Media pipeline (client)

Voice: getUserMedia → optional RNNoise AudioWorklet → gain → same-room peer routing only.
Camera: separate video track; same-room filter.
Screen share: on-demand via data-channel SCREEN_SHARE_REQUEST; platform-specific capture (browser getDisplayMedia, Electron picker, Linux PulseAudio routing).

P2P data channel

Carries voice/screen control messages, chat, attachments, and state sync — not server-relayed. Data-channel failure triggers peer renegotiation or full rebuild (see realtime README). When a failed control channel is replaced (replaceDataChannel), the old channel is closed first so its SCTP resources are released.

Media memory invariants (2026-07-14, RAM investigation)

removePeer / closeAllPeers clear all four remote stream maps, including remotePeerCameraStreams (previously leaked per departed peer).
Video tiles (voice-workspace-stream-tile) pause and null srcObject in ngOnDestroy (voice-workspace-stream-video.rules.ts) so Chromium releases decoder/frame buffers immediately.
debug-network-metrics drops a peer's entry when the peer is fully removed and caps the store at MAX_TRACKED_DEBUG_NETWORK_PEERS = 200 (oldest evicted).
Electron registers setDisplayMediaRequestHandler once per app run (guarded in create-window.ts), not on every window recreation.

Mobile / Capacitor

Background voice uses Android foreground service + iOS audio/CallKit bridges — mobile-capacitor.md. Screen share is limited on mobile WebViews.

signaling.md — envelope catalog
direct-messaging.md — private calls share PeerDeliveryService
toju-app/src/app/infrastructure/realtime/README.md — negotiation, recovery, RNNoise
Domain READMEs: voice-connection, voice-session, screen-share

Changelog

Date	Change
2026-07-14	Media memory invariants: camera-stream map cleanup, tile `srcObject` release, debug-metrics cap, single display-media handler registration, replaced data channels closed
2026-07-05	Initial cross-context voice/WebRTC contract

3.6 KiB Raw Blame History