feat: Add webcam basic support

This commit is contained in:
2026-03-30 03:10:44 +02:00
parent 727059fb52
commit b7d4bf20e3
40 changed files with 1042 additions and 296 deletions

View File

@@ -1,13 +1,13 @@
# Voice Connection Domain
Bridges the application layer to the low-level realtime infrastructure for voice calls. Provides speaking detection via Web Audio analysis and per-peer volume control for playback. The actual WebRTC plumbing lives in `infrastructure/realtime`; this domain wraps it with a clean facade.
Bridges the application layer to the low-level realtime infrastructure for voice calls and in-channel camera transport. Provides speaking detection via Web Audio analysis and per-peer volume control for playback. The actual WebRTC plumbing lives in `infrastructure/realtime`; this domain wraps it with a clean facade.
## Module map
```
voice-connection/
├── application/
│ ├── voice-connection.facade.ts Proxy to RealtimeSessionFacade for voice signals and methods
│ ├── voice-connection.facade.ts Proxy to RealtimeSessionFacade for voice and camera signals/methods
│ ├── voice-activity.service.ts RMS-based speaking detection via AnalyserNode (per-user signals)
│ └── voice-playback.service.ts Per-peer GainNode chain, 0-200% volume, deafen support
@@ -42,13 +42,17 @@ graph TD
`VoiceConnectionFacade` exposes signals and methods from `RealtimeSessionFacade` without leaking infrastructure details into feature components. It covers:
- Connection state: `isVoiceConnected`, `isMuted`, `isDeafened`, `hasConnectionError`
- Stream access: `getRemoteVoiceStream`, `getLocalStream`, `getRawMicStream`
- Controls: `enableVoice`, `disableVoice`, `toggleMute`, `toggleDeafen`, `toggleNoiseReduction`
- Connection state: `isVoiceConnected`, `isMuted`, `isDeafened`, `isCameraEnabled`, `hasConnectionError`
- Stream access: `getRemoteVoiceStream`, `getRemoteCameraStream`, `getLocalStream`, `getLocalCameraStream`, `getRawMicStream`
- Controls: `enableVoice`, `disableVoice`, `enableCamera`, `disableCamera`, `toggleMute`, `toggleDeafen`, `toggleNoiseReduction`
- Audio tuning: `setOutputVolume`, `setInputVolume`, `setAudioBitrate`, `setLatencyProfile`
- Peer events: `onRemoteStream`, `onPeerConnected`, `onPeerDisconnected`
- Heartbeat: `startVoiceHeartbeat`, `stopVoiceHeartbeat`
## Camera transport
Camera capture is treated as voice-adjacent transport, not screen share. The underlying realtime layer routes webcam video only to peers in the same active voice channel, exposes remote camera streams through `getRemoteCameraStream(peerId)`, and keeps webcam senders separate from screen-share senders so both features can run at the same time.
## Speaking detection
`VoiceActivityService` monitors audio levels for local and remote streams using the Web Audio API. Each tracked stream gets its own `AudioContext` with an `AnalyserNode`. A single `requestAnimationFrame` loop polls all analysers.

View File

@@ -8,6 +8,7 @@ export class VoiceConnectionFacade {
readonly isVoiceConnected = inject(RealtimeSessionFacade).isVoiceConnected;
readonly isMuted = inject(RealtimeSessionFacade).isMuted;
readonly isDeafened = inject(RealtimeSessionFacade).isDeafened;
readonly isCameraEnabled = inject(RealtimeSessionFacade).isCameraEnabled;
readonly isNoiseReductionEnabled = inject(RealtimeSessionFacade).isNoiseReductionEnabled;
readonly hasConnectionError = inject(RealtimeSessionFacade).hasConnectionError;
readonly connectionErrorMessage = inject(RealtimeSessionFacade).connectionErrorMessage;
@@ -36,10 +37,18 @@ export class VoiceConnectionFacade {
return this.realtime.getRemoteVoiceStream(peerId);
}
getRemoteCameraStream(peerId: string): MediaStream | null {
return this.realtime.getRemoteCameraStream(peerId);
}
getLocalStream(): MediaStream | null {
return this.realtime.getLocalStream();
}
getLocalCameraStream(): MediaStream | null {
return this.realtime.getLocalCameraStream();
}
getRawMicStream(): MediaStream | null {
return this.realtime.getRawMicStream();
}
@@ -52,6 +61,14 @@ export class VoiceConnectionFacade {
this.realtime.disableVoice();
}
async enableCamera(): Promise<MediaStream> {
return await this.realtime.enableCamera();
}
disableCamera(): void {
this.realtime.disableCamera();
}
async setLocalStream(stream: MediaStream): Promise<void> {
await this.realtime.setLocalStream(stream);
}