myxelium/Toju
1
0
Fork 0
Code Issues 11 Pull Requests Actions Packages Projects 2 Releases 22 Wiki Activity

feat: Add webcam basic support

Browse Source
This commit is contained in:
SocksOnHead
2026-03-30 03:10:44 +02:00
parent 727059fb52
commit b7d4bf20e3
40 changed files with 1042 additions and 296 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
toju-app/src/app/infrastructure/realtime/README.md
View File

@@ -29,10 +29,10 @@ realtime/
│ ├── recovery/
│ │ └── peer-recovery.ts Disconnect grace period + reconnect loop
│ └── streams/
│ └── remote-streams.ts Classifies incoming tracks (voice vs screen)
│ └── remote-streams.ts Classifies incoming tracks (voice vs camera vs screen)
│
├── media/ Local capture and processing
│ ├── media.manager.ts getUserMedia, mute, deafen, gain pipeline
│ ├── media.manager.ts getUserMedia, mute, deafen, camera capture, same-room routing, gain pipeline
│ ├── noise-reduction.manager.ts RNNoise AudioWorklet graph
│ ├── voice-session-controller.ts Higher-level wrapper over MediaManager
│ ├── screen-share.manager.ts Screen capture + per-peer track distribution
@@ -229,12 +229,44 @@ graph LR
click Peers "media/media.manager.ts" "MediaManager.bindLocalTracksToAllPeers()" _blank
```
`MediaManager` grabs the mic with `getUserMedia`, optionally pipes it through the RNNoise AudioWorklet for noise reduction (48 kHz, loaded from `rnnoise-worklet.js`), optionally runs it through a `GainNode` for input volume control, and then routes the resulting audio track only to peers that currently belong to the same active voice channel.
`MediaManager` grabs the mic with `getUserMedia`, optionally pipes it through the RNNoise AudioWorklet for noise reduction (48 kHz, loaded from `rnnoise-worklet.js`), optionally runs it through a `GainNode` for input volume control, and then routes the resulting audio track only to peers that currently belong to the same active voice channel. The same manager also owns camera capture as a separate video-only stream, attaches it to its own video transceiver, and applies the same voice-channel routing rules so webcam video only reaches peers in the active voice room.
Mute just disables the audio track (`track.enabled = false`), the connection stays up. Deafen suppresses incoming audio playback on the local side.
Because peers stay connected across the server for shared state and chat, voice-channel isolation is enforced in both transport and playback: outgoing mic audio is only attached to peers whose voice membership matches the local user's current channel, and remote voice audio plus join/leave cues are only active when the remote peer's announced `voiceState.roomId` and `voiceState.serverId` match the local user's current voice channel.
### Camera
```mermaid
sequenceDiagram
participant UI as VoiceControls/UI
participant MM as MediaManager
participant Peer as PeerConnectionManager
participant Remote as Remote peer
participant RS as remote-streams.ts
participant Shell as VoiceWorkspaceComponent
UI->>MM: enableCamera()
Note over MM: getUserMedia({ video: true, audio: false })
Note over MM: Store localCameraStream
MM->>MM: syncCameraRouting()
Note over MM: Attach video track only to same-room peers
MM->>Peer: renegotiate(peerId)
MM->>Remote: broadcast camera-state
Peer->>Remote: offer/answer with camera video transceiver
Remote->>RS: ontrack(video)
Note over RS: Classify as camera, not screen share
RS->>Shell: getRemoteCameraStream(peerId)
Shell->>Shell: Render camera tile in voice workspace
UI->>MM: disableCamera()
MM->>MM: stopLocalCameraStream()
MM->>MM: detach camera sender from peers
MM->>Remote: broadcast camera-state(false)
```
Camera capture is video-only, uses a dedicated camera sender, and follows the same same-room peer filter as outgoing voice audio. Incoming camera video is classified separately from screen-share tracks so the workspace can show both at the same time.
### Screen share
Screen capture uses a platform-specific strategy: