docs: populate initial cross-context feature docs
Add area-level documentation for the five most significant cross-context feature areas under agents-docs/features/: - websocket-envelopes: full envelope catalogue, lifecycle, dispatcher - ipc-bridge: window.electronAPI surface, IPC channels, CQRS dispatch - plugin-system: manifest contract, runtime, capabilities, plugin-support API - server-directory: REST endpoints, CQRS, entities, business rules - voice-signaling: mesh signaling, RNNoise pipeline, domain split Update agents-docs/FEATURES.md index alphabetically and remove the "no cross-context feature docs" placeholder. Each doc records honest TODOs for verified gaps (stale signaling-contracts.ts, window.api vs window.electronAPI mismatch, IPC error envelope drift from CONTEXT.md, missing OpenAPI coverage for server-directory routes, no envelope round-trip test). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
178
agents-docs/features/ipc-bridge.md
Normal file
178
agents-docs/features/ipc-bridge.md
Normal file
@@ -0,0 +1,178 @@
|
||||
# Electron IPC Bridge
|
||||
|
||||
> **Area:** ipc-bridge
|
||||
> **Status:** Active
|
||||
> **Last updated:** 2026-05-25
|
||||
|
||||
## Overview
|
||||
|
||||
The Electron IPC bridge is the only path through which the Angular renderer can reach the desktop runtime — the filesystem, the local SQLite database, OS APIs, the update flow, plugin loading, and the in-process Local API server. The renderer cannot import `electron`, `node:fs`, TypeORM, or any other privileged module directly; every privileged operation crosses the preload `contextBridge` boundary as a typed IPC call. This area documents the surface itself: how it is registered, how it is consumed, and what guarantees do (or do not) hold.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Expose a frozen, allow-listed set of methods on the renderer's global window object via the preload bridge.
|
||||
- Register one `ipcMain` handler per exposed method, grouped by concern (`system`, `window-controls`, `cqrs`).
|
||||
- Provide a CQRS abstraction over the local database (commands + queries dispatched through two generic channels).
|
||||
- Translate main-process operations into renderer-safe values (file paths → URLs, native errors → structured responses where appropriate).
|
||||
|
||||
This area does **not** own:
|
||||
|
||||
- The schema or business logic behind any specific command/query (those live in `electron/cqrs/handlers/` and the affected product-client domains).
|
||||
- The plugin manifest contract (see [plugin-system](./plugin-system.md)) — only the IPC methods that surface it.
|
||||
- WebSocket signaling (see [websocket-envelopes](./websocket-envelopes.md)) — that bypasses Electron entirely.
|
||||
|
||||
## Key concepts
|
||||
|
||||
- **Preload bridge** — `electron/preload.ts`. The sole place `contextBridge.exposeInMainWorld` runs. Adding a method here requires a matching `ipcMain.handle` / `ipcMain.on` on the main side.
|
||||
- **Window surface** — exposed as `window.electronAPI` on the renderer. (Note: `electron/CONTEXT.md` refers to this as `window.api.*` — the documented intent. The literal global today is `electronAPI`. TODO: pick one and align.)
|
||||
- **CQRS channel** — two reserved channels `cqrs:command` and `cqrs:query` route every typed `Command`/`Query` through a single dispatch step, instead of one channel per operation.
|
||||
- **Handler setup function** — registered once at app boot from `electron/ipc/index.ts`: `setupCqrsHandlers`, `setupSystemHandlers`, `setupWindowControlHandlers`.
|
||||
- **Renderer bridge service** — `ElectronBridgeService` in `toju-app/src/app/core/platform/electron/electron-bridge.service.ts` is the Angular-side wrapper; domain services inject it rather than reaching for `window.electronAPI` directly.
|
||||
|
||||
---
|
||||
|
||||
## Surface catalogue
|
||||
|
||||
Defined in `electron/preload.ts`. Approximately 50 methods, grouped below by concern. For exact signatures see the file.
|
||||
|
||||
### Window controls (fire-and-forget)
|
||||
|
||||
- `minimizeWindow`, `maximizeWindow`, `closeWindow` — channels `window-minimize`, `window-maximize`, `window-close`. Implementation: `electron/ipc/window-controls.ts`. Uses `ipcMain.on` (no return value).
|
||||
|
||||
### Screen share & media
|
||||
|
||||
- `getSources` — DesktopCapturer source enumeration.
|
||||
- Linux audio routing for screen-share monitor capture: `prepareLinuxScreenShareAudioRouting`, `activateLinuxScreenShareAudioRouting`, `deactivateLinuxScreenShareAudioRouting`, `startLinuxScreenShareMonitorCapture`, `stopLinuxScreenShareMonitorCapture`.
|
||||
- Event listeners: `onLinuxScreenShareMonitorAudioChunk`, `onLinuxScreenShareMonitorAudioEnded`.
|
||||
|
||||
### Process & game detection
|
||||
|
||||
- `getRunningProcessNames` (via `electron/process-list.ts`).
|
||||
- `getActiveGameCandidate` (via `electron/game-detection/`).
|
||||
- `getIgnoredGameProcesses`, `setIgnoredGameProcesses`.
|
||||
|
||||
### File system
|
||||
|
||||
- `readFile`, `readFileChunk`, `getFileSize`, `writeFile`, `appendFile`, `deleteFile`, `fileExists`, `getFileUrl`, `ensureDir`, `saveFileAs`, `saveExistingFileAs`, `openFilePath`, `readClipboardFiles`.
|
||||
- `getFileUrl` is the canonical way for the renderer to display a local file via `file://` — direct path access is forbidden.
|
||||
|
||||
### Theme & plugins (filesystem-backed)
|
||||
|
||||
- `getSavedThemesPath`, `listSavedThemes`, `readSavedTheme`, `writeSavedTheme`, `deleteSavedTheme`.
|
||||
- `getLocalPluginsPath`, `listLocalPluginManifests`. See [plugin-system](./plugin-system.md) for the manifest contract.
|
||||
|
||||
### Settings & notifications
|
||||
|
||||
- `getDesktopSettings`, `setDesktopSettings`.
|
||||
- `showDesktopNotification`, `requestWindowAttention`, `clearWindowAttention`, `onWindowStateChanged`.
|
||||
|
||||
### Auto-update
|
||||
|
||||
- `getAutoUpdateState`, `getAutoUpdateServerHealth`, `configureAutoUpdateContext`, `checkForAppUpdates`, `restartToApplyUpdate`, `onAutoUpdateStateChanged`.
|
||||
|
||||
### Local API & docs
|
||||
|
||||
- `getLocalApiStatus`, `openLocalApiDocs`, `openDocusaurusDocs`. The Local API server hosts the prebuilt Docusaurus bundle inside the desktop app — see `electron/api/local-api-server.ts`.
|
||||
|
||||
### App & deep links
|
||||
|
||||
- `relaunchApp`, `consumePendingDeepLink`, `onDeepLinkReceived`, `getAppDataPath`, `openCurrentDataFolder`.
|
||||
|
||||
### Data management
|
||||
|
||||
- `exportUserData`, `importUserData`, `eraseUserData`. Backed by `electron/data-archive.ts`.
|
||||
|
||||
### Clipboard & context menu
|
||||
|
||||
- `copyImageToClipboard`, `onContextMenu`, `contextMenuCommand`.
|
||||
|
||||
### Idle state
|
||||
|
||||
- `getIdleState`, `onIdleStateChanged`. Backed by `electron/idle/`.
|
||||
|
||||
### CQRS (typed database access)
|
||||
|
||||
- `command<T>(command: Command) => Promise<T>` → channel `cqrs:command`.
|
||||
- `query<T>(query: Query) => Promise<T>` → channel `cqrs:query`.
|
||||
- Command and query union types live in `electron/cqrs/types.ts`. Handlers are built dynamically per `DataSource` via `buildCommandHandlers(dataSource)` and `buildQueryHandlers(dataSource)` in `electron/ipc/cqrs.ts`.
|
||||
- Current commands: `SaveMessage`, `DeleteMessage`, `UpdateMessage`, `ClearRoomMessages`, `SaveReaction`, `RemoveReaction`, `SaveUser`, `SetCurrentUserId`, `UpdateUser`, `SaveRoom`, `DeleteRoom`, `UpdateRoom`, `SaveBan`, `RemoveBan`, `SaveAttachment`, `DeleteAttachmentsForMessage`, `SavePluginData`, `DeletePluginData`, `SaveMeta`, `ClearAllData`.
|
||||
- Current queries: `GetMessages`, `GetMessagesSince`, `GetMessageById`, `GetReactionsForMessage`, `GetUser`, `GetCurrentUser`, `GetCurrentUserId`, `GetUsersByRoom`, `GetRoom`, `GetAllRooms`, `GetBansForRoom`, `IsUserBanned`, `GetAttachmentsForMessage`, `GetAllAttachments`, `GetPluginData`, `GetMeta`.
|
||||
- Unknown `type` raises `Error("No command/query handler for type: ${type}")`.
|
||||
|
||||
---
|
||||
|
||||
## Renderer consumption
|
||||
|
||||
- **`ElectronBridgeService`** (`toju-app/src/app/core/platform/electron/electron-bridge.service.ts`) — provides `getApi(): ElectronApi | null` and `requireApi(): ElectronApi`. Domain services inject the bridge service, never `window` directly. This also makes the bridge mockable for spec runs and the website preview (where `window.electronAPI` is absent).
|
||||
- **CQRS wrapper**: `toju-app/src/app/infrastructure/persistence/electron-database.service.ts` wraps `api.command()` / `api.query()` with typed helpers; product-client domains use this rather than calling CQRS directly.
|
||||
- **Per-domain consumers**: file I/O (`attachment`), theme (`theme`), profile-avatar, notifications, idle (used by presence), and game-activity domains each inject the bridge to reach their respective IPC slice.
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
`electron/CONTEXT.md` says:
|
||||
|
||||
> IPC handler errors are translated to typed error envelopes before crossing back into the renderer — the renderer never sees a raw `Error` from main.
|
||||
|
||||
In practice today:
|
||||
|
||||
- The CQRS layer throws raw `Error` objects on unknown `type` (caller sees the serialized message).
|
||||
- Most `electron/ipc/system.ts` handlers catch errors and return structured response objects (e.g. `{ opened: false, reason: string }`), but the shape is per-handler, not centralised.
|
||||
- There is no global error-envelope wrapper around `ipcMain.handle`.
|
||||
|
||||
**TODO**: reconcile the CONTEXT.md invariant with reality — either introduce a shared error-envelope wrapper or update the invariant to match the per-handler convention. Until then, treat error shapes as a per-method concern.
|
||||
|
||||
---
|
||||
|
||||
## Technical implementation
|
||||
|
||||
- **Preload**: `electron/preload.ts` (single source of truth for the exposed surface).
|
||||
- **Registration**: `electron/ipc/index.ts` calls three setup functions at app boot — `setupCqrsHandlers`, `setupSystemHandlers`, `setupWindowControlHandlers`.
|
||||
- **System handlers**: `electron/ipc/system.ts` (~40 channels, ~780 lines).
|
||||
- **Window controls**: `electron/ipc/window-controls.ts` (3 channels, fire-and-forget).
|
||||
- **CQRS handlers**: `electron/ipc/cqrs.ts` plus typed command/query unions in `electron/cqrs/types.ts` and per-handler implementations under `electron/cqrs/handlers/`.
|
||||
- **Local SQLite access** is gated behind CQRS — no other channel exposes the database directly. See `electron/data-source.ts` and `electron/entities/`.
|
||||
|
||||
## Invariants
|
||||
|
||||
- The renderer never imports `electron`, Node APIs, or TypeORM directly. (Enforced by Electron's `contextIsolation` + no `nodeIntegration`.)
|
||||
- Every method on `window.electronAPI` has exactly one IPC channel and exactly one main-process handler.
|
||||
- Schema mutations go through a TypeORM migration in `electron/migrations/`; raw SQL never crosses the IPC bridge.
|
||||
- All file access is path-based on the main side, URL-based on the renderer side (`getFileUrl`).
|
||||
|
||||
## Testing
|
||||
|
||||
- `electron/plugin-library.spec.ts` — plugin discovery (touches the same IPC path but tests the library, not the channel).
|
||||
- `electron/idle/idle-monitor.spec.ts` — idle source unit test.
|
||||
- **TODO**: no spec covers `preload.ts` exposure, the system handler set, the CQRS dispatcher, or the error path. Renderer-side `ElectronBridgeService` spec status not verified.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- `contextIsolation: true` + no `nodeIntegration` in the renderer; `electron/preload.ts` is the only crossing.
|
||||
- Adding a channel requires touching both `preload.ts` and `electron/ipc/`. There is no dynamic channel registration.
|
||||
- File-system handlers should validate paths against user-data scope — TODO: audit `system.ts` for path-traversal protections beyond what the plugin loader does.
|
||||
- Deep-link handling: `consumePendingDeepLink` returns a queued URL; validation lives in renderer routing. TODO: confirm allow-list / scheme filtering on the main side.
|
||||
|
||||
## Performance considerations
|
||||
|
||||
- IPC traffic is per-call serialization; large payloads (file chunks, attachment imports) go via `readFileChunk` + offsets instead of single `readFile` to avoid blocking the main process.
|
||||
- CQRS calls hit the local SQLite database synchronously inside the main process. There is no batching layer.
|
||||
|
||||
## Known issues and limitations
|
||||
|
||||
- **Documented vs. actual API name** — the `window` global is `electronAPI`, not `api`. CONTEXT.md uses `window.api.*`. Reconcile in a future cleanup.
|
||||
- **No typed error envelope** despite the CONTEXT.md invariant.
|
||||
- **No preload-surface test** — additions are caught only at runtime / lint.
|
||||
|
||||
## Related features
|
||||
|
||||
- **[plugin-system](./plugin-system.md)** — surfaces `getLocalPluginsPath`, `listLocalPluginManifests`, and plugin data CQRS commands through this bridge.
|
||||
- **[websocket-envelopes](./websocket-envelopes.md)** — the realtime path that bypasses the bridge; included here only to delineate the two surfaces.
|
||||
- **[voice-signaling](./voice-signaling.md)** — uses `getSources` and the Linux audio routing methods for screen-share media capture.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2026-05-25 | Initial documentation |
|
||||
180
agents-docs/features/plugin-system.md
Normal file
180
agents-docs/features/plugin-system.md
Normal file
@@ -0,0 +1,180 @@
|
||||
# Plugin System
|
||||
|
||||
> **Area:** plugin-system
|
||||
> **Status:** Active
|
||||
> **Last updated:** 2026-05-25
|
||||
|
||||
## Overview
|
||||
|
||||
Plugins extend Toju's renderer with bundled or local ES modules that can publish events into a server, register UI contributions (pages, panels, actions, channel sections, embed renderers), store per-user or per-server data, and exchange messages over P2P. They are described by a typed manifest, gated by an explicit capability grant model, and coordinated across three subdomains: the Electron plugin loader (`electron/plugin-library.ts`), the renderer plugin runtime (`toju-app/src/app/domains/plugins/`), and the server's plugin-support surface (`server/src/routes/plugin-support.ts`). This area documents the contract those three sides share.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Define the canonical plugin manifest shape (`TojuPluginManifest`) and its semantic versioning, dependency, and capability requirements.
|
||||
- Discover local plugin manifests from disk in Electron and surface them to the renderer.
|
||||
- Load plugin modules — local file://, http(s)://, or bundled — into the renderer and run their lifecycle hooks.
|
||||
- Gate every host API call by an explicit capability grant.
|
||||
- Persist server-scoped plugin requirements and event definitions on the signaling server, broadcasting changes to connected clients.
|
||||
|
||||
This area does **not** own:
|
||||
|
||||
- The wire format of `plugin_event` envelopes themselves — those live with [websocket-envelopes](./websocket-envelopes.md); this area defines the **validation rules** applied to them.
|
||||
- IPC plumbing for plugin manifest discovery — that's the [ipc-bridge](./ipc-bridge.md) surface.
|
||||
- Per-plugin business logic — that lives in the plugin's own code.
|
||||
|
||||
## Key concepts
|
||||
|
||||
- **Manifest** — `TojuPluginManifest` (`toju-app/src/app/shared-kernel/plugin-system.contracts.ts`). Required: `id`, `title`, `description`, `version`, `apiVersion`, `kind`, `schemaVersion` (fixed `1`), `compatibility.minimumTojuVersion`. Optional: `entrypoint`, `bundle`, `capabilities[]`, `events[]`, `data[]`, `ui`, `settings`, `relationships`, `authors`, `pluginUser`, `scope`, `homepage`, `changelog`, `license`, `readme`, `load.priority`.
|
||||
- **Capability** — a string ID (e.g. `messages.send`, `events.server.publish`, `ui.pages`, `storage.local`). Plugins declare what they need in `capabilities[]`; the host enforces grants per plugin.
|
||||
- **Plugin event** — a declared `{ eventName, direction, scope, schema?, maxPayloadBytes? }` tuple. `direction` is `clientToServer | serverRelay | p2pHint`; `scope` is `server | channel | user | plugin`.
|
||||
- **Capability grant** — an entry in `metoyou_plugin_capability_grants` (localStorage + desktop file) recording user consent for a `(pluginId, capability)` pair.
|
||||
- **Activation context** — `TojuPluginActivationContext` — what a plugin module receives in its `activate(context)` hook: `pluginId`, `manifest`, `api` (the capability-gated `TojuClientPluginApi`), and a `subscriptions[]` cleanup list.
|
||||
- **Local plugin** — a folder under `${app.getPath('userData')}/plugins/<id>/` containing `toju-plugin.json` (preferred) or `plugin.json` and resolved relative assets.
|
||||
|
||||
---
|
||||
|
||||
## Manifest contract
|
||||
|
||||
Declared in `toju-app/src/app/shared-kernel/plugin-system.contracts.ts` (the source of truth on the renderer side; the Electron loader treats the manifest as `unknown` until the renderer validates it).
|
||||
|
||||
Highlights:
|
||||
|
||||
- **`schemaVersion: 1`** — fixed; bump only with a coordinated migration.
|
||||
- **`apiVersion`** — declares which host API surface the plugin expects.
|
||||
- **`kind`** — distinguishes plugin types (currently a string union; see file for exact members).
|
||||
- **`bundle.url`** + optional `bundle.entrypointUrl` — for remote / store-installed plugins.
|
||||
- **`entrypoint`** — relative path within the local plugin folder; rejected if it escapes `pluginRoot`.
|
||||
- **`relationships.{requires,optional,conflicts,before,after}`** — `pluginId` + optional `versionRange`. Resolved at activation; missing required dependencies block activation.
|
||||
- **`events[]`** — registered with the server via `plugin-support` when the plugin is required by a server. The server uses these definitions to validate inbound `plugin_event` envelopes.
|
||||
- **`data[]`** — `{ key, scope: server|channel|user|plugin, storage: local|serverData, schema? }`.
|
||||
- **`load.priority`** — `bootstrap | high | default | low`; controls ordering when several plugins are activated together.
|
||||
- **`pluginUser`** — synthetic user identity for messages a plugin posts on its own behalf.
|
||||
|
||||
---
|
||||
|
||||
## Discovery & loading
|
||||
|
||||
### Local discovery (Electron main)
|
||||
|
||||
`electron/plugin-library.ts`:
|
||||
|
||||
- Scans `${app.getPath('userData')}/plugins/` **one level deep** for plugin folders.
|
||||
- For each folder, reads `toju-plugin.json` (preferred) or `plugin.json` as raw JSON. No schema validation at this layer.
|
||||
- Resolves relative paths (`entrypoint`, `readme`) against the plugin root, rejecting `..`, absolute paths, or anything that escapes the root via `isPathInside()` realpath check.
|
||||
- Returns `LocalPluginDiscoveryResult { plugins, errors, pluginsPath }` where each `plugins[i]` is a `LocalPluginManifestDescriptor` carrying the raw manifest, manifest path, plugin root, plugin-root `file://` URL, optional entrypoint and readme paths, and a `discoveredAt` timestamp.
|
||||
|
||||
Exposed to the renderer through the [ipc-bridge](./ipc-bridge.md) as `listLocalPluginManifests` and `getLocalPluginsPath`.
|
||||
|
||||
### Renderer runtime
|
||||
|
||||
`toju-app/src/app/domains/plugins/`:
|
||||
|
||||
- **`PluginHostService`** — orchestrates lifecycle: `discoverLocalPlugins`, `registerLocalManifest`, `activate`, `deactivate`, `reload`, `loadPluginModule`.
|
||||
- **`PluginClientApiService`** — constructs the capability-gated facade `TojuClientPluginApi` per plugin. Subsystems: `channels`, `events`, `messages`, `messageBus`, `p2p`, `profile`, `users`, `roles`, `server`, `attachments`, `media`, `storage`, `serverData`, `clientData`, `ui`, `logger`, `context`.
|
||||
- **`PluginCapabilityService`** — `grant(pluginId, capability)`, `revoke()`, `grantAll(manifest)`, `assert(pluginId, capability)`. Storage in `metoyou_plugin_capability_grants` (localStorage + desktop file).
|
||||
- **`PluginMessageBusService`** — plugin-scoped pub/sub with topic, optional channel/peer targeting, optional message replay.
|
||||
- **`PluginStorageService`** — split storage paths for `local` and `serverData` scopes.
|
||||
- **`PluginUiRegistryService`** — central registry of UI contributions consumed by `plugin-render-host`, `plugin-page-host`, `plugin-action-menu`.
|
||||
- **`PluginRequirementStateService`**, **`PluginDesktopStateService`** — state slices.
|
||||
|
||||
### Module loading
|
||||
|
||||
- Entrypoint URL is `file://` for local plugins, `http(s)://` for remote, or the `bundle.url` for bundled.
|
||||
- Bytes are fetched, wrapped in a `Blob`-backed object URL, then imported via dynamic `import()` so devtools and stack traces resolve.
|
||||
- `GuardedPluginMutationObserver` wraps observer callbacks to catch plugin errors and break infinite redispatch loops.
|
||||
|
||||
### Lifecycle states
|
||||
|
||||
From `plugin-runtime.models.ts`: `discovered → validated → blocked | loading → ready → loaded → failed | unloading → unloaded → disabled`.
|
||||
|
||||
### Module contract
|
||||
|
||||
```ts
|
||||
export interface TojuClientPluginModule {
|
||||
activate?(context: TojuPluginActivationContext): void | Promise<void>;
|
||||
deactivate?(context: TojuPluginActivationContext): void | Promise<void>;
|
||||
ready?(context: TojuPluginActivationContext): void | Promise<void>;
|
||||
onServerRequirementsChanged?(context, snapshot): void | Promise<void>;
|
||||
onPluginDataChanged?(context, event): void | Promise<void>;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Server surface (`server/src/routes/plugin-support.ts`)
|
||||
|
||||
Each endpoint scoped to `:serverId`. Permission `manageServer` enforced for mutations via `server-permissions.service.ts`.
|
||||
|
||||
- `GET /:serverId/plugins` → `PluginRequirementsSnapshot` — full set of required/optional plugins + event definitions for the server.
|
||||
- `PUT /:serverId/plugins/:pluginId/requirement` — upsert a requirement: `status: required | optional | recommended | blocked | incompatible`, `installUrl?`, `sourceUrl?`, `manifest?`, `versionRange?`.
|
||||
- `DELETE /:serverId/plugins/:pluginId/requirement` — remove.
|
||||
- `PUT /:serverId/plugins/:pluginId/events/:eventName` — register or update an event definition (`direction`, `scope`, `schema?`, `maxPayloadBytes?`).
|
||||
- `DELETE /:serverId/plugins/:pluginId/events/:eventName` — delete.
|
||||
- `GET|PUT|DELETE /:serverId/plugins/:pluginId/data/:key` — **disabled** on the signaling server (returns HTTP 410). Plugin data on the server is intentionally out of scope.
|
||||
|
||||
Identifier patterns:
|
||||
|
||||
- `pluginId`: `/^[a-z0-9][a-z0-9.-]{1,126}[a-z0-9]$/`
|
||||
- `eventName`: `/^[a-z][a-z0-9.:-]{1,126}[a-z0-9]$/`
|
||||
|
||||
Changes broadcast to connected clients via the [websocket-envelopes](./websocket-envelopes.md) — the matching `PluginRequirementsSnapshot` is also delivered as part of the `join_server` and `view_server` responses.
|
||||
|
||||
---
|
||||
|
||||
## Plugin event validation
|
||||
|
||||
`server/src/services/plugin-support.service.ts` exposes `validatePluginEventEnvelope()`. The `plugin_event` envelope handler (`server/src/websocket/handler.ts`) calls it before broadcasting. Validation checks:
|
||||
|
||||
- `eventName` is registered for `pluginId` on `serverId`.
|
||||
- `direction` permits the source (clientToServer vs p2pHint vs serverRelay).
|
||||
- `payload` size ≤ `maxPayloadBytes` (default 64 KB).
|
||||
- If `schema` was declared in the manifest, the payload conforms — TODO: confirm the schema dialect (looks like JSON Schema subset).
|
||||
|
||||
## Security considerations
|
||||
|
||||
- **Plugins run in the renderer's JS context.** There is no true sandbox: plugins have DOM access, can read/write `localStorage`, and can issue `fetch` requests subject to the renderer's CSP.
|
||||
- **Capability model is the primary security boundary.** Every method on `TojuClientPluginApi` calls `PluginCapabilityService.assert(pluginId, capability)`. Missing grant → `PluginCapabilityError`.
|
||||
- **Path traversal** in the local plugin loader is blocked by `isPathInside()` realpath checks (`electron/plugin-library.ts`).
|
||||
- **Payload bounds** on plugin events: default 64 KB, configurable per event definition.
|
||||
- **No code signing or integrity verification.** Plugins are trusted to the extent the user granted capabilities. The plugin store flow is documented in `docs-site/docs/plugin-development/`.
|
||||
- **TODO**: review whether `bundle.url` fetches go through a CSP / allow-list or are unbounded.
|
||||
|
||||
## Configuration
|
||||
|
||||
- **Local plugins path**: `${app.getPath('userData')}/plugins/` (Electron). Exposed as `getLocalPluginsPath`.
|
||||
- **Capability grants**: `metoyou_plugin_capability_grants` in localStorage; mirrored to a desktop file via `PluginDesktopStateService`.
|
||||
- **Server-side persistence**: plugin requirements and event definitions are stored in the server's database; entities and migrations live alongside other server entities (TODO: enumerate the specific entities).
|
||||
|
||||
## Testing
|
||||
|
||||
- **Electron**: `electron/plugin-library.spec.ts` — covers valid/invalid JSON, path traversal rejection, escaping entrypoints. Uses fixture `TEST_PLUGIN_FIXTURE_DIR`.
|
||||
- **Renderer**: `plugin-host.service.spec.ts`, `plugin-store.service.spec.ts`, `local-plugin-discovery.service.spec.ts`.
|
||||
- **Server**: `server/src/websocket/handler-plugin.spec.ts` — `plugin_event` validation flow.
|
||||
- **E2E**: `e2e/tests/.../plugin-support-api.spec.ts`, `plugin-manager-ui.spec.ts`, `plugin-api-two-users.spec.ts`.
|
||||
|
||||
## Known issues and limitations
|
||||
|
||||
- **Server-side plugin data is intentionally disabled** (`410 Gone` on `*/data/:key` routes). Plugins must currently treat `serverData` storage as not yet implemented on the signaling server. TODO: clarify whether this is a permanent boundary or scheduled work.
|
||||
- **No true sandbox** for plugin execution. The capability model is the only restraint between a plugin and the renderer's globals.
|
||||
- **Manifest validation is renderer-side.** The Electron loader treats manifests as raw JSON; malformed or hostile manifests are caught when the renderer registers them.
|
||||
- **No host-side plugin allow-list** beyond per-server requirements — a user can install any local plugin.
|
||||
|
||||
## Related features
|
||||
|
||||
- **[ipc-bridge](./ipc-bridge.md)** — surfaces `listLocalPluginManifests`, `getLocalPluginsPath`, and the plugin CQRS commands (`SavePluginData`, `DeletePluginData`, `GetPluginData`).
|
||||
- **[websocket-envelopes](./websocket-envelopes.md)** — defines the `plugin_event` envelope this area validates.
|
||||
- **[server-directory](./server-directory.md)** — `join_server` / `view_server` responses include `PluginRequirementsSnapshot` for the joined server.
|
||||
|
||||
## Documentation for plugin authors
|
||||
|
||||
Author-facing docs ship with the desktop app via Docusaurus:
|
||||
|
||||
- `docs-site/docs/plugin-development/create-a-plugin.md`
|
||||
- `docs-site/docs/user-guide/plugins.md`
|
||||
- `docs-site/docs/developer/llm-plugin-builder-guide.md`
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2026-05-25 | Initial documentation |
|
||||
174
agents-docs/features/server-directory.md
Normal file
174
agents-docs/features/server-directory.md
Normal file
@@ -0,0 +1,174 @@
|
||||
# Server Directory
|
||||
|
||||
> **Area:** server-directory
|
||||
> **Status:** Active
|
||||
> **Last updated:** 2026-05-25
|
||||
|
||||
## Overview
|
||||
|
||||
The Server Directory is the public REST surface that lists joinable Toju chat servers, manages invites and join requests, gates membership (passwords, bans, ownership), and exposes moderation actions (kick / ban / unban). It is the only feature where the signaling server holds non-ephemeral, multi-user state: the persistent catalog of servers, their access rules, their memberships, and their pending join requests. The renderer's `server-directory` domain consumes this surface to render the "find a server" experience and to drive the join flow that eventually opens a WebSocket (see [websocket-envelopes](./websocket-envelopes.md)).
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Persist the catalog of servers, their access policy (public/private, password, max users), and ownership.
|
||||
- Mint invites and accept invite redemptions.
|
||||
- Track join requests on private servers and route owner decisions back to the requester.
|
||||
- Track memberships and bans; enforce them on join attempts.
|
||||
- Provide moderation primitives: kick, ban, unban — gated by role/owner permissions.
|
||||
- Emit user-targeted notifications when a join request changes state.
|
||||
|
||||
This area does **not** own:
|
||||
|
||||
- Realtime presence, chat, or voice — those flow over the WebSocket once a user has joined (see [websocket-envelopes](./websocket-envelopes.md), [voice-signaling](./voice-signaling.md)).
|
||||
- Per-channel permissions logic (lives in `server/src/services/server-permissions.service.ts` and is consumed by this area, but is reused beyond it).
|
||||
|
||||
## Key concepts
|
||||
|
||||
- **Server** — a joinable chat server. Persisted as `ServerEntity` (`servers` table).
|
||||
- **Public / private** — `isPrivate` flag. Public servers appear in directory listings; private servers do not.
|
||||
- **Invite** — an opaque token (`ServerInviteEntity`) that grants short-lived access to a specific server. Expires after `SERVER_INVITE_EXPIRY_MS` (10 days).
|
||||
- **Join request** — a pending request on a private server (`JoinRequestEntity`), `pending → approved | denied`.
|
||||
- **Membership** — a `ServerMembershipEntity` row, indexed by `serverId` + `userId`.
|
||||
- **Ban** — a `ServerBanEntity` row, optionally `expiresAt`. Auto-pruned on the next join attempt for the banned user.
|
||||
- **Heartbeat** — periodic `POST /:id/heartbeat` from the server owner's client that updates `lastSeen` and `currentUsers` on the directory entry.
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
All HTTP routes; no auth header — caller identity is supplied per-request in the body (`ownerId`, `actorUserId`, `userId`, `requesterUserId`). Identity is whatever the client claims; authorization is enforced against persisted state. Request body validation is **manual / defensive** (no zod or class-validator).
|
||||
|
||||
### `server/src/routes/servers.ts`
|
||||
|
||||
| Method | Path | Purpose | Auth |
|
||||
|--------|------|---------|------|
|
||||
| `GET` | `/` | List public servers. Query: `q`, `tags`, `limit`, `offset`. | None |
|
||||
| `POST` | `/` | Create a server. Required body: `name`, `ownerId`, `ownerPublicKey`. | Self-asserted |
|
||||
| `GET` | `/:id` | Fetch a single server. 404 if missing. | None |
|
||||
| `PUT` | `/:id` | Update a server. Required body: `currentOwnerId`. Permission check via `canManageServerUpdate`. | Owner / role |
|
||||
| `POST` | `/:id/join` | Join a server. Required body: `userId`. Optional: `password`, `inviteId`. Returns `signalingUrl`. | Self-asserted + access rules |
|
||||
| `POST` | `/:id/invites` | Create an invite. Required body: `requesterUserId`. Delegates to `createServerInvite`. | Role permission |
|
||||
| `POST` | `/:id/moderation/kick` | Kick a user. Required: `actorUserId`, `targetUserId`. Permission: `canModerateServerMember`. | Role permission |
|
||||
| `POST` | `/:id/moderation/ban` | Ban a user. Required: `actorUserId`, `targetUserId`. Optional: `banId`, `reason`, `expiresAt`. | Role permission |
|
||||
| `POST` | `/:id/moderation/unban` | Unban a user. Required: `actorUserId`. Permission: `manageBans`. | Role permission |
|
||||
| `POST` | `/:id/leave` | Leave a server. Required body: `userId`. | Self-asserted |
|
||||
| `POST` | `/:id/heartbeat` | Update `lastSeen` and `currentUsers`. Optional body: `currentUsers`. | None (TODO: confirm) |
|
||||
| `DELETE` | `/:id` | Delete a server. Required body: `ownerId` (must match `server.ownerId`). | Owner |
|
||||
| `GET` | `/:id/requests` | List pending join requests. Query: `ownerId`. | Owner |
|
||||
|
||||
### `server/src/routes/invites.ts`
|
||||
|
||||
- `GET /invites/:id` (API) — fetch invite metadata; `404` for expired or unknown invite.
|
||||
- `GET /invites/:id` (page router) — server-rendered HTML preview of the invite (server info, owner, expiry); renders an offline state when the server is unreachable.
|
||||
|
||||
### `server/src/routes/join-requests.ts`
|
||||
|
||||
- `PUT /requests/:id` — update join-request status. Body: `ownerId`, `status`. Permission: `manageServer`. On success, calls `notifyUser` (WebSocket fan-out, see below).
|
||||
|
||||
### Standard error codes
|
||||
|
||||
`SERVER_NOT_FOUND`, `MISSING_USER`, `NOT_AUTHORIZED`, `BANNED`, `PASSWORD_REQUIRED`, `INVITE_EXPIRED`, plus 400 for missing required fields.
|
||||
|
||||
---
|
||||
|
||||
## CQRS handlers
|
||||
|
||||
`server/src/cqrs/` backs every mutation; routes are thin adapters around CQRS dispatch.
|
||||
|
||||
**Queries** (`server/src/cqrs/queries/handlers/`):
|
||||
|
||||
- `getAllPublicServers` — filtered by `isPrivate = 0`, loads relations.
|
||||
- `getServerById`
|
||||
- `getJoinRequestById`
|
||||
- `getPendingRequestsForServer`
|
||||
|
||||
**Commands** (`server/src/cqrs/commands/handlers/`):
|
||||
|
||||
- `upsertServer` — also calls `replaceServerRelations` to sync `tags`, `channels`, `roles`, `roleAssignments`, `channelPermissions` atomically.
|
||||
- `deleteServer`
|
||||
- `createJoinRequest`
|
||||
- `updateJoinRequestStatus` — emits a `notifyUser` event so the requesting user's client learns the outcome over WebSocket.
|
||||
|
||||
All handlers run inside TypeORM transactions where multi-table changes are involved.
|
||||
|
||||
---
|
||||
|
||||
## Persistence
|
||||
|
||||
### Entities (`server/src/entities/`)
|
||||
|
||||
- `ServerEntity` (table `servers`) — `id`, `name`, `description`, `ownerId`, `ownerPublicKey`, `passwordHash`, `isPrivate`, `maxUsers`, `currentUsers`, `icon`, `iconUpdatedAt`, `slowModeInterval`, `createdAt`, `lastSeen`.
|
||||
- `ServerInviteEntity` (`server_invites`) — `id`, `serverId` (indexed), `createdBy`, `createdByDisplayName`, `createdAt`, `expiresAt` (indexed).
|
||||
- `JoinRequestEntity` (`join_requests`) — `id`, `serverId` (indexed), `userId`, `userPublicKey`, `displayName`, `status` (default `pending`), `createdAt`.
|
||||
- `ServerMembershipEntity` (`server_memberships`) — `id`, `serverId` (indexed), `userId` (indexed), `joinedAt`, `lastAccessAt`.
|
||||
- `ServerBanEntity` (`server_bans`) — `id`, `serverId` (indexed), `userId` (indexed), `bannedBy`, `displayName`, `reason`, `expiresAt` (nullable), `createdAt`.
|
||||
|
||||
Related (referenced by `replaceServerRelations`): `ServerChannelEntity`, `ServerRoleEntity`, `ServerUserRoleEntity`, `ServerTagEntity`, `ServerChannelPermissionEntity`.
|
||||
|
||||
### Migrations (`server/src/migrations/`)
|
||||
|
||||
- `1000000000000-InitialSchema.ts` — `servers`, `users`.
|
||||
- `1000000000001-ServerAccessControl.ts` — adds `passwordHash` to `servers`; creates `server_memberships`, `server_invites`, `server_bans` with indices.
|
||||
- `1000000000002-ServerChannels.ts` — `server_channels`.
|
||||
- `1000000000005-ServerRoleAccessControl.ts` — role/permission tables.
|
||||
- TODO: locate the migration that created `join_requests` (not obvious from filenames; likely folded into an earlier migration).
|
||||
|
||||
---
|
||||
|
||||
## Renderer side
|
||||
|
||||
`toju-app/src/app/domains/server-directory/`:
|
||||
|
||||
- **API client**: `infrastructure/services/server-directory-api.service.ts` — `ServerDirectoryApiService` exposes `searchServers`, `getServers`, `getServer`, `findServerAcrossActiveEndpoints`, `registerServer`, `updateServer`, `requestJoin`, `createInvite`, `getInvite`, `kickServerMember`, `banServerMember`, `unbanServerMember`, `notifyLeave`, `sendHeartbeat`. Defensive coercion (`getNumberValue` / `getStringValue` / `getBooleanValue`) is used instead of schema validation.
|
||||
- **State**: signal-based via `ServerEndpointStateService` (servers, active server) — not NgRx for this slice.
|
||||
- **Facade**: `application/services/server-directory.service.ts` plus `application/facades/`.
|
||||
- **Multi-endpoint awareness**: Toju supports several federated signaling endpoints; `findServerAcrossActiveEndpoints` queries each and merges results.
|
||||
|
||||
---
|
||||
|
||||
## Business rules
|
||||
|
||||
- **Public-only listing**: `GET /` only returns servers with `isPrivate = 0`. Private servers must be reached by ID + invite.
|
||||
- **Owner immutability**: only `currentOwnerId` matching `server.ownerId` may update; only `ownerId` matching `server.ownerId` may delete.
|
||||
- **Join order of checks** (on `POST /:id/join`): existence → ban check (auto-prune expired bans) → password check (if `passwordHash`) → invite check (if private and no invite) → membership upsert → return `signalingUrl`.
|
||||
- **Invite expiry**: 10 days (`SERVER_INVITE_EXPIRY_MS = 10 * 24 * 60 * 60 * 1000`). Expired invites are pruned on access via `pruneExpiredServerAccessArtifacts()`.
|
||||
- **Ban expiry**: optional `expiresAt`; auto-deleted on next join attempt for that user.
|
||||
- **Join request notifications**: on `PUT /requests/:id`, after CQRS dispatch, `notifyUser` pushes the new status over WebSocket to any open connection for `userPublicKey` / `userId`.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- **No authentication header.** All identity is self-asserted in the request body. Authorization is enforced by checking the claimed identity against persisted role/owner state.
|
||||
- **Password storage**: `passwordHash` only; never the cleartext. TODO: confirm the hashing algorithm (likely bcrypt / scrypt — verify in `server/src/services/`).
|
||||
- **SSRF**: routes in this area do not fetch user-supplied URLs, so the SSRF guard does not apply here (it applies to link-metadata, klipy, proxy).
|
||||
- **No rate limiting** on directory or moderation routes — TODO: add brute-force protection on `POST /:id/join` for password attempts.
|
||||
- **No CSRF** (REST + JSON body, no cookies in scope), but spam protection on `POST /` (server creation) is also TODO.
|
||||
|
||||
## Configuration
|
||||
|
||||
- `SERVER_INVITE_EXPIRY_MS` — currently hardcoded at 10 days. Not exposed via `data/variables.json`.
|
||||
- Per-server `maxUsers`, `slowModeInterval`, `isPrivate`, `passwordHash` are operator-configurable via `PUT /:id`.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Server-side**: no direct route specs for `servers.ts`, `invites.ts`, `join-requests.ts`. WebSocket-side handlers (`handler-status.spec.ts`, `handler-plugin.spec.ts`) cover adjacent concerns.
|
||||
- **Renderer-side**: `application/services/server-endpoint-state.service.spec.ts`.
|
||||
- **E2E**: TODO — verify whether the Playwright suite covers join / invite / moderation end-to-end.
|
||||
- **Gap**: routes that mutate persistent state and accept self-asserted identity should ideally have integration tests against a real DB.
|
||||
|
||||
## Known issues and limitations
|
||||
|
||||
- **OpenAPI coverage is incomplete.** `server/src/routes/openapi-docs.ts` currently documents plugin-support endpoints only; server-directory endpoints are not listed.
|
||||
- **No structured request validation library.** Inline manual checks are error-prone; consider zod once the team is ready.
|
||||
- **No rate limiting / spam protection** on server creation or join attempts.
|
||||
- **`join_requests` migration is undocumented** (file not located by inspection); confirm during the next schema change.
|
||||
|
||||
## Related features
|
||||
|
||||
- **[websocket-envelopes](./websocket-envelopes.md)** — `join_server` envelope re-uses this area's access rules via `authorizeWebSocketJoin`. `notifyUser` fan-out for join-request decisions is delivered over the same WebSocket.
|
||||
- **[plugin-system](./plugin-system.md)** — `join_server` responses include the joined server's `PluginRequirementsSnapshot`.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2026-05-25 | Initial documentation |
|
||||
177
agents-docs/features/voice-signaling.md
Normal file
177
agents-docs/features/voice-signaling.md
Normal file
@@ -0,0 +1,177 @@
|
||||
# Voice & WebRTC Signaling
|
||||
|
||||
> **Area:** voice-signaling
|
||||
> **Status:** Active
|
||||
> **Last updated:** 2026-05-25
|
||||
|
||||
## Overview
|
||||
|
||||
Voice and screen-share in Toju are pure WebRTC mesh: peers establish RTCPeerConnections directly, while the signaling server only forwards SDP and ICE messages. This area covers the end-to-end flow — envelope routing, peer election, RTCPeerConnection lifecycle, RNNoise denoising, and the relationships between the three product-client domains involved: `voice-session`, `voice-connection`, and `direct-call`. Screen-share rides on the same peer connection; its UI orchestration is its own domain but the signaling path is shared.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Negotiate WebRTC sessions between peers using `offer` / `answer` / `ice_candidate` envelopes forwarded by the signaling server.
|
||||
- Elect an initiator deterministically when multiple peers arrive simultaneously, with a non-initiator fallback timer.
|
||||
- Maintain the local audio pipeline: mic capture → optional RNNoise denoising → RTCPeerConnection sender.
|
||||
- Track per-peer playback gain, mute, deafen, and speaking-activity state on the receive side.
|
||||
- Mirror voice presence (`voice_state`) and direct-call signalling (`direct-call`) to other peers via the WebSocket.
|
||||
|
||||
This area does **not** own:
|
||||
|
||||
- The WebSocket envelope shape (see [websocket-envelopes](./websocket-envelopes.md)).
|
||||
- Screen-share UI orchestration (its own domain at `toju-app/src/app/domains/screen-share/`); only the peer connection plumbing is shared.
|
||||
- Persistent user settings beyond `voiceSettingsStorage` (audio device IDs, volumes, bitrate, latency profile, noise-reduction toggle, persisted to localStorage).
|
||||
|
||||
## Key concepts
|
||||
|
||||
- **Mesh** — every participant holds an `RTCPeerConnection` per other participant. No SFU / MCU.
|
||||
- **Voice session** — high-level "user is currently in voice room X" state. Owned by `voice-session` domain.
|
||||
- **Voice connection** — low-level transport/peer concerns: speaking detection, per-peer gain, mute / deafen state. Owned by `voice-connection` domain.
|
||||
- **Direct call** — 1:1 voice/video call with an optional group-upgrade path. Owned by `direct-call` domain.
|
||||
- **Initiator** — the peer responsible for sending the first `offer`. Elected first-peer-wins; non-initiators wait `NON_INITIATOR_GIVE_UP_MS` (≈5 s) before generating their own offer.
|
||||
- **Data channel** — `chat`-labelled data channel established alongside each peer connection for P2P chat fallback and direct-message delivery.
|
||||
- **Noise suppressor worklet** — RNNoise WASM running in an `AudioWorkletNode` (`NoiseSuppressorWorklet`), loaded from `rnnoise-worklet.js` at the app root.
|
||||
|
||||
---
|
||||
|
||||
## Signaling envelopes (consumed)
|
||||
|
||||
Defined in [websocket-envelopes](./websocket-envelopes.md). Voice-relevant types:
|
||||
|
||||
- `offer`, `answer`, `ice_candidate` — forwarded by the server to `targetUserId` without inspection.
|
||||
- `direct-call` — forwarded; payload carries call-scoped events (ring, participant join/leave, call end).
|
||||
- `voice_state` — broadcast to a server. Payload includes `roomId`, `voiceGateway`, mute/deafen flags.
|
||||
- `server_users` — full peer roster on join; seeds the initial offer fan-out.
|
||||
- `user_joined` — schedules a fallback offer after a grace delay (`USER_JOINED_FALLBACK_OFFER_DELAY_MS`, ≈1 s).
|
||||
- `user_left` — peer teardown, with special handling that preserves peers still under an active voice session.
|
||||
- `connected` / `access_denied` — connection lifecycle (server bootstrap and authorization).
|
||||
|
||||
The server is **purely signaling**: it does not track which `oderId` is in which voice room. Voice membership is derived client-side from the `voice_state` broadcasts observed on the server.
|
||||
|
||||
---
|
||||
|
||||
## Session establishment flow
|
||||
|
||||
A new participant joining a voice room produces this exchange (initiator perspective; symmetrical when both arrive at once):
|
||||
|
||||
1. Local user clicks "Join voice" → `VoiceSessionFacade.startSession()` populates the session model and asks `voice-connection` to ready peer transport.
|
||||
2. Server broadcasts `user_joined` to existing peers.
|
||||
3. Each existing peer evaluates: am I the elected initiator for the (me, new-peer) pair? If yes, the peer-connection manager calls `doCreateAndSendOffer()`.
|
||||
4. Initiator constructs `new RTCPeerConnection({ iceServers })` (`infrastructure/realtime/peer-connection-manager/.../create-peer-connection.ts`), adds local tracks, creates the data channel `chat`, generates an SDP offer, and sends it via the signaling transport.
|
||||
5. Responder receives `offer` → `doHandleOffer()` sets remote description, generates SDP answer, sends `answer`.
|
||||
6. Initiator receives `answer` → `doHandleAnswer()` sets remote description.
|
||||
7. Both sides emit `ice_candidate` as they gather candidates via `onicecandidate`.
|
||||
8. `iceConnectionState` reaches `connected` / `completed` → media flows.
|
||||
9. Either side may open the `chat` data channel for P2P text payloads (direct messages, etc.).
|
||||
|
||||
If the elected initiator never sends an offer within `NON_INITIATOR_GIVE_UP_MS`, the non-initiator promotes itself and initiates instead — preserves liveness across asymmetric drop-outs.
|
||||
|
||||
`user_left` is treated carefully: the `signaling-message-handler.spec.ts` covers the case where a peer is still required by an active voice session and must not be torn down, even if other parts of the system think the peer has disconnected.
|
||||
|
||||
---
|
||||
|
||||
## Domain responsibilities
|
||||
|
||||
### `voice-session` (`toju-app/src/app/domains/voice-session/`)
|
||||
|
||||
- `VoiceSessionFacade` (`application/facades/voice-session.facade.ts`) — owns the active session metadata (`serverId`, `roomId`, `participantIds`); drives a `showFloatingControls` signal when the user navigates away from the room.
|
||||
- `VoiceWorkspaceService` (`application/services/voice-workspace.service.ts`) — UI state for the workspace (hidden / expanded / minimized), focused stream ID, mini-window position.
|
||||
- `voiceSettingsStorage` (`infrastructure/util/voice-settings-storage.util.ts`) — localStorage persistence: input/output device IDs, output volume (0–100), bitrate (32–256 kbps), latency profile (`low | balanced | high`), noise-reduction toggle.
|
||||
- Joining a new voice target first calls `endSession()` so transitions cannot leak peer connections.
|
||||
|
||||
### `voice-connection` (`toju-app/src/app/domains/voice-connection/`)
|
||||
|
||||
Bridges the application layer to the low-level WebRTC infrastructure under `toju-app/src/app/infrastructure/realtime/`.
|
||||
|
||||
- **`VoiceActivityService`** — RMS-based speaking detection via `AnalyserNode` (fftSize 256, RMS ≥ 0.015, 8-frame grace period).
|
||||
- **`VoicePlaybackService`** — per-peer `GainNode` chains (0–200% range), localStorage-persisted; deafen sets all gains to 0.
|
||||
- **`VoiceConnectionFacade`** — exposes signals like `isVoiceConnected`, `isMuted`; methods like `toggleMute()`, `toggleNoiseReduction()`, `setOutputVolume()`.
|
||||
|
||||
Per the domain README, voice-connection does **not** own RTCPeerConnection construction or signaling — those live in `infrastructure/realtime/peer-connection-manager`.
|
||||
|
||||
### `direct-call` (`toju-app/src/app/domains/direct-call/`)
|
||||
|
||||
- Initiator flow (`DirectCallService.startCall()`): create/reuse the 1:1 DM, start a call-scoped voice session, send a `direct-call` "ring" envelope via `PeerDeliveryService`.
|
||||
- Recipient flow: store incoming session, ring `assets/audio/call.wav` (unless DND), show in-app modal + desktop notification.
|
||||
- Group upgrade: adding a third participant spawns a new group conversation; the active call swaps its chat panel to the new conversation but original DM history is preserved.
|
||||
- Invariant: incoming `direct-call` events are ignored unless the local user is in `participantIds`.
|
||||
|
||||
### Screen share (`toju-app/src/app/domains/screen-share/`)
|
||||
|
||||
- Adds dedicated `MediaStreamTrack` senders to the existing peer connection (does not open a new one).
|
||||
- Request / response model: a receiver sends `screen-share-request`; the sender attaches the share track; `screen-share-stop` tears it down.
|
||||
- Quality presets: `low` / `balanced` / `high` (resolution + FPS).
|
||||
- On Electron, `ScreenShareSourcePickerService` drives a Promise-based picker over `getSources` (see [ipc-bridge](./ipc-bridge.md)).
|
||||
|
||||
---
|
||||
|
||||
## RNNoise pipeline
|
||||
|
||||
Manager: `infrastructure/realtime/media/noise-reduction.manager.ts`.
|
||||
|
||||
```
|
||||
Raw mic → MediaStreamAudioSourceNode → NoiseSuppressorWorklet (AudioWorkletNode) → MediaStreamAudioDestinationNode → clean stream → RTCPeerConnection sender
|
||||
```
|
||||
|
||||
- AudioContext at 48 kHz.
|
||||
- Worklet loaded from `rnnoise-worklet.js` (built from `@timephy/rnnoise-wasm`, output written to `toju-app/public/`).
|
||||
- If worklet load fails, the raw stream is passed through unchanged.
|
||||
- Mute takes priority — when muted, noise reduction is also disabled.
|
||||
|
||||
## Technical implementation
|
||||
|
||||
- **Envelope types**: see [websocket-envelopes](./websocket-envelopes.md).
|
||||
- **Signaling adapter (renderer)**: `toju-app/src/app/infrastructure/realtime/signaling/signaling-message-handler.ts` (and `signaling-transport-handler.ts`).
|
||||
- **Peer-connection manager**: `toju-app/src/app/infrastructure/realtime/peer-connection-manager/` — `create-peer-connection.ts`, recovery (grace timers, reconnect), data-channel plumbing.
|
||||
- **Voice settings**: `domains/voice-session/infrastructure/util/voice-settings-storage.util.ts`.
|
||||
- **Noise reduction**: `infrastructure/realtime/media/noise-reduction.manager.ts`.
|
||||
- **Worklet asset**: `toju-app/public/rnnoise-worklet.js`.
|
||||
- **Server side**: signaling only — `server/src/websocket/handler.ts::forwardRtcMessage`.
|
||||
|
||||
## Invariants
|
||||
|
||||
- The server forwards `offer` / `answer` / `ice_candidate` / `direct-call` envelopes opaquely and never persists media or call state.
|
||||
- Switching voice rooms always tears down the prior session before starting the new one.
|
||||
- Mute overrides noise reduction (the manager disables the worklet path when muted).
|
||||
- Direct-call events with the local user absent from `participantIds` are ignored.
|
||||
|
||||
## Testing
|
||||
|
||||
- `toju-app/src/app/infrastructure/realtime/signaling/signaling-message-handler.spec.ts` — `user_left` peer preservation under active voice.
|
||||
- `toju-app/src/app/infrastructure/realtime/peer-connection-manager/recovery/peer-recovery.spec.ts` — reconnect, grace timers, exponential backoff.
|
||||
- `toju-app/src/app/infrastructure/realtime/peer-connection-manager/messaging/data-channel.spec.ts`.
|
||||
- `toju-app/src/app/domains/direct-call/application/services/direct-call.service.spec.ts`.
|
||||
- E2E: `e2e/tests/voice/multi-signal-eight-user-voice.spec.ts`, `e2e/tests/voice/direct-call.spec.ts` (verify exact filenames in the suite — TODO).
|
||||
|
||||
## Security considerations
|
||||
|
||||
- WebRTC bypasses the server entirely once connected — peer IPs may be exposed to other participants via ICE candidates. Standard WebRTC privacy caveat.
|
||||
- Signaling envelopes are forwarded without verifying that source and target share a server — TODO: confirm whether `forwardRtcMessage` enforces membership.
|
||||
- The data channel `chat` carries P2P text payloads; integrity / authentication of those payloads is owned by the chat/direct-message domains, not by this area.
|
||||
- RNNoise runs entirely client-side; mic audio never leaves the local AudioContext until it enters the encrypted RTCPeerConnection.
|
||||
|
||||
## Performance considerations
|
||||
|
||||
- Mesh topology — N×(N-1)/2 peer connections per voice room. Practical ceiling is bound by client CPU and uplink; no documented soft cap.
|
||||
- Bitrate is client-controlled (32–256 kbps); no server-enforced QoS.
|
||||
- Voice activity detection runs at fftSize 256 with an 8-frame grace period — chosen to minimise CPU while staying responsive to natural speech.
|
||||
- The signaling server's only cost is envelope forwarding (O(1) per envelope).
|
||||
|
||||
## Known issues and limitations
|
||||
|
||||
- **No SFU / MCU.** Large rooms scale linearly with participant count on each client.
|
||||
- **No recording or server-side mixing** for voice or screen.
|
||||
- **Bitrate is not enforced server-side** — adversarial clients could ignore the suggested range.
|
||||
- **No documented call-quality telemetry pipeline.**
|
||||
|
||||
## Related features
|
||||
|
||||
- **[websocket-envelopes](./websocket-envelopes.md)** — owns the wire types this area consumes.
|
||||
- **[ipc-bridge](./ipc-bridge.md)** — `getSources` and the Linux audio-routing methods are used by screen-share.
|
||||
- **[plugin-system](./plugin-system.md)** — plugins may participate as observers via `voice_state` broadcasts (subject to capability grants); no direct call control surface today.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2026-05-25 | Initial documentation |
|
||||
164
agents-docs/features/websocket-envelopes.md
Normal file
164
agents-docs/features/websocket-envelopes.md
Normal file
@@ -0,0 +1,164 @@
|
||||
# WebSocket Envelopes
|
||||
|
||||
> **Area:** websocket-envelopes
|
||||
> **Status:** Active
|
||||
> **Last updated:** 2026-05-25
|
||||
|
||||
## Overview
|
||||
|
||||
The WebSocket envelope contract is the realtime wire-format boundary between the signaling server and every connected client. Every realtime concern in Toju — presence, chat broadcasts, typing indicators, voice state, WebRTC offer/answer/ICE forwarding, direct messages, server icon P2P sync, and plugin events — travels as a typed envelope over a single WebSocket connection per client. Drift between the server definition and the client-side mirror is treated as a wire-protocol break.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Define the canonical shape of every realtime message exchanged between `toju-app` (renderer) and `server`.
|
||||
- Route incoming envelopes to a single dedicated handler on the server.
|
||||
- Provide a stable identity for the connection (`connectionId`, `oderId`, `connectionScope`) and a lazy authorization model on `join_server`.
|
||||
- Forward peer-targeted envelopes (WebRTC signaling, direct messages, server-icon peer transfers) without inspecting their payload.
|
||||
|
||||
This area does **not** own:
|
||||
|
||||
- The HTTP/REST surface (see [server-directory](./server-directory.md)).
|
||||
- WebRTC media transport or session orchestration (see [voice-signaling](./voice-signaling.md) — the envelope contract is shared, but session lifecycle lives there).
|
||||
- Persistence (server entities are owned by the server subdomain; the envelope is the contract, not the entity).
|
||||
|
||||
## Key concepts
|
||||
|
||||
- **Envelope** — a `{ type, ...payload }` message routed by `type`. Defined in `server/src/websocket/types.ts`.
|
||||
- **ConnectedUser** — server-side state record per WebSocket: `connectionId`, `oderId`, `connectionScope`, `displayName`, `description`, `status`, `serverIds`, `lastPong`.
|
||||
- **`oderId`** — opaque user identity. Set by the client in `identify`; falls back to a UUID if absent. Multiple connections may share an `oderId` (e.g. multiple devices) — broadcasts are deduplicated per `oderId`.
|
||||
- **`connectionScope`** — typically the signal URL; disambiguates several connections from the same `oderId`.
|
||||
- **Handler** — server-side function mapped to one envelope `type` in `server/src/websocket/handler.ts`.
|
||||
- **Forwarded envelope** — peer-to-peer envelopes the server relays untouched to a specific `targetUserId` (offer / answer / ice_candidate / direct-call / direct-message family / server_icon_peer_*).
|
||||
|
||||
---
|
||||
|
||||
## Envelope catalogue
|
||||
|
||||
Defined on the server in `server/src/websocket/types.ts` and dispatched by the switch in `server/src/websocket/handler.ts`. Groups below match the dispatch shape, not a literal grouping in code.
|
||||
|
||||
### Connection & presence
|
||||
|
||||
- `identify` — client → server. Profile + `connectionScope`. Required before any other envelope is meaningful.
|
||||
- `connected` — server → client. Sent automatically on connect: `{ connectionId, serverTime }`.
|
||||
- `keepalive` — client ↔ server. Resets `lastPong`. See lifecycle below.
|
||||
- `status_update` — broadcast presence: `online | away | busy | offline`.
|
||||
- `access_denied` — server → client when `join_server` authorization fails.
|
||||
|
||||
### Server membership
|
||||
|
||||
- `join_server` — client requests membership for a `serverId`. Authorization checked via `authorizeWebSocketJoin` (`server/src/services/server-access.service.ts`). Response includes `server_users` + `plugin_requirements`.
|
||||
- `view_server` — client marks a server as viewed (fetch roster + plugin requirements without joining).
|
||||
- `leave_server` — client leaves; broadcasts `user_left` to remaining members.
|
||||
- `server_users` — server → client. Full peer roster for a joined server (used as the seed for P2P offers).
|
||||
- `user_joined` / `user_left` — broadcast presence changes.
|
||||
|
||||
### Chat & typing
|
||||
|
||||
- `chat_message` — broadcast to a server. Payload: `{ message, senderId, senderName, timestamp }`.
|
||||
- `typing` — broadcast: `{ isTyping, channelId, oderId, displayName }`.
|
||||
|
||||
### Voice presence
|
||||
|
||||
- `voice_state` — broadcast user voice state (mute/deafen/room metadata). Pure signaling — the server does not store voice room membership.
|
||||
|
||||
### WebRTC signaling (forwarded)
|
||||
|
||||
- `offer` / `answer` / `ice_candidate` — forwarded to `targetUserId` via `forwardRtcMessage()`.
|
||||
- `direct-call` — forwarded; semantic call lifecycle lives in the `direct-call` product-client domain.
|
||||
|
||||
### Direct messages (forwarded)
|
||||
|
||||
- `direct-message`, `direct-message-status`, `direct-message-mutation`, `direct-message-sync`, `direct-message-sync-request` — forwarded to `targetUserId`.
|
||||
|
||||
### Server icon P2P sync
|
||||
|
||||
- `server_icon_available` — client announces it has an icon at version `iconUpdatedAt`.
|
||||
- `server_icon_sync_request` — client asks the server which peers have a newer icon.
|
||||
- `server_icon_sync_peers` — server → client. Peer list offering newer icons.
|
||||
- `server_icon_peer_request` / `server_icon_peer_data` — P2P transfer, forwarded.
|
||||
|
||||
### Plugins
|
||||
|
||||
- `plugin_event` — validated against the plugin's registered event schema (see [plugin-system](./plugin-system.md) and `server/src/services/plugin-support.service.ts`), then broadcast within the server scope. Payload: `{ serverId, pluginId, eventName, payload, sourcePluginUserId, sourceUserId, emittedAt }`.
|
||||
|
||||
---
|
||||
|
||||
## Connection lifecycle
|
||||
|
||||
Implemented in `server/src/websocket/index.ts`.
|
||||
|
||||
1. Client opens WebSocket → server generates `connectionId` (UUID), creates the `ConnectedUser` record, sends `{ type: 'connected', connectionId, serverTime }`.
|
||||
2. Client sends `identify` with `oderId`, `displayName`, `connectionScope`, optional `description` / `profileUpdatedAt`. Server normalizes and stores.
|
||||
3. Client sends `join_server` (or `view_server`) per server they care about. Each `join_server` is authorized independently.
|
||||
4. Heartbeat: server pings every **30 s** (`PING_INTERVAL_MS`). Any incoming message also refreshes `lastPong`. Connections without a pong for **45 s** (`PONG_TIMEOUT_MS`) are terminated.
|
||||
5. On close: server emits `user_left` to every server the connection had joined. Broadcasts are **deduplicated by `oderId`**, so multi-device users only generate one departure event per logical identity.
|
||||
|
||||
---
|
||||
|
||||
## Authentication model
|
||||
|
||||
There is no bearer token or signed envelope. Identity is whatever the client claims in `identify`. Authorization is **per-`join_server`**, evaluated by `authorizeWebSocketJoin` against persisted server access rules (private flag, password hash, bans, invite/join-request state). `access_denied` is returned when authorization fails; the connection itself stays open.
|
||||
|
||||
---
|
||||
|
||||
## Technical implementation
|
||||
|
||||
### Server
|
||||
|
||||
- **Types**: `server/src/websocket/types.ts` — `WsMessage` (union over `type`), `ConnectedUser`, `ConnectionScope`.
|
||||
- **Dispatcher**: `server/src/websocket/handler.ts` — `handleWebSocketMessage(connectionId, message)`. Single switch (~16 dedicated handler functions plus `forwardRtcMessage`).
|
||||
- **Lifecycle**: `server/src/websocket/index.ts` — `ws` server, ping/pong, connection registry, dead-connection reaping.
|
||||
- **Plugin event validation**: `server/src/services/plugin-support.service.ts` — async `validatePluginEventEnvelope()` (runtime schema check).
|
||||
|
||||
### Client (renderer)
|
||||
|
||||
- **Shared types**: `toju-app/src/app/shared-kernel/signaling-contracts.ts` — **stale**, only declares a generic `SignalingMessage` and an obsolete `SignalingMessageType` enum. Not the active wire-format definition.
|
||||
- **Active envelope shapes** are defined inline as `IncomingSignalingMessage` in `toju-app/src/app/infrastructure/realtime/signaling/signaling-message-handler.ts`.
|
||||
- **Constants**: `toju-app/src/app/infrastructure/realtime/realtime.constants.ts` — every envelope `type` string lives here as `SIGNALING_TYPE_*`.
|
||||
- **Transport**: `toju-app/src/app/infrastructure/realtime/signaling/signaling-transport-handler.ts` — socket lifecycle, sends `identify`, `join_server`, raw envelopes.
|
||||
- **Coordinator**: `toju-app/src/app/infrastructure/realtime/signaling/server-signaling-coordinator.ts` — maps `serverId` to signal URL (Toju supports multiple federated signaling endpoints).
|
||||
- **Inbound dispatch**: `signaling-message-handler.ts` — `handleConnectedSignalingMessage`, `handleServerUsersSignalingMessage`, `handleUserJoinedSignalingMessage`, `handleUserLeftSignalingMessage`, `handleOfferSignalingMessage`, `handleAnswerSignalingMessage`, `handleIceCandidateSignalingMessage`, `handleAccessDeniedSignalingMessage`. Domain envelopes (chat/typing/direct-message/etc.) are consumed in the respective product-client domains, not in this central adapter — TODO: enumerate exact subscription points.
|
||||
|
||||
### Versioning
|
||||
|
||||
No `version` field on envelopes. No `Accept-Version` header. Drift between server and client is enforced only by code review (per `server/CONTEXT.md` invariants).
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
- `server/src/websocket/handler-status.spec.ts` — `status_update` broadcast and profile metadata in `user_joined` / `server_users`.
|
||||
- `server/src/websocket/handler-plugin.spec.ts` — `plugin_event` validation and broadcast.
|
||||
- `toju-app/src/app/infrastructure/realtime/signaling/signaling-message-handler.spec.ts` — inbound handler unit tests (notably `user_left` preserving peers under voice).
|
||||
- **TODO**: no round-trip envelope-shape test between server `WsMessage` and client `IncomingSignalingMessage`. Drift can only be caught by E2E or manual review today.
|
||||
|
||||
## Security considerations
|
||||
|
||||
- No transport-level auth — identity is self-asserted via `identify`. The server trusts `oderId` for routing but checks authorization on every `join_server`.
|
||||
- WebRTC signaling envelopes (`offer` / `answer` / `ice_candidate`) are forwarded **without inspection**. The server does not verify that the sender is a member of the same server as the target — TODO: confirm whether `forwardRtcMessage` enforces server-membership before forwarding.
|
||||
- `plugin_event` payloads are bounded by the plugin's declared `maxPayloadBytes` (default 64 KB) and validated against the plugin's declared event schema. See [plugin-system](./plugin-system.md).
|
||||
- Multi-connection identities: a single `oderId` may have many open sockets. Broadcasts dedupe by `oderId`, but per-connection state (e.g. `voice_state`) does not — TODO: document the cross-connection invariants.
|
||||
|
||||
## Performance considerations
|
||||
|
||||
- Single WebSocket per client. No fan-out worker; broadcast is in-process via the in-memory connection map.
|
||||
- Ping cadence 30 s / pong timeout 45 s. Reaping is per-connection on next tick.
|
||||
- TODO: no documented soft cap on connected users per signaling server.
|
||||
|
||||
## Known issues and limitations
|
||||
|
||||
- **Stale shared-kernel contract.** `toju-app/src/app/shared-kernel/signaling-contracts.ts` does not enumerate the live envelope set; client code uses `IncomingSignalingMessage` in `signaling-message-handler.ts` instead. Update or replace this file when adjacent work touches the wire format.
|
||||
- **No envelope versioning.** Any field rename is an immediate break for older clients.
|
||||
- **TODO — operator concerns**: rate limits, max-message-size, and backpressure are not documented.
|
||||
|
||||
## Related features
|
||||
|
||||
- **[voice-signaling](./voice-signaling.md)** — consumes `offer` / `answer` / `ice_candidate` / `voice_state` / `direct-call`.
|
||||
- **[plugin-system](./plugin-system.md)** — defines and validates `plugin_event`.
|
||||
- **[server-directory](./server-directory.md)** — REST counterpart for server discovery, joining, and moderation; `join_server` envelope authorization reuses the same access rules.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2026-05-25 | Initial documentation |
|
||||
Reference in New Issue
Block a user