212 lines
13 KiB
Markdown
212 lines
13 KiB
Markdown
# Messaging
|
|
|
|
> **Area:** messaging
|
|
> **Status:** Active
|
|
> **Last updated:** 2026-07-13
|
|
|
|
## Overview
|
|
|
|
Messaging in MetoYou covers two transports that share inventory-sync concepts and (for DMs) a monotonic delivery state machine. **Server-channel chat** is broadcast by the signaling server over WebSocket (`chat_message`) as a narrow fallback when P2P data channels are down — the server does not persist message bodies. **Direct messages** (1:1 and group DMs) are primarily peer-to-peer over the WebRTC ordered data channel, with WebSocket signaling relay when no channel is open and an offline queue when neither path succeeds.
|
|
|
|
On both transports the client maintains local history (Electron SQLite / browser IndexedDB for server channels; user-scoped `localStorage` for DMs) and a **chunked inventory-sync protocol** so peers reconcile missing rows without flooding the link.
|
|
|
|
This document is the cross-context contract: envelope names, sync protocol, delivery states, edit/delete rules, and storage boundaries. Internal NgRx orchestration lives in [`toju-app/src/app/domains/chat/README.md`](../../toju-app/src/app/domains/chat/README.md) and [`toju-app/src/app/domains/direct-message/README.md`](../../toju-app/src/app/domains/direct-message/README.md). WebSocket relay rules: [signaling.md](signaling.md). Signed revision chains: [message-integrity.md](message-integrity.md).
|
|
|
|
## Responsibilities
|
|
|
|
- Send server-channel chat over WebSocket fallback (`chat_message`) and primarily over P2P (`chat-message`, `edit-message`, `delete-message`, `message-revision`).
|
|
- Send, edit, delete, and react in direct messages over the data channel with signaling fallback.
|
|
- Carry typing indicators: server channels (`typing` → `user_typing`) and DMs (`direct-message-typing`).
|
|
- Reconcile peer history via the inventory protocol (`chat-inventory` / `chat-sync-batch`; DM `direct-message-sync`).
|
|
- Drive a monotonic DM delivery state machine: `QUEUED → SENT → DELIVERED → ACKNOWLEDGED`.
|
|
- Relay multi-device chat via `account_sync` (`chat-message`, `message-revision`, `chat-sync-batch`).
|
|
|
|
This area does **not** own:
|
|
|
|
- Attachment payloads or chunked file transfer → [attachments.md](attachments.md).
|
|
- WebRTC session setup and data-channel lifecycle → [voice-webrtc.md](voice-webrtc.md).
|
|
- Write permission resolution (`writeMessages`, `manageMessages`, bans) → `toju-app/src/app/domains/access-control/README.md`.
|
|
- Full WebSocket envelope catalog (identity, voice, plugins) → [signaling.md](signaling.md).
|
|
|
|
## Key concepts
|
|
|
|
- **Server-channel message** — room-scoped text in a saved chat-server. Primary path: P2P `chat-message` on the data channel. Fallback: server broadcasts `chat_message` to other connections in the room.
|
|
- **Direct message** — 1:1 or group PM. Persisted per user under `metoyou_direct_message_*` keys (domain-owned storage, not the global messages CQRS table).
|
|
- **Conversation** — DM thread (`direct` or `group`). Upgrading a 1:1 call to a group creates a **new** group conversation; the original 1:1 history is not copied.
|
|
- **Inventory event** — `chat-inventory` (P2P): sender announces message ids plus integrity fields (`ts`, `rc`, `ac`, `revision`, `headHash`); receiver requests missing or stale ids.
|
|
- **Sync batch** — `chat-sync-batch`: chunked response, **200 messages per envelope** (`CHUNK_SIZE` in `message-sync.rules.ts`).
|
|
- **Delivery state** — DM-only enum: `QUEUED (0) → SENT (1) → DELIVERED (2) → ACKNOWLEDGED (3)`. Advanced only via `advanceDirectMessageStatus` (never backwards).
|
|
- **Peer delivery** — `PeerDeliveryService` tries data channel, then signaling forward, then offline queue.
|
|
|
|
---
|
|
|
|
## Transports
|
|
|
|
### Server-channel chat
|
|
|
|
**P2P (primary):** `chat-message`, `edit-message`, `delete-message`, `message-revision`, reactions, and inventory events on the ordered data channel. See [message-integrity.md](message-integrity.md) for dual-emit revision behavior.
|
|
|
|
**WebSocket (fallback):** Client sends `chat_message`; `handleChatMessage` (`server/src/websocket/handler.ts`) broadcasts to other connections in the room. The server does **not** handle `edit_message` or `delete_message` on the wire — edits and deletes are P2P (and `account_sync` for sibling devices).
|
|
|
|
**Typing:** Client sends `typing`; server broadcasts `user_typing` (transient, no persistence).
|
|
|
|
**Multi-device:** Sibling tabs receive live chat via `account_sync` payloads (`chat-message`, `message-revision`, `chat-sync-batch`). See [authentication.md](authentication.md).
|
|
|
|
### Direct messages
|
|
|
|
**P2P (primary):** Events on the shared ordered data channel (same peer connections as voice/chat).
|
|
|
|
**WebSocket (fallback):** `PeerDeliveryService.sendViaSignaling` forwards these types to `targetUserId` without requiring shared server membership:
|
|
|
|
| type | Purpose |
|
|
|------|---------|
|
|
| `direct-message` | New message |
|
|
| `direct-message-status` | Delivery / ack |
|
|
| `direct-message-mutation` | Edit, delete, reactions |
|
|
| `direct-message-typing` | Typing indicator |
|
|
| `direct-message-sync-request` | Request snapshot |
|
|
| `direct-message-sync` | Bounded history merge |
|
|
|
|
**Offline queue:** When both paths fail, `OfflineMessageQueueService` retains message ids; replay runs on `peerConnected$` / `networkRestored$` (no scheduled retry timer).
|
|
|
|
### Storage
|
|
|
|
| Data | Where |
|
|
|------|--------|
|
|
| Server-channel messages | `DatabaseService` → Electron SQLite or browser IndexedDB (`messages` store) |
|
|
| Direct messages | `metoyou_direct_message_*` via direct-message repositories |
|
|
| Signaling server | **No message bytes** — broadcast/relay only |
|
|
|
|
---
|
|
|
|
## Inventory / sync protocol
|
|
|
|
Shared shapes in `toju-app/src/app/shared-kernel/chat-events.ts`:
|
|
|
|
| Event | Role |
|
|
|-------|------|
|
|
| `chat-inventory-request` | Ask peer for inventory |
|
|
| `chat-inventory` | Announce ids + integrity snapshots |
|
|
| `chat-sync-request` | Request specific missing ids |
|
|
| `chat-sync-batch` | Up to **200** messages per envelope |
|
|
| `direct-message-sync-request` / `direct-message-sync` | DM-scoped snapshot merge |
|
|
|
|
Rules (`message-sync.rules.ts`, `message-integrity.rules.ts`):
|
|
|
|
- Merges are **additive** — sparser peers never wipe richer local history.
|
|
- `findMissingIds` compares remote inventory to local `revision` / `headHash` (and legacy `ts` / `rc` / `ac`).
|
|
- `INVENTORY_LIMIT` = `FULL_SYNC_LIMIT` = **20_000** (2026-07-14, RAM investigation; previously 1_000_000). Building an inventory or full-sync batch loads full message rows into memory, so the ceiling must stay bounded. Only the most recent 20k messages per room are reconciled peer-to-peer; older messages stay local-only. `ACCOUNT_SYNC_MESSAGE_LIMIT` follows `FULL_SYNC_LIMIT`.
|
|
- Sync polling: 10 s when catching up, 15 min after a clean cycle (`SYNC_POLL_FAST_MS` / `SYNC_POLL_SLOW_MS`).
|
|
- NgRx store retention: on room switch, inactive rooms are pruned to the most recent `CACHED_INACTIVE_ROOM_MESSAGE_LIMIT` = **100** messages each (`messages.reducer.ts`), keeping return-visit rendering instant while bounding store growth across many rooms. The active room is never pruned; the local DB keeps full history.
|
|
|
|
---
|
|
|
|
## Delivery state machine (DMs only)
|
|
|
|
| Value | Numeric | Meaning |
|
|
|-------|---------|---------|
|
|
| `QUEUED` | 0 | Composed locally; no successful send yet |
|
|
| `SENT` | 1 | Data channel or signaling forward accepted the payload |
|
|
| `DELIVERED` | 2 | At least one recipient acknowledged receipt |
|
|
| `ACKNOWLEDGED` | 3 | Full recipient set acknowledged (1:1: the peer; group: every participant) |
|
|
|
|
`advanceDirectMessageStatus` only moves forward (`direct-message.logic.ts`). Server-channel messages have no application-level delivery enum; the UI treats them as sent once the transport accepts the event.
|
|
|
|
---
|
|
|
|
## Edit and delete
|
|
|
|
**Server channels:** Outgoing edits check `canEditMessage(message, userId)` before broadcast. Incoming P2P `edit-message` / `delete-message` merge via NgRx handlers; signed paths prefer `message-revision` when integrity is enabled.
|
|
|
|
**DMs:** `direct-message-mutation` with types `edit`, `delete`, `reaction-add`, `reaction-remove`. `applyMutation` in `DirectMessageService` updates by `messageId` but **does not verify** the mutator is the original author — a non-cooperating peer could mutate another user's row. Server chat enforces authorship on **outgoing** edits only.
|
|
|
|
Deletes keep tombstone semantics (`isDeleted`, empty `content`) so inventory sync can converge.
|
|
|
|
---
|
|
|
|
## Business rules and invariants
|
|
|
|
- The signaling server is **not authoritative** for message content — it relays `chat_message` and DM types opaquely.
|
|
- DM events are **ignored** unless the local user is in `recipients` / `participants` or already has the conversation locally.
|
|
- Recipient matching (DM **and** `direct-call`) must accept **every local identity alias** — home id, entity id, peer id, and each provisioned signal-server actor id from `SignalServerCredentialStoreService` — because senders who met the recipient on a foreign signal server address them by the provisioned actor id (`direct-message-identity.rules.ts`, `direct-call-participant-identity.rules.ts`).
|
|
- DM status transitions are **monotonic**.
|
|
- Inventory merges never downgrade a row with a newer `revision` / `headHash`.
|
|
- 1:1 → group upgrade **does not copy** private history into the new group thread.
|
|
- Unread counts are **idempotent by message id** — re-sync does not double-increment.
|
|
- Incoming DMs raise a system notification via `NotificationsFacade.handleIncomingDirectMessage` (title = sender name; `shouldDeliverDirectMessageNotification` suppresses only when the conversation is on screen in an active window, notifications are disabled, or the user is busy). System messages (e.g. call-started) and deletions never notify. On Capacitor this flows through the same `DesktopNotificationService` → LocalNotifications routing as server chat.
|
|
|
|
---
|
|
|
|
## Technical implementation
|
|
|
|
### Server
|
|
|
|
- `server/src/websocket/handler.ts` — `handleChatMessage`, `handleTyping`, DM forward via `forwardRtcMessage` / `DIRECT_SIGNALING_TYPES`.
|
|
- No message CQRS or entities on the server.
|
|
|
|
### Product client
|
|
|
|
| Area | Location |
|
|
|------|----------|
|
|
| Server chat effects / handlers | `store/messages/`, `domains/chat/` |
|
|
| DM service / queue | `domains/direct-message/application/services/` |
|
|
| Sync rules | `domains/chat/domain/rules/message-sync.rules.ts` |
|
|
| Wire types | `shared-kernel/chat-events.ts`, `direct-message-contracts.ts` |
|
|
| Account sync relay | `infrastructure/realtime/account-sync/` |
|
|
|
|
### Electron
|
|
|
|
- Server-channel rows: TypeORM `Message` entity + CQRS `save-message` / `delete-message`.
|
|
- DMs: renderer `localStorage` repositories (not the main message table).
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
- Unit: `message-sync.rules.spec.ts`, `message-integrity.rules.spec.ts`, `message.rules.spec.ts`, `direct-message.service.spec.ts`, `direct-message.logic` specs, `messages-incoming.handlers.spec.ts`, `account-sync-chat.helper.spec.ts`.
|
|
- E2E: `e2e/tests/chat/chat-message-features.spec.ts`, `multi-client-chat-sync.spec.ts`, `dm-flow.spec.ts`, `multi-device-attachment-sharing.spec.ts`, `e2e/tests/voice/dm-header-call-ring.spec.ts` (DM-header call ring, incl. cross-signal actor-id addressing).
|
|
|
|
---
|
|
|
|
## Performance considerations
|
|
|
|
- Sync batches: **200 messages per `chat-sync-batch` envelope**.
|
|
- `chat_message` broadcast is O(connections in room) per send.
|
|
- Group DMs: O(recipients) transport attempts per message.
|
|
|
|
---
|
|
|
|
## Security considerations
|
|
|
|
- **No end-to-end encryption** for message bodies. WebRTC data channels use DTLS; signaling fallback is TLS WebSocket; local DBs store plaintext.
|
|
- **DM `applyMutation` does not verify authorship** on incoming mutations.
|
|
- **No server-side rate limit** on `chat_message` volume.
|
|
|
|
---
|
|
|
|
## Known issues and limitations
|
|
|
|
- **No server-side chat log** — late joiners depend on peers with local history or `account_sync` from a sibling device.
|
|
- **DM mutation authorship** not verified on receive.
|
|
- **Offline queue** replays only on peer connect / network restore events.
|
|
|
|
---
|
|
|
|
## Related features
|
|
|
|
- [signaling.md](signaling.md) — WebSocket relay and ordering invariants
|
|
- [message-integrity.md](message-integrity.md) — signed revision chains
|
|
- [attachments.md](attachments.md) — file payloads alongside chat events
|
|
- [voice-webrtc.md](voice-webrtc.md) — data channel transport
|
|
- [authentication.md](authentication.md) — `account_sync` multi-device relay
|
|
- [direct-messaging.md](direct-messaging.md) — short index (defers here)
|
|
|
|
## Changelog
|
|
|
|
| Date | Change |
|
|
|------|--------|
|
|
| 2026-07-14 | RAM bounds: `INVENTORY_LIMIT`/`FULL_SYNC_LIMIT` lowered to 20k (most recent messages reconcile); NgRx prunes inactive rooms to 100 cached messages on room switch |
|
|
| 2026-07-13 | Incoming DMs raise system notifications through the notifications domain (previously unread-badge only) |
|
|
| 2026-07-13 | Recipient matching for DM and `direct-call` events must span all local identity aliases (provisioned actor ids included) |
|
|
| 2026-07-05 | Initial comprehensive messaging contract (replaces thin direct-messaging summary) |
|