Toju/emergency-fix/05-signaling-multi-server.md

05 — Signaling & multi-signal-server connection

User theme: “I’m connected but nobody sees me / rooms flicker / different signal servers break everything.”
Severity: P0 for presence and as root cause of voice + chat visibility.

---

How signaling is supposed to work (user view)

• When you open a community, you appear in the member list for everyone in that community.
• Leaving / going offline shows you as offline (without duplicate leave spam for multi-device).
• Switching communities updates who you see without wiping presence for other saved servers.
• If the signal host restarts or Wi‑Fi blips, you come back automatically.
• Communities hosted on different signal hosts still work as long as your client is authorized on that host (see 04).
• Users should be able to see/sync chats/files/profile images/server states info change/plugin states and events/talk without losing connectivity or only some users are seen and other aren't.

---

How signaling is supposed to work (system)

1. One WebSocket per signal URL (SignalingManager).
2. Identify first with that URL’s token + actor oderId + clientInstanceId.
3. Then join_server / view_server for rooms on that URL.
4. Server serializes handlers per connection so join cannot race mid-identify.
5. Client reIdentifyAndRejoin on reconnect; rooms effects resync as safety net.
6. Room traffic prefers room sourceUrl; temporary fallback to other compatible endpoints on outage.
7. Non-federated: Signal A does not share peer registry with Signal B. Same human on two hosts = two actor ids (client glue).

Canonical catalog: agents-docs/features/signaling.md.
Client map: infrastructure/realtime/README.md + signaling/.

---

Failure modes

1 — Identify / join race or skipped identify

Symptom: Local UI shows room open; others don’t see you; you miss user_joined / chat_message; “alone”.

Causes

• Join sent before identify (fixed server-side serialization + client reIdentify — regressions still dangerous).
• getIdentifyCredentialsForSignalUrl returns null (no credential) → no identify.
• Identify cache empty on fresh socket without store fallback (lesson: must fall back to credential store).

2 — Wrong socket / wrong affinity

Symptom: Presence or RTC relay never reaches peers in the room; works only when everyone shares the same endpoint URL alias.

Causes

• Room sourceUrl stale vs directory serverInstanceId canonicalization.
• Fallback broadcast regressions (raw room messages must not spam every manager when route unknown — README describes current intended behavior).
• Cold start reconnect before health probes collapse aliases.

3 — Multi-signal leave tears down wrong peers

Symptom: Leaving a room on signal-sweden drops a peer still shared via signal.toju.app.

Intended: user_left subtracts only that cluster’s shared servers; preserve routes while P2P still live.

4 — Half-open / zombie sockets

Symptom: UI “connected”; no events after server process restart.

Mitigations in tree: keepalive ack timeout after first ack; /api/health probe; serverInstanceId change forces new WS. Regressions: skipping first heartbeat tick, treating 521/522 as incompatible.

5 — Auth failure cascade

Foreign auth_required / auth_error without silent re-provision → authorize UI or invisible membership (04).

6 — Multi-device eviction loop

Shared clientInstanceId in localStorage across tabs → server evicts sibling on identify. Must stay in sessionStorage.

---

Different signal servers — user-visible contract

Situation	Expected
Two users, same room, same signal URL	Discover each other; RTC relay OK
Two users, “same” community mirrored on different signals	Not supported as one mesh — they are different rooms unless directory/affinity converges them
One user, rooms on A and B	Two sockets; two actor ids; local profile one
DM/call across people who met on foreign rooms	Must address connected actor id (see 09)

---

Key files

• signaling.manager.ts, signaling-transport-handler.ts, server-signaling-coordinator.ts
• signaling-message-handler.ts, server-membership-signaling-handler.ts
• room-signaling-connection.ts, rooms.effects.ts
• server-directory health / canonical endpoint rules
• Server: server/src/websocket/handler.ts (out of default scope — ask)

---

Proof of done (signaling)

1. Two clients join same room: both appear in member lists within seconds of identify.
2. Kill signal process; both recover presence without manual leave/rejoin.
3. User with rooms on two URLs: leave on URL A does not remove voice peer still shared only via URL B.
4. Fresh connect: first outbound after open is identify; join never accepted unauthenticated (server log / test).
5. Alias URLs with same serverInstanceId collapse before room reconnect.

---

Fix order notes

Usually fix auth provision (04) before deep signaling surgery — many “connection” bugs are missing identify. Then harden identify credential resolution and affinity. Instrument connectionScope, actor id, and signal URL on every join in debug builds.