137 lines
8.7 KiB
Markdown
137 lines
8.7 KiB
Markdown
# Attachments
|
|
|
|
> **Area:** attachments
|
|
> **Status:** Active
|
|
> **Last updated:** 2026-07-05
|
|
|
|
## Overview
|
|
|
|
Attachments move file bytes peer-to-peer over the WebRTC ordered data channel using a announce → request → chunk protocol. Chat and DMs attach metadata to messages; the signaling server does not store or relay file payloads. Sibling devices learn attachment **metadata** via `account_sync` `chat-sync-batch` but must still download bytes from a peer that has them.
|
|
|
|
Domain internals: [`toju-app/src/app/domains/attachment/README.md`](../../toju-app/src/app/domains/attachment/README.md).
|
|
|
|
## Responsibilities
|
|
|
|
- Chunked P2P transfer with flow control and cancel semantics.
|
|
- Auto-download when policy allows; disk streaming on Electron/Capacitor.
|
|
- Ownership vs "shared from your device" UI rules.
|
|
- Persist attachment rows + filesystem paths on desktop/mobile.
|
|
|
|
This area does **not** own:
|
|
|
|
- Message envelopes or delivery states → [messaging.md](messaging.md).
|
|
- WebRTC negotiation → [voice-webrtc.md](voice-webrtc.md).
|
|
|
|
## Key concepts
|
|
|
|
- **Announce** — sender advertises `fileId`, name, size, mime without sending bytes.
|
|
- **Mirror host** — peer that holds a complete copy and can serve chunks.
|
|
- **Buffered send** — waits for data-channel back-pressure (4 MB high / 1 MB low water marks on chat channel).
|
|
|
|
---
|
|
|
|
## P2P protocol
|
|
|
|
| type | Purpose |
|
|
|------|---------|
|
|
| `file-announce` | Metadata only |
|
|
| `file-request` | Receiver starts download |
|
|
| `file-chunk` | Base64 chunk (`index`, `total`, `data`) |
|
|
| `file-chunk-ack` | Per-chunk flow control |
|
|
| `file-cancel` | Abort in flight |
|
|
| `file-not-found` | Host lacks bytes |
|
|
|
|
**Chunk size:** `FILE_CHUNK_SIZE_BYTES` = **64 KB** (`attachment-transfer.constants.ts`).
|
|
|
|
**Electron send path:** reads one chunk at a time from disk via IPC (`append-file-bytes` / read chunk) to avoid loading whole files into renderer memory.
|
|
|
|
---
|
|
|
|
## Persistence
|
|
|
|
| Runtime | Metadata | Bytes |
|
|
|---------|----------|-------|
|
|
| Browser | In-memory / optional save | Below **10 MB** auto-save cap (`MAX_AUTO_SAVE_SIZE_BYTES`) |
|
|
| Electron | SQLite `attachments` + CQRS | `user/<username>/…` via `AttachmentStorageService` / IPC |
|
|
| Capacitor | SQLite | App-private attachment directory — [mobile-capacitor.md](mobile-capacitor.md) |
|
|
|
|
---
|
|
|
|
## Download / export to user location
|
|
|
|
`AttachmentDownloadService.downloadToUserLocation` picks the runtime-appropriate export path:
|
|
|
|
| Runtime | Behavior |
|
|
|---------|----------|
|
|
| Electron | `saveExistingFileAs` (disk-backed) or `saveFileAs` (blob) native save dialog |
|
|
| Browser | Anchor `download` click on the object URL |
|
|
| Capacitor | `CapacitorAttachmentExportService.exportToDevice`: copies the disk file from `Directory.Data` into `Directory.Documents` (or fetches the object URL and writes base64) using `buildAttachmentExportFileName` (timestamp suffix so exports never collide — Android 11+ rejects overwrites of files the app did not create). Anchor downloads do nothing in the Android WebView. |
|
|
|
|
## Multi-device
|
|
|
|
`chat-sync-batch` in `account_sync` carries an `attachments` map (local paths stripped). Sibling devices discover files exist; P2P `file-request` still required for bytes.
|
|
|
|
---
|
|
|
|
## Business rules and invariants
|
|
|
|
- Transfers are between connected peers only (no server CDN).
|
|
- Receive strategy is decided once at request time by `canReceiveAttachment` (`attachment.logic.ts`): ≤ 10 MB assembles in memory everywhere; > 10 MB streams to disk on Electron/Capacitor, assembles in memory on the browser up to its 50 MB persist cap, and is rejected with a visible `fileTooLarge` error beyond that. `handleFileChunk` must accept whatever the request gate admitted — a stricter chunk-time size cap silently drops chunks and stalls the transfer.
|
|
- Visibility-based blob lifecycle on desktop: revoke `blob:` URLs when messages scroll off-screen if disk can rehydrate.
|
|
- Startup hydration is an availability boundary: file requests and host re-announcements wait for persisted metadata before inspecting local files. A host re-announces persisted files after reload and on peer connection even from non-chat routes, and can recover an original Electron source path by copying it into app data on demand before serving.
|
|
- Startup database hydration merges persisted metadata into the live runtime attachment map; it must preserve attachments announced during initialization and completed runtime state (`available`, progress, display URL) while filling missing local paths. Replacing the map can regress a completed download to Retry, spinner, or 100% after navigation.
|
|
- Starting a request updates the runtime version immediately so inline cards and galleries show pending/download state at zero bytes; exhausting all candidate peers surfaces `fileNotFound` instead of silently clearing the pending request. A repeat host announce re-queues guarded auto-download recovery for eligible media.
|
|
- Display-blob memory invariants (added 2026-07-14, RAM investigation):
|
|
- Inline hydration (`chat-message-item` effect) only runs for messages that are visible or within the `IntersectionObserver` root margin — gated by `attachment-hydration-visibility.rules.ts`. Off-screen rows never load blobs.
|
|
- Visibility observation uses the rendered message row (`componentHost.firstElementChild`), not the boxless Angular component host; otherwise returning to a channel leaves revoked attachments on permanent spinners.
|
|
- Disk-to-blob hydration is deduplicated per attachment and capped at two active tasks. Offscreen/destroy lifecycle cancellation is checked after every IPC read and before object-URL assignment, so rapid channel switches cannot accumulate stale full-file buffers or orphaned blobs; pinned fullscreen/gallery attachments are exempt.
|
|
- Disk-hydrated blobs are **not** duplicated into `AttachmentRuntimeStore.originalFiles`; peer requests are served from the disk path (`streamRequestedFile` prefers `resolveExistingPath`). `originalFiles` only holds uploads/downloads that have no disk copy yet.
|
|
- `revokeAttachmentDisplayBlob` also drops the `originalFiles` entry when `savedPath` exists, so revocation actually frees the bytes.
|
|
- Message rows always revoke their display blobs on destroy (pins are respected), not only when they were visible.
|
|
- Room switch sweeps display blobs of all other rooms (`releaseDisplayBlobsForInactiveRooms`, driven by `collectMessageIdsForInactiveRoomBlobRelease`). Messages with unknown room mapping are left alone.
|
|
- "Shared from your device" badge only when bytes are local to the viewing user.
|
|
- Blob-backed chat thumbnails use plain `src` with native lazy loading and async decoding. `NgOptimizedImage` is forbidden for these URLs because Angular rejects `blob:` inputs; fullscreen images remain eager.
|
|
|
|
---
|
|
|
|
## Technical implementation
|
|
|
|
- Facade: `AttachmentFacade` → `AttachmentManagerService`
|
|
- Protocol: `AttachmentTransferService` + `AttachmentTransferTransportService`
|
|
- Electron IPC: `read-file-chunk`, `append-file-bytes`, `write-file`, `delete-file`, etc.
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
- Domain logic specs under `attachment/`
|
|
- E2E: `e2e/tests/chat/chat-message-features.spec.ts`, `local-attachment-persistence.spec.ts`, `multi-device-attachment-sharing.spec.ts`, `large-generic-file-transfer.spec.ts` (browser receiver, generic file above the 10 MB auto-save cap)
|
|
|
|
---
|
|
|
|
## Security considerations
|
|
|
|
- No server-side virus scanning; peers trust senders they are connected to.
|
|
- Files stay in user data directories (Electron path jail).
|
|
|
|
---
|
|
|
|
## Related features
|
|
|
|
- [messaging.md](messaging.md) — message + attachment metadata coupling
|
|
- [authentication.md](authentication.md) — `account_sync` batches
|
|
- [mobile-capacitor.md](mobile-capacitor.md) — mobile storage
|
|
|
|
## Changelog
|
|
|
|
| Date | Change |
|
|
|------|--------|
|
|
| 2026-07-14 | Bounded display hydration to two deduplicated tasks, cancelled stale channel work before blob assignment, and added native lazy/async thumbnail hints |
|
|
| 2026-07-14 | Fixed channel-return hydration by observing the rendered message row instead of the boxless component host |
|
|
| 2026-07-14 | Made zero-byte requests visible, surfaced async peer-exhaustion failures, and retried eligible media when a host re-announces |
|
|
| 2026-07-14 | Preserved live attachment/download state when startup database hydration completes after realtime events |
|
|
| 2026-07-14 | Prevented reload-time `file-not-found` responses by waiting for metadata hydration, re-announcing hosts outside chat routes, and recovering persisted source paths on demand |
|
|
| 2026-07-14 | Blob-memory invariants: visibility-gated hydration, no `originalFiles` duplication for disk-backed blobs, revoke-on-destroy, inactive-room blob sweep |
|
|
| 2026-07-13 | Capacitor download/export to public `Documents` via `CapacitorAttachmentExportService` |
|
|
| 2026-07-05 | Expanded to full contract style |
|