Prompt for microphone, camera, and notification runtime permissions during Capacitor startup, and fall back to WebView getUserMedia when the native preflight bridge fails so voice joins still surface Android permission dialogs. Co-authored-by: Cursor <cursoragent@cursor.com>
275 lines
18 KiB
Markdown
275 lines
18 KiB
Markdown
# Mobile Capacitor
|
|
|
|
Cross-context mobile shell for the Angular product client (`toju-app/`). Wraps the existing SPA in Ionic Capacitor native projects (`toju-app/android/`, `toju-app/ios/`) while keeping Capacitor APIs behind `toju-app/src/app/infrastructure/mobile/`.
|
|
|
|
## Responsibilities
|
|
|
|
- Detect runtime shell (`browser`, `capacitor`, `electron`) without importing native plugins in domain code. Capacitor packages and adapters load only on `capacitor` shells via dynamic `import()` so Electron/desktop startup never evaluates `@capacitor/*` modules.
|
|
- Expose facades for notifications, in-call controls, media/attachments, stream pop-out, background audio session, CallKit, and native persistence.
|
|
- Integrate with direct-call, voice-workspace, and chat composer flows.
|
|
|
|
## Boundaries
|
|
|
|
| Layer | Owns |
|
|
|-------|------|
|
|
| `infrastructure/mobile/` | Platform detection, plugin lazy-loading, web/Capacitor adapters |
|
|
| `infrastructure/persistence/` | `DatabaseService` routing (`browser` / `capacitor-sqlite` / `electron`) |
|
|
| Domains (`direct-call`, `chat`, `voice-session`) | Business orchestration; inject mobile facades only |
|
|
| `core/platform/PlatformService` | Adds `isCapacitor` flag for persistence routing |
|
|
| Capacitor native projects | OS permissions, push certificates, store packaging |
|
|
|
|
## Build & run
|
|
|
|
```bash
|
|
# Production web bundle (Capacitor webDir)
|
|
npm run build:prod
|
|
|
|
# Copy web assets into native projects
|
|
npm run cap:sync
|
|
|
|
# Open IDE
|
|
npm run cap:open:android
|
|
npm run cap:open:ios
|
|
|
|
### Linux: Android Studio path
|
|
|
|
Capacitor defaults to `/usr/local/android-studio/bin/studio.sh`. If Android Studio is installed elsewhere (common with **Flatpak** from Flathub), `npm run cap:open:android` uses `tools/resolve-android-studio-path.js` to locate `studio.sh` (Flatpak `active` symlink, Toolbox, snap, `/opt`, etc.). Override anytime with `CAPACITOR_ANDROID_STUDIO_PATH`.
|
|
|
|
# Convenience (build + sync + open)
|
|
npm run cap:build:android
|
|
npm run cap:build:ios
|
|
|
|
# CI / Linux: production web bundle + Capacitor sync + Gradle debug APK
|
|
npm run cap:apk:android
|
|
# → toju-app/android/app/build/outputs/apk/debug/app-debug.apk
|
|
```
|
|
|
|
Config: `toju-app/capacitor.config.ts` (`webDir: ../dist/client/browser`).
|
|
|
|
### CI (Gitea)
|
|
|
|
Release workflow `.gitea/workflows/release-draft.yml` builds a debug Android APK on every push to `main` / `master` (job `build-android`), stages it as `Toju-<version>-android-debug.apk`, and uploads it to the same draft Gitea release as the desktop `.exe` / `.deb` assets via `tools/gitea-release.js`.
|
|
|
|
Manual-only workflow `.gitea/workflows/build-android-apk.yml` (**workflow_dispatch**) repeats the same build and release upload on demand from any branch.
|
|
|
|
Both jobs install JDK 21 and Android SDK platform 36 inside the `node:22` container and run `tools/build-android-apk.sh`. No signing keystore is configured — output is a **debug** APK suitable for sideloading and QA.
|
|
|
|
Optional `google-services.json` is not injected in CI; push registration in artifact builds follows the same optional-Firebase behavior as local unsigned debug builds.
|
|
|
|
After dependency or plugin changes, run `npm run build:prod && npm run cap:sync` so native projects register `@capacitor/app`, `@capacitor-community/sqlite`, `@capawesome/capacitor-app-update`, push plugins, and `MetoyouMobile`.
|
|
|
|
## App icon & splash (Android brand assets)
|
|
|
|
The Capacitor shell must ship the Toju brand mark, not the stock Ionic/Capacitor placeholder. Brand resources are generated from `images/icon-new-rounded.png` (circular cat-on-purple disc) into `toju-app/android/app/src/main/res/`:
|
|
|
|
```bash
|
|
npm run cap:assets:android # → tools/generate-android-app-icons.mjs (uses sharp)
|
|
```
|
|
|
|
This produces, for every density (`mdpi … xxxhdpi`):
|
|
|
|
- `mipmap-*/ic_launcher.png` + `ic_launcher_round.png` — legacy launcher bitmaps (the brand disc).
|
|
- `mipmap-*/ic_launcher_foreground.png` — full-bleed adaptive foreground; the adaptive layers in `mipmap-anydpi-v26/ic_launcher*.xml` reference `@mipmap/ic_launcher_foreground` (the stock `drawable-v24/ic_launcher_foreground.xml` vector is removed).
|
|
- `values/ic_launcher_background.xml` — adaptive background colour set to the **brand purple `#4A217A`**, not stock white.
|
|
- `drawable*/splash.png` (port + land per density, plus the base) — brand mark centred on a purple field for the launch splash.
|
|
|
|
Invariants are encoded in `toju-app/src/app/infrastructure/mobile/logic/mobile-android-launcher-icon.rules.ts` (required file set, brand background colour, and the SHA-256 of every stock Capacitor placeholder that must never reappear). Coverage:
|
|
|
|
- Unit: `mobile-android-launcher-icon.rules.spec.ts` — asserts every density is present, no resource matches a stock placeholder hash, and the adaptive background is the brand purple.
|
|
- E2E: `e2e/tests/mobile/android-app-icon.spec.ts` — same contract plus pixel checks (launcher ring is purple, centre is the white cat; splash corner is purple, centre is the cat). Deterministic; no emulator.
|
|
|
|
Re-run `npm run cap:assets:android` whenever `images/icon-new-rounded.png` changes; `npm run cap:sync` is **not** needed (resources live in the native project, not `webDir`).
|
|
|
|
## Feature status
|
|
|
|
| Feature | Status | Notes |
|
|
|---------|--------|-------|
|
|
| Push/local notifications | **Working (partial)** | Local notifications always available; remote push (FCM/APNs) registers only when Firebase/APNs is configured — app starts normally without `google-services.json` |
|
|
| Server push dispatch | **Working (configured)** | Tokens persist in server SQLite; outbound FCM/APNs via env credentials |
|
|
| In-call notifications | **Working (Capacitor)** | Persistent notification with answer/mute/hang-up actions |
|
|
| Stream pop-out (PiP) | **Working (partial)** | Document PiP when WebView supports it; Android native PiP fallback via `MetoyouMobile` plugin |
|
|
| Background voice | **Working (partial)** | Android foreground service; iOS `UIBackgroundModes` audio + CallKit active-call bridge |
|
|
| iOS CallKit | **Working (partial)** | `MetoyouMobile.startCallKitSession` reports active calls; requires Xcode target wiring after `cap:sync` |
|
|
| Screensharing | **Limited** | Disabled on iOS WebView; Android `getDisplayMedia` may work |
|
|
| Composer attachments | **Working** | Mobile attachment button + hidden file input |
|
|
| Camera sharing | **Working** | Existing `getUserMedia` camera path in WebRTC stack |
|
|
| Speakerphone | **Working (partial)** | Android `AudioManager` via `MetoyouMobile`; iOS `@capgo/capacitor-audio-session`; direct-call speaker toggle on native mobile |
|
|
| Local DB (SQLite) | **Working** | `DatabaseService` routes Capacitor shells to `CapacitorDatabaseService` (native SQLite CRUD) |
|
|
| Store app updates | **Working (partial)** | `@capawesome/capacitor-app-update` via `MobileAppUpdateService`; Android in-app updates when Play allows, iOS opens App Store |
|
|
|
|
## Platform limitations
|
|
|
|
- **iOS background WebRTC:** OS may still suspend peer connections when backgrounded despite `audio` background mode and CallKit reporting.
|
|
- **iOS CallKit:** Plugin Swift source ships in `ios/App/App/MetoyouMobilePlugin.swift`; add it to the Xcode target if not auto-linked. Incoming-call UI is not fully bridged to WebRTC answer/hang-up yet.
|
|
- **iOS screenshare:** `getDisplayMedia` is not available in WKWebView.
|
|
- **Android PiP:** Native PiP enters activity-level PiP; WebView video may not always render inside PiP on all OEM WebViews.
|
|
- **Production discovery:** `signal.toju.app` may not expose `/api/servers/featured` or `/trending`; client skips those calls for known hosts.
|
|
- **Push delivery:** Requires FCM service account and APNs key configuration on the signaling server.
|
|
|
|
## Push notification setup (FCM / APNs)
|
|
|
|
### Android (FCM)
|
|
|
|
The app starts without Firebase. `MobilePushRegistrationService` probes `MetoyouMobile.isRemotePushConfigured()` (Firebase `FirebaseApp` on Android) before calling `PushNotifications.register()`; when unconfigured it logs a single warning and skips registration.
|
|
|
|
1. Create a Firebase project and add an Android app with package `com.metoyou.app`.
|
|
2. Copy `toju-app/android/app/google-services.json.example` to `google-services.json` (gitignored) and fill in your Firebase values.
|
|
3. Run `npm run cap:sync` so the Google Services Gradle plugin applies when the file is present (`build.gradle` applies it only when the JSON exists).
|
|
4. Rebuild with `npm run cap:build:android`.
|
|
5. Ensure `POST_NOTIFICATIONS`, `RECORD_AUDIO`, `MODIFY_AUDIO_SETTINGS`, `CAMERA`, and foreground-service permissions are granted on Android 13+.
|
|
6. Verify `MobilePushRegistrationService` logs a registration token after login.
|
|
|
|
### Android runtime permissions (voice / camera)
|
|
|
|
Capacitor's WebView requests `RECORD_AUDIO` **and** `MODIFY_AUDIO_SETTINGS` together for microphone capture. If `MODIFY_AUDIO_SETTINGS` is missing from `AndroidManifest.xml`, users can accept the prompt and `getUserMedia` still fails.
|
|
|
|
Declared in `toju-app/android/app/src/main/AndroidManifest.xml`:
|
|
|
|
| Permission | Purpose |
|
|
|------------|---------|
|
|
| `RECORD_AUDIO` | Microphone capture for voice calls and channels |
|
|
| `MODIFY_AUDIO_SETTINGS` | Required by Capacitor WebChromeClient alongside `RECORD_AUDIO` |
|
|
| `CAMERA` | WebRTC camera sharing and WebView file capture |
|
|
| `BLUETOOTH_CONNECT` | Bluetooth headset routing during calls (Android 12+) |
|
|
| `POST_NOTIFICATIONS` | Incoming/active call notifications |
|
|
| `FOREGROUND_SERVICE` / `FOREGROUND_SERVICE_MICROPHONE` | Background voice session |
|
|
|
|
Before WebRTC capture, the client calls `MobileMediaService.ensureVoiceCapturePermissions()` / `ensureCameraCapturePermissions()`, which delegate to `MetoyouMobile.requestVoiceCapturePermissions()` / `requestCameraCapturePermissions()` on Capacitor shells. If the native plugin is unavailable or the bridge call fails, capture preflight defers to the WebView `getUserMedia` permission flow instead of aborting voice/camera joins.
|
|
|
|
On Capacitor startup, `MobileRuntimePermissionsService` (via `MobileAppLifecycleService.initialize()`) proactively prompts for microphone, camera, local-notification, and push-notification runtime permissions so Android 13+ shells do not keep every permission in the "Not allowed" state until the user joins voice or receives a call.
|
|
|
|
### iOS (APNs)
|
|
|
|
1. Enable Push Notifications capability in Xcode for the `App` target.
|
|
2. Upload your APNs key/certificate in Apple Developer portal.
|
|
3. `Info.plist` includes `remote-notification`, `audio`, and `voip` background modes.
|
|
4. Run on a physical device; simulator push registration is limited.
|
|
|
|
### Server token storage & dispatch
|
|
|
|
Clients POST:
|
|
|
|
```http
|
|
POST /api/users/device-tokens
|
|
Content-Type: application/json
|
|
|
|
{ "userId": "<uuid>", "platform": "android|ios", "token": "<fcm-or-apns-token>" }
|
|
```
|
|
|
|
Tokens persist in server SQLite (`device_tokens` table). Outbound push uses repository-root `.env` credentials:
|
|
|
|
| Variable | Purpose |
|
|
|----------|---------|
|
|
| `FCM_SERVICE_ACCOUNT_PATH` or `FCM_SERVICE_ACCOUNT_JSON` | Android FCM HTTP v1 |
|
|
| `APNS_KEY_PATH`, `APNS_KEY_ID`, `APNS_TEAM_ID` | iOS APNs HTTP/2 |
|
|
| `APNS_BUNDLE_ID` | Defaults to `com.metoyou.app` |
|
|
| `APNS_USE_SANDBOX` | `true` for development builds |
|
|
|
|
Manual dispatch (ops/testing):
|
|
|
|
```http
|
|
POST /api/users/device-tokens/:userId/dispatch
|
|
{ "title": "Incoming call", "body": "Alice is calling" }
|
|
```
|
|
|
|
## Android foreground service
|
|
|
|
`VoiceCallForegroundService` starts when `MobileCallSessionService` begins an active call. Required manifest permissions:
|
|
|
|
- `FOREGROUND_SERVICE`
|
|
- `FOREGROUND_SERVICE_MICROPHONE`
|
|
- `RECORD_AUDIO`
|
|
- `MODIFY_AUDIO_SETTINGS`
|
|
- `POST_NOTIFICATIONS`
|
|
|
|
The service shows a low-importance ongoing notification while a call is active.
|
|
|
|
## SQLite persistence (Capacitor)
|
|
|
|
- Schema rules: `infrastructure/mobile/logic/mobile-sqlite-schema.rules.ts` (mirrors Electron entities).
|
|
- Statement execution: `infrastructure/mobile/logic/mobile-sqlite-execute.rules.ts` — `@capacitor-community/sqlite` `execute()` accepts **one** SQL statement per call; migrations run each DDL statement separately (never concatenated).
|
|
- Row mapping: `infrastructure/mobile/logic/mobile-sqlite-row-mapper.rules.ts`.
|
|
- CRUD service: `infrastructure/persistence/capacitor-database.service.ts`.
|
|
- Routing: `infrastructure/persistence/database-backend.rules.ts` — Capacitor uses SQLite, not IndexedDB.
|
|
- Per-user database files: `metoyou__<userId>` via `mobile-sqlite-database-name.rules.ts`.
|
|
- First launch runs DDL migrations stored in the `meta` table. Schema init failures are cached per database file so the client does not retry in a loop.
|
|
|
|
## Capacitor plugin loading
|
|
|
|
- `infrastructure/mobile/adapters/capacitor/capacitor-plugin-loader.ts` uses **static** `@capacitor/*` imports and `Capacitor.isPluginAvailable()` before returning a plugin. Do not `import()` plugin modules dynamically or `await` plugin objects (Capacitor proxies expose a throwing `.then()` stub).
|
|
- After adding or upgrading Capacitor plugins, run `npm run build:prod && npm run cap:sync` so Android/iOS native projects register `App`, `AppUpdate`, `LocalNotifications`, push, and SQLite.
|
|
|
|
## Safe area (Android)
|
|
|
|
- Capacitor `SystemBars` injects `--safe-area-inset-*` CSS variables into `document.documentElement`. `index.html` sets `viewport-fit=cover` and default inset values; `main.ts` calls `applyMobileSafeAreaDefaults()` so injection never hits a missing root element after the WebView loads.
|
|
- `capacitor.config.ts` sets `plugins.SystemBars.insetsHandling: 'css'` so Android WebView versions that mis-report `env(safe-area-inset-*)` still receive correct insets.
|
|
- Global `styles.scss` applies inset padding on `html` (with `env()` fallback) and sizes `app-root` to `height: 100%` so content stays below the status bar and above the navigation bar in edge-to-edge mode.
|
|
|
|
## Self-hosted HTTPS signal servers (Android)
|
|
|
|
Electron and desktop browsers accept the repo's self-signed `.certs/localhost.crt` because Electron runs with `ignore-certificate-errors` when `SSL=true`, and browsers let users bypass the warning once. **Android WebView does neither** — it only trusts system CAs (release) or system + user-installed CAs (debug builds).
|
|
|
|
| Runtime | Trust behavior |
|
|
|---------|----------------|
|
|
| Electron (`SSL=true`) | Ignores certificate errors (`electron/app/flags.ts`) |
|
|
| Browser | User accepts warning or imports CA |
|
|
| Android debug APK | System CAs + **user-installed CAs** (`src/debug/res/xml/network_security_config.xml`) |
|
|
| Android release APK | **System CAs only** — use Let's Encrypt or another public CA |
|
|
|
|
### Certificate requirements
|
|
|
|
1. **Trust:** Install `.certs/localhost.crt` on the Android device as a **CA certificate** (Settings → Security → Encryption & credentials → Install a certificate → CA certificate). Debug APKs pick this up automatically; release builds ignore user CAs.
|
|
2. **SAN:** The cert must list every host clients use. Regenerate with the server IP in the SAN when connecting by IP:
|
|
|
|
```bash
|
|
rm -rf .certs
|
|
SERVER_IP=46.59.68.77 ./generate-cert.sh
|
|
```
|
|
|
|
Restart the signaling server after regenerating certs.
|
|
|
|
3. **HTTPS only:** `AndroidManifest.xml` sets `android:usesCleartextTraffic="false"`. Server URLs must use `https://` (matching `environment.ts` / saved server endpoints).
|
|
|
|
### Troubleshooting
|
|
|
|
| Symptom | Likely cause |
|
|
|---------|----------------|
|
|
| `ERR_CERT_AUTHORITY_INVALID` / silent fetch failure | CA not installed on device, or testing a **release** APK with a self-signed cert |
|
|
| `ERR_CERT_COMMON_NAME_INVALID` | Cert SAN missing the IP/hostname (regenerate with `SERVER_IP`) |
|
|
| `ERR_CONNECTION_REFUSED` | Port unreachable from the phone (firewall, NAT, server not listening on `0.0.0.0`) — verify with `curl -k https://46.59.68.77:3001/api/health` from the device browser first |
|
|
| Works in Chrome on phone, fails in app | Chrome may use a different trust store path; ensure the CA is installed at the **system** level, not only per-browser |
|
|
|
|
Network security configs:
|
|
|
|
- `android/app/src/main/res/xml/network_security_config.xml` — release (system CAs, no cleartext)
|
|
- `android/app/src/debug/res/xml/network_security_config.xml` — debug (+ user CAs for dev)
|
|
|
|
**Do not commit** `.certs/*.crt`, `.certs/*.key`, or device-specific credential files.
|
|
|
|
## Integration points
|
|
|
|
- `DirectCallService` — incoming/active call notifications, ring-queue on user hydration, notification action routing.
|
|
- `PrivateCallComponent` — speakerphone toggle on native mobile shells.
|
|
- `ChatMessageComposerComponent` — `shouldShowAttachmentButton` + `pickAttachmentsFromDevice()`.
|
|
- `VoiceWorkspaceStreamTileComponent` — PiP when focused stream tile backgrounds.
|
|
- `MobileCallSessionService` — CallKit + foreground service + in-call notifications.
|
|
- `App` bootstrap — initializes mobile persistence, lifecycle, app-update polling, call-session, and push registration wiring.
|
|
- `MobileAppUpdateService` — periodic Play Store / App Store checks (30 min) and settings UI actions; mirrors Electron `DesktopAppUpdateService` polling but uses native store APIs instead of release manifests.
|
|
|
|
## Phase 3 completion notes
|
|
|
|
Phase 3 delivered:
|
|
|
|
1. Full `CapacitorDatabaseService` CRUD with `DatabaseService` routing on `isCapacitor`.
|
|
2. Server SQLite persistence for device tokens plus FCM/APNs outbound dispatch.
|
|
3. iOS CallKit bridge (partial) via `MetoyouMobile` plugin and `MobileCallKitService`.
|
|
4. Android Firebase Gradle wiring with `google-services.json.example` (real file gitignored).
|
|
5. Capacitor plugin availability checks to avoid hard failures when plugins are missing pre-sync.
|
|
6. Discovery endpoint skip for production signal hosts without featured/trending routes.
|
|
|
|
Remaining work:
|
|
|
|
- Wire CallKit answer/end actions back into `DirectCallService`.
|
|
- Migrate legacy IndexedDB mobile data into SQLite where needed.
|
|
- Deploy featured/trending routes to production signal servers or add capability negotiation in health checks.
|