diff options
| -rw-r--r-- | README.md | 91 | ||||
| -rw-r--r-- | Scrypted-Viewport-v2-Claude-Code-Implementation-Guide.md | 172 |
2 files changed, 194 insertions, 69 deletions
@@ -32,7 +32,10 @@ Waveshare ESP32-P4-ETH-POE Power -> DHCP -> mDNS (_scrypted-viewport._tcp.local) --> Wait +-> If unconfigured: show IP + hostname on screen +-> Wait for `/config` and `/frame` + +Unconfigured display shows the device IP and `viewport.local` so the operator can register it in Scrypted via `POST /config`. Once configured, the screen goes blank and waits for the first `/frame`. ## Resolution @@ -79,13 +82,15 @@ Returns `200 OK` with JSON: ```json { "display": "mudroom", - "callback": "http://scrypted.local:11080/api/viewport/touch" + "callback": "http://scrypted.local:11080/api/viewport/touch", + "idle_timeout_ms": 60000 } ``` - Persisted to NVS, survives reboot. - Idempotent; subsequent calls atomically replace prior config. - `display` must be non-empty; `callback` must be `http://...`. +- `idle_timeout_ms` optional, default `60000`. Sets the device-side idle timeout. Scrypted should use the same value for its own per-stream timeout so both ends agree (but they time independently — either can end the session). - Response: `204 No Content`. Invalid body: `400`. ### POST /frame @@ -93,7 +98,7 @@ Returns `200 OK` with JSON: - `Content-Type: image/jpeg`, body is raw JPEG bytes. - Image must be 800x480 baseline JPEG. Device does not scale or letterbox. - Max size: 1 MB. -- Wakes display (backlight on) and resets the idle timer. +- Wakes display (backlight on) and resets the 60s idle timer. - Single in-flight frame; concurrent posts may be rejected with `503`. - Returns `204` once decoded and pushed to the panel. - `400` malformed JPEG, `413` over size, `500` decode/display failure. On failure the previous frame stays on screen. @@ -114,25 +119,74 @@ Returns `200 OK` with JSON: - Persisted to NVS. Applied immediately if awake, otherwise on next wake. - Response: `204`. -## Touch Callback +## Wake / Sleep + +Wake and sleep couple the device backlight with the JPEG stream from Scrypted. + +- **Wake** = backlight on, frames being received from Scrypted. +- **Sleep** = backlight off, no frames expected. + +The device owns wake/sleep state. Scrypted does not — it just receives imperatives: + +- `wake` → "start streaming to this display now" +- `sleep` → "stop streaming to this display now" + +Both are idempotent on Scrypted's side. Scrypted does not track per-viewport state; it acts on the event and forgets. -Device POSTs to the `callback` URL registered via `/config`: +Transitions: + +| Trigger | Action | Callback to Scrypted | +| --- | --- | --- | +| Tap while asleep | Backlight on, show loading screen | `wake` | +| Tap while awake | Backlight off | `sleep` | +| Idle timer (`idle_timeout_ms` no `/frame`) | Backlight off | `sleep` | +| Scrypted `POST /sleep` | Backlight off | none | +| Scrypted `POST /frame` while asleep | Backlight on, paint frame | none | + +Only `tap` is detected on the touchscreen — long-press and swipes are out of scope. `tap` is internal; the callback carries the resulting imperative. + +Callback body: ```json { "display": "mudroom", - "event": "tap", + "event": "wake", "timestamp": 1730000000 } ``` -Events: `tap`, `long_press`, `swipe_left`, `swipe_right`. No coordinates — the device emits gestures, not points. +`event` is `wake` or `sleep`. Delivery: best-effort, ~1s timeout, no retry. Events before `/config` registers a callback are dropped. + +## Idle + +After `idle_timeout_ms` (default 60s) with no `/frame`, the device sleeps and POSTs `sleep`. Scrypted should be configured with the same timeout so its per-stream cutoff matches, but they run independently — either side can end the session, whichever notices first. -Delivery: best-effort, ~1 second timeout, no retry queue. Events before `/config` registers a callback are dropped. The device does not interpret events; Scrypted decides what to show next. +## BOOT button + +- **Short press**: overlay the IP screen for 15 seconds, then return to the prior state. Useful for identifying or re-registering a device that's already configured. Wakes the backlight temporarily; does not change the wake/sleep state or send a callback. +- **Hold ≥5s**: factory reset — clear NVS, reboot. The device comes back unconfigured, showing the IP screen until Scrypted POSTs `/config`. + +## Local rendering + +The device renders exactly two things itself; everything else is a JPEG from Scrypted: + +- **IP screen**: IP address and `viewport.local` as plain centered text. Shown on first boot until `/config`, after factory reset, and as a 15s overlay when BOOT is short-pressed. +- **Loading screen**: shown between a wake and the next `/frame` arriving. Plain "Loading…" text. + +Both use a small embedded bitmap font. No LVGL, no general text engine. ## Scrypted Integration -Scrypted owns a static list of viewports and pushes frames to each. On startup, register every viewport: +The Scrypted side is **code, not configuration** — Scrypted has no built-in concept of a network framebuffer. The code is small and lives inside Scrypted: + +- **v1**: a Scrypted Script (in the Scripts plugin) — listens for camera events, calls `takePicture()`, POSTs the JPEG to `/frame`. Exposes the `/api/viewport/touch` endpoint via the EndpointManager. ~50 lines of TypeScript, no package install. +- **v2** (planned, not in this spec): a small custom plugin using FFmpeg via `MediaManager` to pipe MJPEG into a chunked `POST /stream`. Adds live-feel frame rates. + +Either way, no Scrypted core changes and no external service. + +Scrypted owns a static list of viewports, each **bound to one Scrypted camera device** in the script/plugin. The binding tells Scrypted which camera's events drive that viewport (doorbell press, person/motion detection, etc.) and which camera's frames to push to it. + +On startup, register every viewport: ```ts await fetch(`${viewport}/config`, { @@ -145,7 +199,7 @@ await fetch(`${viewport}/config`, { }); ``` -Push a frame: +Stream a session of frames to a viewport (in response to a camera event or a `wake` callback): ```ts await fetch(`${viewport}/frame`, { @@ -155,17 +209,22 @@ await fetch(`${viewport}/frame`, { }); ``` -Handle touch at `POST /api/viewport/touch`. The handler is the entire interaction layer — camera selection, page cycling, snapshot vs live, sleep — all in Scrypted, keyed off `display` and `event`. +Handle wake/sleep at `POST /api/viewport/touch`: + +- `wake` → look up the camera bound to `display`, start streaming frames. +- `sleep` → stop streaming frames to `display`. + +Both are idempotent. Don't track viewport state on the Scrypted side; act on the event and forget. -Idle behavior is device-local: the viewport sleeps the backlight after ~30s with no new frame. Scrypted does not need to send `/sleep` unless it wants to dim early. +Scrypted should use the same timeout value it sent in `/config` as its own per-stream cutoff. The two timers run independently — either side can cut a session, whichever notices first. If the viewport's `sleep` callback is lost, the Scrypted-side timeout still ends the stream. ## Ops - Firmware updates: reflash over USB. No OTA in v1. -- Factory reset: hold BOOT during power-on to clear NVS (display name, callback, brightness). -- Boot screen: black until first `/frame`. `/health` is available as soon as DHCP completes. -- No DHCP lease: keep retrying; do not reboot. -- Ethernet disconnect: reconnect automatically; keep last frame on screen. +- Provisioning: flash the same firmware to every device. On first boot the screen shows its IP; register it from Scrypted via `POST /config`. +- Factory reset: hold BOOT for 5s during normal operation to clear NVS (display name, callback, brightness) and reboot. The device returns to the IP screen. +- No DHCP lease: keep retrying; do not reboot. Screen shows "no network" if unconfigured. +- Ethernet disconnect: reconnect automatically. If Scrypted is unreachable, displays go stale — nothing the device can do about it. ## Build diff --git a/Scrypted-Viewport-v2-Claude-Code-Implementation-Guide.md b/Scrypted-Viewport-v2-Claude-Code-Implementation-Guide.md index 74f5945..de34800 100644 --- a/Scrypted-Viewport-v2-Claude-Code-Implementation-Guide.md +++ b/Scrypted-Viewport-v2-Claude-Code-Implementation-Guide.md @@ -82,12 +82,14 @@ Boot flow: power on -> initialize logging -> initialize NVS + -> load saved config (display name, callback, brightness) -> initialize Ethernet -> DHCP -> initialize mDNS -> initialize display/backlight/touch -> start HTTP server -> advertise _scrypted-viewport._tcp.local + -> if unconfigured: render IP + viewport.local on screen -> wait for API calls ``` @@ -103,16 +105,43 @@ Scrypted renders 800x480 JPEG -> reset idle timer ``` +Wake/sleep model: + +Wake and sleep couple the backlight with Scrypted's frame stream. The two move together, never independently. + +```text +wake = backlight on + Scrypted streaming frames +sleep = backlight off + Scrypted not streaming +``` + Touch flow: ```text -user taps display - -> firmware detects gesture - -> POST JSON event to configured callback URL - -> Scrypted decides what to do - -> Scrypted may POST a new frame/sleep/brightness +tap while asleep: + backlight on + render loading screen + POST {"event":"wake"} -> Scrypted starts streaming + next /frame replaces loading screen + +tap while awake: + backlight off + POST {"event":"sleep"} -> Scrypted stops streaming + +idle timer fires (60s with no /frame): + backlight off + POST {"event":"sleep"} -> Scrypted stops streaming ``` +The callback events are `wake` and `sleep`; `tap` itself is internal. The events are idempotent imperatives — "start streaming" / "stop streaming" — not state notifications. Scrypted does not track per-viewport state; it acts on the event and forgets. + +Scrypted-initiated `/sleep` and `/frame` do not echo a callback. + +Only `tap` is detected on the touchscreen. Long-press and swipes are out of scope for v1. + +BOOT button: +- Short press: overlay IP screen for 15s, then return to prior state. Wakes backlight temporarily but does not change wake/sleep state and does not POST a callback. +- Hold ≥5s: clear NVS and reboot. Device comes back unconfigured. + --- ## 5. API Contract @@ -172,7 +201,8 @@ Request body: ```json { "display": "mudroom", - "callback": "http://scrypted.local:11080/api/viewport/touch" + "callback": "http://scrypted.local:11080/api/viewport/touch", + "idle_timeout_ms": 60000 } ``` @@ -180,7 +210,8 @@ Behavior: - Store display name. - Store callback URL. -- Persist to NVS so it survives reboot. +- Store idle timeout (optional in body, default 60000). +- Persist all to NVS so it survives reboot. - May be called repeatedly. - Replaces old config atomically. @@ -317,18 +348,19 @@ Example request body: Supported event names: ```text -tap -long_press -swipe_left -swipe_right +wake +sleep ``` +`tap` is the input the touchscreen detects; the callback event is the resulting state (`wake` or `sleep`). Idle-timer-driven sleep also sends `sleep`. + Rules: -- Scrypted Viewport does not interpret events. -- Scrypted Viewport does not switch cameras. -- Scrypted Viewport does not decide whether to stream. -- Scrypted owns behavior. +- The device owns wake/sleep state. Scrypted does not track it. +- Scrypted owns the viewport→camera binding and decides which stream goes to which viewport. +- `wake`/`sleep` callbacks are idempotent imperatives: "start streaming" / "stop streaming". Scrypted acts on receipt and forgets. +- Scrypted enforces its own per-stream timeout independently of the device's idle timer. Either can end a session, whichever notices first. +- Scrypted-initiated `/sleep` and `/frame` do not echo a callback. Callback delivery: @@ -382,6 +414,9 @@ scrypted-viewport/ idle_timer.h idle_timer.c + local_screens.h + local_screens.c + nvs_config.h nvs_config.c ``` @@ -508,15 +543,14 @@ Output: Responsibilities: - Initialize capacitive touch controller. -- Detect basic gestures: - - tap - - long press - - swipe left - - swipe right +- Detect `tap` only (touch-down + release within ~500ms, ignore movement). Long-press and swipes are out of scope. - Debounce. -- Send events to callback client. - -Gesture accuracy does not need to be perfect. The primary event is tap. +- On tap: + - If asleep: transition to wake — backlight on, render loading screen, POST `{"event":"wake"}`. + - If awake: transition to sleep — backlight off, POST `{"event":"sleep"}`. +- Also handle BOOT button: + - Short press: ask `local_screens` to overlay the IP screen for 15s, then restore prior state. No callback. + - Hold ≥5s: clear NVS via `nvs_config_reset()` and reboot. ### 8.8 `callback_client.c` @@ -531,12 +565,25 @@ Responsibilities: Responsibilities: -- Reset timer on `/frame`. -- Turn off backlight after timeout. -- Default idle timeout can be hardcoded initially, e.g. 30 seconds. -- Future `/config` may add timeout, but not required now. +- Reset timer on `/frame` and on tap-driven wake. +- On expiry: transition to sleep (backlight off, POST `{"event":"sleep"}`). +- Idle timeout: read from NVS (`idle_timeout_ms`), default 60000 ms. Set via `/config`. Scrypted is expected to use the same value as its own per-stream cutoff, but timers run independently. + +### 8.10 `local_screens.c` + +The only application UI the firmware draws. Two screens, both via a small embedded bitmap font (no LVGL, no general text engine). + +Responsibilities: + +- `local_screens_show_ip(ip)` — centered IP address and `viewport.local`. Shown: + - on boot when no callback URL is in NVS (persistent until `/config` arrives), and + - as a 15s overlay on BOOT short-press (then restore prior state). +- `local_screens_show_loading()` — centered "Loading…" text. Shown on wake until the next `/frame` lands. +- Render into the same RGB565 framebuffer the JPEG decoder targets, then push to the panel. + +Keep it small. Hard-code the font glyphs for digits, dots, colons, and a-z. No string formatting library — write tiny inline blitters. -### 8.10 `nvs_config.c` +### 8.11 `nvs_config.c` Responsibilities: @@ -545,10 +592,7 @@ Persist: - display name - callback URL - brightness - -Optional: - -- idle timeout +- idle timeout (ms) Do not persist frame data. @@ -635,44 +679,49 @@ Acceptance criterion: ## 12. Scrypted Integration Assumptions -Scrypted script owns a static display list: +Scrypted owns a list of viewports, each bound to one camera device: ```ts -const displays = [ - "http://viewport-mudroom.local", - "http://viewport-kitchen.local", +const viewports = [ + { url: "http://viewport-mudroom.local", display: "mudroom", camera: "frontDoor" }, + { url: "http://viewport-kitchen.local", display: "kitchen", camera: "driveway" }, ]; ``` +The binding lives in Scrypted config. The device knows nothing about cameras. + Registration on startup: ```ts -await fetch(`${display}/config`, { - method: "POST", - headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ - display: "mudroom", - callback: "http://scrypted.local:11080/api/viewport/touch" - }) -}); +for (const v of viewports) { + await fetch(`${v.url}/config`, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ + display: v.display, + callback: "http://scrypted.local:11080/api/viewport/touch", + idle_timeout_ms: 60000, + }), + }); +} ``` -Frame push: +Stream a session of frames to a viewport — triggered by either a camera event (doorbell, person, motion) or a `wake` callback from the viewport: ```ts -await fetch(`${display}/frame`, { +await fetch(`${v.url}/frame`, { method: "POST", headers: { "Content-Type": "image/jpeg" }, - body: jpegBuffer + body: jpegBuffer, }); ``` -Touch callback handler decides: +Callback handler at `/api/viewport/touch`: -- latest snapshot -- short live preview -- cycle page/camera -- sleep +- `wake` → look up viewport→camera binding, start streaming. +- `sleep` → stop streaming to that viewport. + +Both are idempotent. Don't track viewport state. Apply a Scrypted-side per-stream timeout using the same `idle_timeout_ms` value sent in `/config`, so streams end even if the `sleep` callback is dropped. Timers run independently on each side. --- @@ -774,6 +823,19 @@ Acceptance: Tap generates callback to test HTTP endpoint. ``` +### Milestone 8: Local Screens + +- Embed minimal bitmap font. +- Render unconfigured screen (IP + viewport.local) on boot when NVS has no callback. +- Render loading screen on tap-wake; replaced by next `/frame`. + +Acceptance: + +```text +Fresh device boots to an IP-display screen. +Tap on a sleeping configured device shows "Loading…" until next frame. +``` + --- ## 14. Test Images @@ -866,7 +928,11 @@ The project is successful when: 9. `/sleep` works. 10. `/brightness` works. 11. Tap sends callback JSON to Scrypted/test server. -12. Firmware has no Matter, HomeKit, polling, local UI rendering, or config portal. +12. Unconfigured device shows its IP on screen so it can be registered from Scrypted. +13. Tap toggles the device between wake (backlight on + Scrypted streaming) and sleep (backlight off + Scrypted not streaming), via `wake`/`sleep` callback events. +14. Idle timer fires `sleep` after `idle_timeout_ms` with no `/frame` (default 60s, set by Scrypted via `/config`). +15. BOOT short-press overlays the IP screen for 15s; BOOT hold ≥5s factory-resets. +16. Firmware has no Matter, HomeKit, polling, business logic, or config portal. The only locally-rendered UI is the IP screen and the loading screen. --- |