diff options
| -rw-r--r-- | README.md | 2 | ||||
| -rw-r--r-- | scrypted/README.md | 83 | ||||
| -rw-r--r-- | scrypted/scrypted-viewport.ts | 338 |
3 files changed, 422 insertions, 1 deletions
@@ -335,7 +335,7 @@ Both use a small embedded bitmap font. No LVGL, no general text engine. The Scrypted side is **code, not configuration** — Scrypted has no built-in concept of a network framebuffer. The code is small and lives inside Scrypted: -- **v1**: a Scrypted Script (in the Scripts plugin) — listens for camera events, calls `takePicture()`, POSTs the JPEG to `/frame`. Exposes a `POST /state` handler at the plugin's endpoint root (e.g. `http://scrypted.local:11080/endpoint/scrypted-viewport/state`) via the EndpointManager. ~50 lines of TypeScript, no package install. +- **v1** (in this repo at [`scrypted/scrypted-viewport.ts`](scrypted/scrypted-viewport.ts), install instructions in [`scrypted/README.md`](scrypted/README.md)): a Scrypted Script (in the Scripts plugin) — listens for camera events, calls `takePicture()`, POSTs the JPEG to `/frame`. Exposes a `POST /state` handler at the plugin's endpoint root (e.g. `http://scrypted.local:11080/endpoint/scrypted-viewport/state`) via the EndpointManager. ~250 lines of TypeScript, no package install. - **Next milestone after v1 (`/stream`)**: add `POST /stream` with `multipart/x-mixed-replace` chunked body for live frame rates. Scrypted side becomes a small custom plugin using FFmpeg via `MediaManager` to pipe MJPEG. `/frame` stays for snapshots and debug. Either way, no Scrypted core changes and no external service. diff --git a/scrypted/README.md b/scrypted/README.md new file mode 100644 index 0000000..f279ca9 --- /dev/null +++ b/scrypted/README.md @@ -0,0 +1,83 @@ +# Scrypted-side script (v1) + +The ESP32 firmware needs something on the Scrypted side that: +- registers each viewport via `POST /config` on startup, +- receives device-initiated `POST /state` callbacks (tap, idle timeout), +- starts a JPEG stream to a viewport when its bound camera fires an event (doorbell ring, motion, person), and +- stops the stream when the viewport reports `sleep` or its own per-stream timer expires. + +`scrypted-viewport.ts` in this directory does all of that as a single-file TypeScript script for the **Scripts plugin**. v2 will replace this with a packaged plugin doing `POST /stream` over MJPEG; for now this is enough to bring up end-to-end functionality. + +## Install + +1. In Scrypted's web UI: **Plugins → search "Scripts" → install** (if you don't already have it). +2. **+ Add Device → Scripts plugin → New Script**. +3. Paste the entire contents of `scrypted-viewport.ts` into the editor and **Save**. +4. Edit the `BINDINGS` array at the top of the script for your setup: + + ```ts + const BINDINGS: Binding[] = [ + { + name: "mudroom", // matches /config viewport + host: "192.168.1.42", // viewport IP (LAN-resolvable) + cameraId: "abcdef0123456789", // Scrypted device id + orientation: "portrait", // 480x800 + }, + ]; + ``` + + - **`name`**: must match what the script POSTs to the device's `/config` (also drives `viewport-<name>.local`). + - **`host`**: viewport's current IP or hostname. v1 has no mDNS-SD discovery — set it manually and update if DHCP renumbers (or use a DHCP reservation). + - **`cameraId`**: the Scrypted device id (not the human name) of the camera to bind. Grab it from the URL bar on the camera's settings page in Scrypted. + - **`orientation`**: `portrait` (480×800) or `landscape` (800×480). Sent to the device via `/config` and used to size the snapshots. + +5. Save. The script's `start()` runs immediately. Within a few seconds you should see lines in the Scrypted log: + + ``` + Scrypted Viewport script up. Callback URL base: http://scrypted.local:11080/endpoint/<scriptId> + Bindings: 1 viewport(s) + subscribed to "Front Door" events for viewport "mudroom" + ``` + +6. On the viewport, `GET /config` should now show the populated `scrypted` URL and the `viewport` name. `GET /state` should show `configured: true` and `state: "asleep"`. + +## Tuning constants + +Defined right above the class: + +| Constant | Default | What it controls | +| --- | --- | --- | +| `IDLE_TIMEOUT_MS` | 60000 | Sent to the device in `/config`. Both sides use this value independently. | +| `STREAM_TIMEOUT_MS` | == `IDLE_TIMEOUT_MS` | Scrypted-side per-stream cutoff. Keep it the same as the device's idle timeout so both ends agree. | +| `FRAME_INTERVAL_MS` | 1000 | Snapshot push cadence during an active stream. ~1 fps is fine for ambient camera viewing. | +| `REREGISTER_INTERVAL_MS` | 300000 | Re-issue `/config` to every viewport every 5 minutes so a viewport that rebooted or moved IPs re-syncs without manual intervention. | +| `HTTP_TIMEOUT_MS` | 1000 | Per-call timeout for outbound POSTs. Matches the device's own 1 s timeout. | + +## What the script does on each event + +- **Camera event (doorbell ring / motion / person)** → call `startStream(binding.name)`: + - cancel any previous stream for that viewport, + - POST `{state: "wake"}` to the device, + - start the snapshot interval, + - arm a `STREAM_TIMEOUT_MS` safety timer. +- **Device-initiated `{state: "wake"}` callback** (operator tapped the panel) → same `startStream` path. +- **Device-initiated `{state: "sleep"}` callback** → `stopStream` with `sendSleep=false` (the device already knows). +- **Per-stream safety timer fires** → `stopStream` with `sendSleep=true` (tell the device to sleep). +- **`POST /frame` returns 409** → device went to sleep on its own (tap-to-sleep, or its idle timer); `stopStream` with `sendSleep=false`. + +## v1 limitations + +- Snapshot-rate only (~1 fps). Live MJPEG over `POST /stream` is v2. +- Manual IP per viewport (no mDNS-SD discovery yet — that's v1.1). +- Camera must respect `picture.width` / `picture.height` in `PictureOptions` or be paired with a snapshot plugin that resizes. If the camera returns the wrong size, the device rejects `/frame` with 400 and you'll see warning logs. Workaround: configure the camera plugin's snapshot size, or wait for v2 which will resize Scrypted-side via FFmpeg. +- No retry on transport errors. Best-effort matches the device's own semantics; the next event or callback re-syncs. + +## End-to-end smoke test + +Once installed and bindings are filled in: + +1. **Viewport boots** with no `/config` yet → IP screen on panel. +2. Script runs → `POST /config` lands → viewport → `state: asleep`, backlight off. +3. Tap viewport → device POSTs `{viewport: "mudroom", state: "wake"}` → script logs `recv mudroom -> wake` → snapshots start flowing for `STREAM_TIMEOUT_MS`. +4. Tap viewport again → device POSTs `{state: "sleep"}` → script stops streaming. +5. Trigger the bound camera (doorbell, motion sensor, or person detection) → script POSTs `{state: "wake"}` to viewport → snapshots flow for `STREAM_TIMEOUT_MS`, then both sides time out and sleep. diff --git a/scrypted/scrypted-viewport.ts b/scrypted/scrypted-viewport.ts new file mode 100644 index 0000000..026a3e2 --- /dev/null +++ b/scrypted/scrypted-viewport.ts @@ -0,0 +1,338 @@ +// Scrypted Viewport — v1 Scripts-plugin script +// +// What this does +// -------------- +// One Scrypted Script binds N viewport devices (the ESP32-P4 panels) to N +// Scrypted cameras. On a camera event (doorbell ring, motion, person), it +// wakes the bound viewport and streams snapshots until the viewport tells +// us to stop (or the per-stream timeout cuts us off). When the operator +// taps the viewport, the device POSTs us a wake/sleep call and we mirror +// the stream state. +// +// How to install +// -------------- +// 1. In Scrypted's web UI, install the "Scripts" plugin if you haven't. +// 2. Create a new script device under that plugin. +// 3. Paste this whole file into the script editor and save. +// 4. Edit the BINDINGS constant below — one entry per viewport device, +// matching the `viewport` name the device will use in /config and +// the Scrypted device ID of the camera you want it bound to. +// 5. The script registers each viewport with `/config` on load and again +// every REREGISTER_INTERVAL_MS to handle DHCP renumbering / reboots. +// +// Protocol contract is in this repo's README.md and TESTING.md. +// +// Limitations of this v1 +// ---------------------- +// - Snapshot-rate (~1 fps) via Camera.takePicture(). Live-rate streaming +// over POST /stream is v2 (custom plugin with FFmpeg/MJPEG). +// - Viewport IP is operator-configured, not mDNS-discovered. Add IPs to +// each BINDINGS entry; if a viewport gets a new DHCP lease, update the +// IP and re-save. (Proper bonjour-service discovery is a v1.1 addition.) +// - Camera must respect `picture.width`/`picture.height` in +// PictureOptions, OR be paired with a snapshot plugin that resizes. +// If the camera returns a different size, /frame will reject with 400 +// and the next attempt will fail the same way until the camera is +// reconfigured. + +import sdk, { + HttpRequest, + HttpRequestHandler, + HttpResponse, + ScryptedDeviceBase, + ScryptedInterface, + EventListenerRegister, +} from "@scrypted/sdk"; + +const { systemManager, endpointManager, mediaManager } = sdk; + +// ============================================================================ +// CONFIG — edit these for your setup +// ============================================================================ + +interface Binding { + /** Viewport name; matches the device's /config `viewport` field and its + * mDNS hostname (viewport-<name>.local). Must be unique. */ + name: string; + /** Operator-set IP or hostname of the viewport. Plain IP is simplest. */ + host: string; + /** Scrypted device id (not name) of the camera. Get it from the URL + * bar of the camera's settings page in Scrypted. */ + cameraId: string; + /** Physical orientation of the panel. `portrait` = 480x800, `landscape` + * = 800x480. Frames are sent at this resolution. */ + orientation: "portrait" | "landscape"; +} + +const BINDINGS: Binding[] = [ + // { + // name: "mudroom", + // host: "192.168.1.42", + // cameraId: "abcdef0123456789", + // orientation: "portrait", + // }, +]; + +/** Sent to the device in /config. The device's idle timer and Scrypted's + * per-stream timer both use this value, independently. */ +const IDLE_TIMEOUT_MS = 60_000; + +/** Hard ceiling on a stream session. Stops streaming this many ms after + * the most recent wake event. Matches the device's idle timeout so the + * two sides cut off together. */ +const STREAM_TIMEOUT_MS = IDLE_TIMEOUT_MS; + +/** Time between snapshot pushes during an active stream. 1 fps is fine + * for v1 ambient viewing; tune as desired. */ +const FRAME_INTERVAL_MS = 1_000; + +/** Re-issue POST /config to every viewport at this cadence so a viewport + * that rebooted (or got a new IP) re-syncs without manual intervention. */ +const REREGISTER_INTERVAL_MS = 5 * 60_000; + +/** Outbound HTTP timeout per call. */ +const HTTP_TIMEOUT_MS = 1_000; + +// ============================================================================ +// Implementation +// ============================================================================ + +class ScryptedViewportScript extends ScryptedDeviceBase implements HttpRequestHandler { + private bindings = new Map<string, Binding>(); + private listeners: EventListenerRegister[] = []; + private streams = new Map<string, { + interval: NodeJS.Timeout; + timeout: NodeJS.Timeout; + abort: AbortController; + }>(); + private scryptedBase = ""; + + constructor(nativeId?: string) { + super(nativeId); + this.start().catch(e => this.console.error("start failed", e)); + } + + private async start() { + for (const b of BINDINGS) this.bindings.set(b.name, b); + + // Endpoint URL the devices will POST callbacks to. + const raw = await endpointManager.getInsecurePublicLocalEndpoint(this.id); + this.scryptedBase = raw.replace(/\/$/, ""); + this.console.log(`Scrypted Viewport script up. Callback URL base: ${this.scryptedBase}`); + this.console.log(`Bindings: ${BINDINGS.length} viewport(s)`); + + // Register every viewport on boot and on a slow refresh timer. + await this.registerAll(); + setInterval(() => this.registerAll().catch(() => {}), REREGISTER_INTERVAL_MS); + + // Subscribe to camera events for every binding. + for (const b of BINDINGS) { + const camera = systemManager.getDeviceById(b.cameraId); + if (!camera) { + this.console.warn(`camera ${b.cameraId} for viewport "${b.name}" not found`); + continue; + } + const ifaces = [ + ScryptedInterface.BinarySensor, // doorbell + ScryptedInterface.MotionSensor, // motion + ScryptedInterface.ObjectDetector, // person/vehicle/... + ]; + const reg = camera.listen(ifaces, (source, details, data) => { + this.handleCameraEvent(b, details, data); + }); + this.listeners.push(reg); + this.console.log(`subscribed to "${camera.name}" events for viewport "${b.name}"`); + } + } + + private viewportUrl(host: string, path: string) { + return `http://${host}${path}`; + } + + private async registerAll() { + for (const b of BINDINGS) { + try { + await this.postJSON(this.viewportUrl(b.host, "/config"), { + viewport: b.name, + scrypted: this.scryptedBase, + idle_timeout_ms: IDLE_TIMEOUT_MS, + orientation: b.orientation, + brightness: 80, + }); + } catch (e) { + this.console.warn(`register ${b.name} (${b.host}) failed:`, (e as Error).message); + } + } + } + + // ------------------------------------------------------------------------ + // Camera event → wake the bound viewport + // ------------------------------------------------------------------------ + + private handleCameraEvent(b: Binding, details: any, data: any) { + const iface = details.eventInterface as ScryptedInterface; + let trigger = false; + + if (iface === ScryptedInterface.BinarySensor && data === true) trigger = true; + if (iface === ScryptedInterface.MotionSensor && data === true) trigger = true; + if (iface === ScryptedInterface.ObjectDetector) { + const detections = data?.detections ?? []; + // Heuristic: any person detection wakes the viewport. Customize + // here if you want to filter by zone, confidence, class, etc. + if (detections.some((d: any) => d?.className === "person")) trigger = true; + } + if (!trigger) return; + + this.console.log(`event ${iface} on ${b.name} -> wake + stream`); + this.startStream(b.name).catch(e => this.console.error("startStream failed", e)); + } + + // ------------------------------------------------------------------------ + // Stream control + // ------------------------------------------------------------------------ + + private async startStream(name: string) { + const b = this.bindings.get(name); + if (!b) return; + + // Race rule from spec: cancel pending operations on every callback. + // Stopping an already-running stream first ensures the safety timeout + // restarts cleanly and we don't double-send /state {wake}. + this.stopStream(name, /*sendSleep=*/ false); + + // Wake the device first so it shows the loading screen before frames arrive. + await this.postJSON(this.viewportUrl(b.host, "/state"), { state: "wake" }); + + const abort = new AbortController(); + const interval = setInterval(() => { + this.pushFrame(b, abort).catch(e => { + if (!abort.signal.aborted) this.console.warn(`pushFrame ${name} failed:`, (e as Error).message); + }); + }, FRAME_INTERVAL_MS); + + const timeout = setTimeout(() => { + this.console.log(`${name} stream timeout — stopping`); + this.stopStream(name); + }, STREAM_TIMEOUT_MS); + + this.streams.set(name, { interval, timeout, abort }); + } + + private stopStream(name: string, sendSleep = true) { + const s = this.streams.get(name); + if (!s) return; + s.abort.abort(); + clearInterval(s.interval); + clearTimeout(s.timeout); + this.streams.delete(name); + + if (sendSleep) { + const b = this.bindings.get(name); + if (b) { + this.postJSON(this.viewportUrl(b.host, "/state"), { state: "sleep" }) + .catch(() => {}); + } + } + } + + private async pushFrame(b: Binding, abort: AbortController) { + if (abort.signal.aborted) return; + + const camera: any = systemManager.getDeviceById(b.cameraId); + if (!camera) return; + + const w = b.orientation === "portrait" ? 480 : 800; + const h = b.orientation === "portrait" ? 800 : 480; + + // Ask the camera for a JPEG at the panel's effective resolution. Many + // snapshot providers honor this; some ignore it and return native + // resolution (in which case the device will reject with 400). + const picture = await camera.takePicture({ + picture: { width: w, height: h }, + reason: "event", + }); + const buf: Buffer = await mediaManager.convertMediaObjectToBuffer(picture, "image/jpeg"); + + const res = await fetch(this.viewportUrl(b.host, "/frame"), { + method: "POST", + headers: { "Content-Type": "image/jpeg" }, + body: buf, + signal: abort.signal, + }); + + if (res.status === 409) { + this.console.log(`${b.name} returned 409 — device went to sleep, stopping stream`); + this.stopStream(b.name, /*sendSleep=*/ false); + } else if (res.status === 400) { + // Most likely a dimension mismatch — log once per stream window. + const reason = await res.text().catch(() => ""); + this.console.warn(`${b.name} returned 400: ${reason}`); + } else if (!res.ok) { + this.console.warn(`${b.name} /frame failed: ${res.status}`); + } + } + + // ------------------------------------------------------------------------ + // Inbound: device -> Scrypted POST /state + // ------------------------------------------------------------------------ + + async onRequest(request: HttpRequest, response: HttpResponse) { + if (request.method !== "POST") { + response.send("", { code: 405 }); + return; + } + if (!request.url.endsWith("/state")) { + response.send("", { code: 404 }); + return; + } + + let body: any; + try { + body = JSON.parse(request.body); + } catch { + response.send("invalid JSON", { code: 400 }); + return; + } + const { viewport, state } = body ?? {}; + if (typeof viewport !== "string" || !this.bindings.has(viewport)) { + response.send(`unknown viewport: ${viewport}`, { code: 404 }); + return; + } + if (state !== "wake" && state !== "sleep") { + response.send(`state must be wake or sleep`, { code: 400 }); + return; + } + + this.console.log(`recv ${viewport} -> ${state} (device-initiated)`); + + if (state === "wake") { + // Race rule: startStream cancels pending operations first. + await this.startStream(viewport); + } else { + // Race rule: stopStream clears its safety timeout. Don't echo a + // /state {sleep} back to the device — it already knows. + this.stopStream(viewport, /*sendSleep=*/ false); + } + + response.send("", { code: 204 }); + } + + // ------------------------------------------------------------------------ + // Tiny HTTP helper + // ------------------------------------------------------------------------ + + private async postJSON(url: string, body: any) { + const res = await fetch(url, { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(body), + signal: AbortSignal.timeout(HTTP_TIMEOUT_MS), + }); + if (!res.ok && res.status !== 204) { + const text = await res.text().catch(() => ""); + throw new Error(`POST ${url} -> ${res.status} ${text}`); + } + } +} + +export default ScryptedViewportScript; |