| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Decouples network upload from firmware decode-and-paint. Before, the
inFlight boolean guard meant Scrypted sent frame N, waited for the
full ~80ms response (~40ms body upload + ~20ms decode + ~50µs paint +
ack), THEN sent frame N+1. The next ffmpeg frame arriving during that
window got dropped.
After:
- ScryptedViewportProvider keeps a per-host undici Agent with
keepAliveTimeout 30s, connections:2, pipelining:0. Sockets stay
open across /frame, /state, /config — saves the SYN+SYN-ACK+ACK
round-trip every POST (small on LAN but real on Wi-Fi).
- Stream loop's inFlight boolean is now a counter capped at 2 so
frame N+1 can begin uploading on socket B while frame N is still
being decoded on the device via socket A. Roughly doubles effective
throughput when body upload time dominates.
Firmware side:
- esp_http_server max_open_sockets bumped 7→4 explicitly: 2 for
pipelined /frame + 2 spare for concurrent /state and /config slots.
The JPEG decoder mutex still serialises decode (only one /frame
can be decoding at a time); this change only unblocks the network
half of the pipeline.
|
|
The single warning conflated two unrelated symptoms:
- HTTP/POST backpressure (drops accumulating because the firmware or
network can't keep up) — fix is to raise frame_interval_ms.
- Source-limited rate (ffmpeg's fps filter or the camera substream
produces fewer frames than requested) — raising the interval makes
this worse, not better.
Now: drops > 25% of target prints "backpressure" advice; otherwise
delivered < 60% of target prints "source-limited" with the correct
hint that the camera/filter is the limit, not the firmware. The
600ms-interval case where we get 1.5 fps delivered with zero drops
no longer prints the (wrong) "raise interval" suggestion.
|
|
Previous threshold was "log if drops > 25% of target rate". At 500ms
interval (2 fps target) that fired at 0.5+ dropped fps — but a
displayed rate of 1.5 fps is fine, the panel still updates every
~666ms. The warning told the user to "raise frame_interval_ms" when
nothing was actually wrong.
Track sentFrames separately and only warn when delivered rate falls
under 75% of target. Drop rate is now informational context in the
log line, not the trigger.
|
|
ffmpeg's fps filter sometimes emits timestamp-clumped pairs at low
rates; the single-flight POST guard correctly drops the second one
and we logged a "raise frame_interval_ms" warning for every such
event. At 500ms interval (2 fps target) a single drop produced a
0.6 fps warning even though we were still painting the requested
rate.
Suppress the message unless drops exceed a quarter of the configured
rate. Above that threshold there's real backpressure worth knowing
about; below it's just decimation noise.
|
|
substream + dropping queued frames
Two compounding causes of the lag:
- destination:"remote-recorder" returned the high-bitrate main encoder
with the camera's own ~10s prebuffer baked in. Walk substreams
low-resolution → medium-resolution → local → remote → remote-recorder
and take the first that resolves. The low-res preview is what
Scrypted itself uses for grid views — it's near-realtime.
- ffmpeg's output queue had no drop policy, so whenever the encode +
push pipeline fell behind for a second the queue just kept growing,
and we'd play back a steadily-older view of the world. Added
-fps_mode drop so late frames hit the floor.
Plus a few input-side latency knobs (-probesize 32, -analyzeduration 0,
-avioflags direct, -fflags +discardcorrupt) so ffmpeg stops sitting
on the first chunk to probe the stream layout.
|
|
type:"button" Settings get silently dropped by Scrypted's renderer —
they were in the model but never appeared in the UI. Switch both to
type:"boolean" so they show as toggles. The toggle acts as a one-shot
trigger: flipping it on fires the wake/sleep action and the next
getSettings() call returns value:false so it's armed again.
|
|
Triage signal for the persistent "expected 800x480, got 480x800"
dimension mismatch. The error has survived two filter rewrites; logs
on the user's side still reference old script.js line numbers, which
suggests the Scrypted Script editor isn't picking up re-pasted code.
Emit the orientation + panel dims + actual `-vf` string once per
stream start. If the log shows up, we know the latest code is live
and the problem is the filter; if it doesn't, the script never
updated.
|
|
The portrait filter chain was scale=480:800,transpose=1. Output frame
dimensions came out 800x480 (correct) but the mjpeg encoder was
occasionally writing the pre-transpose 480x800 into the JPEG SOF
marker, so the firmware's dimension check rejected the frame:
expected 800x480, got 480x800
Reordering to transpose=1,scale=800:480,setsar=1 makes the post-filter
dims explicit and unambiguous — transpose first changes the frame to
800x480, scale re-asserts it, setsar clears any leftover non-1:1
aspect ratio metadata. The encoder now writes 800x480 into SOF every
frame.
|
|
Adds an "Actions" group with two buttons:
- Wake now: starts a stream for the bound camera right now, bypassing
trigger filters. Useful for verifying the panel + camera link
without having to walk in front of a motion sensor.
- Sleep now: tears down the active stream and POSTs /state {sleep}.
Drops the private modifier from streams/streamStarting/startStream/
stopStream so the child Viewport can drive them — same package, no
encapsulation lost.
|
|
onBindingChanged previously stopped the active stream and waited for
the next camera event to relaunch. Changing orientation, frame
interval, or camera meant the user saw no immediate effect — the
display sat dark until the next motion/doorbell trigger.
Now if a stream was live at change-time we relaunch it right after
registerViewport pushes the new /config. Guarded by streamStarting so
the relaunch doesn't race with a concurrent camera-event-triggered
start.
|
|
Two changes to handle a burst of motion events without piling up
startStream calls and tripping the 1s /state-POST timeout:
- HTTP_TIMEOUT_MS 1000 → 5000. The firmware's single httpd task
processes one TCP connection at a time; under heavy /frame streaming
a /state {wake} can queue behind 1–3 in-flight /frames before
landing. 1s was too tight.
- handleCameraEvent now ignores repeat triggers if a stream is already
live for that viewport (this.streams.has(name)) or already starting
(streamStarting set). Sustained motion fires MotionSensor every
~500ms; we used to launch a fresh startStream each time, racing
with the previous one's ffmpeg spawn + state POST.
|
|
wall-clock
|
|
once
|
|
Cameras occasionally drop their H.264 stream mid-event — RTSP source
rotation, brief network glitch, etc. ffmpeg exits clean and the script
previously logged the exit and stopped feeding /frame, leaving the
panel stuck on the last successful frame until the next camera event
fired a fresh startStream.
Restructure the spawn into a closure-captured spawnFfmpeg() that the
on('close') handler can call again. Rolling restart cap of 5 per 60 s
prevents tight loops if the source is genuinely down.
On clean exit during an active stream: log + setTimeout(spawnFfmpeg, 250).
On too-many restarts: log a warning, stopStream, wait for the next
camera event.
The streams map no longer stores the ffmpeg proc (it changes across
restarts); abort.signal kills whatever's current via its listener.
|
|
|
|
Two fixes plus a README refresh:
1. scrypted: pushStreamFrame previously reset the per-stream idle timer
on every successful /frame response. That made the timer anchored to
"frames are flowing" rather than to "the camera event that triggered
the stream", so a continuously-streaming source would never let the
stream time out. Removed the reset. The startStream → stopStream(false)
cancel-and-replace path on repeated events still keeps the stream
alive while the event keeps firing; idle (no new events) now actually
ends the stream at idle_timeout_ms.
2. firmware: break the previous coarse recv/dec/paint timing into
lock : try_lock returned
ttfb : first httpd_req_recv chunk landed
body : remaining bytes received
dec : hardware JPEG decode
paint : esp_lcd_panel_draw_bitmap returned
post : state-counter bookkeeping + unlock
Logged every 10 frames at INFO. Splits the previously-fat recv bucket
into TCP/HTTP handshake overhead (ttfb) vs wire-time (body), and
surfaces any tail bookkeeping cost.
3. README: replace the stale "5 fps ceiling caused by CPU RGB conversion"
guess with the actual measured per-phase budget and re-rank the
backlog accordingly. Double-buffering the panel (paint 24 ms → ~2 ms)
is now the highest-value next move; the previously-listed DMA-2D
rewrite is moot because the CPU loop is already gone.
|
|
Architectural rework of the /frame hot path. Scrypted now ships every
JPEG already scaled + rotated to the panel's native dimensions (read
from the firmware's /state response so nothing is hardcoded on the
Scrypted side); the firmware decodes the JPEG straight into a BGR888
buffer that's directly draw_bitmap'able by the DSI driver, with zero
CPU pixel work and zero rotation work in between.
Firmware
- jpeg_decoder now uses JPEG_DECODE_OUT_FORMAT_RGB888 +
JPEG_DEC_RGB_ELEMENT_ORDER_BGR. Output buffer sized for 800*480*3.
- New display_present_bgr888() is a one-liner that hands the decoder's
output straight to esp_lcd_panel_draw_bitmap.
- /frame handler validates dimensions against the panel-native
VIEWPORT_PANEL_WIDTH x VIEWPORT_PANEL_HEIGHT (was effective_dims
branching on orientation). Returns 400 if it's anything else.
- /state JSON adds panel_width + panel_height so Scrypted can read them
without hardcoding board-specific knowledge.
- display_present_rgb565 + s_rot_buf stay for the local-screens cold
path (info screen, loading) which still does its own CPU conversion +
rotation — infrequent enough that it's not worth the rewrite.
Scrypted
- startStream() GETs /state at stream-start time, caches panel_width
and panel_height in viewport storage, and uses them as the ffmpeg
scale target. Falls back to cached or 800x480 if /state is mid-reboot.
- For portrait viewports the ffmpeg pipeline now does
scale=H:W:flags=lanczos,transpose=1 so the JPEG arrives pre-rotated
90° CW into panel-native dimensions. Landscape is just scale=W:H.
- No more in-firmware rotation; Scrypted is the single source of truth
for "how do I get this camera frame into a panel-shaped JPEG".
Expected ceiling lift: ~5 fps → ~10 fps, gated by the JPEG decoder
hardware throughput instead of the CPU rgb565→bgr888 + rotation loop.
|
|
|
|
Replace the setInterval(cam.takePicture) loop with a single ffmpeg
child per active stream that:
1. pulls the camera's video stream via getVideoStream + converts to
ffmpeg input args via ScryptedMimeTypes.FFmpegInput,
2. transcodes inline: scale=W:H:flags=lanczos, fps=N, c:v=mjpeg q:v=2,
-f image2pipe to stdout,
3. demuxes JPEG frames from stdout (FFD8…FFD9 boundaries),
4. POSTs each frame to the firmware's existing /frame endpoint with
the same single-flight semantics as before — surplus frames are
dropped silently and counted for a periodic skip-rate log so the
operator can dial frame_interval_ms to where the firmware sustains.
Wins vs. the snapshot path:
- The camera's main H.264 encoder is producing keyframes for free; we
pay near-zero on the source side instead of triggering a full snapshot
fetch per frame (the cost that capped us at ~2 fps).
- ffmpeg sustains the requested fps as long as the firmware can decode
+ paint at that rate. Measured ceiling is the firmware, not the
source, so the actual fps lands near the panel's true limit.
- Lanczos + q:v 2 keeps the picture sharp end-to-end.
stopStream now SIGTERMs the ffmpeg child alongside aborting outstanding
fetches. Each successful paint also nudges the Scrypted-side idle timer
so a healthy stream stays open as long as frames are landing.
Drop the now-unused pushFrame + resizeJpegHQ + the old setInterval
machinery.
|
|
Stopped asking cam.takePicture for a specific dimension. Most camera
plugins default snapshot JPEGs to q≈75 and do a quick bilinear downscale
from native (1920x1080 or so) to whatever we requested, which looked
visibly worse than the H.264 keyframe from the same camera at the same
panel resolution.
pushFrame now:
1. takePicture without picture.{width,height} → native-res JPEG
2. spawn ffmpeg child_process: scale w:h:flags=lanczos, mjpeg q:v 2
3. POST the result.
q:v 2 is near-lossless; Lanczos is a sharper downscale than the camera
plugin's default. Adds ~10-20 ms ffmpeg work per frame on a real CPU
(noise-level on a Pi 4), well under the snapshot-source ceiling that
already gates pushFrame at ~500 ms.
|
|
snapshot-based pipeline
|
|
The script-reload re-discovery passed `name: nativeId` to
onDeviceDiscovered, which Scrypted honored by renaming the existing
"kitchen" device to its vp_xxx nativeId. The subsequent registerViewport
fallback then saw a non-empty `v.name` (the nativeId) and used it as the
viewport name when POSTing /config, sending the firmware /config with
viewport="vp_mqek8i55_3z3s" instead of "kitchen".
Fix two things:
1. start()'s re-discovery now passes the persisted display_name (set on
createDevice + every Settings save) as the device's `name`, so the
user-chosen kitchen name survives script reloads.
2. registerViewport now prefers storage's display_name over v.name —
the storage value is the authoritative one, v.name is just a render
that can briefly drift to the nativeId during reload.
|
|
|
|
|
|
mDNS auto-resolve
- Move "frame push interval" from the parent's global setting to a
per-viewport child setting (different cameras want different rates).
Clamped to ≥33 ms (~30 fps max). Default still 1000 ms (1 fps).
- New "Wake triggers" multi-select on each viewport: doorbell, motion,
person. Default = all three. Clear all of them for tap-only mode.
handleCameraEvent now gates by this set; clearing it cleanly turns a
viewport into a tap-only display with no auto-wake.
- Add a "Status (live)" group at the bottom of each viewport's Settings
page that fetches /state + /config in parallel (1.5 s timeout) and
surfaces every field — name, mac, ip, awake/asleep, configured,
uptime, frame + error counters, resolution, free heap/psram, firmware
version, registered scrypted URL. Offline devices say so cleanly.
- Drop mDNS auto-resolve: no more lookupMdns / refreshHostFromMdns,
no per-viewport "auto-resolve via mDNS" toggle, no `dns` require.
The host field is just an operator-set string now. The README
documents the manual lookup commands (dns-sd, avahi-resolve) so
finding the device IP stays a one-liner.
- Parent's Settings page is now informational-only (viewport count +
callback base URL); per-viewport tuning lives on each child.
|
|
|
|
After the first camera event fires on a freshly-created viewport, v.name
sometimes resolves as "" (Scrypted device-record load racing with event
delivery). registerViewport then POSTs /config with viewport="" and the
firmware returns 400 "viewport must be a non-empty string". The handler
retries on the next callback so it eventually self-heals, but the log is
noisy and we'd needlessly fail one register cycle.
Mirror the canonical display name into device storage (display_name)
on createDevice + on every successful register, and have registerViewport
+ attachListener fall back to that storage value when v.name is empty.
No more empty-name POSTs to the firmware.
|
|
|
|
script reload
|
|
endpointManager.getInsecurePublicLocalEndpoint
|
|
|
|
they're injected
The probe in scrypted/diagnostic.ts confirmed @scrypted/core 0.3.147
injects every name we need (ScryptedDeviceBase, ScryptedDeviceType,
ScryptedInterface, systemManager, deviceManager, mediaManager,
endpointManager, log, device, require, exports) as top-level scope
variables. Notably sdk.ScryptedDeviceBase IS undefined — the classes
are separate injections, not properties on the sdk object.
So the correct pattern is `declare const X: any` for each (declarations
erase at compile time → free identifiers at runtime → bound to the
scryptedEval-injected values), not `const X = sdk.X` (which was
undefined and tripped "Class extends value undefined").
|
|
|
|
being a separate injection
|
|
The scryptedEval sandbox pre-injects SDK names (ScryptedDeviceBase,
ScryptedDeviceType, ScryptedInterface, systemManager, deviceManager,
mediaManager, endpointManager, log, device, sdk, require) into the
script's scope. The previous attempt destructured them from `sdk`,
but on user's Scrypted 0.143.0 / @scrypted/core 0.3.147 something in
the compile pipeline was still emitting a `require('@scrypted/sdk')`
that failed to resolve.
Switch to `declare const <name>: any` for each runtime value — the
declarations fully erase at compile time, no require is emitted, and
the values come straight from the sandbox scope.
|
|
@scrypted/sdk import
The Scripts plugin sandbox (in @scrypted/core) doesn't resolve ESM
imports of npm modules — saving the script with `import sdk, {...} from
'@scrypted/sdk'` produced:
Error: Cannot find module '@scrypted/sdk'
...
at t.scryptedEval (.../scrypted-eval.ts:102:29)
Fix: destructure runtime values from the `sdk` global the plugin
injects, use `require('dns').promises` for the Node mDNS lookup, and
declare runtime type aliases as `any` so the file still parses on a
machine without @scrypted/sdk installed locally.
Also drop the `as ScryptedInterface` cast in handleCameraEvent — it was
the only place an SDK enum was used as a type rather than a value, and
just storing the string is fine.
Editor-only diagnostics (NodeJS namespace, Buffer) remain — they
resolve in the Scrypted Node runtime and only show up in clangd/tsc
without @types/node locally.
|
|
|
|
The host field is now optional in "+ Add Device". The script tries
dns.promises.lookup("viewport-<name>.local") via the OS resolver
(Bonjour on macOS, nss-mdns on Linux, host networking on Docker) on:
- every registerViewport call (plugin start, child instantiation,
settings change, periodic 5-min refresh),
- right after createDevice so a fresh viewport's host field is
populated by the time the operator opens its settings page.
On a successful lookup that differs from the stored host, the
resolved IP is written back to the child's storage. POST /config and
POST /frame then use the resolved value.
Falls back gracefully to the operator-entered host (with a one-line
warning if both are empty). A per-viewport "Auto-resolve via mDNS"
toggle (default on) opts out — useful for cross-VLAN setups or hosts
where mDNS doesn't reach.
scrypted/README.md adds a "How mDNS auto-resolve works" section
covering the per-OS resolver requirements and the Docker host-
networking note.
|
|
Convert the Scripts-plugin script from a hardcoded BINDINGS constant
into a DeviceProvider + DeviceCreator + HttpRequestHandler. Each
viewport is now a child Scrypted device under the parent script with
its own Settings page; operators add, edit, and delete viewports
entirely through the Scrypted UI.
Parent (ScryptedViewportProvider):
- DeviceProvider: getDevice(nativeId) instantiates a Viewport, attaches
its camera event listener, and posts /config. releaseDevice tears
down stream + listener + child storage entry.
- DeviceCreator: "+ Add Device" on the parent's page shows a small form
(name / host / camera picker filtered to Camera interfaces /
orientation choice). createDevice() pre-populates the child's storage
via deviceManager.getDeviceStorage(nativeId) and registers it under
the parent.
- Tracks known child nativeIds in its own storage as a JSON array so it
can eagerly instantiate every child on plugin start (each
registration + camera subscription happens at load time, not lazily).
- 5-min re-register loop catches devices that rebooted or got new DHCP
leases.
- HttpRequestHandler routes POST <base>/state on the parent's endpoint;
body {viewport, state} is matched against child names. Honors the
spec race rules: every callback cancels any prior stream + safety
timer for that viewport before applying the new state, and a /frame
409 stops the stream without echoing sleep back.
- Global tuning (frame_interval_ms) lives on the parent's Settings.
Child (Viewport):
- Settings: host (string), camera (type=device with deviceFilter for
the Camera interface — the UX win), orientation (choices), idle
timeout, brightness. All persisted via this.storage.
- putSetting fires onBindingChanged() on the parent so re-register +
re-subscribe happen immediately when any field changes.
scrypted/README.md rewritten for the UI-driven flow — install + add
device + edit + remove + global tuning + smoke test — no more "edit
BINDINGS and re-save."
scrypted/package.json + tsconfig.json: optional `npm install` so
editors can resolve @scrypted/sdk types. Nothing here ships — install
remains "paste into Scripts plugin." node_modules ignored.
|
|
scrypted/scrypted-viewport.ts — single-file TypeScript script for
Scrypted's Scripts plugin. Binds N viewports to N Scrypted cameras and
implements the full Scrypted side of the protocol:
Outbound (Scrypted -> device):
- POST /config on script load + every 5 min (re-syncs after device
reboot or DHCP renumber).
- POST /state {wake} when a bound camera fires an event or when the
device's own POST /state {wake} arrives (operator tap).
- POST /frame at FRAME_INTERVAL_MS (1 fps default) via
camera.takePicture({picture:{width,height}}).
- POST /state {sleep} when the Scrypted-side per-stream timer
expires.
Inbound (device -> Scrypted POST <base>/state):
- {state:wake} -> startStream() (cancels pending sleep timer + any
prior interval, then begins pushing frames).
- {state:sleep} -> stopStream(sendSleep=false) (no echo back).
- Unknown viewport -> 404. Bad body -> 400.
Race rules from the spec are honored:
- Every callback cancels prior pending state for that viewport
before applying the new state (cancelPendingSleep semantics).
- startStream / stopStream are idempotent — re-entering is safe.
- On /frame returning 409, stream stops immediately without
echoing /state {sleep} back (the device already slept).
Configuration is a single BINDINGS array at the top of the file —
edit name/host/cameraId/orientation per viewport and save in the
Scrypted UI. No package install. Tuning constants
(IDLE_TIMEOUT_MS, FRAME_INTERVAL_MS, REREGISTER_INTERVAL_MS, etc.)
are right above the class.
scrypted/README.md walks through Scripts-plugin install, the BINDINGS
schema, what each constant controls, an event-by-event description of
the script's behavior, the v1 limitations (snapshot-rate, manual IP,
camera must respect picture dims), and an end-to-end smoke test.
Top-level README's Scrypted Integration section now links the script
file and the install README directly.
|
