| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
backpressure-blind + lwIP TCP window bump
Three changes that work together to make the stream "always paint
what's freshest, never sit on a stale frame":
#1 — Firmware FIONREAD skip in stream_server
Right after the body of frame N comes off the wire (and before we
unlock the decoder + spend ~6ms on decode + paint), check the
kernel receive buffer with ioctl(FIONREAD). If at least one more
header (8 bytes) is queued, frame N is no longer the freshest
possible — skip its decode + paint and loop back to read frame
N+1. The TCP recv cost is unavoidable (bytes still have to cross
the wire) but the decoder + paint cost is saved on every superseded
frame. Glass-to-glass latency on the latest frame drops by however
many frames had backed up.
#2 — Scrypted: keep writing past kernel-buffer backpressure
Previously: when sock.write() returned false we dropped the next
ffmpeg frame at source. New: we keep writing through. Node buffers
internally; under our load (~16 MB/s ffmpeg → ~5-7 MB/s firmware)
the buffer rarely exceeds a frame or two. With the firmware now
silently skipping decode on backed-up frames (#1), excess frames
get shed for free on the device side. Scrypted's job is just to
hand the firmware the freshest bytes as fast as possible.
socketBackpressured is still tracked for the diagnostic log.
#3 — lwIP TCP window bump (revisiting earlier regression)
The previous attempt at LWIP_TCP_WND_DEFAULT=32k regressed under
HTTP because every /frame opened a fresh socket and we paid the
slow-start cost repeatedly. The streaming pivot eliminated that:
the socket is long-lived, slow-start runs exactly once, then we
ride the full window for the rest of the session. Bumping to
65535 (max for stock lwIP), SND_BUF to match, RECVMBOX to 16, SACK
on. Expected: recv throughput ceiling moves up from ~5.3 MB/s,
which directly raises the fps ceiling (recv is currently 37ms of
the 43ms per-frame total).
|
|
Three independent improvements landing together because they all
target the post-streaming-pivot "where do we spend the wall clock?"
question.
#1 — Snapshot: sharp → mediaManager native → ffmpeg cascade
pushSnapshot now tries three transforms in order of cost:
- sharp (~5-15ms, libvips bindings, handles resize + rotate)
- mediaManager.convertMediaObjectToBuffer with image/jpeg;width=W
;height=H mime hint (~10-30ms, Scrypted's native converter,
used only for landscape since rotation isn't standard)
- ffmpeg one-shot (~500-700ms cold start, the old slow path)
A `path=...` field in the snapshot log identifies which transform
actually ran so the user can confirm the fast paths are reachable
in their Scrypted runtime. takePicture, transform, and POST timings
are each broken out so we can see exactly where the snapshot wall
goes.
#2 — Firmware: windowed min/avg/max breakdown + idle-gap
Stream server replaces the per-frame single-sample log with a 30-
frame window summary:
N frames over Xs: Yfps Z MB/s avg-jpeg=KB |
lock min/avg/max | recv min/avg/max | dec min/avg/max |
paint min/avg/max | idle min/avg/max
- lock = mutex acquire time (sanity check; should be ~0us with one
client owning the decoder)
- recv = body bytes off the wire
- dec = HW JPEG decode
- paint = backbuffer flip + DMA queue
- idle = gap between previous paint completing and next header
landing (= upstream slack). Large idle means we're waiting on
ffmpeg/network; near-zero means we're the bottleneck.
#3 — Camera substream picker
"Stream-source choice drives end-to-end latency more than anything
else" — the existing hardcoded low-latency-first walk lands on the
camera's preview substream which is typically capped at 5-8 fps.
Adds a per-viewport setting under Display:
Camera substream: auto | low-resolution | medium-resolution
| local | remote | remote-recorder
auto keeps the current behavior; the pinned options let the user
force a higher-fps source when they want stream rate > preview
rate. The chosen destination is logged at stream start.
|
|
Replaces the per-frame HTTP POST loop with a single long-lived TCP
connection on port 81. The HTTP control plane (/state, /config,
/frame for snapshot) stays unchanged.
Wire protocol (big-endian, repeating until connection close):
[4 bytes jpeg_len][4 bytes seq][jpeg_len bytes JPEG]
Why
---
The HTTP path hit a measured floor of ~37ms p50 with intermittent
~230ms p95 spikes that survived every Nagle/keep-alive fix attempt.
Each frame paid: TCP setup (or pool churn), HTTP parsing, body recv,
decode, paint, response write, response ACK. Streaming removes
everything except recv + decode + paint.
Firmware (main/stream_server.[ch], new)
---------------------------------------
- TCP listen on configurable port (81) in its own FreeRTOS task,
one client at a time (matches the one-stream-per-device model).
- Per accepted socket: TCP_NODELAY on, then loop reading 8-byte
header → jpeg body → through the existing jpeg_decoder + display
paths. Same stale-seq guard as the HTTP /frame handler (reset per
connection so each session starts at seq 1).
- Frames received while asleep are still drained (to stay framed)
but not painted. The HTTP control-plane POST /state {wake}
resumes painting on the next frame.
- Every 30 painted frames a structured serial log shows per-stage
timing + sustained MB/s — replaces the cross-side Server-Timing
header (no HTTP response to attach it to anymore).
Scrypted side
-------------
- net.createConnection({ host, port: 81, noDelay: true }) opened
once per stream session. On disconnect/error/close we auto-
reconnect after 500ms.
- ffmpeg stdout demux writes [header][body] directly to the
socket. sock.write() returning false sets a backpressured flag
that drops incoming ffmpeg frames until 'drain' fires — natural
TCP backpressure handles "firmware can't keep up" without us
modeling it manually.
- Stripped the entire fetch-based timing infrastructure
(pushStreamFrame, fetchSamples, parseServerTiming, depth
histogram, Server-Timing parser). Replaced with one stream-shaped
log every 10s: fps + MB/s + socket.write p50/p95/max + drop count
+ backpressured flag.
- pushSnapshot (one-shot first-paint) still uses the HTTP /frame
endpoint — small and infrequent, not worth reworking.
- postJSON still uses node:http for /state and /config.
State machine status flags
--------------------------
Extended boot-time flags array from 6 → 7 slots so the stream
server's bring-up shows up alongside ETH/MDNS/HTTP. Layout is now
E M H S D J T (was E M H D J T).
|
