luke/esp32-poe-scrypted-viewport/main/http_api.c, branch v1.0.0

firmware: build fixes — esp_app_format component + forward-declare s_last_painted_seq

2026-06-20T00:31:23+00:00

Two build breaks discovered when actually compiling the prior commits: - main/CMakeLists.txt missing esp_app_format requirement, so esp_app_desc.h couldn't be resolved when app_main.c included it for the boot-time git-hash log. - s_last_painted_seq is referenced inside state_post_handler (reset to 0 on /state wake) but the static was declared further down the file alongside the other /frame state. C requires declaration before use — forward declare it above state_post_handler and keep a stub comment at the original location.

disable Nagle on /frame socket + add min/max/worst-frame to script log

2026-06-20T00:15:28+00:00

#1: TCP_NODELAY for /frame ESP-IDF lwIP defaults Nagle ON. /frame is the worst Nagle workload: ~140KB body POST followed by an empty 204 response, no follow-up data either way. Both the final partial-MTU body packet and the response packet can sit in the kernel send buffer up to 40ms waiting for an ACK that the other side is delaying-ACKing — silently adding tens of ms to every frame's wall time and showing up as a fat net_up bucket on the Scrypted side. setsockopt(TCP_NODELAY) at handler entry on every /frame. Cheap and idempotent. Includes the /state and /config sockets too — those are infrequent but small responses also benefit. #2: min/max + worst-frame decomposition in the Scrypted log The p50/p95 line answered "what's typical" but not "what happened in the worst frame this window". Now every 10-fetch log adds: - min and max columns alongside p50/p95 per bucket - a worst-frame breakdown row: identifies the single slowest fetch in the window and decomposes its wall time across all stages. Answers "did the slow frame get gated by emit→post (ffmpeg backed up), net_up (TCP/handshake spike), or fw_dec (large JPEG)?" directly, instead of us inferring from percentiles. Together these are the prerequisites for evaluating whether further optimization (HTTP keep-alive, chunked streaming) is worth pursuing.

unified end-to-end per-frame instrumentation + version stamps

2026-06-20T00:07:55+00:00

Cross-side timing was opaque: script saw "req=70ms" but couldn't split TCP/dispatch overhead from firmware decode time, and firmware serial logs and Scrypted console logs couldn't be correlated. Firmware /frame handler now emits Server-Timing on every response with per-stage breakdown: Server-Timing: recv;dur=X, dec;dur=Y, paint;dur=Z, post;dur=W, handle;dur=total Script parses it, joins by X-Frame-Seq implicitly (one POST per seq), derives net_up = req − fw_total, and logs a multi-line p50/p95 breakdown every 10 fetches: fetch "kitchen" #10 (jpeg=137KB) wall p50=65ms p95=140ms emit→post p50=0ms p95=2ms (queue wait) req p50=62ms p95=135ms (fetch → Response headers) net_up p50=43ms p95=110ms (TCP + body wire + dispatch) fw_recv p50=12ms p95=18ms (body off the wire) fw_dec p50=6ms p95=8ms (hardware JPEG) fw_paint p50=0.1ms p95=0.2ms (backbuffer flip) fw_post p50=0.4ms p95=0.6ms body-read p50=1ms (drain — empty body) inflight d0=8 d1=2 d2=0 stale-drops=0 Plus version stamps on both sides for the "is the user on the right code" question: - CMakeLists: PROJECT_VER = git short hash + -dirty marker if dirty. esp_app_get_description()->version exposes it at runtime. Boot log: "Scrypted Viewport boot (v0.1.0 build=4cf36e2-dirty)". - TS: SCRIPT_VERSION const at the top, bumped per commit, logged at script-eval: "Scrypted Viewport up (script=4cf36e2). Callback ..."

add X-Frame-Seq monotonic ordering for pipelined /frame POSTs

2026-06-15T14:32:57+00:00

With two /frame POSTs in flight on separate sockets, the firmware's JPEG decoder mutex serialises decode but FreeRTOS semaphore acquisition is not FIFO — under jitter the later-arriving older frame can grab the lock first and paint over the newer one, producing brief "panel travels backward in time" glitches. Scrypted side: - Per-viewport monotonic frameSeq counter, sent as X-Frame-Seq on every /frame POST. Reset on stopStream so each new stream starts at 1. Firmware side: - s_last_painted_seq tracks the highest seq we've painted. Frame arrives, mutex acquired, body received — if seq <= s_last_painted_seq the frame is dropped (200 OK + X-Frame-Drop: stale-seq header) and the mutex released without touching the back buffer. - s_last_painted_seq resets to 0 on POST /state {wake} so the next stream's seq=1 isn't rejected because the previous session reached a higher counter. - Missing/zero X-Frame-Seq (legacy clients) skips the check entirely — preserves pre-pipelining behaviour.

pipeline two /frame POSTs + HTTP keep-alive on the Scrypted side

2026-06-15T14:29:52+00:00

Decouples network upload from firmware decode-and-paint. Before, the inFlight boolean guard meant Scrypted sent frame N, waited for the full ~80ms response (~40ms body upload + ~20ms decode + ~50µs paint + ack), THEN sent frame N+1. The next ffmpeg frame arriving during that window got dropped. After: - ScryptedViewportProvider keeps a per-host undici Agent with keepAliveTimeout 30s, connections:2, pipelining:0. Sockets stay open across /frame, /state, /config — saves the SYN+SYN-ACK+ACK round-trip every POST (small on LAN but real on Wi-Fi). - Stream loop's inFlight boolean is now a counter capped at 2 so frame N+1 can begin uploading on socket B while frame N is still being decoded on the device via socket A. Roughly doubles effective throughput when body upload time dominates. Firmware side: - esp_http_server max_open_sockets bumped 7→4 explicitly: 2 for pipelined /frame + 2 spare for concurrent /state and /config slots. The JPEG decoder mutex still serialises decode (only one /frame can be decoding at a time); this change only unblocks the network half of the pipeline.

instrumentation: firmware idle gap between frames + Scrypted per-fetch wall-clock

2026-06-15T13:25:02+00:00

firmware: double-buffer the panel + zero-copy decode → ~22 fps ceiling

2026-06-15T12:50:23+00:00

Per-frame paint cost drops from ~24 ms to ~45 µs (≈500× faster) by enabling num_fbs=2 on the DPI panel and decoding straight into the back framebuffer. Measured on the bench: before: lock=7us ttfb=370us body=40ms dec=6ms paint=24ms post=35us = ~70ms / ~14fps after : lock=7us ttfb=330us body=38ms dec=6ms paint=42us post=25us = ~45ms / ~22fps How the win actually lands: - num_fbs=2 in the esp_lcd_dpi_panel_config_t makes the IDF driver allocate two framebuffers and stream from one while we fill the other. - display_back_buffer() returns the inactive fb pointer + its size. - jpeg_decoder_decode() now accepts a caller-provided destination buffer instead of owning its own scratch. http_api passes the panel back-fb so the hardware JPEG decoder writes BGR888 pixels straight into where the DSI will eventually scan from. Zero memcpy in the hot path. - display_flip_back_buffer() calls esp_lcd_panel_draw_bitmap with the fb pointer. Because the buffer is inside the panel's own fb range, the IDF driver skips its memcpy and just does a cache writeback + swaps cur_fb_index. The actual flip happens on the next vsync, asynchronously — the call returns in microseconds. The remaining ceiling is network body time (~38 ms for ~210 KB JPEGs) and the hardware decoder (~6 ms). Per-viewport JPEG quality (smaller files = shorter body) is the next lever; everything firmware-side is already at or near floor. Also drop the old static jpeg output scratch + JPEG_DECODER_MAX_OUTPUT_BYTES constant — nothing references them anymore.

scrypted: don't reset idle timer on each painted frame + finer timing

2026-06-15T12:41:45+00:00

Two fixes plus a README refresh: 1. scrypted: pushStreamFrame previously reset the per-stream idle timer on every successful /frame response. That made the timer anchored to "frames are flowing" rather than to "the camera event that triggered the stream", so a continuously-streaming source would never let the stream time out. Removed the reset. The startStream → stopStream(false) cancel-and-replace path on repeated events still keeps the stream alive while the event keeps firing; idle (no new events) now actually ends the stream at idle_timeout_ms. 2. firmware: break the previous coarse recv/dec/paint timing into lock : try_lock returned ttfb : first httpd_req_recv chunk landed body : remaining bytes received dec : hardware JPEG decode paint : esp_lcd_panel_draw_bitmap returned post : state-counter bookkeeping + unlock Logged every 10 frames at INFO. Splits the previously-fat recv bucket into TCP/HTTP handshake overhead (ttfb) vs wire-time (body), and surfaces any tail bookkeeping cost. 3. README: replace the stale "5 fps ceiling caused by CPU RGB conversion" guess with the actual measured per-phase budget and re-rank the backlog accordingly. Double-buffering the panel (paint 24 ms → ~2 ms) is now the highest-value next move; the previously-listed DMA-2D rewrite is moot because the CPU loop is already gone.

http_api: log per-frame timing breakdown every 10 frames (recv / decode / paint)

2026-06-15T12:30:37+00:00

firmware: zero-copy JPEG → BGR888 → DSI hot path; Scrypted pre-rotates

2026-06-15T12:23:47+00:00

Architectural rework of the /frame hot path. Scrypted now ships every JPEG already scaled + rotated to the panel's native dimensions (read from the firmware's /state response so nothing is hardcoded on the Scrypted side); the firmware decodes the JPEG straight into a BGR888 buffer that's directly draw_bitmap'able by the DSI driver, with zero CPU pixel work and zero rotation work in between. Firmware - jpeg_decoder now uses JPEG_DECODE_OUT_FORMAT_RGB888 + JPEG_DEC_RGB_ELEMENT_ORDER_BGR. Output buffer sized for 800*480*3. - New display_present_bgr888() is a one-liner that hands the decoder's output straight to esp_lcd_panel_draw_bitmap. - /frame handler validates dimensions against the panel-native VIEWPORT_PANEL_WIDTH x VIEWPORT_PANEL_HEIGHT (was effective_dims branching on orientation). Returns 400 if it's anything else. - /state JSON adds panel_width + panel_height so Scrypted can read them without hardcoding board-specific knowledge. - display_present_rgb565 + s_rot_buf stay for the local-screens cold path (info screen, loading) which still does its own CPU conversion + rotation — infrequent enough that it's not worth the rewrite. Scrypted - startStream() GETs /state at stream-start time, caches panel_width and panel_height in viewport storage, and uses them as the ffmpeg scale target. Falls back to cached or 800x480 if /state is mid-reboot. - For portrait viewports the ffmpeg pipeline now does scale=H:W:flags=lanczos,transpose=1 so the JPEG arrives pre-rotated 90° CW into panel-native dimensions. Landscape is just scale=W:H. - No more in-firmware rotation; Scrypted is the single source of truth for "how do I get this camera frame into a panel-shaped JPEG". Expected ceiling lift: ~5 fps → ~10 fps, gated by the JPEG decoder hardware throughput instead of the CPU rgb565→bgr888 + rotation loop.