luke/esp32-poe-scrypted-viewport/main/jpeg_decoder.c, branch main

main: code-review fixes — brightness units, OTA stall cap, stats + locking

2026-07-17T23:56:03+00:00

- display: s_last_pwm cached the raw 0-100 percentage at init but display_wake writes it straight to REG_PWM (0-255 duty) — first wake ran visibly dim until a /config brightness change. Shared pct_to_duty helper now converts in both paths. - http_api: cap consecutive OTA recv timeouts (a stalled client spun forever holding s_ota_in_progress, wedging OTA until reboot); cap viewport name at 54 chars so viewport- fits the 63-byte mDNS label; log mdns_service_refresh failures; /state builds JSON from a snapshot instead of holding the state lock across ~25 cJSON allocs; respond_400 delegates to respond_status. - stream_server: bytes_in_window now counts painted frames only (frames discarded while asleep inflated the first post-wake window's MB/s, avg-jpeg and chunk/wire averages; recv_bytes folded in); so_rcvbuf carried into the stats snapshot under the mux; drop dead HEADER_BYTES; merge read_body_instrumented into read_n. - state_machine: transition mutex serializes concurrent wake/sleep (display side-effects ran after the state lock dropped, so racing callers could leave the backlight contradicting st->state); wake-path placeholder paint now takes the decoder lock like the stream path (concurrent esp_lcd_panel_draw_bitmap from two tasks isn't safe). - local_screens: overlay paint takes the decoder lock too; check the overlay timer create; panel dims from viewport_state.h. - jpeg_decoder: try_lock before init returns busy instead of passing a NULL semaphore to xSemaphoreTake. - net_eth: clear cached IP string on link down (stale /state + info). - dead code: display_present_bgr888, TOUCH_FT5426_ADDR, touch s_task, JPEG_DECODER_MAX_OUTPUT_BYTES; doc drift in nvs_config.h / jpeg_decoder.h / app_main flag legend.

firmware: split stream recv into its own task with 3-buffer ping-pong

2026-06-21T01:50:43+00:00

handle_client previously ran recv → decode → paint serially on one FreeRTOS task. The kernel TCP buffer filled during decode+paint (~6ms), and against the IDF-default 5760-byte window the sender naturally stop-go-rate-limited to ~consumption. Raising the window to 65535 (previous experiment) regressed g2g from ~100ms to 17s growing unbounded — the sender pumped 45+ segments per round into a kernel buffer the app couldn't drain in time, and there was no way to skip-oldest on the kernel queue. This commit decouples recv from decode+paint: recv-task: owns the socket. Reads header + body into one of three preallocated PSRAM body buffers. On body complete, swaps the just-filled buffer into a 1-deep pending slot and picks a free buffer for the next recv. If the slot already held a frame (decode is slow), drops oldest in place — mirror of the Scrypted-side skip-oldest from e5acf93. decode-task: waits on a binary semaphore. On signal, claims pending, then decodes + paints without holding any shared lock. Frees its prior buffer implicitly by overwriting s_decode_idx on the next claim. 3 PSRAM body buffers (~3MB of 28MB free) ensure the invariant {recv_idx, pending_idx, decode_idx} are pairwise distinct without ever blocking recv. jpeg_decoder.c grew an alloc_input_buffer helper + jpeg_decoder_decode now takes an explicit input pointer so the stream and http_api snapshot paths don't share scratch. New stats: - recv_dropped_oldest: per-window count of pending-slot overwrites - decode_idle_min/avg/max_us: time decode-task spent waiting on signal Measurement at IDF-default 5760 window, Unifi medium substream: before split: recv_avg=32ms recv_max~44ms fps=22-26 (recv blocked during 6ms decode+paint; chunk_max capped at 5760) after split: recv_avg=17ms recv_max=18-37ms fps=21-29 steady, decode_idle_avg=27-40ms (decode mostly waiting), drop_oldest=0, painted at source rate The bottleneck moved from 'decode+paint serializes recv' to the wire's own send rate. Bigger windows are now safe (recv-task drains continuously, can't bury us), but won't add fps until source rate goes up — that's a separate conversation.

firmware: double-buffer the panel + zero-copy decode → ~22 fps ceiling

2026-06-15T12:50:23+00:00

Per-frame paint cost drops from ~24 ms to ~45 µs (≈500× faster) by enabling num_fbs=2 on the DPI panel and decoding straight into the back framebuffer. Measured on the bench: before: lock=7us ttfb=370us body=40ms dec=6ms paint=24ms post=35us = ~70ms / ~14fps after : lock=7us ttfb=330us body=38ms dec=6ms paint=42us post=25us = ~45ms / ~22fps How the win actually lands: - num_fbs=2 in the esp_lcd_dpi_panel_config_t makes the IDF driver allocate two framebuffers and stream from one while we fill the other. - display_back_buffer() returns the inactive fb pointer + its size. - jpeg_decoder_decode() now accepts a caller-provided destination buffer instead of owning its own scratch. http_api passes the panel back-fb so the hardware JPEG decoder writes BGR888 pixels straight into where the DSI will eventually scan from. Zero memcpy in the hot path. - display_flip_back_buffer() calls esp_lcd_panel_draw_bitmap with the fb pointer. Because the buffer is inside the panel's own fb range, the IDF driver skips its memcpy and just does a cache writeback + swaps cur_fb_index. The actual flip happens on the next vsync, asynchronously — the call returns in microseconds. The remaining ceiling is network body time (~38 ms for ~210 KB JPEGs) and the hardware decoder (~6 ms). Per-viewport JPEG quality (smaller files = shorter body) is the next lever; everything firmware-side is already at or near floor. Also drop the old static jpeg output scratch + JPEG_DECODER_MAX_OUTPUT_BYTES constant — nothing references them anymore.

firmware: zero-copy JPEG → BGR888 → DSI hot path; Scrypted pre-rotates

2026-06-15T12:23:47+00:00

Architectural rework of the /frame hot path. Scrypted now ships every JPEG already scaled + rotated to the panel's native dimensions (read from the firmware's /state response so nothing is hardcoded on the Scrypted side); the firmware decodes the JPEG straight into a BGR888 buffer that's directly draw_bitmap'able by the DSI driver, with zero CPU pixel work and zero rotation work in between. Firmware - jpeg_decoder now uses JPEG_DECODE_OUT_FORMAT_RGB888 + JPEG_DEC_RGB_ELEMENT_ORDER_BGR. Output buffer sized for 800*480*3. - New display_present_bgr888() is a one-liner that hands the decoder's output straight to esp_lcd_panel_draw_bitmap. - /frame handler validates dimensions against the panel-native VIEWPORT_PANEL_WIDTH x VIEWPORT_PANEL_HEIGHT (was effective_dims branching on orientation). Returns 400 if it's anything else. - /state JSON adds panel_width + panel_height so Scrypted can read them without hardcoding board-specific knowledge. - display_present_rgb565 + s_rot_buf stay for the local-screens cold path (info screen, loading) which still does its own CPU conversion + rotation — infrequent enough that it's not worth the rewrite. Scrypted - startStream() GETs /state at stream-start time, caches panel_width and panel_height in viewport storage, and uses them as the ffmpeg scale target. Falls back to cached or 800x480 if /state is mid-reboot. - For portrait viewports the ffmpeg pipeline now does scale=H:W:flags=lanczos,transpose=1 so the JPEG arrives pre-rotated 90° CW into panel-native dimensions. Landscape is just scale=W:H. - No more in-firmware rotation; Scrypted is the single source of truth for "how do I get this camera frame into a panel-shaped JPEG". Expected ceiling lift: ~5 fps → ~10 fps, gated by the JPEG decoder hardware throughput instead of the CPU rgb565→bgr888 + rotation loop.

M5 colors fixed: JPEG byte order + DPI memory layout

2026-06-14T23:08:50+00:00

Two channel-swap bugs were stacked, hiding each other on symmetric pixels (white text on black background) and showing up only on saturated colour JPEGs: 1. jpeg_decoder: rgb_order was JPEG_DEC_RGB_ELEMENT_ORDER_RGB which empirically emits the RGB565 word in big-endian byte order (the IDF header comment "color component in big endian" is correct; the enum's "RGB" name is misleading). Our painter reads native LE uint16_t, so the bytes were always swapped. Flipped to _BGR which emits LE — verified by dumping decoded bytes for solid red/green/blue JPEGs. 2. display: rgb565_to_rgb888 wrote bytes [R, G, B], but the ESP32-P4 DSI engine + TC358762 + Pi panel expect [B, G, R] in memory despite the "RGB888" label. (MIPI DSI defines bit order, not byte position in memory.) Verified with an embedded solid-blue test JPEG: with [R, G, B] the panel showed solid red; with [B, G, R] it shows solid blue. Diagnosed by embedding a 480x800 blue test JPEG in firmware, decoding + dumping the buffer at boot, and reading the actual bytes over USB serial (ethernet was disconnected mid-investigation). The temporary self-test + debug dump are removed; the fixes themselves are tiny.

Info screen: full /config + /state dump, replaces identity screen

2026-06-14T22:37:51+00:00

Previously the 4-line "identity" screen showed only viewport name, mDNS host, IP, and state. Expanded to 15 lines covering the full GET /config + GET /state output (name, host, ip, state, configured, scrypted, orientation, brightness, idle, fw, uptime, frames, errs, free heap, free PSRAM), label/value pairs left-aligned with auto-scaled font. Renames "identity" → "info" throughout — symbol, log messages, README/TESTING references. Also: - Move the info-screen render's big locals (~1.6 KiB: lines[16][80] + scrypt[256] + vp_name[64]) to BSS. The touch task's 3 KiB stack was overflowing on every long-press, leaving whatever frame was previously on the panel — gave the appearance of a "blue screen" after the M5 test pattern. - Drop the ≥5 s touch factory-reset gesture and remove all stale references to it in docs and comments. NVS wipe is now a USB-side `idf.py erase-flash` operation only. - M5 follow-up TODO in jpeg_decoder.c: solid green renders ~black and solid blue renders green; not a byte-order issue (BGR setting turns red into blue). Tracked as a known M5 gap.

M5: POST /frame — hardware JPEG decode + orientation-aware paint

2026-06-14T03:48:48+00:00

jpeg_decoder.{h,c} wraps esp_driver_jpeg (ESP32-P4 hardware decoder). - One-time engine + DMA-aligned PSRAM scratch buffer setup (1 MB input, 768 KB output @ panel native 800x480 RGB565). - try_lock + unlock so concurrent /frame POSTs get 503 instead of queueing, per spec. - jpeg_decoder_get_info() reports the dimensions; the http handler validates them against the effective resolution before painting. display.h adds display_present_rgb565(src, w, h): - Landscape: src is 800x480, memcpy 1:1 into the panel framebuffer. - Portrait: src is 480x800, software rotate 90° CW into the 800x480 panel framebuffer. (PPA / 2D-DMA hardware rotation is a later optimization if portrait latency matters.) http_api.c adds POST /frame: - Content-Type must be image/jpeg → else 400. - Empty body → 400. > 1 MB → 413. Display not initialized → 500. - jpeg_decoder_try_lock(0) for concurrency: second post returns 503. - Body streamed into the decoder's input buffer in chunks. - Decode failure or dimension mismatch → 400 + decode_errors++. - Paint failure → 500. - Success → frames_received++, last_frame_us = esp_timer_get_time(), 204 No Content. app_main initializes the JPEG decoder after display_init(). Both are best-effort: failures log a warning and leave the rest of the firmware running. CMakeLists.txt: add esp_driver_jpeg to REQUIRES. Known gap (M6 closes it): /frame currently paints regardless of wake/sleep state. The 409-when-asleep rule lands with POST /state in M6. Build clean against ESP-IDF 5.4 (binary ~640 KB). TESTING.md M5 expanded with portrait/landscape test commands, ImageMagick test-image recipes, the full negative matrix (wrong Content-Type, oversize, wrong dims, concurrent, garbage), and the M6 dependency note.