| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Per-frame paint cost drops from ~24 ms to ~45 µs (≈500× faster) by
enabling num_fbs=2 on the DPI panel and decoding straight into the
back framebuffer. Measured on the bench:
before: lock=7us ttfb=370us body=40ms dec=6ms paint=24ms post=35us = ~70ms / ~14fps
after : lock=7us ttfb=330us body=38ms dec=6ms paint=42us post=25us = ~45ms / ~22fps
How the win actually lands:
- num_fbs=2 in the esp_lcd_dpi_panel_config_t makes the IDF driver
allocate two framebuffers and stream from one while we fill the
other.
- display_back_buffer() returns the inactive fb pointer + its size.
- jpeg_decoder_decode() now accepts a caller-provided destination
buffer instead of owning its own scratch. http_api passes the panel
back-fb so the hardware JPEG decoder writes BGR888 pixels straight
into where the DSI will eventually scan from. Zero memcpy in the
hot path.
- display_flip_back_buffer() calls esp_lcd_panel_draw_bitmap with the
fb pointer. Because the buffer is inside the panel's own fb range,
the IDF driver skips its memcpy and just does a cache writeback +
swaps cur_fb_index. The actual flip happens on the next vsync,
asynchronously — the call returns in microseconds.
The remaining ceiling is network body time (~38 ms for ~210 KB JPEGs)
and the hardware decoder (~6 ms). Per-viewport JPEG quality (smaller
files = shorter body) is the next lever; everything firmware-side is
already at or near floor.
Also drop the old static jpeg output scratch + JPEG_DECODER_MAX_OUTPUT_BYTES
constant — nothing references them anymore.
|
|
Architectural rework of the /frame hot path. Scrypted now ships every
JPEG already scaled + rotated to the panel's native dimensions (read
from the firmware's /state response so nothing is hardcoded on the
Scrypted side); the firmware decodes the JPEG straight into a BGR888
buffer that's directly draw_bitmap'able by the DSI driver, with zero
CPU pixel work and zero rotation work in between.
Firmware
- jpeg_decoder now uses JPEG_DECODE_OUT_FORMAT_RGB888 +
JPEG_DEC_RGB_ELEMENT_ORDER_BGR. Output buffer sized for 800*480*3.
- New display_present_bgr888() is a one-liner that hands the decoder's
output straight to esp_lcd_panel_draw_bitmap.
- /frame handler validates dimensions against the panel-native
VIEWPORT_PANEL_WIDTH x VIEWPORT_PANEL_HEIGHT (was effective_dims
branching on orientation). Returns 400 if it's anything else.
- /state JSON adds panel_width + panel_height so Scrypted can read them
without hardcoding board-specific knowledge.
- display_present_rgb565 + s_rot_buf stay for the local-screens cold
path (info screen, loading) which still does its own CPU conversion +
rotation — infrequent enough that it's not worth the rewrite.
Scrypted
- startStream() GETs /state at stream-start time, caches panel_width
and panel_height in viewport storage, and uses them as the ffmpeg
scale target. Falls back to cached or 800x480 if /state is mid-reboot.
- For portrait viewports the ffmpeg pipeline now does
scale=H:W:flags=lanczos,transpose=1 so the JPEG arrives pre-rotated
90° CW into panel-native dimensions. Landscape is just scale=W:H.
- No more in-firmware rotation; Scrypted is the single source of truth
for "how do I get this camera frame into a panel-shaped JPEG".
Expected ceiling lift: ~5 fps → ~10 fps, gated by the JPEG decoder
hardware throughput instead of the CPU rgb565→bgr888 + rotation loop.
|
|
Two channel-swap bugs were stacked, hiding each other on symmetric
pixels (white text on black background) and showing up only on
saturated colour JPEGs:
1. jpeg_decoder: rgb_order was JPEG_DEC_RGB_ELEMENT_ORDER_RGB which
empirically emits the RGB565 word in big-endian byte order (the
IDF header comment "color component in big endian" is correct;
the enum's "RGB" name is misleading). Our painter reads native
LE uint16_t, so the bytes were always swapped. Flipped to
_BGR which emits LE — verified by dumping decoded bytes for
solid red/green/blue JPEGs.
2. display: rgb565_to_rgb888 wrote bytes [R, G, B], but the
ESP32-P4 DSI engine + TC358762 + Pi panel expect [B, G, R]
in memory despite the "RGB888" label. (MIPI DSI defines bit
order, not byte position in memory.) Verified with an
embedded solid-blue test JPEG: with [R, G, B] the panel
showed solid red; with [B, G, R] it shows solid blue.
Diagnosed by embedding a 480x800 blue test JPEG in firmware, decoding
+ dumping the buffer at boot, and reading the actual bytes over USB
serial (ethernet was disconnected mid-investigation). The temporary
self-test + debug dump are removed; the fixes themselves are tiny.
|
|
Previously the 4-line "identity" screen showed only viewport name, mDNS
host, IP, and state. Expanded to 15 lines covering the full GET /config
+ GET /state output (name, host, ip, state, configured, scrypted,
orientation, brightness, idle, fw, uptime, frames, errs, free heap,
free PSRAM), label/value pairs left-aligned with auto-scaled font.
Renames "identity" → "info" throughout — symbol, log messages,
README/TESTING references.
Also:
- Move the info-screen render's big locals (~1.6 KiB: lines[16][80] +
scrypt[256] + vp_name[64]) to BSS. The touch task's 3 KiB stack
was overflowing on every long-press, leaving whatever frame was
previously on the panel — gave the appearance of a "blue screen"
after the M5 test pattern.
- Drop the ≥5 s touch factory-reset gesture and remove all stale
references to it in docs and comments. NVS wipe is now a USB-side
`idf.py erase-flash` operation only.
- M5 follow-up TODO in jpeg_decoder.c: solid green renders ~black and
solid blue renders green; not a byte-order issue (BGR setting turns
red into blue). Tracked as a known M5 gap.
|
|
jpeg_decoder.{h,c} wraps esp_driver_jpeg (ESP32-P4 hardware decoder).
- One-time engine + DMA-aligned PSRAM scratch buffer setup (1 MB input,
768 KB output @ panel native 800x480 RGB565).
- try_lock + unlock so concurrent /frame POSTs get 503 instead of
queueing, per spec.
- jpeg_decoder_get_info() reports the dimensions; the http handler
validates them against the effective resolution before painting.
display.h adds display_present_rgb565(src, w, h):
- Landscape: src is 800x480, memcpy 1:1 into the panel framebuffer.
- Portrait: src is 480x800, software rotate 90° CW into the 800x480
panel framebuffer. (PPA / 2D-DMA hardware rotation is a later
optimization if portrait latency matters.)
http_api.c adds POST /frame:
- Content-Type must be image/jpeg → else 400.
- Empty body → 400. > 1 MB → 413. Display not initialized → 500.
- jpeg_decoder_try_lock(0) for concurrency: second post returns 503.
- Body streamed into the decoder's input buffer in chunks.
- Decode failure or dimension mismatch → 400 + decode_errors++.
- Paint failure → 500.
- Success → frames_received++, last_frame_us = esp_timer_get_time(),
204 No Content.
app_main initializes the JPEG decoder after display_init(). Both are
best-effort: failures log a warning and leave the rest of the firmware
running.
CMakeLists.txt: add esp_driver_jpeg to REQUIRES.
Known gap (M6 closes it): /frame currently paints regardless of
wake/sleep state. The 409-when-asleep rule lands with POST /state
in M6.
Build clean against ESP-IDF 5.4 (binary ~640 KB).
TESTING.md M5 expanded with portrait/landscape test commands,
ImageMagick test-image recipes, the full negative matrix (wrong
Content-Type, oversize, wrong dims, concurrent, garbage), and the
M6 dependency note.
|
