| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
backpressure-blind + lwIP TCP window bump
Three changes that work together to make the stream "always paint
what's freshest, never sit on a stale frame":
#1 — Firmware FIONREAD skip in stream_server
Right after the body of frame N comes off the wire (and before we
unlock the decoder + spend ~6ms on decode + paint), check the
kernel receive buffer with ioctl(FIONREAD). If at least one more
header (8 bytes) is queued, frame N is no longer the freshest
possible — skip its decode + paint and loop back to read frame
N+1. The TCP recv cost is unavoidable (bytes still have to cross
the wire) but the decoder + paint cost is saved on every superseded
frame. Glass-to-glass latency on the latest frame drops by however
many frames had backed up.
#2 — Scrypted: keep writing past kernel-buffer backpressure
Previously: when sock.write() returned false we dropped the next
ffmpeg frame at source. New: we keep writing through. Node buffers
internally; under our load (~16 MB/s ffmpeg → ~5-7 MB/s firmware)
the buffer rarely exceeds a frame or two. With the firmware now
silently skipping decode on backed-up frames (#1), excess frames
get shed for free on the device side. Scrypted's job is just to
hand the firmware the freshest bytes as fast as possible.
socketBackpressured is still tracked for the diagnostic log.
#3 — lwIP TCP window bump (revisiting earlier regression)
The previous attempt at LWIP_TCP_WND_DEFAULT=32k regressed under
HTTP because every /frame opened a fresh socket and we paid the
slow-start cost repeatedly. The streaming pivot eliminated that:
the socket is long-lived, slow-start runs exactly once, then we
ride the full window for the rest of the session. Bumping to
65535 (max for stock lwIP), SND_BUF to match, RECVMBOX to 16, SACK
on. Expected: recv throughput ceiling moves up from ~5.3 MB/s,
which directly raises the fps ceiling (recv is currently 37ms of
the 43ms per-frame total).
|
|
Three independent improvements landing together because they all
target the post-streaming-pivot "where do we spend the wall clock?"
question.
#1 — Snapshot: sharp → mediaManager native → ffmpeg cascade
pushSnapshot now tries three transforms in order of cost:
- sharp (~5-15ms, libvips bindings, handles resize + rotate)
- mediaManager.convertMediaObjectToBuffer with image/jpeg;width=W
;height=H mime hint (~10-30ms, Scrypted's native converter,
used only for landscape since rotation isn't standard)
- ffmpeg one-shot (~500-700ms cold start, the old slow path)
A `path=...` field in the snapshot log identifies which transform
actually ran so the user can confirm the fast paths are reachable
in their Scrypted runtime. takePicture, transform, and POST timings
are each broken out so we can see exactly where the snapshot wall
goes.
#2 — Firmware: windowed min/avg/max breakdown + idle-gap
Stream server replaces the per-frame single-sample log with a 30-
frame window summary:
N frames over Xs: Yfps Z MB/s avg-jpeg=KB |
lock min/avg/max | recv min/avg/max | dec min/avg/max |
paint min/avg/max | idle min/avg/max
- lock = mutex acquire time (sanity check; should be ~0us with one
client owning the decoder)
- recv = body bytes off the wire
- dec = HW JPEG decode
- paint = backbuffer flip + DMA queue
- idle = gap between previous paint completing and next header
landing (= upstream slack). Large idle means we're waiting on
ffmpeg/network; near-zero means we're the bottleneck.
#3 — Camera substream picker
"Stream-source choice drives end-to-end latency more than anything
else" — the existing hardcoded low-latency-first walk lands on the
camera's preview substream which is typically capped at 5-8 fps.
Adds a per-viewport setting under Display:
Camera substream: auto | low-resolution | medium-resolution
| local | remote | remote-recorder
auto keeps the current behavior; the pinned options let the user
force a higher-fps source when they want stream rate > preview
rate. The chosen destination is logged at stream start.
|
|
Replaces the per-frame HTTP POST loop with a single long-lived TCP
connection on port 81. The HTTP control plane (/state, /config,
/frame for snapshot) stays unchanged.
Wire protocol (big-endian, repeating until connection close):
[4 bytes jpeg_len][4 bytes seq][jpeg_len bytes JPEG]
Why
---
The HTTP path hit a measured floor of ~37ms p50 with intermittent
~230ms p95 spikes that survived every Nagle/keep-alive fix attempt.
Each frame paid: TCP setup (or pool churn), HTTP parsing, body recv,
decode, paint, response write, response ACK. Streaming removes
everything except recv + decode + paint.
Firmware (main/stream_server.[ch], new)
---------------------------------------
- TCP listen on configurable port (81) in its own FreeRTOS task,
one client at a time (matches the one-stream-per-device model).
- Per accepted socket: TCP_NODELAY on, then loop reading 8-byte
header → jpeg body → through the existing jpeg_decoder + display
paths. Same stale-seq guard as the HTTP /frame handler (reset per
connection so each session starts at seq 1).
- Frames received while asleep are still drained (to stay framed)
but not painted. The HTTP control-plane POST /state {wake}
resumes painting on the next frame.
- Every 30 painted frames a structured serial log shows per-stage
timing + sustained MB/s — replaces the cross-side Server-Timing
header (no HTTP response to attach it to anymore).
Scrypted side
-------------
- net.createConnection({ host, port: 81, noDelay: true }) opened
once per stream session. On disconnect/error/close we auto-
reconnect after 500ms.
- ffmpeg stdout demux writes [header][body] directly to the
socket. sock.write() returning false sets a backpressured flag
that drops incoming ffmpeg frames until 'drain' fires — natural
TCP backpressure handles "firmware can't keep up" without us
modeling it manually.
- Stripped the entire fetch-based timing infrastructure
(pushStreamFrame, fetchSamples, parseServerTiming, depth
histogram, Server-Timing parser). Replaced with one stream-shaped
log every 10s: fps + MB/s + socket.write p50/p95/max + drop count
+ backpressured flag.
- pushSnapshot (one-shot first-paint) still uses the HTTP /frame
endpoint — small and infrequent, not worth reworking.
- postJSON still uses node:http for /state and /config.
State machine status flags
--------------------------
Extended boot-time flags array from 6 → 7 slots so the stream
server's bring-up shows up alongside ETH/MDNS/HTTP. Layout is now
E M H S D J T (was E M H D J T).
|
|
s_last_painted_seq
Two build breaks discovered when actually compiling the prior commits:
- main/CMakeLists.txt missing esp_app_format requirement, so
esp_app_desc.h couldn't be resolved when app_main.c included it for
the boot-time git-hash log.
- s_last_painted_seq is referenced inside state_post_handler (reset
to 0 on /state wake) but the static was declared further down the
file alongside the other /frame state. C requires declaration
before use — forward declare it above state_post_handler and keep
a stub comment at the original location.
|
|
#1: TCP_NODELAY for /frame
ESP-IDF lwIP defaults Nagle ON. /frame is the worst Nagle workload:
~140KB body POST followed by an empty 204 response, no follow-up data
either way. Both the final partial-MTU body packet and the response
packet can sit in the kernel send buffer up to 40ms waiting for an
ACK that the other side is delaying-ACKing — silently adding tens of
ms to every frame's wall time and showing up as a fat net_up bucket
on the Scrypted side.
setsockopt(TCP_NODELAY) at handler entry on every /frame. Cheap and
idempotent. Includes the /state and /config sockets too — those are
infrequent but small responses also benefit.
#2: min/max + worst-frame decomposition in the Scrypted log
The p50/p95 line answered "what's typical" but not "what happened in
the worst frame this window". Now every 10-fetch log adds:
- min and max columns alongside p50/p95 per bucket
- a worst-frame breakdown row: identifies the single slowest fetch
in the window and decomposes its wall time across all stages.
Answers "did the slow frame get gated by emit→post (ffmpeg
backed up), net_up (TCP/handshake spike), or fw_dec (large
JPEG)?" directly, instead of us inferring from percentiles.
Together these are the prerequisites for evaluating whether further
optimization (HTTP keep-alive, chunked streaming) is worth pursuing.
|
|
Cross-side timing was opaque: script saw "req=70ms" but couldn't
split TCP/dispatch overhead from firmware decode time, and firmware
serial logs and Scrypted console logs couldn't be correlated.
Firmware /frame handler now emits Server-Timing on every response
with per-stage breakdown:
Server-Timing: recv;dur=X, dec;dur=Y, paint;dur=Z,
post;dur=W, handle;dur=total
Script parses it, joins by X-Frame-Seq implicitly (one POST per seq),
derives net_up = req − fw_total, and logs a multi-line p50/p95
breakdown every 10 fetches:
fetch "kitchen" #10 (jpeg=137KB)
wall p50=65ms p95=140ms
emit→post p50=0ms p95=2ms (queue wait)
req p50=62ms p95=135ms (fetch → Response headers)
net_up p50=43ms p95=110ms (TCP + body wire + dispatch)
fw_recv p50=12ms p95=18ms (body off the wire)
fw_dec p50=6ms p95=8ms (hardware JPEG)
fw_paint p50=0.1ms p95=0.2ms (backbuffer flip)
fw_post p50=0.4ms p95=0.6ms
body-read p50=1ms (drain — empty body)
inflight d0=8 d1=2 d2=0
stale-drops=0
Plus version stamps on both sides for the "is the user on the right
code" question:
- CMakeLists: PROJECT_VER = git short hash + -dirty marker if dirty.
esp_app_get_description()->version exposes it at runtime. Boot
log: "Scrypted Viewport boot (v0.1.0 build=4cf36e2-dirty)".
- TS: SCRIPT_VERSION const at the top, bumped per commit, logged at
script-eval: "Scrypted Viewport up (script=4cf36e2). Callback ..."
|
|
With two /frame POSTs in flight on separate sockets, the firmware's
JPEG decoder mutex serialises decode but FreeRTOS semaphore
acquisition is not FIFO — under jitter the later-arriving older
frame can grab the lock first and paint over the newer one,
producing brief "panel travels backward in time" glitches.
Scrypted side:
- Per-viewport monotonic frameSeq counter, sent as X-Frame-Seq on
every /frame POST. Reset on stopStream so each new stream starts
at 1.
Firmware side:
- s_last_painted_seq tracks the highest seq we've painted. Frame
arrives, mutex acquired, body received — if seq <= s_last_painted_seq
the frame is dropped (200 OK + X-Frame-Drop: stale-seq header) and
the mutex released without touching the back buffer.
- s_last_painted_seq resets to 0 on POST /state {wake} so the next
stream's seq=1 isn't rejected because the previous session reached
a higher counter.
- Missing/zero X-Frame-Seq (legacy clients) skips the check entirely
— preserves pre-pipelining behaviour.
|
|
Decouples network upload from firmware decode-and-paint. Before, the
inFlight boolean guard meant Scrypted sent frame N, waited for the
full ~80ms response (~40ms body upload + ~20ms decode + ~50µs paint +
ack), THEN sent frame N+1. The next ffmpeg frame arriving during that
window got dropped.
After:
- ScryptedViewportProvider keeps a per-host undici Agent with
keepAliveTimeout 30s, connections:2, pipelining:0. Sockets stay
open across /frame, /state, /config — saves the SYN+SYN-ACK+ACK
round-trip every POST (small on LAN but real on Wi-Fi).
- Stream loop's inFlight boolean is now a counter capped at 2 so
frame N+1 can begin uploading on socket B while frame N is still
being decoded on the device via socket A. Roughly doubles effective
throughput when body upload time dominates.
Firmware side:
- esp_http_server max_open_sockets bumped 7→4 explicitly: 2 for
pipelined /frame + 2 spare for concurrent /state and /config slots.
The JPEG decoder mutex still serialises decode (only one /frame
can be decoding at a time); this change only unblocks the network
half of the pipeline.
|
|
wall-clock
|
|
Per-frame paint cost drops from ~24 ms to ~45 µs (≈500× faster) by
enabling num_fbs=2 on the DPI panel and decoding straight into the
back framebuffer. Measured on the bench:
before: lock=7us ttfb=370us body=40ms dec=6ms paint=24ms post=35us = ~70ms / ~14fps
after : lock=7us ttfb=330us body=38ms dec=6ms paint=42us post=25us = ~45ms / ~22fps
How the win actually lands:
- num_fbs=2 in the esp_lcd_dpi_panel_config_t makes the IDF driver
allocate two framebuffers and stream from one while we fill the
other.
- display_back_buffer() returns the inactive fb pointer + its size.
- jpeg_decoder_decode() now accepts a caller-provided destination
buffer instead of owning its own scratch. http_api passes the panel
back-fb so the hardware JPEG decoder writes BGR888 pixels straight
into where the DSI will eventually scan from. Zero memcpy in the
hot path.
- display_flip_back_buffer() calls esp_lcd_panel_draw_bitmap with the
fb pointer. Because the buffer is inside the panel's own fb range,
the IDF driver skips its memcpy and just does a cache writeback +
swaps cur_fb_index. The actual flip happens on the next vsync,
asynchronously — the call returns in microseconds.
The remaining ceiling is network body time (~38 ms for ~210 KB JPEGs)
and the hardware decoder (~6 ms). Per-viewport JPEG quality (smaller
files = shorter body) is the next lever; everything firmware-side is
already at or near floor.
Also drop the old static jpeg output scratch + JPEG_DECODER_MAX_OUTPUT_BYTES
constant — nothing references them anymore.
|
|
Two fixes plus a README refresh:
1. scrypted: pushStreamFrame previously reset the per-stream idle timer
on every successful /frame response. That made the timer anchored to
"frames are flowing" rather than to "the camera event that triggered
the stream", so a continuously-streaming source would never let the
stream time out. Removed the reset. The startStream → stopStream(false)
cancel-and-replace path on repeated events still keeps the stream
alive while the event keeps firing; idle (no new events) now actually
ends the stream at idle_timeout_ms.
2. firmware: break the previous coarse recv/dec/paint timing into
lock : try_lock returned
ttfb : first httpd_req_recv chunk landed
body : remaining bytes received
dec : hardware JPEG decode
paint : esp_lcd_panel_draw_bitmap returned
post : state-counter bookkeeping + unlock
Logged every 10 frames at INFO. Splits the previously-fat recv bucket
into TCP/HTTP handshake overhead (ttfb) vs wire-time (body), and
surfaces any tail bookkeeping cost.
3. README: replace the stale "5 fps ceiling caused by CPU RGB conversion"
guess with the actual measured per-phase budget and re-rank the
backlog accordingly. Double-buffering the panel (paint 24 ms → ~2 ms)
is now the highest-value next move; the previously-listed DMA-2D
rewrite is moot because the CPU loop is already gone.
|
|
|
|
Architectural rework of the /frame hot path. Scrypted now ships every
JPEG already scaled + rotated to the panel's native dimensions (read
from the firmware's /state response so nothing is hardcoded on the
Scrypted side); the firmware decodes the JPEG straight into a BGR888
buffer that's directly draw_bitmap'able by the DSI driver, with zero
CPU pixel work and zero rotation work in between.
Firmware
- jpeg_decoder now uses JPEG_DECODE_OUT_FORMAT_RGB888 +
JPEG_DEC_RGB_ELEMENT_ORDER_BGR. Output buffer sized for 800*480*3.
- New display_present_bgr888() is a one-liner that hands the decoder's
output straight to esp_lcd_panel_draw_bitmap.
- /frame handler validates dimensions against the panel-native
VIEWPORT_PANEL_WIDTH x VIEWPORT_PANEL_HEIGHT (was effective_dims
branching on orientation). Returns 400 if it's anything else.
- /state JSON adds panel_width + panel_height so Scrypted can read them
without hardcoding board-specific knowledge.
- display_present_rgb565 + s_rot_buf stay for the local-screens cold
path (info screen, loading) which still does its own CPU conversion +
rotation — infrequent enough that it's not worth the rewrite.
Scrypted
- startStream() GETs /state at stream-start time, caches panel_width
and panel_height in viewport storage, and uses them as the ffmpeg
scale target. Falls back to cached or 800x480 if /state is mid-reboot.
- For portrait viewports the ffmpeg pipeline now does
scale=H:W:flags=lanczos,transpose=1 so the JPEG arrives pre-rotated
90° CW into panel-native dimensions. Landscape is just scale=W:H.
- No more in-firmware rotation; Scrypted is the single source of truth
for "how do I get this camera frame into a panel-shaped JPEG".
Expected ceiling lift: ~5 fps → ~10 fps, gated by the JPEG decoder
hardware throughput instead of the CPU rgb565→bgr888 + rotation loop.
|
|
(33 ms)
|
|
On wake, display_wake() turned the backlight on first and then the
local_screens_show_loading / show_info call repainted the framebuffer.
For a few ms the user saw the previous /frame's contents (the last
camera snapshot from the prior wake cycle) before the new screen
landed. Swap the order: paint the placeholder first, then turn the
backlight on.
|
|
- Seed viewport_name with the full base MAC, colons stripped
(e8:f6:0a:e0:90:94 → "e8f60ae09094"). Stable across reboots, globally
unique, 12 alphanumeric chars — well inside the mDNS hostname limit
even with the "viewport-" prefix.
- Add a mac_str field to viewport_state and expose it as the new "mac"
key in GET /state and as a "mac" line on the info screen.
- "Configured" no longer requires a viewport_name (it always has the
MAC default); the scrypted URL alone gates outbound POSTs and the
loading-vs-info screen choice.
- Strip viewport_name[0] fallbacks in http_api / mdns / local_screens
now that the field is never empty.
- Replace the wake-on-boot-stays-on behaviour with a ~600 ms flash
followed by sleep — long enough for the FT5426 touch IC to come out
of its initial unresponsive state, short enough that an idle device
doesn't burn the backlight.
- Drop the touch defensive cruft added when we were chasing a PoE-power
theory: the DEV_MODE=0x00 write at init, the touch_reset_pulse() call
at init, and the runtime wedge-self-heal in the polling loop didn't
actually fix anything — the wake-on-boot flash did. Also delete the
display_touch_reset_pulse() helper since it now has zero callers.
|
|
Symptom: after a few cleanup cycles + power cycles, fresh boots left
touch dead — the FT5426 ack'd, polling read TD_STATUS=0x00 cleanly,
but taps never registered. Curl-waking the screen made touch work
again instantly; tapping the dark backlight-off panel did nothing.
Root cause: on this panel the FT5426 firmware only reports contact
points while the DSI link is actively streaming with the backlight on.
Booting straight into ASLEEP per the spec leaves taps silently unread,
and there's no built-in way to wake without touch (no hardware button
on this board — see prior commit). The fix is to wake on boot; the
idle timer cuts back to sleep after idle_timeout_ms if nothing happens.
Also defensively write DEV_MODE=0x00 (Working Mode) at touch_init,
and add a runtime self-heal: if the polling loop sees stuck TD_STATUS
above the 5-touch ceiling for ~3 s, pulse PC_RST_TP_N via the panel
MCU and retry. Combined with display_touch_reset_pulse() the chip
recovers from cold-boot wedges without manual intervention.
|
|
After a long bench session of taps + power cycles, the FT5426 on this
unit started returning dev_mode=0xff on every read. The polling loop
interpreted that as 15 simultaneous touches forever (td_status bits
0..3 all set), wedged `was_down=true` permanently, and no tap or
long-press ever fired — the screen looked dead.
Two layers of defence in touch_init:
1. retry the dev_mode read up to 10 × 30 ms in case the chip is just
slow to come up after PC_RST_TP_N is released.
2. if still stuck, pulse PC_RST_TP_N via a new
display_touch_reset_pulse() helper (drop reset for 20 ms then
release + 50 ms settle), then retry the read another 10 × 30 ms.
Plus a polling-loop sanity check: an FT5x06 supports max 5 simultaneous
points, so a TD_STATUS lower-nibble value > 5 is bogus and the poll
cycle is skipped. That keeps a wedged chip from producing phantom
"15 touches" reports if it slips into 0xff during runtime.
If both attempts still report 0xff, we log loudly and let the polling
task run — it harmlessly drops cycles via the >5 sanity check. Wake
still works via POST /state from Scrypted; this only disables touch
gestures on a hardware-stuck boot.
|
|
Code-review pass after the M5 colour fix. -126 net lines (-215 / +89).
Dead code removed:
- display_fill, display_test_pattern — M3 bring-up self-tests, no callers
- net_eth_is_up — no callers
- nvs_config_reset — no callers (factory-reset gesture was removed earlier;
NVS wipe now goes through `idf.py erase-flash`)
Helpers inlined or collapsed (each had one call site):
- state_name / orientation_name — ternaries at the cJSON site
- fmt_bytes / fmt_uptime — inline scope in local_screens_show_info
- expected_dims (http_api) + effective_dims (local_screens) — merged into
viewport_state_effective_dims() in viewport_state, one canonical source
state_machine cleanup:
- arm_idle_timer_unlocked / disarm_idle_timer collapsed to one arm_idle_timer(ms)
that takes the snapshotted timeout (ms==0 disables). Removes the misleading
name and the second viewport_state lock acquisition per painted frame.
- state_machine_set + state_machine_frame_painted now snapshot state + idle_ms
under one lock.
mdns_service cleanup:
- snapshot_state / apply_hostname / apply_txt collapsed into a single
apply_state(include_hostname) helper; mdns_service_start grabs hostname
once for hostname_set + log instead of going under the lock three times.
http_api cleanup:
- POST /config brightness-changed path uses the local `bright` it already
validated instead of re-locking + re-reading st->brightness.
- Trimmed verbose bring-up comments (stack-size justification, M5 saga,
DSI shadow struct) to one line each — kept the load-bearing facts.
Smoke-tested on hardware: boot clean, display + JPEG + touch all up
([DJT]), info screen renders, no crashes.
|
|
\`state\` now reports only the screen's runtime state (awake or asleep).
Whether a viewport is set up to talk to Scrypted is a separate
\`configured\` flag, derived from \`viewport_name && scrypted_url\`.
There's no third state.
Behaviour changes:
- POST /state always succeeds; the previous 409 "device unconfigured"
path is gone. The screen toggles regardless of /config status.
- POST /config now sets \`configured\` directly from the derived
predicate instead of mutating the state enum.
- Outbound state-client POST to Scrypted is still gated on a scrypted
URL being present — that's the only thing the configured flag now
actually controls in the runtime path.
GET /state JSON unchanged in shape, but \`state\` is now never
"unconfigured" — that's reported through the existing \`configured\`
boolean instead.
|
|
Two channel-swap bugs were stacked, hiding each other on symmetric
pixels (white text on black background) and showing up only on
saturated colour JPEGs:
1. jpeg_decoder: rgb_order was JPEG_DEC_RGB_ELEMENT_ORDER_RGB which
empirically emits the RGB565 word in big-endian byte order (the
IDF header comment "color component in big endian" is correct;
the enum's "RGB" name is misleading). Our painter reads native
LE uint16_t, so the bytes were always swapped. Flipped to
_BGR which emits LE — verified by dumping decoded bytes for
solid red/green/blue JPEGs.
2. display: rgb565_to_rgb888 wrote bytes [R, G, B], but the
ESP32-P4 DSI engine + TC358762 + Pi panel expect [B, G, R]
in memory despite the "RGB888" label. (MIPI DSI defines bit
order, not byte position in memory.) Verified with an
embedded solid-blue test JPEG: with [R, G, B] the panel
showed solid red; with [B, G, R] it shows solid blue.
Diagnosed by embedding a 480x800 blue test JPEG in firmware, decoding
+ dumping the buffer at boot, and reading the actual bytes over USB
serial (ethernet was disconnected mid-investigation). The temporary
self-test + debug dump are removed; the fixes themselves are tiny.
|
|
Previously the 4-line "identity" screen showed only viewport name, mDNS
host, IP, and state. Expanded to 15 lines covering the full GET /config
+ GET /state output (name, host, ip, state, configured, scrypted,
orientation, brightness, idle, fw, uptime, frames, errs, free heap,
free PSRAM), label/value pairs left-aligned with auto-scaled font.
Renames "identity" → "info" throughout — symbol, log messages,
README/TESTING references.
Also:
- Move the info-screen render's big locals (~1.6 KiB: lines[16][80] +
scrypt[256] + vp_name[64]) to BSS. The touch task's 3 KiB stack
was overflowing on every long-press, leaving whatever frame was
previously on the panel — gave the appearance of a "blue screen"
after the M5 test pattern.
- Drop the ≥5 s touch factory-reset gesture and remove all stale
references to it in docs and comments. NVS wipe is now a USB-side
`idf.py erase-flash` operation only.
- M5 follow-up TODO in jpeg_decoder.c: solid green renders ~black and
solid blue renders green; not a byte-order issue (BGR setting turns
red into blue). Tracked as a known M5 gap.
|
|
Holding the touch panel for 5 s to wipe NVS is dangerous (easy to
trigger by accident — phone resting on the panel, kids, etc.) and
useless as a recovery path (factory reset alone doesn't undo a bricked
config; a USB reflash does). Factory reset should be an explicit API
or USB-side action.
Touch gestures now: tap = toggle wake/sleep, ≥1.5 s hold = identity
overlay.
|
|
POST /config has ~2.4 KiB of stack locals (2 KiB body buffer + the
scrypted URL + viewport name + cJSON parser frames) which overran the
default 4 KiB httpd task stack and tripped the stack-protect canary
mid-handler. The request body was applied to RAM + NVS, but the
handler crashed before sending the response, so curl saw a connection
reset and the device rebooted into "Stack protection fault".
M4 + M6 ✅ verified 2026-06-14 after the bump:
- POST /config full / partial → 204; survives reboot via NVS
- 5 validation failure modes → 400
- POST /state wake/sleep → 204; idempotent repeats → 204
- POST /frame while asleep → 409 with expected body
- idle timer fires after idle_timeout_ms; 0 correctly disables
|
|
The Waveshare ESP32-P4-ETH board exposes BOOT on GPIO 35, but at runtime
that pin is owned by the EMAC TXD1 signal. No usable GPIO is wired to a
separate user button, so the BOOT-button task could never fire.
Move both BOOT-button behaviours onto the touch panel:
- ≥1.5s hold → 15s identity overlay (was BOOT short-press)
- ≥5s hold → factory reset (was BOOT long-hold)
Short tap (<500ms) still toggles wake/sleep. Long-press fires while the
finger is still down so the user gets immediate feedback at each
threshold.
Also strip the R/G/B 6 s boot test sequence — the panel now renders
correctly, so it's no longer useful diagnostically.
Boot subsystem flags drop the trailing B column: [EMHDJT].
|
|
The Hosyond 5" panel uses Pi 7" v1.1 architecture (ATTINY ID 0xC3 +
TC358762 DSI-to-DPI bridge). Getting it to paint needed three things
that ESP-IDF doesn't expose:
1. TC358762 bridge configuration — 16 register writes via DSI Generic
Long Write (DT=0x29) packets, transcribed from Linux's tc358762.c.
2. Non-burst video mode — IDF hardcodes BURST_WITH_SYNC_PULSES, but
TC358762 requires NON_BURST_WITH_SYNC_PULSES. Overridden via the
mipi_dsi_host_ll_* API after esp_lcd_new_panel_dpi() and before
panel_init().
3. ATTINY v1.1 power-on sequence + SPI-proxy bridge wake. The legacy
REG_POWERON(0x85) path doesn't apply to 0xC3 firmware; instead
write PORTC/PORTA/PORTB/PORTC in order, then later release bridge
reset and proxy-write TC358762 SYSPMCTRL=0 through the ATTINY's
ADDR_H/L + WR_DATA_H/L registers.
Reaching the LL/HAL APIs requires shadowing esp_lcd_dsi_bus_t so we can
pick the mipi_dsi_hal_context_t out of the private struct. Documented
the layout dependency at the shadow definition.
Tuned config (observed stable on ESP32-P4 per embenix's reference):
- 1 data lane @ 600 Mbps
- DPI 26 MHz (Linux modeline is 25.98)
- timings HSW=2 HBP=46 HFP=210 / VSW=20 VBP=4 VFP=22
- RGB888 end-to-end, R,G,B byte order (BGR was wrong)
- disable_lp=0 so LP windows are available for the bridge writes
|
|
DSI / panel:
- LDO_VO3 acquired at 2500 mV (VDD_MIPI_DPHY) — without this the PHY
PLL busy-waits forever inside esp_lcd_new_dsi_bus.
- PSRAM bumped to 200 MHz (CONFIG_IDF_EXPERIMENTAL_FEATURES) to keep
the DPI fed without underruns.
- Pi-7"-style 800x480 panel init (REG_POWERON, PORTA/PORTB/PWM/PORTC)
+ 16-bit RGB565 framebuffer with portrait-mode rotation buffer in
PSRAM. Currently shows garbled output — known issue, next commit
switches to the TC358762-bridge-aware init sequence.
State + UX:
- viewport_state simplified to AWAKE/ASLEEP only; "unconfigured" is
a flag, not a state. Tap always toggles; content choice (identity
vs frame) is driven by the configured flag.
- BOOT button arms a 15 s identity overlay via local_screens_overlay;
expired callback returns to prior state.
- Boot-done indicator: two backlight flashes (no usable LED GPIO on
the Waveshare board — GPIO 35 BOOT is shared with EMAC TXD1).
- Best-effort subsystem init in app_main; display init deferred to
its own task so a panel hang can't block networking.
Display test pattern: solid R/G/B for 2 s each on boot to characterize
the garble independent of text rendering.
|
|
First flash to real hardware (Waveshare ESP32-P4-ETH on USB power,
no LAN cable, no panel attached) exposed three bugs and the need to
degrade gracefully when peripherals are missing.
1. net_eth.c: RXD0 / RXD1 GPIO swap. ESP32-P4's EMAC iomux table
(components/soc/esp32p4/emac_periph.c) fixes RXD0 to GPIO 29 and
RXD1 to GPIO 30. The Waveshare-wiki/ESPHome research had them
transposed. Symptom was:
E (esp.emac.gpio): invalid RXD0 GPIO number
E (esp.emac): esp_eth_mac_new_esp32 failed
-> ESP_ERROR_CHECK abort
Fix: swap the two #defines. CRS_DV / TXD0 / TXD1 / TX_EN / REF_CLK
pinout was already correct.
2. mdns_service.c: mdns_service_add() rejects with INVALID_ARG when
the hostname isn't set yet (see mdns_responder.c:771 — first guard
in mdns_service_add_for_host). We were setting hostname AFTER
service_add inside apply_records(). Restructure mdns_service_start
to: init -> hostname -> service_add -> txt. apply_hostname() and
apply_txt() helpers reuse the same snapshot under viewport_state's
mutex; mdns_service_refresh() re-applies both.
3. app_main.c: every subsystem now best-effort instead of
ESP_ERROR_CHECK abort. A single missing peripheral can't take down
the rest of the firmware. End-of-boot summary line:
boot complete — subsystems [EMHdJ-B] ip=(no link)
Uppercase letter = up, lowercase = down. E=Ethernet M=mDNS H=HTTP
D=Display J=JPEG T=Touch B=BootButton. Touch shows '-' when display
didn't come up (it shares the panel I2C bus). On a stripped board
(just ESP32-P4-ETH on USB, no panel, no LAN) the line above prints
and the device serves /state, /config, and mDNS over the loopback;
plugging Ethernet in later picks up DHCP without a reboot.
Side-effects: the DHCP timeout in app_main shortened from 30s to 15s
so a no-cable boot finishes quickly. Reaching the "ready" state with
no link is fine — the driver keeps the link-up event handler armed and
gets the IP whenever a cable appears.
Verified on hardware (bare ESP32-P4-ETH, no LAN, no panel):
- ESP-IDF v5.4.1, esp32p4, 32 MB PSRAM detected, 16 MB flash config
(actual 32 MB silkscreen — keeping 16 MB until access-beyond-16MB
support lands in flash driver).
- Ethernet driver started — MAC e8:f6:0a:e0:90:94.
- mDNS up advertising viewport.local on _scrypted-viewport._tcp:80.
- HTTP up listening :80.
- JPEG decoder ready.
- Display reports "panel MCU @0x45 unreachable" — exactly correct
for no-panel state.
- Touch skipped as designed.
- BOOT button registered on GPIO 0 (still a guess; harmless if wrong).
This is enough to flip M1's "code" status to ✅ on hardware once
Ethernet is plugged in. M2 (mDNS browse + GET /state) ready to verify.
|
|
local_screens_show_ip() now renders four lines instead of two —
all the device's runtime identity at once:
line 1: viewport name ("mudroom" / "viewport" if unconfigured)
line 2: mDNS hostname ("viewport-mudroom.local" / "viewport.local")
line 3: IP ("192.168.1.42" / "no network")
line 4: state ("awake" / "asleep" / "unconfigured")
Same screen is shown on first boot, after factory reset, and as a
15s BOOT short-press overlay — operator always has the device's full
identity one button-press away (find it on the LAN, confirm name +
configuration without curling).
Font scale is auto-picked: the largest integer scale (1×–6×) where
the longest line fits within 90% of width AND all four lines plus
inter-line spacing fit within 90% of height. Works for both portrait
(480×800) and landscape (800×480) without separate code paths.
Font expanded to cover the new strings:
- All lowercase a–z (added b, f, h, j, k, m, q, s, u, x, y, z).
- Punctuation: dash, slash (digits, period, colon already covered).
Fallback policy unchanged: unsupported chars render as blank, which
keeps the table tight and the failure mode visible rather than
crashing.
README "Local rendering" + TESTING.md M8 updated to describe the
four-line identity layout and show both the unconfigured-boot and
configured-overlay example outputs.
Build verified clean against ESP-IDF 5.4 (binary ~873 KB, +1.6 KB
for the expanded font).
|
|
local_screens.{h,c}:
- Embedded 8x8 bitmap font sized to a 95-char table; only the glyphs
used by today's strings (viewport.local, the IPv4 string, Loading...)
are populated. Unsupported chars render blank.
- 768 KB PSRAM scratch FB at panel-native dims.
- local_screens_show_ip() — two centered lines:
"viewport.local"
<current IP>
Scale 3x portrait / 4x landscape; centered vertically.
- local_screens_show_loading() — centered "Loading..." at scale 4x/5x.
- local_screens_restore_for_state() — repaint after BOOT-overlay expiry:
UNCONFIGURED -> IP screen; AWAKE/ASLEEP -> black FB (next /frame or
sleep handles the rest).
- All paths go through display_present_rgb565() so orientation rotation
is automatic.
button.{h,c}:
- Polling task at 30ms, active-low with internal pull-up.
- Short press (<5s, released) → backlight on, IP overlay for 15s via
esp_timer one-shot, then restore.
- Hold ≥ 5s → nvs_config_reset() + esp_restart().
- PIN_BOOT_BUTTON = GPIO 0 is a TODO placeholder. ESP32-P4 strap pin
GPIO35 is owned by RMII TXD1 at runtime so Waveshare must expose a
separate user button on a free pin; confirm against the schematic.
Fail-soft if mis-wired: input reads stuck-high and the task never
fires.
state_machine on AWAKE transition now calls local_screens_show_loading()
right after display_wake() — so every wake (tap or POST /state) flashes
a clear "Loading..." until the next /frame paints over it.
app_main on boot:
- Calls local_screens_init() after display_init().
- UNCONFIGURED → paint IP screen (replaces the M3 test pattern).
- ASLEEP → backlight off (unchanged).
- After all subsystems: button_init().
Build clean against ESP-IDF 5.4 (binary ~870 KB; the font/glyph data is
under 800 bytes).
TESTING.md M8 documents the visual checks, the BOOT-pin placeholder
caveat, the AWAKE-overlay-overwrite behavior, the open follow-up about
post-overlay backlight when ASLEEP, and the font fallback policy.
|
|
state_client.{h,c}:
- Worker task drains a depth-1 queue (xQueueOverwrite gives the
replace-on-full semantics from the spec — in-flight POST is never
cancelled; the next queued entry is overwritten by newer state).
- esp_http_client POST to <scrypted>/state with Content-Type application/
json, User-Agent ScryptedViewport/<version>, Connection: close, 1s
timeout. Body: {"viewport":"<name>","state":"wake"|"sleep"}.
- Any non-2xx or transport error increments state_post_failures and is
otherwise ignored. Silently drops if no Scrypted URL is configured.
touch.{h,c}:
- FT5426 capacitive touch on the shared I2C bus at 0x38.
- 30ms polling task; tracks down/up transitions, detects taps as
down-then-up within 500ms with a 150ms debounce.
- On tap, toggles wake/sleep via state_machine_set_local(), which drives
the local transition AND fires state_client_post() at Scrypted.
state_machine adds state_machine_set_local(): runs the same transition
as state_machine_set() then enqueues an outbound POST. Idle-timer
expiry now uses this path so Scrypted sees idle-driven sleeps too
(the spec's "tighten the race with /frame 409" path stays in place as
the fallback).
display.h exposes display_i2c_bus(); touch.c uses it instead of
re-initializing the same I2C port.
app_main starts state_client right after the state machine and starts
touch after display init (touch is skipped if display isn't up since
they share the bus).
CMakeLists.txt: add esp_http_client to REQUIRES.
Build clean against ESP-IDF 5.4 (binary ~860 KB; jumped ~210 KB from
M6 because esp_http_client + cJSON path pulls in tcp_transport, mbedtls,
http_parser).
TESTING.md M7: flask-based test receiver, tap dispatch verification,
failure-path check (kill receiver, confirm counter increments),
queue-coalescing behavior, no-POST-on-Scrypted-initiated, no-POST-
when-unconfigured.
|
|
state_machine.{h,c} — central wake/sleep transitions:
- state_machine_init() creates the esp_timer one-shot for the idle timer.
- state_machine_set(target) is idempotent and atomic. On AWAKE: backlight
on, idle timer (re)armed. On ASLEEP: idle timer cancelled, backlight off.
Rejects with INVALID_STATE when the device is unconfigured.
- state_machine_frame_painted() restarts the idle timer if awake; called
by /frame after each successful paint.
- Idle-timer callback transitions to ASLEEP. TODO M7 hook: outbound POST
{viewport, state:sleep} to <scrypted>/state.
http_api.c:
- POST /state: parse {state}, accept "wake"/"sleep", reject others 400.
Unconfigured device → 409 "device unconfigured". Already-in-state → 204
(idempotent no-op). Successful transition → 204.
- POST /frame: 409 Conflict when state != AWAKE. After successful paint,
call state_machine_frame_painted() so the idle clock keeps resetting
while frames stream.
app_main:
- Initialize state_machine before http_api so the route handler can drive
it from request 0.
- After display_init(), reconcile the panel with the boot state:
UNCONFIGURED → test pattern (placeholder until M8 IP screen)
ASLEEP → display_sleep() so a configured device boots dark
AWAKE → leave on (not reached on fresh boot)
Disable path: idle_timeout_ms=0 in /config means the timer is never armed
and a wake state persists until /state {sleep} or a power cycle.
Build clean against ESP-IDF 5.4 (binary ~645 KB).
TESTING.md M6 expands with idempotency checks, 409-when-asleep, 409-when-
unconfigured, idle-timer firing within idle_timeout_ms+slack, /frame
restarting the idle timer, idle-timer disable via idle_timeout_ms=0, and
the M7 dependency note about the missing outbound sleep POST.
|
|
jpeg_decoder.{h,c} wraps esp_driver_jpeg (ESP32-P4 hardware decoder).
- One-time engine + DMA-aligned PSRAM scratch buffer setup (1 MB input,
768 KB output @ panel native 800x480 RGB565).
- try_lock + unlock so concurrent /frame POSTs get 503 instead of
queueing, per spec.
- jpeg_decoder_get_info() reports the dimensions; the http handler
validates them against the effective resolution before painting.
display.h adds display_present_rgb565(src, w, h):
- Landscape: src is 800x480, memcpy 1:1 into the panel framebuffer.
- Portrait: src is 480x800, software rotate 90° CW into the 800x480
panel framebuffer. (PPA / 2D-DMA hardware rotation is a later
optimization if portrait latency matters.)
http_api.c adds POST /frame:
- Content-Type must be image/jpeg → else 400.
- Empty body → 400. > 1 MB → 413. Display not initialized → 500.
- jpeg_decoder_try_lock(0) for concurrency: second post returns 503.
- Body streamed into the decoder's input buffer in chunks.
- Decode failure or dimension mismatch → 400 + decode_errors++.
- Paint failure → 500.
- Success → frames_received++, last_frame_us = esp_timer_get_time(),
204 No Content.
app_main initializes the JPEG decoder after display_init(). Both are
best-effort: failures log a warning and leave the rest of the firmware
running.
CMakeLists.txt: add esp_driver_jpeg to REQUIRES.
Known gap (M6 closes it): /frame currently paints regardless of
wake/sleep state. The 409-when-asleep rule lands with POST /state
in M6.
Build clean against ESP-IDF 5.4 (binary ~640 KB).
TESTING.md M5 expanded with portrait/landscape test commands,
ImageMagick test-image recipes, the full negative matrix (wrong
Content-Type, oversize, wrong dims, concurrent, garbage), and the
M6 dependency note.
|
|
nvs_config.{h,c} — persist the runtime config (viewport, scrypted,
idle_timeout_ms, orientation, brightness) under a single NVS namespace.
nvs_config_load() applies persisted values over the in-RAM defaults on
boot and flips state from UNCONFIGURED to ASLEEP once both name and
Scrypted URL are present. nvs_config_save() commits the whole record
atomically.
http_api.c — add GET /config and POST /config:
- GET serializes viewport_state to the spec's JSON shape, with null
for unset string fields and defaults filled in for the rest.
- POST is partial: each field is optional; only present fields are
validated and applied. Validation runs on a staged copy and errors
short-circuit with 400 + reason before any state mutation, so a
rejected request leaves the device untouched.
- Validation rules: viewport non-empty <64 chars; scrypted starts with
http:// and <256 chars; idle_timeout_ms 0 or >=5000; orientation in
{portrait,landscape}; brightness 0..100.
- Side-effects fire after the lock + save: brightness change pushes
PWM to the panel MCU; viewport/orientation change reapplies mDNS
hostname + TXT.
- 204 on success; 400 with a single-line reason on validation error.
app_main calls nvs_config_load() right after viewport_state_init(), so
mdns_service_start() and display_init() see the persisted hostname,
orientation, and brightness from the first packet/PWM.
Build clean against ESP-IDF 5.4 (binary ~620 KB).
TESTING.md M3 now documents the Hosyond jumper wiring (5V/GND/SDA=GPIO7/
SCL=GPIO8 from board to panel header; DSI FPC carries only the high-
speed lanes). M4 entry expands the validation matrix and side-effects
to verify.
|
|
display.{h,c} brings up the Hosyond 5\" 800x480 panel via the Raspberry
Pi 7\" touchscreen architecture:
- I2C init to the panel-side MCU at 0x45 (Pi 7\" register map ported
from drivers/gpu/drm/panel/panel-raspberrypi-touchscreen.c).
- POWERON write + 120ms settle.
- ESP32-P4 MIPI-DSI bring-up in DPI video mode: 2-lane, 480 Mbps,
800x480 @ 60Hz, RGB565, canonical Pi 7\" timings (HSW=18/HBP=20/HFP=62,
VSW=4/VBP=27/VFP=18, 30 MHz pixel clock).
- Gamma-corrected (^2.2) PWM brightness via REG_PWM (0x86).
- display_sleep / display_wake hooks for M6.
- display_fill() and display_test_pattern() (8 vertical color bars)
for M3 acceptance and reuse by M8 local screens.
app_main calls display_init() as best-effort — if the panel isn't
attached or wiring is wrong, the rest of the firmware (Ethernet, mDNS,
/state) keeps running. On success it paints the test pattern.
Pin assignments (PIN_I2C_SDA=7, PIN_I2C_SCL=8) match Waveshare's
bundled-panel BSP convention but are flagged TODO until the user's
specific FPC adapter wiring is confirmed.
main/CMakeLists.txt: add driver, esp_driver_i2c, esp_lcd to REQUIRES.
Build verified clean against ESP-IDF 5.4 for target esp32p4. M3
acceptance (color bars on screen) cannot be verified without a
flashed board + connected panel.
|
|
- viewport_state.{h,c}: shared state struct (config, run-state, counters,
orientation, timestamps) behind a FreeRTOS mutex. Modules update it
under viewport_state_lock(); GET /state serializes a snapshot.
- http_api.{h,c}: starts esp_http_server on :80, registers GET /state.
Returns the full JSON shape from the spec (name, version, configured,
state, uptime_ms, last_frame_ms_ago, frames_received, decode_errors,
state_post_failures, resolution, ip, free_heap, free_psram).
- mdns_service.{h,c}: mdns_init + advertise _scrypted-viewport._tcp.local
on :80 with version/resolution/orientation/name TXT records.
mdns_service_refresh() reapplies hostname + TXT after /config writes
(called from M4 onward).
- main/CMakeLists.txt: add esp_http_server, esp_timer, json, mdns to
REQUIRES. espressif/mdns added as managed component via
main/idf_component.yml.
app_main calls viewport_state_init early so any module can read defaults
before /config arrives.
Also adds TESTING.md tracking per-milestone verification status and an
integration test plan to run after M9 (races, failure modes, longevity,
power cycles, negative protocol, multi-viewport).
M1 promoted to "🟡 builds clean" in TESTING.md; combined HW verification
of M1+M2 will be one flash session. Build verified against ESP-IDF 5.4
for target esp32p4 (binary ~550 KB).
|
|
net_eth module brings up the Waveshare ESP32-P4-ETH-POE Ethernet
interface (internal EMAC + IP101GRI PHY) and waits for a DHCP lease.
Pin map confirmed via Waveshare wiki + ESPHome's working config:
MDC=31, MDIO=52, REF_CLK=50 (CLK_EXT_IN from PHY)
TX_EN=49, TXD0=34, TXD1=35
CRS_DV=28, RXD0=30, RXD1=29
PHY reset/enable=51, PHY addr=1
app_main initializes NVS, netif, the default event loop, then starts
the Ethernet driver and waits up to 30s for an IP. On success it logs
the IP; on timeout it logs a warning and proceeds (the driver keeps
retrying in the background).
sdkconfig: declare 16 MB flash so partitions.csv (6.1 MB) fits.
main/CMakeLists.txt: explicit REQUIRES esp_eth esp_event esp_netif
nvs_flash.
Acceptance (per M1 in the impl guide): device gets a DHCP lease over
Ethernet and prints its IP. Build is clean against ESP-IDF 5.4 for
target esp32p4.
|
|
|
|
Plain ESP-IDF project targeting Waveshare ESP32-P4-ETH-POE with a 5" 800x480
MIPI-DSI capacitive touch panel. Stub app_main with TODOs for Ethernet, mDNS,
HTTP API (/health, /config, /frame, /sleep, /brightness), JPEG decode, and
touch callback delivery per the v1 spec in README.md.
|
