| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
WND=23040 alone regressed: sender fps 20 -> 14, drop-oldest 3x,
recv bimodal (min 20ms but 200-450ms stalls; firmware wire_min
6.2Mbps / wire_max 75Mbps, recv_max 253ms). The stalls are RTO
recovery: the default EMAC RX pool (20 x 512B = 10KB) is smaller
than the 23KB the window now invites in flight, so a full-window
burst overruns the RX descriptors, the burst tail is dropped with
no dup-ACKs behind it, and the sender waits out a ~200ms min-RTO.
1600B buffers fit one MSS frame per buffer (1 descriptor/packet
instead of 3); 24 of them = 38.4KB >= window + slack. ~44KB more
internal RAM (491KB free).
|
|
Instrumented baseline at WND=5760 (kitchen, ~202KB frames, 20fps
sender) confirmed the receive window as the dominant throttle:
- wire 53.0/58.8 Mbps avg/max = the WND/RTT ceiling, vs ~94 Mbps
line rate on the 10/100 PHY
- recv_chunk_max pinned at exactly 5760 (drains window-quantized)
- recv_calls avg 60/frame; queued_at_body always 0
- sender backpressured 44-61% of frames, drop-oldest 17-54/window
- meanwhile pend_age avg 58us: decode idles, receive starves it
Expected: recv_avg ~30ms -> ~20ms, wire toward line rate, sender
bp%/drop-oldest down, g2g 73ms -> ~60ms. Tripwires (revert if hit):
pend_age or recv_dropped_oldest growing window-over-window, g2g
regressing. RECVMBOX scales with the window: 23040 = 16 segments
in flight, a 6-deep mbox would silently become the new cap.
|
|
regressed g2g
Step-2 experiment from the iterative tuning plan: bumped
CONFIG_LWIP_TCP_WND_DEFAULT 5760 → 65535 (and matching SND_BUF +
RECVMBOX 6 → 16) under the hypothesis that the 4×MSS window
throttled receive throughput.
Instrumentation showed the window WAS the per-call throttle
(recv_chunk_max was exactly 5760 before, grew to 33-60KB after).
But end-to-end got dramatically worse:
- g2g exploded from ~100ms steady to 17 SECONDS growing unbounded
(1964 → 6010 → 10632 → 14725 → 17889ms over consecutive windows).
- painted dropped from ~24fps to 15-20fps.
- recv_avg rose from 32ms steady to 50-90ms with frequent
230-500ms outliers (the retransmit / RTO signature).
- chunk_min fell to 61 bytes, chunk_avg dropped from 3415 to ~2200
— fragmentation as lwIP RX struggled with the bigger bursts.
Root cause: the single recv→decode→paint task serializes; the
kernel buffer fills against ANY window during the ~6ms
decode+paint, but with 65535 bytes available the sender pumps
~45 segments before stopping vs ~4 before. That overruns the
lwIP RX path (RECVMBOX=16 mailboxes, default pbuf pool) → drops
→ retransmits → multi-frame stalls. And Scrypted has no way to
skip-oldest on the kernel queue, so frames just pile up on the
wire / in the ESP32 kernel buffer → g2g grows linearly with time.
Lesson: window size isn't the right knob until the receiver can
drain at line rate independent of decode+paint. Next iteration:
split recv into its own FreeRTOS task with a 1-deep latest-frame
slot to the decode/paint task — only then revisit window.
|
|
Streams the raw .bin to the inactive ota_0/ota_1 slot via esp_ota_*, flips otadata, replies 200, reboots after 500 ms. Single-shot guarded by atomic_flag (409 on concurrent). CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE armed: new images boot pending-verify and ota_arm_healthy_timer marks them valid after 30 s of healthy uptime; otherwise the bootloader reverts on next reset. /state gains ota_state.
|
|
This reverts commit 6e36cbc14a06b8c0e15bfe78afb8adb43f8909ba.
|
|
Hypothesis confirmed by the symptoms: with WND_DEFAULT=65535,
SND_BUF=65535, RECVMBOX=16, the firmware boots fine and the stream
server logs "listening on tcp/:81", but Scrypted's first SYN never
produces a "client connected from ..." entry. The script-side
socket stays stuck in pending-connect — neither a connect nor an
error event fires — and every ffmpeg-emitted frame gets dropped.
Root cause is almost certainly lwIP's PBUF / MEMP pools not being
scaled to back the 64KB windows on multiple sockets (httpd has 4,
stream has 1, that's 5 × 64KB = 320KB of buffer commitment from a
heap that's also feeding PSRAM-backed framebuffers, JPEG scratch,
and Ethernet DMA). lwIP doesn't surface the allocation failure as
an accept error — it just refuses to complete the handshake.
Backing off the changes for now. The streaming path already hits
~14-20 fps at our measured 5.3 MB/s ingest ceiling under the lwIP
defaults — that's plenty above the 12-fps goal. Revisit window
tuning later by also bumping CONFIG_LWIP_PBUF_POOL_SIZE and
MEMP_NUM_TCP_PCB so the buffer commitment is actually allocatable.
Keeping the FIONREAD skip in stream_server (it's orthogonal to lwIP
buffer sizes and only helps).
|
|
backpressure-blind + lwIP TCP window bump
Three changes that work together to make the stream "always paint
what's freshest, never sit on a stale frame":
#1 — Firmware FIONREAD skip in stream_server
Right after the body of frame N comes off the wire (and before we
unlock the decoder + spend ~6ms on decode + paint), check the
kernel receive buffer with ioctl(FIONREAD). If at least one more
header (8 bytes) is queued, frame N is no longer the freshest
possible — skip its decode + paint and loop back to read frame
N+1. The TCP recv cost is unavoidable (bytes still have to cross
the wire) but the decoder + paint cost is saved on every superseded
frame. Glass-to-glass latency on the latest frame drops by however
many frames had backed up.
#2 — Scrypted: keep writing past kernel-buffer backpressure
Previously: when sock.write() returned false we dropped the next
ffmpeg frame at source. New: we keep writing through. Node buffers
internally; under our load (~16 MB/s ffmpeg → ~5-7 MB/s firmware)
the buffer rarely exceeds a frame or two. With the firmware now
silently skipping decode on backed-up frames (#1), excess frames
get shed for free on the device side. Scrypted's job is just to
hand the firmware the freshest bytes as fast as possible.
socketBackpressured is still tracked for the diagnostic log.
#3 — lwIP TCP window bump (revisiting earlier regression)
The previous attempt at LWIP_TCP_WND_DEFAULT=32k regressed under
HTTP because every /frame opened a fresh socket and we paid the
slow-start cost repeatedly. The streaming pivot eliminated that:
the socket is long-lived, slow-start runs exactly once, then we
ride the full window for the rest of the session. Bumping to
65535 (max for stock lwIP), SND_BUF to match, RECVMBOX to 16, SACK
on. Expected: recv throughput ceiling moves up from ~5.3 MB/s,
which directly raises the fps ceiling (recv is currently 37ms of
the 43ms per-frame total).
|
|
Bumping WND/SND_BUF to 32 KiB + RECVMBOX to 16 + SACK on regressed
ttfb from ~350µs to ~1.2ms across every frame and introduced periodic
250+ms body stalls (TCP retransmit on the larger window). Mean body
unchanged. Reason: Scrypted's per-frame fetch() opens a fresh TCP
socket, so the larger receive window just slows slow-start on every
new connection rather than helping.
Reverted to lwIP defaults. The real fix is HTTP keep-alive on the
Scrypted side; will revisit window tuning once persistent connections
are in place. README backlog updated with what we learned.
|
|
DSI / panel:
- LDO_VO3 acquired at 2500 mV (VDD_MIPI_DPHY) — without this the PHY
PLL busy-waits forever inside esp_lcd_new_dsi_bus.
- PSRAM bumped to 200 MHz (CONFIG_IDF_EXPERIMENTAL_FEATURES) to keep
the DPI fed without underruns.
- Pi-7"-style 800x480 panel init (REG_POWERON, PORTA/PORTB/PWM/PORTC)
+ 16-bit RGB565 framebuffer with portrait-mode rotation buffer in
PSRAM. Currently shows garbled output — known issue, next commit
switches to the TC358762-bridge-aware init sequence.
State + UX:
- viewport_state simplified to AWAKE/ASLEEP only; "unconfigured" is
a flag, not a state. Tap always toggles; content choice (identity
vs frame) is driven by the configured flag.
- BOOT button arms a 15 s identity overlay via local_screens_overlay;
expired callback returns to prior state.
- Boot-done indicator: two backlight flashes (no usable LED GPIO on
the Waveshare board — GPIO 35 BOOT is shared with EMAC TXD1).
- Best-effort subsystem init in app_main; display init deferred to
its own task so a panel hang can't block networking.
Display test pattern: solid R/G/B for 2 s each on boot to characterize
the garble independent of text rendering.
|
|
net_eth module brings up the Waveshare ESP32-P4-ETH-POE Ethernet
interface (internal EMAC + IP101GRI PHY) and waits for a DHCP lease.
Pin map confirmed via Waveshare wiki + ESPHome's working config:
MDC=31, MDIO=52, REF_CLK=50 (CLK_EXT_IN from PHY)
TX_EN=49, TXD0=34, TXD1=35
CRS_DV=28, RXD0=30, RXD1=29
PHY reset/enable=51, PHY addr=1
app_main initializes NVS, netif, the default event loop, then starts
the Ethernet driver and waits up to 30s for an IP. On success it logs
the IP; on timeout it logs a warning and proceeds (the driver keeps
retrying in the background).
sdkconfig: declare 16 MB flash so partitions.csv (6.1 MB) fits.
main/CMakeLists.txt: explicit REQUIRES esp_eth esp_event esp_netif
nvs_flash.
Acceptance (per M1 in the impl guide): device gets a DHCP lease over
Ethernet and prints its IP. Build is clean against ESP-IDF 5.4 for
target esp32p4.
|
|
Plain ESP-IDF project targeting Waveshare ESP32-P4-ETH-POE with a 5" 800x480
MIPI-DSI capacitive touch panel. Stub app_main with TODOs for Ethernet, mDNS,
HTTP API (/health, /config, /frame, /sleep, /brightness), JPEG decode, and
touch callback delivery per the v1 spec in README.md.
|
