src.nth.io/

summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorLuke Hoersten <[email protected]>2026-06-15 07:52:34 -0500
committerLuke Hoersten <[email protected]>2026-06-15 07:52:34 -0500
commitd949b19a6b89b9e74e8706bafa1d1396ac8f17f8 (patch)
tree48108f2f50e7cb6f3b95b452b78f419e964eda70
parent3f53ced44ea112a39eec70216c8c72fa6867726e (diff)
README: refresh per-frame budget with double-buffered measurements; network body is now the only big lever
-rw-r--r--README.md45
1 files changed, 23 insertions, 22 deletions
diff --git a/README.md b/README.md
index 41cb4e1..64e0f6f 100644
--- a/README.md
+++ b/README.md
@@ -490,35 +490,36 @@ Every endpoint is idempotent; every failure leaves the device in a sane state.
## What's next
-M1 – M8 are all ✅ on hardware (see [`TESTING.md`](TESTING.md) for verification details). End-to-end Scrypted streaming via ffmpeg + the zero-copy `JPEG → BGR888 → DSI` hot path is live and sustains **~6.5 fps painted at the 100 ms target tick**, gated by per-frame work below.
+M1 – M8 are all ✅ on hardware (see [`TESTING.md`](TESTING.md) for verification details). End-to-end Scrypted streaming via ffmpeg + the zero-copy `JPEG → BGR888 → DSI` hot path sustains **~22 fps at full quality** with the double-buffered panel.
-### Measured per-frame budget (target ≥ 22 fps requires shrinking these)
+### Measured per-frame budget
-The firmware logs a `frame N: lock=… ttfb=… body=… dec=… paint=… post=… total=…ms` line every 10 frames. Steady-state on the bench (Waveshare ESP32-P4-ETH + Hosyond 5" panel, ~210 KB JPEGs at `-q:v 2`):
+The firmware logs a `frame N: lock=… ttfb=… body=… dec=… paint=… post=… total=…ms` line every 10 frames. Steady-state on the bench (Waveshare ESP32-P4-ETH + Hosyond 5" panel, ~215 KB JPEGs at `-q:v 2`):
-| Phase | Time | What it is |
-|---|---|---|
-| `lock` | ~tens of µs | `jpeg_decoder_try_lock` (mutex acquire) |
-| `ttfb` | ~1–2 ms | Time-to-first-byte after lock — TCP/HTTP handshake overhead |
-| `body` | ~38 ms | Wire time for the rest of the JPEG body (~210 KB at ~42 Mbit/s effective) |
-| `dec` | ~6 ms | Hardware JPEG decode → BGR888 (way faster than the textbook 50–80 ms guess) |
-| `paint` | ~24 ms | `esp_lcd_panel_draw_bitmap` + ~1.5 DSI refresh cycles at 60 Hz |
-| `post` | < 1 ms | State counter bookkeeping + unlock |
-| **total** | **~70 ms** | **Theoretical ceiling ~14 fps if Scrypted ticks faster than the 100 ms default** |
+| Phase | Time | Share | What it is |
+|---|---|---|---|
+| `lock` | 6–10 µs | < 0.1% | `jpeg_decoder_try_lock` mutex acquire |
+| `ttfb` | 300–650 µs | < 1% | First byte landing after lock — TCP handshake noise |
+| `body` | 37–44 ms | **~85%** | Wire time for the JPEG body (~215 KB at ~45 Mbit/s effective on 100 Mbit Ethernet) |
+| `dec` | ~6 ms | ~13% | Hardware JPEG decode → BGR888 (way faster than the textbook 50–80 ms guess) |
+| `paint` | 35–55 µs | < 0.1% | `esp_lcd_panel_draw_bitmap` — cache writeback + index swap thanks to `num_fbs = 2` and zero-copy decode into the back fb |
+| `post` | 20–30 µs | < 0.1% | State counter bookkeeping + unlock |
+| **total** | **43–49 ms** | | **Ceiling ~22 fps** |
-So the firmware ceiling at full quality is ~14 fps, but Scrypted's `frame_interval_ms = 100` only fires 10 ticks/sec, leaving ~30 ms idle between frames → effective ~6.5 fps. Lower the per-viewport interval to ~75 ms to chase the ceiling.
+Network body is now ~85% of every frame; everything else is at or near hardware floor.
### Current backlog, in rough priority order
-1. **Double-buffer the panel (`num_fbs = 2`)** — *high impact*. Today `paint` is 24 ms because the synchronous `esp_lcd_panel_draw_bitmap` waits for a full DSI refresh cycle. With a second framebuffer the DSI engine streams from FB-A while the next frame fills FB-B; paint becomes a pointer swap (~1–2 ms). Total drops to ~46 ms, ceiling jumps to ~22 fps at full quality. Watch out for the TC358762 bridge's `non-burst with sync pulses` constraint — `flags.use_dma2d` and `num_fbs` interact non-obviously with the bridge wake sequence, so this needs careful re-bring-up.
-2. **HTTP persistent connection** — *medium impact, low effort*. Would shave the ~1–2 ms `ttfb` per frame by keeping the TCP connection open across frames (currently Scrypted's `fetch` opens fresh each POST). Real win is consistency, not absolute speed; useful once #1 lands.
-3. **OTA firmware updates** — *low effort, gateway to fleet ops*. Partition table already has `ota_0` / `ota_1`. Add `esp_https_ota` (or a one-shot `POST /firmware` using `esp_ota_*`) so reflashing doesn't require USB. Critical once you have more than one viewport in production.
-4. **Task watchdog + crash counters** — *low effort*. Enable the ESP-IDF task watchdog, surface its bite count in `/state` alongside the existing `decode_errors` / `state_post_failures`. Good hygiene.
-5. **Multi-camera per viewport** — *medium effort*. Let one viewport listen to events from N cameras, picking which one to stream based on which fired. Useful for "show whichever doorbell rang" or zone monitoring.
-6. **Per-viewport JPEG quality knob** — *trivial*. Add a `jpeg_quality` setting that drives ffmpeg's `-q:v` arg. Trade fidelity for fps if anyone wants. Only matters once we want >22 fps from a single viewport.
-7. **MJPEG-over-WebSocket** instead of `POST /frame` per JPEG — *medium effort, marginal win*. Would shave per-frame HTTP overhead but the firmware decode-paint mutex is the new ceiling. Skip unless we find a use case the per-frame path can't cover.
-8. **Boot info-screen flash polish** — *low effort*. Keep the brief wake-on-boot (the FT5426 needs it to start reporting touches) but smoothen the ~600 ms flash so it doesn't visibly flicker on power-up.
-9. **Production sealing** — *eventual*. Configurable LAN scope (cross-VLAN, mDNS-via-Unicast), Scrypted-side mutual auth, replay protection for `/state` callbacks.
+1. **Network body shrink** — *the only remaining big lever*. The firmware is reading ~215 KB JPEGs at ~45 Mbit/s effective on a 100 Mbit Ethernet link — that's about 45% of theoretical. Two routes:
+ - **TCP tuning in `sdkconfig`**: bump `CONFIG_LWIP_TCP_WND_DEFAULT` and `CONFIG_LWIP_TCP_RCVMBOX_SIZE`, plus enable `CONFIG_LWIP_TCP_RTO_MIN` adjustments. With BDP at LAN-1 ms × 100 Mbit ≈ 12.5 KB, the default 5.7 KB window is half what we need; doubling it should add 30–50% throughput.
+ - **Per-viewport JPEG quality** (`-q:v` on the ffmpeg side): -q:v 5 halves file size at modest visual cost. `body` halves with it.
+ Combined, total → ~22 ms → **~45 fps ceiling**.
+2. **OTA firmware updates** — *low effort, gateway to fleet ops*. Partition table already has `ota_0` / `ota_1`. Add `esp_https_ota` (or a one-shot `POST /firmware` using `esp_ota_*`) so reflashing doesn't require USB. Critical once you have more than one viewport in production.
+3. **Task watchdog + crash counters** — *low effort*. Enable the ESP-IDF task watchdog, surface its bite count in `/state` alongside the existing `decode_errors` / `state_post_failures`. Good hygiene.
+4. **Multi-camera per viewport** — *medium effort*. Let one viewport listen to events from N cameras, picking which one to stream based on which fired. Useful for "show whichever doorbell rang" or zone monitoring.
+5. **MJPEG-over-WebSocket** instead of `POST /frame` per JPEG — *medium effort, marginal win*. Skip unless we find a use case the per-frame path can't cover.
+6. **Boot info-screen flash polish** — *low effort*. Keep the brief wake-on-boot (the FT5426 needs it to start reporting touches) but smoothen the ~600 ms flash so it doesn't visibly flicker on power-up.
+7. **Production sealing** — *eventual*. Configurable LAN scope (cross-VLAN, mDNS-via-Unicast), Scrypted-side mutual auth, replay protection for `/state` callbacks.
## Philosophy