diff options
| author | Luke Hoersten <[email protected]> | 2026-06-15 07:52:34 -0500 |
|---|---|---|
| committer | Luke Hoersten <[email protected]> | 2026-06-15 07:52:34 -0500 |
| commit | d949b19a6b89b9e74e8706bafa1d1396ac8f17f8 (patch) | |
| tree | 48108f2f50e7cb6f3b95b452b78f419e964eda70 /README.md | |
| parent | 3f53ced44ea112a39eec70216c8c72fa6867726e (diff) | |
README: refresh per-frame budget with double-buffered measurements; network body is now the only big lever
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 45 |
1 files changed, 23 insertions, 22 deletions
@@ -490,35 +490,36 @@ Every endpoint is idempotent; every failure leaves the device in a sane state. ## What's next -M1 – M8 are all ✅ on hardware (see [`TESTING.md`](TESTING.md) for verification details). End-to-end Scrypted streaming via ffmpeg + the zero-copy `JPEG → BGR888 → DSI` hot path is live and sustains **~6.5 fps painted at the 100 ms target tick**, gated by per-frame work below. +M1 – M8 are all ✅ on hardware (see [`TESTING.md`](TESTING.md) for verification details). End-to-end Scrypted streaming via ffmpeg + the zero-copy `JPEG → BGR888 → DSI` hot path sustains **~22 fps at full quality** with the double-buffered panel. -### Measured per-frame budget (target ≥ 22 fps requires shrinking these) +### Measured per-frame budget -The firmware logs a `frame N: lock=… ttfb=… body=… dec=… paint=… post=… total=…ms` line every 10 frames. Steady-state on the bench (Waveshare ESP32-P4-ETH + Hosyond 5" panel, ~210 KB JPEGs at `-q:v 2`): +The firmware logs a `frame N: lock=… ttfb=… body=… dec=… paint=… post=… total=…ms` line every 10 frames. Steady-state on the bench (Waveshare ESP32-P4-ETH + Hosyond 5" panel, ~215 KB JPEGs at `-q:v 2`): -| Phase | Time | What it is | -|---|---|---| -| `lock` | ~tens of µs | `jpeg_decoder_try_lock` (mutex acquire) | -| `ttfb` | ~1–2 ms | Time-to-first-byte after lock — TCP/HTTP handshake overhead | -| `body` | ~38 ms | Wire time for the rest of the JPEG body (~210 KB at ~42 Mbit/s effective) | -| `dec` | ~6 ms | Hardware JPEG decode → BGR888 (way faster than the textbook 50–80 ms guess) | -| `paint` | ~24 ms | `esp_lcd_panel_draw_bitmap` + ~1.5 DSI refresh cycles at 60 Hz | -| `post` | < 1 ms | State counter bookkeeping + unlock | -| **total** | **~70 ms** | **Theoretical ceiling ~14 fps if Scrypted ticks faster than the 100 ms default** | +| Phase | Time | Share | What it is | +|---|---|---|---| +| `lock` | 6–10 µs | < 0.1% | `jpeg_decoder_try_lock` mutex acquire | +| `ttfb` | 300–650 µs | < 1% | First byte landing after lock — TCP handshake noise | +| `body` | 37–44 ms | **~85%** | Wire time for the JPEG body (~215 KB at ~45 Mbit/s effective on 100 Mbit Ethernet) | +| `dec` | ~6 ms | ~13% | Hardware JPEG decode → BGR888 (way faster than the textbook 50–80 ms guess) | +| `paint` | 35–55 µs | < 0.1% | `esp_lcd_panel_draw_bitmap` — cache writeback + index swap thanks to `num_fbs = 2` and zero-copy decode into the back fb | +| `post` | 20–30 µs | < 0.1% | State counter bookkeeping + unlock | +| **total** | **43–49 ms** | | **Ceiling ~22 fps** | -So the firmware ceiling at full quality is ~14 fps, but Scrypted's `frame_interval_ms = 100` only fires 10 ticks/sec, leaving ~30 ms idle between frames → effective ~6.5 fps. Lower the per-viewport interval to ~75 ms to chase the ceiling. +Network body is now ~85% of every frame; everything else is at or near hardware floor. ### Current backlog, in rough priority order -1. **Double-buffer the panel (`num_fbs = 2`)** — *high impact*. Today `paint` is 24 ms because the synchronous `esp_lcd_panel_draw_bitmap` waits for a full DSI refresh cycle. With a second framebuffer the DSI engine streams from FB-A while the next frame fills FB-B; paint becomes a pointer swap (~1–2 ms). Total drops to ~46 ms, ceiling jumps to ~22 fps at full quality. Watch out for the TC358762 bridge's `non-burst with sync pulses` constraint — `flags.use_dma2d` and `num_fbs` interact non-obviously with the bridge wake sequence, so this needs careful re-bring-up. -2. **HTTP persistent connection** — *medium impact, low effort*. Would shave the ~1–2 ms `ttfb` per frame by keeping the TCP connection open across frames (currently Scrypted's `fetch` opens fresh each POST). Real win is consistency, not absolute speed; useful once #1 lands. -3. **OTA firmware updates** — *low effort, gateway to fleet ops*. Partition table already has `ota_0` / `ota_1`. Add `esp_https_ota` (or a one-shot `POST /firmware` using `esp_ota_*`) so reflashing doesn't require USB. Critical once you have more than one viewport in production. -4. **Task watchdog + crash counters** — *low effort*. Enable the ESP-IDF task watchdog, surface its bite count in `/state` alongside the existing `decode_errors` / `state_post_failures`. Good hygiene. -5. **Multi-camera per viewport** — *medium effort*. Let one viewport listen to events from N cameras, picking which one to stream based on which fired. Useful for "show whichever doorbell rang" or zone monitoring. -6. **Per-viewport JPEG quality knob** — *trivial*. Add a `jpeg_quality` setting that drives ffmpeg's `-q:v` arg. Trade fidelity for fps if anyone wants. Only matters once we want >22 fps from a single viewport. -7. **MJPEG-over-WebSocket** instead of `POST /frame` per JPEG — *medium effort, marginal win*. Would shave per-frame HTTP overhead but the firmware decode-paint mutex is the new ceiling. Skip unless we find a use case the per-frame path can't cover. -8. **Boot info-screen flash polish** — *low effort*. Keep the brief wake-on-boot (the FT5426 needs it to start reporting touches) but smoothen the ~600 ms flash so it doesn't visibly flicker on power-up. -9. **Production sealing** — *eventual*. Configurable LAN scope (cross-VLAN, mDNS-via-Unicast), Scrypted-side mutual auth, replay protection for `/state` callbacks. +1. **Network body shrink** — *the only remaining big lever*. The firmware is reading ~215 KB JPEGs at ~45 Mbit/s effective on a 100 Mbit Ethernet link — that's about 45% of theoretical. Two routes: + - **TCP tuning in `sdkconfig`**: bump `CONFIG_LWIP_TCP_WND_DEFAULT` and `CONFIG_LWIP_TCP_RCVMBOX_SIZE`, plus enable `CONFIG_LWIP_TCP_RTO_MIN` adjustments. With BDP at LAN-1 ms × 100 Mbit ≈ 12.5 KB, the default 5.7 KB window is half what we need; doubling it should add 30–50% throughput. + - **Per-viewport JPEG quality** (`-q:v` on the ffmpeg side): -q:v 5 halves file size at modest visual cost. `body` halves with it. + Combined, total → ~22 ms → **~45 fps ceiling**. +2. **OTA firmware updates** — *low effort, gateway to fleet ops*. Partition table already has `ota_0` / `ota_1`. Add `esp_https_ota` (or a one-shot `POST /firmware` using `esp_ota_*`) so reflashing doesn't require USB. Critical once you have more than one viewport in production. +3. **Task watchdog + crash counters** — *low effort*. Enable the ESP-IDF task watchdog, surface its bite count in `/state` alongside the existing `decode_errors` / `state_post_failures`. Good hygiene. +4. **Multi-camera per viewport** — *medium effort*. Let one viewport listen to events from N cameras, picking which one to stream based on which fired. Useful for "show whichever doorbell rang" or zone monitoring. +5. **MJPEG-over-WebSocket** instead of `POST /frame` per JPEG — *medium effort, marginal win*. Skip unless we find a use case the per-frame path can't cover. +6. **Boot info-screen flash polish** — *low effort*. Keep the brief wake-on-boot (the FT5426 needs it to start reporting touches) but smoothen the ~600 ms flash so it doesn't visibly flicker on power-up. +7. **Production sealing** — *eventual*. Configurable LAN scope (cross-VLAN, mDNS-via-Unicast), Scrypted-side mutual auth, replay protection for `/state` callbacks. ## Philosophy |
