# Testing & Verification This file tracks which milestones have been verified on hardware and how. Per-milestone entries below each list: - **Acceptance** — what "done" means (mirrors the impl guide). - **How to verify** — exact commands or actions. - **Status** — ⬜ not verified / 🟡 partial / ✅ verified, with a one-line note (date, board rev, anything weird). Hardware-required milestones can't be verified by a clean build alone — a checked-in build success counts as 🟡 (compiles, not yet run). --- ## M1 — Board Bring-Up **Acceptance**: device gets a DHCP lease over Ethernet and logs its IP. **How to verify** ```bash source ~/Dev/code/git/esp32/env.sh idf.py -p /dev/cu.usbmodem* flash monitor ``` Expected log within ~5–10s of boot: ``` I (xxx) viewport: Scrypted Viewport boot I (xxx) net_eth: ethernet driver started, waiting for link + DHCP I (xxx) net_eth: link up, mac xx:xx:xx:xx:xx:xx I (xxx) net_eth: got ip 192.168.x.x gw 192.168.x.1 netmask 255.255.255.0 I (xxx) viewport: online at 192.168.x.x ``` Cross-check from another host: ```bash ping 192.168.x.x ``` **Status**: 🟡 builds clean against ESP-IDF 5.4 (commit e2ac22e). Not yet flashed to hardware — awaiting first board run. > Combined hardware verification of M1+M2 is fine — they both run from the same flash session. --- ## M2 — HTTP + mDNS **Acceptance**: `GET /state` returns JSON; mDNS service is discoverable. **How to verify** After flash, from a host on the same LAN: ```bash # Direct call by IP (always works): curl http:///state | jq . # mDNS hostname (requires OS-level .local resolution: macOS Bonjour, Linux nss-mdns): curl http://viewport.local/state | jq . # Service discovery browse (preferred — what Scrypted will do, no OS .local resolver needed): dns-sd -B _scrypted-viewport._tcp local. # macOS avahi-browse -r _scrypted-viewport._tcp # Linux ``` Expected `/state` body on a fresh device (unconfigured): ```json { "name": null, "version": "0.1.0", "configured": false, "state": "unconfigured", "uptime_ms": 12345, "last_frame_ms_ago": null, "frames_received": 0, "decode_errors": 0, "state_post_failures": 0, "resolution": "480x800", "ip": "192.168.x.x", "free_heap": 200000, "free_psram": 30000000 } ``` Expected mDNS browse output should show a `_scrypted-viewport._tcp` instance with TXT records `version=`, `resolution=`, `orientation=`, `name=` (empty for `name`). **Status**: 🟡 builds clean against ESP-IDF 5.4 with espressif/mdns managed component. Awaiting first board run. --- ## M3 — Display Bring-Up **Acceptance**: panel powers on, MCU at I2C 0x45 responds, color-bar test pattern renders, brightness control works. **Wiring** (Hosyond 5" 800x480 panel ↔ Waveshare ESP32-P4-ETH): The DSI FPC adapter (15-pin Pi side → 22-pin Waveshare side) carries the DSI data lanes and power. I2C and panel power are jumpered separately from the ESP32-P4 board to the Hosyond's auxiliary header (the same header Pi users normally connect to the Pi's 40-pin GPIO): | Hosyond panel pin | Wire to ESP32-P4 board | | --- | --- | | 5V | board 5V rail | | GND | board GND | | SDA | GPIO 7 | | SCL | GPIO 8 | GPIO 7/8 match Waveshare's BSP convention for touch I2C on their bundled-panel kit. If different pins are more convenient, change `PIN_I2C_SDA` / `PIN_I2C_SCL` in `display.c`. **Bring-up sequence** 1. Power the ESP32-P4 board over USB (don't plug DSI yet). 2. Connect 5V + GND jumpers to the panel. Panel LED (if present) should light. 3. Connect SDA + SCL jumpers. 4. Plug the DSI FPC cable. 5. Flash: ```bash idf.py -p /dev/cu.usbmodem* flash monitor ``` **Expected log** ``` I (xxx) display: panel MCU id 0xC3 — Pi 7" architecture ack'd I (xxx) display: panel powered on I (xxx) display: DSI up: 800x480 30 MHz, 2-lane 480 Mbps I (xxx) viewport: display up — test pattern on screen ``` **Visual check**: 8 vertical color bars (white, yellow, cyan, green, magenta, red, blue, black) across the panel. Brightness should look perceptually mid-range (default 80/100 with gamma). **Failure-mode signals** - `panel MCU @0x45 unreachable` → jumper or pull-up problem on I2C, no need to suspect DSI yet. - I2C ack but no image → DSI cable / FPC adapter orientation. Pi FPCs are easy to install upside-down. - Image but wrong colors → check RGB565 byte order; flip `LCD_COLOR_PIXEL_FORMAT_RGB565` to a variant with swap. - Image but vertical/horizontal sync issues → adjust the `PANEL_*SYNC_*` timings; Pi 7" canonical values used as defaults. **Status**: 🟡 builds clean against ESP-IDF 5.4. Driver ported from Linux `panel-raspberrypi-touchscreen.c`. Awaiting hardware bring-up with confirmed jumper wiring. --- ## M4 — Config Persistence **Acceptance**: `POST /config` persists across reboot; partial-update semantics work. **How to verify** ```bash # Full config: curl -X POST -H "Content-Type: application/json" \ -d '{"viewport":"mudroom","scrypted":"http://host/endpoint/scrypted-viewport","orientation":"landscape"}' \ http:///config # Read back: curl http:///config | jq . # Partial update (only brightness): curl -X POST -H "Content-Type: application/json" \ -d '{"brightness":50}' \ http:///config # Reboot, then re-read — values should survive: curl http:///config | jq . ``` Also verify validation: ```bash # idle_timeout_ms below 5000 (and non-zero) — expect 400: curl -i -X POST -H "Content-Type: application/json" \ -d '{"idle_timeout_ms":1000}' http:///config # bogus orientation — expect 400: curl -i -X POST -H "Content-Type: application/json" \ -d '{"orientation":"sideways"}' http:///config # brightness out of range — expect 400: curl -i -X POST -H "Content-Type: application/json" \ -d '{"brightness":150}' http:///config # scrypted without http:// — expect 400: curl -i -X POST -H "Content-Type: application/json" \ -d '{"scrypted":"scrypted.local"}' http:///config # Garbage JSON — expect 400: curl -i -X POST -H "Content-Type: application/json" \ -d 'not json' http:///config ``` Idle-timer disable (`idle_timeout_ms: 0`) is intentionally allowed. Side-effects to confirm: - After `POST /config` with `brightness`: panel brightness changes immediately (if display is up). - After `POST /config` with `viewport` or `orientation`: mDNS TXT records update; `viewport-.local` resolves; browse shows new TXT. - After `POST /config` with both `viewport` and `scrypted` (any order, on any subsequent call): `GET /state` shows `configured: true`, `state: "asleep"`. **Status**: 🟡 builds clean against ESP-IDF 5.4. Logic exercised in code but unverified on hardware. --- ## M5 — JPEG Frame Push **Acceptance**: `POST /frame` paints a 480x800 (portrait, default) or 800x480 (landscape) JPEG. Orientation determines the expected dimensions and the panel-side rotation. **How to verify** ```bash # Default orientation (portrait) — send 480x800: curl -X POST -H "Content-Type: image/jpeg" \ --data-binary @test-480x800.jpg \ http:///frame # Then switch to landscape and re-test with an 800x480 JPEG: curl -X POST -H "Content-Type: application/json" \ -d '{"orientation":"landscape"}' http:///config curl -X POST -H "Content-Type: image/jpeg" \ --data-binary @test-800x480.jpg \ http:///frame ``` Visual: image fills the panel correctly oriented. After paint, `GET /state` shows `frames_received` incremented and `last_frame_ms_ago` populated. Generate test JPEGs with ImageMagick: ```bash # 8 vertical color bars at 480x800 portrait: convert -size 60x800 gradient:white-black -duplicate 7 +append test-480x800.jpg # or just any 480x800 JPEG: convert -size 480x800 plasma: test-480x800.jpg convert -size 800x480 plasma: test-800x480.jpg ``` Negative tests: ```bash # Wrong Content-Type — expect 400: curl -i -X POST -H "Content-Type: image/png" --data-binary @anything \ http:///frame # Oversize — expect 413: dd if=/dev/urandom of=big.bin bs=1M count=2 curl -i -X POST -H "Content-Type: image/jpeg" --data-binary @big.bin \ http:///frame # Wrong dimensions — expect 400 with "expected WxH, got WxH": convert -size 640x480 plasma: test-640x480.jpg curl -i -X POST -H "Content-Type: image/jpeg" --data-binary @test-640x480.jpg \ http:///frame # Concurrent posts — second gets 503: curl -X POST -H "Content-Type: image/jpeg" --data-binary @test-480x800.jpg \ http:///frame & curl -i -X POST -H "Content-Type: image/jpeg" --data-binary @test-480x800.jpg \ http:///frame # expect 503 if first is still in flight # Garbage bytes claiming to be JPEG — expect 400: curl -i -X POST -H "Content-Type: image/jpeg" -d 'not a jpeg' \ http:///frame ``` After every error: `decode_errors` in `GET /state` should increment. **Known gap (M6 closes this)**: M5 paints regardless of wake/sleep state. The `/frame` → `409` when asleep rule is added with `POST /state` in M6. Until then, a configured device boots ASLEEP per spec but `/frame` still paints because the state guard isn't wired yet. **Status**: 🟡 builds clean against ESP-IDF 5.4. Awaiting hardware verification (the test pattern from M3 confirms paint works; M5 adds the JPEG path). --- ## M6 — State + Idle Timer **Acceptance**: `POST /state` toggles wake/sleep; `/frame` is 409 when asleep; idle timer fires after `idle_timeout_ms`. **How to verify** First make sure the device is configured (M4 must be done): ```bash curl -X POST -H "Content-Type: application/json" \ -d '{"viewport":"mudroom","scrypted":"http://host/cb","idle_timeout_ms":10000}' \ http:///config ``` Wake / sleep: ```bash # Backlight off: curl -i -X POST -H "Content-Type: application/json" \ -d '{"state":"sleep"}' http:///state # expect: 204 # Backlight on (loading-screen placeholder until M8): curl -i -X POST -H "Content-Type: application/json" \ -d '{"state":"wake"}' http:///state # expect: 204 # Idempotency: repeating either is also 204 with no side effects: curl -X POST -H "Content-Type: application/json" \ -d '{"state":"wake"}' http:///state curl -X POST -H "Content-Type: application/json" \ -d '{"state":"wake"}' http:///state # Both: 204 ``` `/frame` while asleep returns 409: ```bash curl -X POST -H "Content-Type: application/json" \ -d '{"state":"sleep"}' http:///state curl -i -X POST -H "Content-Type: image/jpeg" \ --data-binary @test-480x800.jpg \ http:///frame # expect: HTTP/1.1 409 Conflict, body "device asleep — POST /state ..." ``` Idle timer: ```bash curl -X POST -H "Content-Type: application/json" \ -d '{"state":"wake"}' http:///state sleep 12 # idle_timeout_ms was set to 10000 above curl http:///state | jq .state # expect: "asleep" ``` When the timer fires, the serial log should print: ``` I (xxx) state: idle timer expired — sleeping I (xxx) state: ASLEEP ``` Disabling the idle timer: ```bash curl -X POST -H "Content-Type: application/json" \ -d '{"idle_timeout_ms":0}' http:///config curl -X POST -H "Content-Type: application/json" \ -d '{"state":"wake"}' http:///state sleep 70 curl http:///state | jq .state # expect: still "awake" ``` Resetting on `/frame`: every successful paint restarts the idle timer. ```bash # Set short idle, then keep painting to keep awake: curl -X POST -d '{"idle_timeout_ms":10000}' \ -H "Content-Type: application/json" http:///config curl -X POST -d '{"state":"wake"}' \ -H "Content-Type: application/json" http:///state for i in 1 2 3 4 5; do sleep 8 curl -X POST -H "Content-Type: image/jpeg" \ --data-binary @test-480x800.jpg http:///frame done curl http:///state | jq .state # expect "awake" ``` Unconfigured device rejects `POST /state` with 409: ```bash # (After factory reset / fresh boot, no /config yet) curl -i -X POST -H "Content-Type: application/json" \ -d '{"state":"wake"}' http:///state # expect: 409 Conflict "device unconfigured" ``` Bad input: ```bash curl -i -X POST -d '{"state":"middle"}' \ -H "Content-Type: application/json" http:///state # 400 curl -i -X POST -d 'not json' \ -H "Content-Type: application/json" http:///state # 400 ``` **Known gap (M7 closes this)**: when the idle timer fires the device transitions to ASLEEP locally but does NOT yet POST `{viewport,state:sleep}` to `/state`. That outbound POST lands with `state_client` in M7. **Status**: 🟡 builds clean against ESP-IDF 5.4. Awaiting hardware. --- ## M7 — Touch + Outbound `/state` POST **Acceptance**: tap toggles wake/sleep locally; idle timer firing posts sleep; device POSTs `{viewport,state}` to `/state` for every local transition. **Test receiver** (run on the host you'll point `scrypted` at): ```bash # Single-file flask receiver that echoes every POST: pip install flask cat > /tmp/recv.py <<'EOF' from flask import Flask, request app = Flask(__name__) @app.post("/state") def state(): print("RX", request.get_json()) return "", 204 app.run(host="0.0.0.0", port=11080) EOF python3 /tmp/recv.py ``` Configure the device: ```bash curl -X POST -H "Content-Type: application/json" \ -d '{"viewport":"mudroom","scrypted":"http://:11080","idle_timeout_ms":10000}' \ http:///config ``` **Tap dispatch** - Tap while asleep → device wakes; receiver prints `RX {'viewport': 'mudroom', 'state': 'wake'}`. - Tap while awake → device sleeps; receiver prints same with `state: 'sleep'`. - Idle timer expires (10s here) → receiver prints `state: 'sleep'`. Each successful POST also logs on serial: ``` I (xxx) state_client: POST http://:11080/state {state:wake} -> 204 ``` **Failure path** - Stop the receiver, then tap. Serial: ``` W (xxx) state_client: POST http://:11080/state failed: err=ESP_OK status=-1 ``` Then `GET /state` shows `state_post_failures` incremented. - Local state change still happens (backlight toggles). The POST is best-effort, not a precondition. **Depth-1 queue with replace-on-full** Tap rapidly (faster than the receiver can ack) and confirm the receiver only sees the most recent state transition between in-flight POSTs. Intermediate flips are coalesced. With a 1s receiver delay, only the final state of each ~1s window should hit the receiver. **No POST when Scrypted-initiated** - `curl -X POST -d '{"state":"sleep"}' /state` — receiver should print **nothing** (Scrypted already knows it initiated). - Same for `/state {wake}` and `/frame` (asleep → 409, no callback either). **No POST when unconfigured** - After factory reset, tapping the screen does nothing (no Scrypted URL to call). No queue entries dropped to disk. **Status**: 🟡 builds clean against ESP-IDF 5.4. Hardware verification needs the FT5426 touch jumpered + reachable on I2C 0x38. --- ## M8 — Local Screens + BOOT button **Acceptance**: IP screen on first boot; loading screen on every wake; BOOT button works. **Visual checks (panel-attached)** - Fresh flash (or after factory reset) → screen shows two centered white-on-black lines: ``` viewport.local 192.168.x.y ``` - `POST /config` with viewport + scrypted → device transitions to ASLEEP, backlight off. - `POST /state {state:wake}` (or tap) → backlight on, `Loading...` centered until the first `/frame` lands. - `POST /frame` while AWAKE → loading screen replaced by the JPEG. - `POST /state {state:sleep}` (or idle timeout, or tap-while-awake) → backlight off. **BOOT button** (GPIO 0 placeholder — see Hardware note below) - Short press at any state: backlight wakes if it was off, IP screen overlays for 15s, then `local_screens_restore_for_state()` paints black (Scrypted's next `/frame` repaints if awake; the state machine handles backlight if asleep). Does not change wake/sleep state. - Hold ≥ 5s: serial logs `BOOT held 5000ms → factory reset`, NVS clears via `nvs_config_reset()`, `esp_restart()`. After reboot the device comes back UNCONFIGURED with the IP screen. **Hardware note**: `PIN_BOOT_BUTTON = 0` in `button.c` is a guess. ESP32-P4's official strap pin is GPIO35 but that's owned by RMII TXD1 at runtime. Most Waveshare ESP32-P4 boards expose a separate user button on a free GPIO — confirm against the schematic and update `PIN_BOOT_BUTTON` if needed. If the wrong pin is wired, the button just never triggers (input reads stuck-high) and the rest of M8 still works. **Negative / edge** - BOOT overlay during AWAKE: the overlay paints over the live frame. Scrypted's next `/frame` (within ~1s in a normal stream) overwrites it. Acceptable: the operator sees the IP for ≤ 1 frame interval then the live view resumes. - BOOT overlay during ASLEEP: backlight comes on for the overlay, then `restore_for_state` paints black. The state machine's `display_sleep()` is NOT re-issued — that's an open follow-up if the test pattern stays lit after expiry. - Font fallback: any character outside the supported set (digits, dot, colon, `L`, lowercase a/c/d/e/g/i/l/n/o/p/r/t/v/w, space) renders as blank. Today the only strings drawn are `viewport.local`, the IPv4 string, and `Loading...` — all covered. **Status**: 🟡 builds clean against ESP-IDF 5.4 (binary ~870 KB). Hardware verification awaits panel + BOOT-pin confirmation. --- ## M9 — Live Stream (`POST /stream`) **Acceptance**: ≥ 10 fps multipart MJPEG over a single chunked POST. **How to verify**: ffmpeg-driven test stream from a host, measure fps from the device's frame counter. **Status**: ⬜ pending. --- ## Integration & system tests (post-M9) Run these once all milestones are implemented and individually verified. The point is to exercise races, edge cases, and longevity that single-milestone tests miss. ### A. End-to-end with Scrypted - Real Scrypted-side Script (M7's scope): doorbell-event → `/state {wake}` → frame stream → idle timer fires → `/state {sleep}` callback → Scrypted stops. - Bound camera switching: change the binding in Scrypted, confirm next wake routes to the new camera. ### B. Race conditions - Concurrent tap-while-Scrypted-POSTs-`/state`: spam both, confirm the device converges (mutex serializes; last-write-wins). Use the wake/sleep counters in `/state` to track transitions. - Mid-stream tap-to-sleep: while Scrypted is streaming frames, tap to sleep. Inflight `/frame` should return 409. Confirm no half-painted frames, no re-wake. - Idle-timeout coincides with a fresh tap-wake: the second event wins. Verify by repeating with timing jitter. - Stale Scrypted sleep timer races a fresh wake callback: Scrypted-side `cancelPendingSleep` must work — if a stale sleep lands after a wake, viewport sleeps; user taps again; recovery in one extra tap. ### C. Failure modes - Cable pull mid-frame: device idle-sleeps after `idle_timeout_ms`. On reconnect, mDNS re-advertises; Scrypted re-finds and continues. - Scrypted unreachable on tap: device still toggles backlight, `state_post_failures` increments. Recovery: Scrypted comes back, next tap syncs. - DHCP lease change: device gets new IP, re-advertises via mDNS. Scrypted's periodic browse picks up the new address. - `/state_post_failures` count should be observable via `GET /state`. ### D. Longevity - 24h soak with a slow camera stream (~1 frame every 10s): verify no memory leaks (`free_heap` / `free_psram` stable in `/state`), no decode_errors growing. - Frame storm: 30 minutes at max sustainable fps. Verify watchdog never fires, no resets. ### E. Power & boot - PoE injector cycle: device boots clean, gets DHCP, re-registers via mDNS, no manual intervention needed. - Brown-out (PoE on edge of spec): device should reboot cleanly, not corrupt NVS. ### F. Negative protocol - Garbage JSON to `/config` → 400. - `/frame` with `Content-Type: image/png` → 400. - `/state` with `{"state":"middle"}` → 400. - `/frame` with body > 1 MB → 413. - Two concurrent `/frame` posts → one wins, the other 503. ### G. Multi-viewport Once at least two physical units are configured: confirm each routes its callback to Scrypted with its own `viewport` name; confirm Scrypted's discovery map has both IPs; confirm a camera event on the wrong-named viewport is correctly ignored on the right one.