feat: event-based walker detector tuned to real 7' overhead mount

Replace per-track line-crossing counter with a single event state machine
gated by foreground pixel count (ENTER=250, EXIT=150) and finalized by
quiet-exit or timeout. Direction inferred from centroid excursion
(up_score vs down_score) on quiet-exit fires, and from net displacement
(last_c vs first_c) on timeout fires.

Tuning reflects bench data at the intended 7' overhead mount: walkers
produce smaller centroid excursions than originally modelled, so
EXTENT gates, MIN_TRAJ, MAX_FRAMES and REFRACTORY were all relaxed from
their initial guesses. Constants and rationale live in firmware/lib/cv/cv.h.

Bench results (8 isolated walks, 4 entries + 4 exits):
  * Event detection: 8/8 (100%)
  * Aggregate entries+exits split: 4+4 (matches)
  * Per-walk direction labelling: 4/8 (~50%)

Document explicitly that per-walk direction is unreliable at this mount
and that downstream analytics should trust only gross traffic
(entries + exits). Recovering direction would require a physical mount
change or a richer signal; both are out of scope for v1.

Tooling:
  * tools/replay_logs.py — replay event state machine against captured
    [F] diagnostic lines, for offline tuning without flash-test loops.
  * firmware/src/main_capture.cpp + tools/capture_frames.py +
    tools/replay_frames.py — raw-frame capture firmware and Python port
    of the detector, kept in tree for future iteration even though the
    TimerCamera-F serial driver stripped specific byte ranges in testing
    and log-based replay became the working path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-17 16:03:36 -07:00
parent 3b471992f2
commit a37207b6ff
12 changed files with 1203 additions and 340 deletions

View File

@@ -13,7 +13,7 @@
[TimerCamera-F Device]
├── Provisioning module — captive portal AP on first boot
├── Config store — NVS: device_id, location_id, HMAC secret, WiFi creds, line_offset
├── Camera + CV module — captures frames, runs line-crossing counter
├── Camera + CV module — captures frames, runs event-based walker detector
├── BLE scanner — continuous passive scan (WiFi coexistence mode)
├── Report buffer — accumulates counts in RAM, flushes hourly
└── HTTP client — HMAC-signed POSTs to logs.research.bike
@@ -89,22 +89,38 @@ Capture → Grayscale → Downscale 96×96 → Frame diff → Threshold → Blob
| Downscale | Bilinear to 96×96 (~11× compute reduction) |
| Frame diff | Absolute difference against rolling background (updated every ~2s when no motion) |
| Threshold | Pixels > 30 intensity delta = foreground |
| Blob detect | Connected components; blobs < 8×8 px discarded as noise |
| Centroid track | Nearest-centroid matching frame-to-frame (max 15px), tracks persist up to 10 missed frames |
| Line crossing | Virtual horizontal line at configurable vertical position (default: 50% of frame height) |
| Directional traversal | Each track records its **spawn y** and fires **at most once**. An event fires only when the track's spawn position and current position are both ≥ `CV_TRAVERSAL_MARGIN_PX` (14 px) from the line and on opposite sides — i.e. a true traversal, not a wobble. |
| Cooldown | Per-direction cooldown between counted events (default 5 frames ≈ 0.8s @ 5 fps) — safety net on top of directional logic |
| Event state machine | Single global state machine (not per-blob). Per-frame `fg_count` (total foreground pixels) gates event start and end. |
| Event start | `fg_count ≥ CV_EVENT_ENTER_THRESH` (250 px) → event becomes active. Background updates freeze for the event's duration so the walker does not blend into the baseline. |
| Event accumulation | Each frame records `first_c` (centroid_y at start), running `min_c` / `max_c` / `last_c`, vertical extents (`min_y_seen`, `max_y_seen`), and frame count. |
| Event end | Either **quiet exit** (`fg_count < CV_EVENT_EXIT_THRESH` (150 px) for `CV_EVENT_QUIET_FRAMES` (3) consecutive frames) or **timeout** (`event_frame_count > CV_EVENT_MAX_FRAMES` (25)). On end, background snaps to the current frame. |
| Fire gates | Duration ≥ `CV_EVENT_MIN_FRAMES` (5), `min_y_seen ≤ CV_EVENT_EXTENT_TOP` (25) AND `max_y_seen ≥ CV_EVENT_EXTENT_BOT` (50) — event must span a large fraction of the frame — AND `max(up_score, down_score) ≥ CV_EVENT_MIN_TRAJ` (5) |
| Refractory | `CV_EVENT_REFRACTORY_FRAMES` (10 ≈ 2s) after a fire, the machine refuses to start a new event — absorbs lingering motion of the just-counted walker. |
**Counting logic:**
- Each track has a `spawn_y` (recorded at creation) and a `counted` flag.
- An event fires only if the track is **not yet counted**, spawned **firm** on one side of the line (|spawn_y line_y| > `CV_TRAVERSAL_MARGIN_PX`), and is **now firm** on the opposite side. On fire, the track is flagged counted — it will not produce another event for its lifetime.
- Spawn firm above + now firm below = **entry**
- Spawn firm below + now firm above = **exit**
- Cooldown (per-direction, independent entries/exits) is a secondary gate: within `CV_CROSSING_COOLDOWN_FRAMES` of the last counted event in that direction, a new event is suppressed even if a different track's traversal would otherwise qualify.
**Direction heuristic (applied after fire gates pass):**
- `up_score = first_c min_c` (peak upward centroid excursion)
- `down_score = max_c first_c` (peak downward centroid excursion)
- **Quiet-exit fires**: `is_entry = (up_score ≥ down_score)`
- **Timeout fires**: `is_entry = (last_c < first_c)` — walker is still in frame at timeout, so net displacement is a better signal than excursion.
**Rationale**: a single person traversing the doorway produces one track with a clear origin and destination — that's one count. Shadows that appear near the line and wobble, or tracks that churn at spawn, lack a firm origin and never count.
Per-mount convention: centroid moving **up through the frame** (y decreasing) = **entry** into the store.
Counts accumulate as `{entries, exits}` in RAM and reset each hour on report.
**Counting surface**: `{entries, exits}` accumulate in RAM and reset each hour on report.
**Directional accuracy is best-effort, not guaranteed.** In bench testing at the intended 7' overhead straight-down mount:
| Metric | Result |
|--------|--------|
| Event detection | 8/8 walks (100%) |
| Aggregate entry/exit split | 4+4 vs ground-truth 4+4 (matches) |
| Per-walk direction labelling | 4/8 (50%) — no better than chance |
At this mount, entries and exits produce nearly identical centroid trajectories: the walker is already large when `fg_count` crosses 250 (so `first_c` is always near mid-frame), their tail is still visible when `fg_count` drops below 150 (so `last_c` is always near mid-frame), and the excursion in between peaks upward for both directions. No statistic computable from (`first_c`, `min_c`, `max_c`, `last_c`, duration) separates them reliably.
**Contract with downstream consumers (API and analytics):**
- **`entries + exits` is the trustworthy number** — it is the count of walkers through the doorway in the hour. Use this as "foot traffic."
- **Individual `entries` and `exits` are reported for API shape compatibility, but should not be relied on for net flow, dwell, or any per-direction analysis.**
Recovering true direction requires either a physical change (tilt or raise the camera so walkers pass fully through the frame edges) or a richer signal (time-resolved centroid trajectory, optical flow, secondary sensor). Both are out of scope for v1.
---