feat: event-based walker detector tuned to real 7' overhead mount

Replace per-track line-crossing counter with a single event state machine
gated by foreground pixel count (ENTER=250, EXIT=150) and finalized by
quiet-exit or timeout. Direction inferred from centroid excursion
(up_score vs down_score) on quiet-exit fires, and from net displacement
(last_c vs first_c) on timeout fires.

Tuning reflects bench data at the intended 7' overhead mount: walkers
produce smaller centroid excursions than originally modelled, so
EXTENT gates, MIN_TRAJ, MAX_FRAMES and REFRACTORY were all relaxed from
their initial guesses. Constants and rationale live in firmware/lib/cv/cv.h.

Bench results (8 isolated walks, 4 entries + 4 exits):
  * Event detection: 8/8 (100%)
  * Aggregate entries+exits split: 4+4 (matches)
  * Per-walk direction labelling: 4/8 (~50%)

Document explicitly that per-walk direction is unreliable at this mount
and that downstream analytics should trust only gross traffic
(entries + exits). Recovering direction would require a physical mount
change or a richer signal; both are out of scope for v1.

Tooling:
  * tools/replay_logs.py — replay event state machine against captured
    [F] diagnostic lines, for offline tuning without flash-test loops.
  * firmware/src/main_capture.cpp + tools/capture_frames.py +
    tools/replay_frames.py — raw-frame capture firmware and Python port
    of the detector, kept in tree for future iteration even though the
    TimerCamera-F serial driver stripped specific byte ranges in testing
    and log-based replay became the working path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-17 16:03:36 -07:00
parent 3b471992f2
commit a37207b6ff
12 changed files with 1203 additions and 340 deletions

View File

@@ -1,6 +1,8 @@
# DoorCounter
Retail door traffic counter using M5Stack TimerCamera-F (ESP32 + OV3660). Counts entries/exits via overhead camera CV, passively scans BLE foot traffic, and reports hourly to `logs.research.bike`.
Retail door traffic counter using M5Stack TimerCamera-F (ESP32 + OV3660). Counts walker traversals via overhead camera CV, passively scans BLE foot traffic, and reports hourly to `logs.research.bike`.
> **Known limitation — directional accuracy.** This firmware reports counts as `{entries, exits}` for API compatibility, but **per-walk direction labelling is not reliable at the current mount (7' overhead, straight down).** In bench testing, event detection was 100% (8/8 walks detected) while per-walk direction matched the physical walk only ~50% of the time — the centroid trajectories produced by entries and exits were nearly indistinguishable. **The number to trust is gross traffic: `entries + exits` ≈ total walkers through the doorway.** The directional split is an unreliable best-effort heuristic. See [Directional counting](#directional-counting) for why.
## Hardware
@@ -21,7 +23,7 @@ pio run -t upload --upload-port /dev/ttyUSB0
| Module | Behavior |
|--------|----------|
| CV pipeline | 5 fps, 96×96 grayscale, blob tracking, directional traversal count (origin→destination, once per track) with per-direction cooldown safety net |
| CV pipeline | 5 fps, 96×96 grayscale, event-based walker detector (foreground-count state machine; centroid-trajectory direction heuristic) with post-fire refractory period |
| Detection LED | Single blink on entry, double blink on exit (preserves upload/no-WiFi status LED) |
| BLE scanner | Continuous passive scan; deinits during hourly upload to free heap |
| Reporter | Hourly HMAC-signed POST; 60s boot report for fast connectivity check |
@@ -33,31 +35,70 @@ pio run -t upload --upload-port /dev/ttyUSB0
- **First report**: 60 seconds after NTP sync (connectivity check)
- **Subsequent reports**: every 3600 seconds
### Directional counting
### Counting model — event-based walker detector
Each tracked blob fires at most **one** event over its lifetime, and only
when it has genuinely traversed the frame — specifically, when its spawn
position and current position are both at least `CV_TRAVERSAL_MARGIN_PX`
(14 px ≈ 15% of the 96×96 frame) from the line, and on opposite sides.
The CV pipeline is a **single event state machine** (no per-blob tracking
for counting). Per-frame foreground pixel count gates event start and end;
centroid trajectory within the active event decides direction.
- Top half → bottom half traversal = **entry**
- Bottom half → top half traversal = **exit**
**Event lifecycle:**
1. **Idle → Active**: `fg_count ≥ CV_EVENT_ENTER_THRESH` (250 px) fires event start.
Background updates freeze while the event is active so the walker does
not get absorbed into the baseline.
2. **Active accumulation**: every frame updates `first_c` (once), `min_c`,
`max_c`, `last_c`, `min_y_seen`, `max_y_seen`, and the frame count.
3. **Active → End** (either):
- **Quiet exit**: `fg_count < CV_EVENT_EXIT_THRESH` (150 px) for
`CV_EVENT_QUIET_FRAMES` (3) consecutive frames — walker has left.
- **Timeout**: `event_frame_count > CV_EVENT_MAX_FRAMES` (25 frames ≈ 5s).
4. On end, the event is finalized: gated by minimum duration, vertical
extent (must span a large fraction of the frame), and minimum centroid
trajectory magnitude. Background snaps to the current frame.
5. A **refractory period** (`CV_EVENT_REFRACTORY_FRAMES` = 10 ≈ 2s) after
a fire blocks a new event from starting — absorbs residual lingering
motion that would otherwise double-count.
A blob that appears near the line and wobbles across it does not count
(both positions are within the margin band). A blob that fully traverses
then reverses under the same track also does not double-count (the track
is flagged `counted`). If tracking churns — the track dies mid-traversal
and respawns on the other side — a new track with a new spawn on the
crossed side is the normal path to a correct count.
**Direction heuristic** (applied only if the event passes all gates):
- `up_score = first_c min_c` (how far centroid excursed upward)
- `down_score = max_c first_c` (how far it excursed downward)
- Quiet-exit events: `is_entry = (up_score ≥ down_score)`
- Timeout events: `is_entry = (last_c < first_c)` — net displacement is
more reliable than excursion when the walker is still in frame at timeout.
See `firmware/lib/cv/cv.h` for margin and `cv.cpp` for the crossing logic.
Per-mount convention: centroid moving **up through the frame** (y decreasing)
= **entry** into the store.
### Crossing cooldown (safety net)
### Directional counting — known limitation
On top of directional counting, each direction enforces a cooldown between
counted events. Default: `CV_CROSSING_COOLDOWN_FRAMES = 5` (≈0.8s at 5 fps).
Entries and exits maintain separate cooldowns, so a real entry immediately
followed by a real exit still counts both.
**Per-walk direction labelling is unreliable at the current mount.** In
bench testing (8 alternating entry/exit walks at 4s intervals, 7' overhead
mount pointing straight down):
- **Event detection**: 8/8 (100%) — every walk produced exactly one event.
- **Aggregate split**: 4 entries + 4 exits — matches the 4+4 ground truth.
- **Per-walk direction**: 4/8 (50%) — essentially a coin flip.
At this mount, entries and exits produce nearly identical centroid
trajectories: both begin near mid-frame (walker is already large when
`fg_count` crosses 250), both reach a peak excursion toward the top, and
both end near mid-frame (walker's tail is still visible when `fg_count`
drops below 150). No heuristic over the recorded centroid statistics
separates them with better than ~50% accuracy on alternating walks.
**What we ship, and what the server should trust:**
- **Gross traffic (`entries + exits`) is accurate.** This is the number
downstream analytics should use as "people through the door this hour."
- **Directional split is reported but unreliable.** Treat individual
`entries` and `exits` values as a best-effort labelling. Do not infer
net flow or dwell from them.
To actually recover per-walk direction would require either a physical
change (raise or tilt the camera so walkers enter/leave through the frame
edges) or a richer signal than centroid statistics (e.g. time-resolved
optical flow, or a second sensor). That work is out of scope for v1.
See `firmware/lib/cv/cv.h` for tuning constants and `cv.cpp` for the
finalize logic.
## Operator Setup
@@ -124,7 +165,7 @@ DoorCounter/
├── firmware/
│ ├── platformio.ini
│ ├── lib/
│ │ ├── cv/ — CV pipeline (blob tracking, line cross, cooldown)
│ │ ├── cv/ — CV pipeline (event state machine, centroid-trajectory direction)
│ │ └── hmac/ — HMAC-SHA256 signing library
│ └── src/
│ ├── main.cpp — FreeRTOS tasks, boot sequence