feat: event-based walker detector tuned to real 7' overhead mount
Replace per-track line-crossing counter with a single event state machine
gated by foreground pixel count (ENTER=250, EXIT=150) and finalized by
quiet-exit or timeout. Direction inferred from centroid excursion
(up_score vs down_score) on quiet-exit fires, and from net displacement
(last_c vs first_c) on timeout fires.
Tuning reflects bench data at the intended 7' overhead mount: walkers
produce smaller centroid excursions than originally modelled, so
EXTENT gates, MIN_TRAJ, MAX_FRAMES and REFRACTORY were all relaxed from
their initial guesses. Constants and rationale live in firmware/lib/cv/cv.h.
Bench results (8 isolated walks, 4 entries + 4 exits):
* Event detection: 8/8 (100%)
* Aggregate entries+exits split: 4+4 (matches)
* Per-walk direction labelling: 4/8 (~50%)
Document explicitly that per-walk direction is unreliable at this mount
and that downstream analytics should trust only gross traffic
(entries + exits). Recovering direction would require a physical mount
change or a richer signal; both are out of scope for v1.
Tooling:
* tools/replay_logs.py — replay event state machine against captured
[F] diagnostic lines, for offline tuning without flash-test loops.
* firmware/src/main_capture.cpp + tools/capture_frames.py +
tools/replay_frames.py — raw-frame capture firmware and Python port
of the detector, kept in tree for future iteration even though the
TimerCamera-F serial driver stripped specific byte ranges in testing
and log-based replay became the working path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
85
README.md
85
README.md
@@ -1,6 +1,8 @@
|
||||
# DoorCounter
|
||||
|
||||
Retail door traffic counter using M5Stack TimerCamera-F (ESP32 + OV3660). Counts entries/exits via overhead camera CV, passively scans BLE foot traffic, and reports hourly to `logs.research.bike`.
|
||||
Retail door traffic counter using M5Stack TimerCamera-F (ESP32 + OV3660). Counts walker traversals via overhead camera CV, passively scans BLE foot traffic, and reports hourly to `logs.research.bike`.
|
||||
|
||||
> **Known limitation — directional accuracy.** This firmware reports counts as `{entries, exits}` for API compatibility, but **per-walk direction labelling is not reliable at the current mount (7' overhead, straight down).** In bench testing, event detection was 100% (8/8 walks detected) while per-walk direction matched the physical walk only ~50% of the time — the centroid trajectories produced by entries and exits were nearly indistinguishable. **The number to trust is gross traffic: `entries + exits` ≈ total walkers through the doorway.** The directional split is an unreliable best-effort heuristic. See [Directional counting](#directional-counting) for why.
|
||||
|
||||
## Hardware
|
||||
|
||||
@@ -21,7 +23,7 @@ pio run -t upload --upload-port /dev/ttyUSB0
|
||||
|
||||
| Module | Behavior |
|
||||
|--------|----------|
|
||||
| CV pipeline | 5 fps, 96×96 grayscale, blob tracking, directional traversal count (origin→destination, once per track) with per-direction cooldown safety net |
|
||||
| CV pipeline | 5 fps, 96×96 grayscale, event-based walker detector (foreground-count state machine; centroid-trajectory direction heuristic) with post-fire refractory period |
|
||||
| Detection LED | Single blink on entry, double blink on exit (preserves upload/no-WiFi status LED) |
|
||||
| BLE scanner | Continuous passive scan; deinits during hourly upload to free heap |
|
||||
| Reporter | Hourly HMAC-signed POST; 60s boot report for fast connectivity check |
|
||||
@@ -33,31 +35,70 @@ pio run -t upload --upload-port /dev/ttyUSB0
|
||||
- **First report**: 60 seconds after NTP sync (connectivity check)
|
||||
- **Subsequent reports**: every 3600 seconds
|
||||
|
||||
### Directional counting
|
||||
### Counting model — event-based walker detector
|
||||
|
||||
Each tracked blob fires at most **one** event over its lifetime, and only
|
||||
when it has genuinely traversed the frame — specifically, when its spawn
|
||||
position and current position are both at least `CV_TRAVERSAL_MARGIN_PX`
|
||||
(14 px ≈ 15% of the 96×96 frame) from the line, and on opposite sides.
|
||||
The CV pipeline is a **single event state machine** (no per-blob tracking
|
||||
for counting). Per-frame foreground pixel count gates event start and end;
|
||||
centroid trajectory within the active event decides direction.
|
||||
|
||||
- Top half → bottom half traversal = **entry**
|
||||
- Bottom half → top half traversal = **exit**
|
||||
**Event lifecycle:**
|
||||
1. **Idle → Active**: `fg_count ≥ CV_EVENT_ENTER_THRESH` (250 px) fires event start.
|
||||
Background updates freeze while the event is active so the walker does
|
||||
not get absorbed into the baseline.
|
||||
2. **Active accumulation**: every frame updates `first_c` (once), `min_c`,
|
||||
`max_c`, `last_c`, `min_y_seen`, `max_y_seen`, and the frame count.
|
||||
3. **Active → End** (either):
|
||||
- **Quiet exit**: `fg_count < CV_EVENT_EXIT_THRESH` (150 px) for
|
||||
`CV_EVENT_QUIET_FRAMES` (3) consecutive frames — walker has left.
|
||||
- **Timeout**: `event_frame_count > CV_EVENT_MAX_FRAMES` (25 frames ≈ 5s).
|
||||
4. On end, the event is finalized: gated by minimum duration, vertical
|
||||
extent (must span a large fraction of the frame), and minimum centroid
|
||||
trajectory magnitude. Background snaps to the current frame.
|
||||
5. A **refractory period** (`CV_EVENT_REFRACTORY_FRAMES` = 10 ≈ 2s) after
|
||||
a fire blocks a new event from starting — absorbs residual lingering
|
||||
motion that would otherwise double-count.
|
||||
|
||||
A blob that appears near the line and wobbles across it does not count
|
||||
(both positions are within the margin band). A blob that fully traverses
|
||||
then reverses under the same track also does not double-count (the track
|
||||
is flagged `counted`). If tracking churns — the track dies mid-traversal
|
||||
and respawns on the other side — a new track with a new spawn on the
|
||||
crossed side is the normal path to a correct count.
|
||||
**Direction heuristic** (applied only if the event passes all gates):
|
||||
- `up_score = first_c − min_c` (how far centroid excursed upward)
|
||||
- `down_score = max_c − first_c` (how far it excursed downward)
|
||||
- Quiet-exit events: `is_entry = (up_score ≥ down_score)`
|
||||
- Timeout events: `is_entry = (last_c < first_c)` — net displacement is
|
||||
more reliable than excursion when the walker is still in frame at timeout.
|
||||
|
||||
See `firmware/lib/cv/cv.h` for margin and `cv.cpp` for the crossing logic.
|
||||
Per-mount convention: centroid moving **up through the frame** (y decreasing)
|
||||
= **entry** into the store.
|
||||
|
||||
### Crossing cooldown (safety net)
|
||||
### Directional counting — known limitation
|
||||
|
||||
On top of directional counting, each direction enforces a cooldown between
|
||||
counted events. Default: `CV_CROSSING_COOLDOWN_FRAMES = 5` (≈0.8s at 5 fps).
|
||||
Entries and exits maintain separate cooldowns, so a real entry immediately
|
||||
followed by a real exit still counts both.
|
||||
**Per-walk direction labelling is unreliable at the current mount.** In
|
||||
bench testing (8 alternating entry/exit walks at 4s intervals, 7' overhead
|
||||
mount pointing straight down):
|
||||
|
||||
- **Event detection**: 8/8 (100%) — every walk produced exactly one event.
|
||||
- **Aggregate split**: 4 entries + 4 exits — matches the 4+4 ground truth.
|
||||
- **Per-walk direction**: 4/8 (50%) — essentially a coin flip.
|
||||
|
||||
At this mount, entries and exits produce nearly identical centroid
|
||||
trajectories: both begin near mid-frame (walker is already large when
|
||||
`fg_count` crosses 250), both reach a peak excursion toward the top, and
|
||||
both end near mid-frame (walker's tail is still visible when `fg_count`
|
||||
drops below 150). No heuristic over the recorded centroid statistics
|
||||
separates them with better than ~50% accuracy on alternating walks.
|
||||
|
||||
**What we ship, and what the server should trust:**
|
||||
- **Gross traffic (`entries + exits`) is accurate.** This is the number
|
||||
downstream analytics should use as "people through the door this hour."
|
||||
- **Directional split is reported but unreliable.** Treat individual
|
||||
`entries` and `exits` values as a best-effort labelling. Do not infer
|
||||
net flow or dwell from them.
|
||||
|
||||
To actually recover per-walk direction would require either a physical
|
||||
change (raise or tilt the camera so walkers enter/leave through the frame
|
||||
edges) or a richer signal than centroid statistics (e.g. time-resolved
|
||||
optical flow, or a second sensor). That work is out of scope for v1.
|
||||
|
||||
See `firmware/lib/cv/cv.h` for tuning constants and `cv.cpp` for the
|
||||
finalize logic.
|
||||
|
||||
## Operator Setup
|
||||
|
||||
@@ -124,7 +165,7 @@ DoorCounter/
|
||||
├── firmware/
|
||||
│ ├── platformio.ini
|
||||
│ ├── lib/
|
||||
│ │ ├── cv/ — CV pipeline (blob tracking, line cross, cooldown)
|
||||
│ │ ├── cv/ — CV pipeline (event state machine, centroid-trajectory direction)
|
||||
│ │ └── hmac/ — HMAC-SHA256 signing library
|
||||
│ └── src/
|
||||
│ ├── main.cpp — FreeRTOS tasks, boot sequence
|
||||
|
||||
Reference in New Issue
Block a user