From 3b471992f25ce72a2f62694140f1779322dc9292 Mon Sep 17 00:00:00 2001 From: Peter Woolery Date: Fri, 17 Apr 2026 09:46:59 -0700 Subject: [PATCH] feat(cv): directional once-per-track counting + detection LED blinks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A single person walking under the overhead camera was generating both an entry and an exit within a few seconds — the line-crossing logic treated a blob's traversal into one side of the frame and out the other as two separate events whenever the track spawned near the line, oscillated against shadows, or churned at creation. Replaced line-crossing semantics with directional traversal: - Each track records spawn_y at creation and a counted flag. - An event fires only if the track is not yet counted, spawned firm on one side of the line (|spawn_y - line_y| > CV_TRAVERSAL_MARGIN_PX), and is now firm on the opposite side. Direction of travel determines entry vs exit. The track is then flagged counted — one trip, one count. - Cooldown remains as a secondary safety net. main.cpp: single/double LED pulse on entry/exit detections. Saves and restores the current LED state so upload (yellow-on) and no-WiFi indicators aren't clobbered. Tests updated to walk blobs beyond the margin and register two new cases: wobble-at-line doesn't count, and a reversed full traversal doesn't double-count on the same track. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 38 ++++-- .../specs/2026-04-13-door-counter-design.md | 13 ++- firmware/lib/cv/cv.cpp | 56 +++++---- firmware/lib/cv/cv.h | 8 ++ firmware/src/main.cpp | 17 ++- firmware/test/test_cv/test_cv.cpp | 110 ++++++++++++++---- 6 files changed, 180 insertions(+), 62 deletions(-) diff --git a/README.md b/README.md index 0298f4f..323115e 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,8 @@ pio run -t upload --upload-port /dev/ttyUSB0 | Module | Behavior | |--------|----------| -| CV pipeline | 5 fps, 96×96 grayscale, blob tracking, line-crossing count with per-direction cooldown | +| CV pipeline | 5 fps, 96×96 grayscale, blob tracking, directional traversal count (origin→destination, once per track) with per-direction cooldown safety net | +| Detection LED | Single blink on entry, double blink on exit (preserves upload/no-WiFi status LED) | | BLE scanner | Continuous passive scan; deinits during hourly upload to free heap | | Reporter | Hourly HMAC-signed POST; 60s boot report for fast connectivity check | | Provisioning | Captive portal AP on first boot for WiFi setup | @@ -32,16 +33,31 @@ pio run -t upload --upload-port /dev/ttyUSB0 - **First report**: 60 seconds after NTP sync (connectivity check) - **Subsequent reports**: every 3600 seconds -### Crossing cooldown +### Directional counting -To suppress double-counts from track churn (a blob briefly dropping below the -minimum-blob-pixel threshold, causing the tracker to kill and respawn a track -that then re-crosses the line), each direction enforces a cooldown window -between counted crossings. Default: `CV_CROSSING_COOLDOWN_FRAMES = 5`, which -suppresses any second crossing in the same direction whose frame gap is `< 5` -— ≈0.8s at 5 fps. Entries and exits maintain separate cooldowns, so a real entry -immediately followed by a real exit still counts both. See -`firmware/lib/cv/cv.h`. +Each tracked blob fires at most **one** event over its lifetime, and only +when it has genuinely traversed the frame — specifically, when its spawn +position and current position are both at least `CV_TRAVERSAL_MARGIN_PX` +(14 px ≈ 15% of the 96×96 frame) from the line, and on opposite sides. + +- Top half → bottom half traversal = **entry** +- Bottom half → top half traversal = **exit** + +A blob that appears near the line and wobbles across it does not count +(both positions are within the margin band). A blob that fully traverses +then reverses under the same track also does not double-count (the track +is flagged `counted`). If tracking churns — the track dies mid-traversal +and respawns on the other side — a new track with a new spawn on the +crossed side is the normal path to a correct count. + +See `firmware/lib/cv/cv.h` for margin and `cv.cpp` for the crossing logic. + +### Crossing cooldown (safety net) + +On top of directional counting, each direction enforces a cooldown between +counted events. Default: `CV_CROSSING_COOLDOWN_FRAMES = 5` (≈0.8s at 5 fps). +Entries and exits maintain separate cooldowns, so a real entry immediately +followed by a real exit still counts both. ## Operator Setup @@ -87,7 +103,7 @@ python tools/ota_push.py \ 3. Connect phone to `DoorCounter-Setup` WiFi 4. Browser opens automatically → enter store WiFi password → done -**LED indicators**: Red = no WiFi · Blue = counting · Yellow = uploading +**LED indicators**: Red = no WiFi · Blue = counting · Yellow = uploading · Brief flash (×1) on entry · Brief flash (×2) on exit ## API diff --git a/docs/superpowers/specs/2026-04-13-door-counter-design.md b/docs/superpowers/specs/2026-04-13-door-counter-design.md index f622c24..d6d0d97 100644 --- a/docs/superpowers/specs/2026-04-13-door-counter-design.md +++ b/docs/superpowers/specs/2026-04-13-door-counter-design.md @@ -92,12 +92,17 @@ Capture → Grayscale → Downscale 96×96 → Frame diff → Threshold → Blob | Blob detect | Connected components; blobs < 8×8 px discarded as noise | | Centroid track | Nearest-centroid matching frame-to-frame (max 15px), tracks persist up to 10 missed frames | | Line crossing | Virtual horizontal line at configurable vertical position (default: 50% of frame height) | -| Cooldown | Per-direction cooldown between counted crossings (default 5 frames ≈ 0.8s @ 5 fps) — suppresses duplicate counts from track churn | +| Directional traversal | Each track records its **spawn y** and fires **at most once**. An event fires only when the track's spawn position and current position are both ≥ `CV_TRAVERSAL_MARGIN_PX` (14 px) from the line and on opposite sides — i.e. a true traversal, not a wobble. | +| Cooldown | Per-direction cooldown between counted events (default 5 frames ≈ 0.8s @ 5 fps) — safety net on top of directional logic | **Counting logic:** -- Centroid crosses line top→bottom = **entry** -- Centroid crosses line bottom→top = **exit** -- After a counted entry (resp. exit), subsequent entries (resp. exits) within `CV_CROSSING_COOLDOWN_FRAMES` are ignored. Entry and exit cooldowns are tracked independently, so an entry immediately followed by an exit (or vice versa) still counts both. +- Each track has a `spawn_y` (recorded at creation) and a `counted` flag. +- An event fires only if the track is **not yet counted**, spawned **firm** on one side of the line (|spawn_y − line_y| > `CV_TRAVERSAL_MARGIN_PX`), and is **now firm** on the opposite side. On fire, the track is flagged counted — it will not produce another event for its lifetime. +- Spawn firm above + now firm below = **entry** +- Spawn firm below + now firm above = **exit** +- Cooldown (per-direction, independent entries/exits) is a secondary gate: within `CV_CROSSING_COOLDOWN_FRAMES` of the last counted event in that direction, a new event is suppressed even if a different track's traversal would otherwise qualify. + +**Rationale**: a single person traversing the doorway produces one track with a clear origin and destination — that's one count. Shadows that appear near the line and wobble, or tracks that churn at spawn, lack a firm origin and never count. Counts accumulate as `{entries, exits}` in RAM and reset each hour on report. diff --git a/firmware/lib/cv/cv.cpp b/firmware/lib/cv/cv.cpp index 4e9f002..e125d35 100644 --- a/firmware/lib/cv/cv.cpp +++ b/firmware/lib/cv/cv.cpp @@ -151,36 +151,46 @@ CVResult cv_process(CVState& state, const uint8_t* frame, uint8_t line_pct) { t.id = state.next_id++; t.x = centroids[i].first; t.y = centroids[i].second; + t.spawn_y = t.y; t.above_line = (t.y < line_y); + t.counted = false; t.missed = 0; state.tracks.push_back(t); } - // Line crossing check + // Directional crossing check. A track counts at most once, and only if it + // spawned clearly on one side of the line AND is now clearly on the other. + // This rejects blobs that wobble around the line (shadows, body straddling + // the line, track churn at spawn) — only a true traversal fires an event. for (auto& track : state.tracks) { - if (track.missed > 0) continue; // only check tracks matched this frame - bool now_above = (track.y < line_y); - if (now_above != track.above_line) { - if (!now_above) { - // was above, now below → entry - bool in_cooldown = state.last_entry_frame != 0 && - (state.frame_index - state.last_entry_frame) < CV_CROSSING_COOLDOWN_FRAMES; - if (!in_cooldown) { - state.entries++; - result.entries_delta++; - state.last_entry_frame = state.frame_index; - } - } else { - // was below, now above → exit - bool in_cooldown = state.last_exit_frame != 0 && - (state.frame_index - state.last_exit_frame) < CV_CROSSING_COOLDOWN_FRAMES; - if (!in_cooldown) { - state.exits++; - result.exits_delta++; - state.last_exit_frame = state.frame_index; - } + if (track.missed > 0) continue; // only check tracks matched this frame + if (track.counted) continue; // one track = one trip + + bool spawned_above = track.spawn_y < (line_y - CV_TRAVERSAL_MARGIN_PX); + bool spawned_below = track.spawn_y > (line_y + CV_TRAVERSAL_MARGIN_PX); + bool now_above_firm = track.y < (line_y - CV_TRAVERSAL_MARGIN_PX); + bool now_below_firm = track.y > (line_y + CV_TRAVERSAL_MARGIN_PX); + + if (spawned_above && now_below_firm) { + bool in_cooldown = state.last_entry_frame != 0 && + (state.frame_index - state.last_entry_frame) < CV_CROSSING_COOLDOWN_FRAMES; + if (!in_cooldown) { + state.entries++; + result.entries_delta++; + state.last_entry_frame = state.frame_index; + track.counted = true; + } + } else if (spawned_below && now_above_firm) { + bool in_cooldown = state.last_exit_frame != 0 && + (state.frame_index - state.last_exit_frame) < CV_CROSSING_COOLDOWN_FRAMES; + if (!in_cooldown) { + state.exits++; + result.exits_delta++; + state.last_exit_frame = state.frame_index; + track.counted = true; } } - track.above_line = now_above; + + track.above_line = (track.y < line_y); } return result; diff --git a/firmware/lib/cv/cv.h b/firmware/lib/cv/cv.h index dc3bc6c..6ab19b1 100644 --- a/firmware/lib/cv/cv.h +++ b/firmware/lib/cv/cv.h @@ -12,6 +12,12 @@ static const int CV_MIN_BLOB_PX = 64; static const float CV_MAX_MOVE = 15.0f; static const int CV_MAX_MISSED = 10; +// Directional counting margin: a track only counts if it spawned and is now +// both at least this far from the line (in pixels). Prevents counting blobs +// that wobble around the line or spawn on top of it. Value chosen at ~15% of +// the 96px frame: 14px ≈ the typical torso half-width overhead. +static const float CV_TRAVERSAL_MARGIN_PX = 14.0f; + // Per-direction crossing cooldown. Any same-direction crossing whose frame gap // is strictly less than this value is dropped. At 5 fps, a value of 5 → ≈0.8s // suppression window. Purpose: mask track churn (blob briefly drops below @@ -21,7 +27,9 @@ static const uint32_t CV_CROSSING_COOLDOWN_FRAMES = 5; struct CVTrack { int id; float x, y; + float spawn_y; // y at track creation — used for directional counting bool above_line; + bool counted; // fires at most once per track (one track = one trip) int missed; }; diff --git a/firmware/src/main.cpp b/firmware/src/main.cpp index c8189cd..bf4a1df 100644 --- a/firmware/src/main.cpp +++ b/firmware/src/main.cpp @@ -24,9 +24,22 @@ static DeviceConfig g_cfg; static CVState g_cv; static SemaphoreHandle_t s_cv_mutex = nullptr; -// LED: simple on/off — blink patterns can be added later static void led_set(bool on) { digitalWrite(LED_PIN, on ? HIGH : LOW); } +// Non-blocking-ish detection blink. Saves and restores the current LED state +// so it doesn't clobber upload/no-wifi indicators. Total duration: ~60ms per +// pulse + 80ms gap between pulses. +static void led_blink_pattern(int pulses) { + bool prev = digitalRead(LED_PIN); + for (int i = 0; i < pulses; i++) { + led_set(true); + vTaskDelay(pdMS_TO_TICKS(60)); + led_set(false); + if (i < pulses - 1) vTaskDelay(pdMS_TO_TICKS(80)); + } + led_set(prev); +} + static void check_factory_reset() { if (digitalRead(BUTTON_PIN) != LOW) return; uint32_t held = millis(); @@ -49,6 +62,8 @@ static void task_camera(void*) { if (r.entries_delta) Serial.printf("[CV] entry +%d (total %d)\n", r.entries_delta, g_cv.entries); if (r.exits_delta) Serial.printf("[CV] exit +%d (total %d)\n", r.exits_delta, g_cv.exits); xSemaphoreGive(s_cv_mutex); + if (r.entries_delta) led_blink_pattern(1); + if (r.exits_delta) led_blink_pattern(2); } } vTaskDelay(pdMS_TO_TICKS(CAM_INTERVAL_MS)); diff --git a/firmware/test/test_cv/test_cv.cpp b/firmware/test/test_cv/test_cv.cpp index 20aa7f3..8e664b1 100644 --- a/firmware/test/test_cv/test_cv.cpp +++ b/firmware/test/test_cv/test_cv.cpp @@ -94,19 +94,22 @@ void test_blob_crossing_line_top_to_bottom_is_entry() { CVState state; cv_init(state); - // Line at 50% = y=48; step ≤14px per frame to stay within CV_MAX_MOVE + // Line at 50% = y=48, traversal margin = 14px. Spawn must be y<34, final y>62. + // Step ≤14px per frame to stay within CV_MAX_MOVE. uint8_t bg[CV_PIXELS]; fill_frame(bg, 100); cv_process(state, bg, 50); // init background - // Walk blob from y=20 toward line; crossing occurs at y=48 (above→below) - // Stop at crossing frame and assert its result - int setup[] = {20, 34}; - for (int i = 0; i < 2; i++) { + int setup[] = {20, 34, 48, 62}; // spawn firm above, walk across, not yet firm below + for (int i = 0; i < 4; i++) { uint8_t f[CV_PIXELS]; make_blob_frame(f, 48, setup[i]); cv_process(state, f, 50); } - uint8_t fcross[CV_PIXELS]; make_blob_frame(fcross, 48, 48); + // Still no count — y=62 is not firm below (needs >62) + TEST_ASSERT_EQUAL_INT(0, state.entries); + + // One more step: y=70 is firm below → entry fires now + uint8_t fcross[CV_PIXELS]; make_blob_frame(fcross, 48, 70); CVResult r = cv_process(state, fcross, 50); TEST_ASSERT_EQUAL_INT(1, r.entries_delta); @@ -121,24 +124,77 @@ void test_blob_crossing_line_bottom_to_top_is_exit() { uint8_t bg[CV_PIXELS]; fill_frame(bg, 100); cv_process(state, bg, 50); - // Walk blob from y=76 toward line; crossing occurs at y=34 (below→above) - // Stop at crossing frame and assert its result - int setup[] = {76, 62, 48}; - for (int i = 0; i < 3; i++) { + // Spawn firm below (y=76 > 62), walk toward and across line (y=48), continue + // until firm above (y<34). Each step ≤14px. + int setup[] = {76, 62, 48, 34}; + for (int i = 0; i < 4; i++) { uint8_t f[CV_PIXELS]; make_blob_frame(f, 48, setup[i]); cv_process(state, f, 50); } - uint8_t fcross[CV_PIXELS]; make_blob_frame(fcross, 48, 34); + // y=34 not firm above (needs <34) — no count yet + TEST_ASSERT_EQUAL_INT(0, state.exits); + + uint8_t fcross[CV_PIXELS]; make_blob_frame(fcross, 48, 22); // firm above CVResult r = cv_process(state, fcross, 50); TEST_ASSERT_EQUAL_INT(0, r.entries_delta); TEST_ASSERT_EQUAL_INT(1, r.exits_delta); } +void test_track_spawned_near_line_does_not_count_on_wobble() { + // Simulates a blob that appears right on the line (e.g. shadow or noise) + // and wobbles across it. With directional margin, no count should fire — + // this is the false-positive pattern the feature guards against. + CVState state; + cv_init(state); + + uint8_t bg[CV_PIXELS]; fill_frame(bg, 100); + cv_process(state, bg, 50); + + // Spawn within margin of line (y=44, margin=14 so 44 ∈ [34,62]) + // then wobble above to y=38, below to y=58. Both within margin. + int setup[] = {44, 38, 58, 42, 56}; + for (int i = 0; i < 5; i++) { + uint8_t f[CV_PIXELS]; make_blob_frame(f, 48, setup[i]); + cv_process(state, f, 50); + } + TEST_ASSERT_EQUAL_INT(0, state.entries); + TEST_ASSERT_EQUAL_INT(0, state.exits); +} + +void test_track_counts_at_most_once_even_if_it_wobbles_back() { + // A track that traverses fully should count once. If it then reverses and + // crosses back, the track should NOT fire again — it's already counted. + // (A separate new track on the return trip would count as exit, but while + // the same track persists, it's one trip.) + CVState state; + cv_init(state); + + uint8_t bg[CV_PIXELS]; fill_frame(bg, 100); + cv_process(state, bg, 50); + + // Full traversal top→bottom + int walk_down[] = {20, 34, 48, 62, 70}; + for (int i = 0; i < 5; i++) { + uint8_t f[CV_PIXELS]; make_blob_frame(f, 48, walk_down[i]); + cv_process(state, f, 50); + } + TEST_ASSERT_EQUAL_INT(1, state.entries); + + // Same track reverses back to top. counted=true prevents a second event. + int walk_up[] = {62, 48, 34, 22}; + for (int i = 0; i < 4; i++) { + uint8_t f[CV_PIXELS]; make_blob_frame(f, 48, walk_up[i]); + cv_process(state, f, 50); + } + TEST_ASSERT_EQUAL_INT(1, state.entries); + TEST_ASSERT_EQUAL_INT(0, state.exits); +} + void test_cooldown_suppresses_rapid_re_entry() { - // Set up post-crossing state with a pre-existing track just above the line. - // Force a second above→below crossing within the cooldown window; second - // must not increment entries. Then advance past cooldown; third crossing must. + // Cooldown is a safety net on top of directional counting. Construct two + // DIFFERENT tracks (each counts once on its own) whose crossings happen + // within the cooldown window — the second should still be suppressed. CVState state; cv_init(state); state.bg_valid = true; @@ -147,22 +203,28 @@ void test_cooldown_suppresses_rapid_re_entry() { state.entries = 1; state.last_entry_frame = 100; + // Track at y=50 (just below line), spawn_y=20 (firm above) — a valid trajectory. CVTrack t; - t.id = 1; t.x = 48; t.y = 40; t.above_line = true; t.missed = 0; + t.id = 1; t.x = 48; t.y = 50; t.spawn_y = 20; + t.above_line = false; t.counted = false; t.missed = 0; state.tracks.push_back(t); - // Frame 101: blob at y=52 (below line=48). Track matches (delta=12 < CV_MAX_MOVE). - // Crossing above→below occurs but cooldown (101-100=1 < 5) suppresses it. - uint8_t f1[CV_PIXELS]; make_blob_frame(f1, 48, 52); + // Frame 101: blob at y=64 (delta=14, matches; firm below line+margin=62). + // Would count but cooldown (101-100=1 < 5) suppresses. + uint8_t f1[CV_PIXELS]; make_blob_frame(f1, 48, 64); CVResult r1 = cv_process(state, f1, 50); TEST_ASSERT_EQUAL_INT(0, r1.entries_delta); TEST_ASSERT_EQUAL_INT(1, state.entries); - // Advance past cooldown, reset track above the line, then cross again. - state.frame_index = 200; - state.tracks[0].y = 40; - state.tracks[0].above_line = true; - uint8_t f2[CV_PIXELS]; make_blob_frame(f2, 48, 52); + // Advance past cooldown; reset a fresh track (previous one had counted=true + // set only if it actually counted — cooldown path leaves counted=false so + // we reuse the same track). + state.frame_index = 200; + state.tracks[0].y = 50; + state.tracks[0].spawn_y = 20; + state.tracks[0].counted = false; + state.tracks[0].above_line = false; + uint8_t f2[CV_PIXELS]; make_blob_frame(f2, 48, 64); CVResult r2 = cv_process(state, f2, 50); TEST_ASSERT_EQUAL_INT(1, r2.entries_delta); TEST_ASSERT_EQUAL_INT(2, state.entries); @@ -194,6 +256,8 @@ int main() { RUN_TEST(test_tracking_spawns_track_for_new_blob); RUN_TEST(test_blob_crossing_line_top_to_bottom_is_entry); RUN_TEST(test_blob_crossing_line_bottom_to_top_is_exit); + RUN_TEST(test_track_spawned_near_line_does_not_count_on_wobble); + RUN_TEST(test_track_counts_at_most_once_even_if_it_wobbles_back); RUN_TEST(test_no_crossing_same_side_no_count); RUN_TEST(test_cooldown_suppresses_rapid_re_entry); return UNITY_END();