Improve scanner repeatability guardrails

This commit is contained in:
AzuTear
2026-07-09 10:21:13 +02:00
parent 13fd46c104
commit c10f17b4ef
13 changed files with 356 additions and 39 deletions
+5
View File
@@ -15,6 +15,7 @@ npm run eval:review-candidates # export unconfirmed review samples for human la
npm test # runs the eval gate alongside the rest of the suite
npm run scan:assessment:test # verifies quality-first scan ranking logic
npm run scan:iterate:compare:validated:wait # 20-artifact live comparison
npm run scan:repeatability:wait # 20/45/100 current-engine repeatability
npm run scan:goal:compare:validated:wait # final 100-artifact live comparison
```
@@ -22,6 +23,10 @@ The report prints exact-match rate, overall field accuracy, a per-field
breakdown (critical fields marked with `*`), and every failing case with an
`expected "..." got "..."` diff.
Repeatability runs are single-engine evidence. Use them to prove that the
visible-inventory scanner stays stable across later sessions, but keep
current-vs-IK claims on `scan:goal:compare:validated:*`.
## How it works
- `src/eval/ocrEvalHarness.ts` - pure metric functions. `runOcrEval(cases)`