Improve scanner repeatability guardrails
This commit is contained in:
@@ -144,6 +144,37 @@ Current status after the 2026-07-09 merge to `main`:
|
||||
end-to-end capture roundtrip time. Use `averageCaptureRoundTripMs` and
|
||||
`averageCaptureRoundTripOverheadMs` in the next `limit=20` live iteration to
|
||||
decide whether the next cut belongs in native capture transport or OCR.
|
||||
- **Repeatability and capture-overhead guardrails (2026-07-09)** -
|
||||
`scripts/live-soak.ps1` now writes capture roundtrip and roundtrip-overhead
|
||||
timing into `scan-performance-assessment.json`. The assessment validator can
|
||||
enforce optional speed budgets with `--max-active-average-ms` and
|
||||
`--max-capture-roundtrip-overhead-ms`, and `npm run scan:repeatability:wait`
|
||||
runs 20/45/100 current-engine passes as repeatability evidence without
|
||||
presenting them as an IK comparison. `-RepeatabilityRun` now sets those limits
|
||||
inside PowerShell, and the script refuses unsafe limits above 1800 so npm/cmd
|
||||
argument parsing cannot accidentally turn `20,45,100` into one oversized run.
|
||||
- **Distinctive partial piece recovery (2026-07-09)** - a live repeatability run
|
||||
exposed four identical OCR misses where the piece name was read as
|
||||
`Wontiroms Creation pan`. The parser now derives a piece only when a long OCR
|
||||
fragment uniquely matches exactly one known artifact piece. This recovered the
|
||||
local case as `Sharpness That Ceased Upon Wondrous Creation` /
|
||||
`Disenchantment in Deep Shadow` without adding a broad fuzzy exception.
|
||||
- **Repeatability live pass after parser fix (2026-07-09)** -
|
||||
`outputs/live-soak/2026-07-09T09-29-11/scan-performance-assessment.json`
|
||||
captured a clean current-engine 20-artifact run: `20/20` parsed, `0` review,
|
||||
`0` misses, `336 ms/artifact` active average, `318 ms` average capture
|
||||
roundtrip, and `138 ms` average roundtrip overhead. The strict 3 artifacts per
|
||||
second budget still failed by 3 ms (`336 ms` vs `333 ms`).
|
||||
- **3/s follow-up experiments (2026-07-09)** - tested and rejected several
|
||||
shortcut-style optimizations because live runs got slower or added risk:
|
||||
skipping Paimon-menu analysis, skipping lock-state as a production shortcut,
|
||||
reducing the artifact-level crop scale, and raising the OCR worker pool to 6.
|
||||
The kept low-risk changes are fast-profile OCR crop priority and avoiding a
|
||||
duplicate DataURL string when the native helper already returns Base64. A
|
||||
follow-up clean 20-artifact run after payload cleanup reached `351 ms/artifact`,
|
||||
`331 ms` capture roundtrip, and `146 ms` roundtrip overhead, so the next
|
||||
credible 3/s work is native capture transport/roundtrip reduction, not UI
|
||||
recommendation work.
|
||||
- **3/s live attempt (2026-07-08)** - the missing-detail-preview review trigger
|
||||
was fixed and tested. The best clean 20-artifact run reached `7285 ms`
|
||||
(`364 ms/artifact`, about `2.75 artifacts/second`) with 0 review and 0 misses.
|
||||
@@ -218,14 +249,17 @@ resolution or without UI work best tested live:
|
||||
substat OCR time, or a larger capture pipeline change that avoids full-frame
|
||||
PNG/Base64 transport without hurting safety checks. The target remains
|
||||
`<= 6667 ms` elapsed for 20 parsed artifacts with 0 misses and no silent OCR
|
||||
review regression.
|
||||
review regression. The latest clean 20-artifact repeatability run reached
|
||||
`336 ms/artifact`, so 3/s remains close but unproven.
|
||||
|
||||
6. **Broader scan soak test** — direct-GDI current-engine runs now passed at
|
||||
`20/20`, `45/45`, and `100/100` with 0 misses. Continue with repeat runs if
|
||||
duplicate rate needs tuning.
|
||||
`20/20`, `45/45`, and `100/100` with 0 misses. Continue with
|
||||
`npm run scan:repeatability:wait` in later sessions to check duplicate rate,
|
||||
scroll behavior, and capture roundtrip timing without changing defaults.
|
||||
7. **Repeatability pass** — repeat the qualified current-vs-IK-traineddata run
|
||||
in a later live session before making major OCR-engine defaults or speed
|
||||
claims beyond this environment.
|
||||
claims beyond this environment. Current-engine-only repeatability is useful
|
||||
evidence, but it is not an IK parity claim.
|
||||
|
||||
Visible-page limits up to 20, scroll/page-transition limit 45, the final
|
||||
100-artifact current-vs-IK-traineddata comparison, equipped footer live smokes,
|
||||
|
||||
Reference in New Issue
Block a user