Prepare scanner branch for merge

This commit is contained in:
AzuTear
2026-07-09 08:44:50 +02:00
parent f791d1464c
commit 8b73c01e46
69 changed files with 6700 additions and 3773 deletions
+110 -25
View File
@@ -14,12 +14,19 @@ Current status:
- The scanner architecture now follows the relevant Inventory Kamera model:
32 artifact targets per page, lookup-derived fields, fast artifact OCR profile,
short readiness gates, page-overlap planning, and queued OCR/store work.
direct detail-fingerprint verification from the OCR capture, page-overlap
planning, and batched store work.
- The live runner can compare `current` and `ik-traineddata` engines and rejects
runs that are fast but fail miss/review quality thresholds.
- The final 100-artifact IK target is not proven yet. The dev-control port is
currently owned by an older elevated Electron build, and the runner correctly
refuses stale timing evidence until the app is restarted with UAC approval.
- The final current-vs-IK-traineddata 100-artifact comparison is now proven for
the current live environment. On 2026-07-08,
`npm run scan:goal:compare:validated` passed with evidence at
`outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json`.
`current` won with `100/100` parsed, `0` review, `0` misses, and
`378 ms/artifact` active average. `ik-traineddata` was rejected at 100 because
it parsed `97/100`, had `5` review and `3` misses. The next optional speed
target remains `3 artifacts/second`, which means `333 ms/artifact` or faster
on clean 20-artifact iterations.
## Done (implemented, unit-tested, build green)
@@ -55,13 +62,13 @@ Current status:
2 review samples, and 0 misses.
- **Auto-scan OCR performance pass** - auto-scan captures now use an artifact
OCR mode that skips inventory-count OCR on each tile, keeps equipped-character
OCR, raises the substat crop to catch artifact level, stores automatic review
samples without full-screen/inventory screenshots, reads only the tail of large
OCR on the real artifact-read captures, raises the substat crop to catch
artifact level, stores automatic review samples without full-screen/inventory screenshots, reads only the tail of large
JSONL files, avoids review noise when only level/equipped is missing, starts
the scan with an OCR-free preflight capture, skips exact visual duplicates
before OCR, prevents repeated startup review reprocessing, omits full-frame
and inventory-preview Base64 payloads from tile captures, and applies
crop-specific Tesseract page-segmentation/whitelist parameters.
the scan with an OCR-free preflight capture, prevents repeated startup review
reprocessing, omits full-frame and inventory-preview Base64 payloads from tile
captures, and applies crop-specific Tesseract page-segmentation/whitelist
parameters.
- **Visible-page live soak helper** - `scripts/live-soak.ps1` now drives the
dev-control health/status, smart-capture, probe-click, bounded scan, and
review-tail endpoints and writes evidence to `outputs/live-soak/`. On
@@ -105,10 +112,77 @@ Current status:
summaries, groups results by limit, identifies timing bottlenecks, and rejects
winners that miss the requested count, exceed 2% misses, or exceed 15% review.
`npm run scan:assessment:test` verifies this ranking logic without Genshin.
The assessment also reports `goal100Decision` and
`goal100.comparisonComplete`, so a single-engine 100-artifact run cannot be
misread as the final IK comparison. Use
`npm run scan:iterate:compare:validated:wait` for the 20-artifact live
iteration and `npm run scan:goal:compare:validated:wait` for the final proof
when starting directly after UAC. The validator `--summary` output includes
the assessment path and timestamp for reporting.
- **State-polled guided entry** - the guided auto-entry waits for Inventory,
artifact grid, and first detail card evidence instead of sleeping the full
fixed delay every time. OCR/review/store work still starts only after artifact
detail preflight passes.
- **Hot-loop speed pass (2026-07-08)** - the scan loop no longer performs a
separate card-ready capture before OCR; the artifact OCR capture itself
verifies detail-fingerprint change. Routine click diagnostics and scan stat
publishes are throttled. Auto-scan artifact captures no longer update the
full preview/topbar UI on every tile. Store writes can be batched so the scan
path avoids per-artifact save/reload churn. Auto-scan artifact captures now
use a direct GDI hot path and skip Electron `desktopCapturer.getSources()` in
the per-artifact loop.
- **3/s instrumentation pass (2026-07-08)** - artifact hot-path captures omit
the detail-preview payload, and scan stats now split inner capture time from
end-to-end capture roundtrip time. Use `averageCaptureRoundTripMs` and
`averageCaptureRoundTripOverheadMs` in the next `limit=20` live iteration to
decide whether the next cut belongs in native capture transport or OCR.
- **3/s live attempt (2026-07-08)** - the missing-detail-preview review trigger
was fixed and tested. The best clean 20-artifact run reached `7285 ms`
(`364 ms/artifact`, about `2.75 artifacts/second`) with 0 review and 0 misses.
The final stable run on `2026-07-08-direct-gdi-reviewfix` completed `20/20`
with 0 review, 0 misses, and `7973 ms` elapsed (`399 ms/artifact`). Detail
region capture, 5 OCR workers, DataURL buffer decode, and substat
`PSM.SINGLE_COLUMN` were tested and rejected as slower.
- **Review-to-eval loop (2026-07-08)** - `npm run eval:review-candidates`
exports the local review queue into `outputs/review-eval-candidates/` as a
human-labeling worklist. The exporter deduplicates samples, surfaces complete
fast-field captures first, marks stale captures, and now surfaces equipped
footer OCR plus `locked=true/false` payload counts for the next ownership/lock
validation pass. Its output is deliberately
ignored by Git and must not be treated as ground truth until fields are
confirmed against the real artifact. Confirmed review labels now have a
dedicated corpus file, `src/eval/corpus/confirmedReviewCorpus.ts`, with tests
that reject duplicate ids, empty labels, and unconfirmed entries. The helper
`npm run eval:prepare-confirmed` generates a paste-ready confirmed-case
snippet only when explicit expected labels are provided.
- **Prepared ownership/learning loop (2026-07-08)** - fast auto-scan no longer
drops the artifact footer by profile alone; it omits footer OCR only when the
capture option explicitly requests that or when the footer marker is absent.
Parser tests cover noisy equipped names, split `Equipped:`/name footers, and
one-letter OCR fragments that must stay `Not detected`. Scanner learning now
persists text replacements, field aliases, constrained fixes, crop adjustment
proposals, and UI-profile adjustment proposals instead of truncating everything
back to text replacements.
- **Visible-inventory merge guard (2026-07-09)** - the normal guided Auto-Scan
start no longer falls back into `auto-entry` when the artifact detail card is
missing. It now blocks and asks the operator to open the Artifact inventory
with a visible detail card. The explicit `auto-entry`, `direct-inventory`, and
`paimon-menu` Dev-Control modes remain available for targeted experiments, but
they are not the merge-ready default path.
- **Ownership live smoke (2026-07-09)** - live artifact detail capture parsed
and stored an equipped footer as `equipped: "Citlali"` and the grey lock state
as `locked: false`. A same-session visible-inventory run with
`/scanner/start?entry=visible-inventory&limit=20&engine=current` completed
`20/20` verified and parsed, `19` stored, `1` duplicate, `0` review, and
`0` misses in `8047 ms` elapsed (`402 ms/artifact`).
- **Locked artifact live proof (2026-07-09)** - a visibly locked artifact was
selected through a read-only inventory tile click. Smart Capture reported
`locked: true` with `lockSignal.ratio: 0.14797913950456323` over threshold
`0.06`, and `/scanner/start?entry=visible-inventory&limit=1&engine=current`
persisted the same artifact with `equipped: "Citlali"` and `locked: true`.
Lock detection now decodes the lock crop PNG before measuring active lock
pixels because Electron's native bitmap channel order was ambiguous in live
captures.
## Remaining — needs the live environment or a UI pass
@@ -122,23 +196,34 @@ resolution or without UI work best tested live:
The current benchmark can use IK-traineddata through Tesseract.js; native
Tesseract integration remains the next implementation step before any engine
default changes.
3. **Validate guided entry live** from world, visible inventory, and Paimon/menu
states with limits 2, 20, and 45. Confirm the artifact-tab coordinate in the
user's current 16:9 layout and keep `visible-inventory` as fallback if the
menu path is blocked.
4. **Validate locked=true** against a known locked artifact — unlocked/grey lock
was live-checked; a gold locked icon still needs a positive sample.
3. **Validate explicit entry modes separately** from world, direct inventory,
and Paimon/menu states with low limits only. These are now Dev-Control
experiments, not the normal merge path; the normal Auto-Scan button blocks
unless the visible artifact detail card is already present.
4. **Repeat locked=true on another page/session** if lock behavior changes.
The first positive live proof passed on 2026-07-09, including store
persistence. Further repeats are useful for confidence but no longer block
the merge.
5. **Broader scan soak test** — after the bounded two-item live scan passed,
the next automation validation should increase the limit gradually and watch
for repeated pages, scroll behavior, duplicate handling, and OCR review rate.
6. **100-artifact IK comparison** — after `/health.appBuild.signature` matches
current source, run `npm run scan:goal:compare` and compare qualified
100-artifact results.
5. **3 artifacts/second iteration** - not reached yet. The next credible path is
either native Tesseract/IK-traineddata integration that materially reduces
substat OCR time, or a larger capture pipeline change that avoids full-frame
PNG/Base64 transport without hurting safety checks. The target remains
`<= 6667 ms` elapsed for 20 parsed artifacts with 0 misses and no silent OCR
review regression.
Visible-page limits up to 20 and a scroll/page-transition limit of 45 have
passed. The remaining soak work is now OCR accuracy, review-rate reduction, and
larger runs after the review corpus has grown.
6. **Broader scan soak test** — direct-GDI current-engine runs now passed at
`20/20`, `45/45`, and `100/100` with 0 misses. Continue with repeat runs if
duplicate rate needs tuning.
7. **Repeatability pass** — repeat the qualified current-vs-IK-traineddata run
in a later live session before making major OCR-engine defaults or speed
claims beyond this environment.
Visible-page limits up to 20, scroll/page-transition limit 45, and the final
100-artifact current-vs-IK-traineddata comparison have passed for the current
environment. Remaining soak work is repeatability, OCR corpus growth, equipped
footer confirmation repeats, locked artifact repeats, and optional 3 artifacts/second
speed work.
## Grow the eval corpus