Improve IK-style artifact scanner pipeline

This commit is contained in:
AzuTear
2026-07-07 22:02:24 +02:00
parent 8ebbe91c39
commit f791d1464c
70 changed files with 7408 additions and 445 deletions
+19 -1
View File
@@ -55,6 +55,9 @@ flowchart LR
| `src/App.tsx` | Main app shell, scan view, triage view, build view, overlay preview |
| `src/lib/artifactOcrParser.ts` | Converts OCR output into a parsed artifact candidate with confidence and notes |
| `src/lib/fuzzyMatch.ts` | Generic fuzzy string matching for OCR text against known game data |
| `src/lib/genshinLookup.ts` | Pure lookup and validation API for generated Genshin data |
| `src/lib/autoScanEntry.ts` | Pure entry-mode planning and auto-scan preflight validation |
| `src/lib/cardReadyGate.ts` | Detail/page fingerprint readiness gate for scan timing |
| `src/lib/scoring.ts` | Recommendation and build scoring logic |
| `src/lib/demoData.ts` | Temporary local demo snapshot |
| `src/data/genshinGameData.json` | Generated local dictionary of characters, artifact sets, slots, and stats |
@@ -131,6 +134,17 @@ sequenceDiagram
[AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
- Click verification: after each click the parsed detail-panel signature should change. An unchanged signature is a soft miss (it can also mean two OCR-identical neighbor pieces, common among +0 artifacts), so it is retried once with a small offset, logged with the stuck artifact name, and then skipped - never fatal on its own. The scan aborts only when the first ~6 clicks of page 1 produce nothing new (diagnosis hint: elevated Genshin blocks SendInput via UIPI, or grid coordinates are wrong) or a later page yields zero new artifacts.
- Scan stats separate clicked (click attempts), parsed (readable captures), stored (persisted), review (review samples), duplicates, and misses, so "scanned" cannot be mistaken for "successfully read".
- Artifact grid automation uses Inventory Kamera's 32-target full-page model
(`8 x 4` safe click targets). The apparent lower fifth row sits in the
bottom control band on 16:9 captures and is not clicked automatically.
- Guided auto-entry is state gated. The normal scan button first performs a
lightweight no-OCR preflight; OCR/store/review work starts only after the
artifact inventory grid and artifact detail card are visually confirmed.
- The entry path tries direct inventory (`B`) first and uses the IK-style
ESC/B fallback only when direct entry does not reach an artifact detail card.
- Card and page waits are fingerprint based. The scanner can proceed as soon as
the expected visual state changes and stabilizes, while still accepting IK-like
200 ms item and 100 ms scroll readiness points.
- Scrolling sends one wheel notch per grid row with the cursor anchored over the inventory (assumption: roughly one row per notch; overlap is absorbed by dedupe, and a page without new artifacts stops the scan).
- The scan never deletes, enhances, feeds, locks, or spends anything; it only selects tiles to read them.
@@ -156,4 +170,8 @@ Parsed artifacts from both modes are persisted into `artifact-store.json` keyed
## Performance
Current OCR is prototype-grade and may be slower than the target scanner. Two batch-scan bottlenecks were removed: input/capture no longer spawn a PowerShell process (and recompile Win32 interop) per action, and the Tesseract worker is created once and reused across captures. The eventual batch scanner should still move expensive capture/OCR/build work into workers or a native sidecar.
Current OCR is still measured against the IK target rather than assumed good.
The app keeps a Tesseract.js worker pool, can use the Inventory-Kamera
`genshin_fast_09_04_21.traineddata` path for comparison, and reports capture,
OCR, card-ready, scroll-ready, active-scan, and projected-100 timings. A default
engine change requires a same-capture benchmark and a qualified live soak result.