feat(scanner): add native artifact pipeline
Add native IK-style capture processing, Artifact Inventory, explicit promotion and single-result review. Confirm the three live OCR corrections in the eval corpus and preserve extraction/value separation.
This commit is contained in:
AzuTear
+71
-33
@@ -16,8 +16,9 @@ This document contains Architecture Decision Records.
|
||||
| ADR-008 | Replace the PowerShell input/capture helper with a C# sidecar | Accepted | 2026-07-05 |
|
||||
| ADR-009 | Resolution-anchored layout profiles and OCR preprocessing over color detection | Accepted | 2026-07-05 |
|
||||
| ADR-010 | Elevated dev runner and bounded live automation probes | Accepted | 2026-07-07 |
|
||||
| ADR-011 | Quality-gated Inventory Kamera comparison before OCR default changes | Accepted | 2026-07-07 |
|
||||
| ADR-011 | Retire alternate OCR comparison paths after current scanner baseline | Superseded | 2026-07-09 |
|
||||
| ADR-012 | Separate live scan results, artifact inventory, and value evaluation | Accepted | 2026-07-09 |
|
||||
| ADR-013 | Move high-speed scanning into the native helper and vendor IK inventorylists | Accepted | 2026-07-09 |
|
||||
|
||||
## ADR-001: Build A Local Electron App First
|
||||
|
||||
@@ -47,7 +48,8 @@ Accepted
|
||||
|
||||
### Context
|
||||
|
||||
The user wants an app that works without Inventory Kamera, Genshin Optimizer, Enka, or HoYoLAB as core dependencies.
|
||||
The user wants an app that works without optimizer imports, external scanner
|
||||
tools, Enka, or HoYoLAB as core dependencies.
|
||||
|
||||
### Decision
|
||||
|
||||
@@ -184,9 +186,8 @@ Accepted
|
||||
The persistent PowerShell helper (ADR-006) still carries Windows PowerShell 5.1
|
||||
quirks (the `Marshal::SizeOf` interop bug), compiles Win32 interop at startup,
|
||||
and captures each frame by writing a PNG to the temp directory and reading it
|
||||
back. Inventory Kamera - the proven reference for automated Genshin scanning -
|
||||
uses a C#/.NET stack with InputSimulator (SendInput) and direct GDI/BitBlt
|
||||
capture.
|
||||
back. A C#/.NET sidecar with SendInput and direct GDI/BitBlt capture is a
|
||||
cleaner fit for the validated Windows automation path.
|
||||
|
||||
### Decision
|
||||
|
||||
@@ -215,10 +216,9 @@ Accepted
|
||||
The current pipeline finds the artifact detail panel with hardcoded orange/green
|
||||
color thresholds (`inferDetailRect`) and then crops fixed percentages of that
|
||||
guessed rectangle. This is brittle against HDR, color profiles, UI scale, aspect
|
||||
ratio, and game UI updates. Inventory Kamera instead requires borderless 16:9 and
|
||||
scales fixed crop coordinates from a reference resolution, then feeds Tesseract
|
||||
preprocessed (grayscale, upscaled, thresholded) crops - which is why general
|
||||
Tesseract is accurate enough for them.
|
||||
ratio, and game UI updates. A stricter borderless 16:9 profile with fixed crop
|
||||
coordinates from a reference resolution is easier to validate and reproduce than
|
||||
per-frame color hunting.
|
||||
|
||||
### Decision
|
||||
|
||||
@@ -289,29 +289,26 @@ Document the workflow in [AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
|
||||
reads, hooks, injection, game-file modification, deleting, feeding, enhancing,
|
||||
locking/unlocking, or spending resources.
|
||||
|
||||
## ADR-011: Quality-Gated Inventory Kamera Comparison Before OCR Default Changes
|
||||
## ADR-011: Retire Alternate OCR Comparison Paths After Current Scanner Baseline
|
||||
|
||||
### Status
|
||||
|
||||
Accepted
|
||||
Superseded
|
||||
|
||||
### Context
|
||||
|
||||
The product target is not merely to click through 100 artifacts quickly. It is
|
||||
to scan the first 100 artifacts with accuracy at least as good as Inventory
|
||||
Kamera and speed equal to or better than Inventory Kamera. A faster scan that
|
||||
creates too many misses, review samples, or false positives is worse than a
|
||||
slower qualified run.
|
||||
|
||||
The app can now compare the current OCR path with Inventory Kamera's
|
||||
`genshin_fast_09_04_21.traineddata` through the same visible crop set. It also
|
||||
has hot-loop timing fields for capture, OCR, card readiness, scroll readiness,
|
||||
active scan time, and projected 100-artifact time.
|
||||
The project previously carried alternate OCR-engine and comparison paths to
|
||||
decide whether the current scanner should change OCR defaults. The current
|
||||
visible-inventory path has now produced clean 100-artifact evidence, and the
|
||||
next product phase is result clarity, artifact inventory, detail evaluation, and
|
||||
corpus growth. Keeping an alternate engine path increases app size and
|
||||
maintenance surface without serving the current user workflow.
|
||||
|
||||
### Decision
|
||||
|
||||
Use a quality-gated live soak and benchmark before changing the default OCR
|
||||
engine or claiming IK parity. A qualified scan result must:
|
||||
Retire the alternate OCR-engine and current-vs-reference comparison paths. Keep a
|
||||
single current OCR/capture path and preserve the quality-gated live soak
|
||||
assessment. A qualified scan result must:
|
||||
|
||||
- finish cleanly,
|
||||
- parse at least the requested count,
|
||||
@@ -324,22 +321,18 @@ engine or claiming IK parity. A qualified scan result must:
|
||||
`scripts/live-soak.ps1` writes the evidence bundle and
|
||||
`scan-performance-assessment.json`. `npm run scan:assessment:test` verifies that
|
||||
the ranking logic rejects fast but low-quality synthetic runs without needing
|
||||
Genshin. The assessment also records `goal100Decision` and
|
||||
`goal100.comparisonComplete`; IK-target claims require a qualified 100-artifact
|
||||
winner and a complete `current` vs. `ik-traineddata` comparison.
|
||||
`npm run scan:assessment:validate -- --summary` prints the assessment path and
|
||||
`createdAt` timestamp so reports can cite the exact evidence file.
|
||||
Genshin. `npm run scan:assessment:validate -- --summary` prints the assessment
|
||||
path and `createdAt` timestamp so reports can cite the exact evidence file.
|
||||
|
||||
### Consequences
|
||||
|
||||
- Speed claims cannot be based on click count or elapsed time alone.
|
||||
- A new OCR engine cannot become the default just because it is theoretically
|
||||
closer to IK; it must win the same-capture benchmark and a qualified live run.
|
||||
- A new OCR engine is out of scope until scanner UX, inventory, detail
|
||||
evaluation, and corpus growth justify reopening that work.
|
||||
- Stale elevated Electron instances are treated as invalid evidence, not as a
|
||||
harmless warning.
|
||||
- The validated `:wait` scan scripts are acceptable for manual post-UAC startup;
|
||||
non-waiting scripts remain useful when automation should fail fast.
|
||||
- The goal remains open until the 100-artifact qualified comparison is captured.
|
||||
|
||||
## ADR-012: Separate Live Scan Results, Artifact Inventory, And Value Evaluation
|
||||
|
||||
@@ -395,5 +388,50 @@ Adopt a split scan/result/inventory model:
|
||||
- Inventory and detail views become the natural place for richer analysis.
|
||||
- Recommendation work has a cleaner dependency chain: trusted scans, compact
|
||||
results, artifact inventory, detail evaluation, then build recommendations.
|
||||
- Speed work remains possible, but it is secondary unless measured timings show
|
||||
a real regression.
|
||||
- Renderer-loop speed work remains secondary unless measured timings show a real
|
||||
regression; native capture speed work is handled by ADR-013.
|
||||
|
||||
## ADR-013: Move High-Speed Scanning Into The Native Helper And Vendor IK Inventorylists
|
||||
|
||||
### Status
|
||||
|
||||
Accepted
|
||||
|
||||
### Context
|
||||
|
||||
Inventory Kamera is fast because the game-control loop is native and the UI does
|
||||
not perform per-artifact work. The Electron renderer path paid for focus,
|
||||
capture, OCR, parse, persistence, status updates, and UI state inside one loop.
|
||||
The user also has Inventory Kamera 1.4.4 locally, including `inventorylists`
|
||||
that can be used 1:1 as scanner reference data.
|
||||
|
||||
### Decision
|
||||
|
||||
Use the C# input helper as the high-speed scanner service. Electron starts,
|
||||
stops, packages, and reports status. React remains a visual control surface. The
|
||||
IK `inventorylists` are copied into `data/ik-inventorylists` and packaged as
|
||||
extra resources. The first native milestone captures artifact detail card crops
|
||||
quickly; OCR, parsing, GOOD persistence, and evaluation are separate downstream
|
||||
steps. Each native run writes `manifest.json`, `capture-jobs.jsonl`, and
|
||||
`status.json`; downstream workers must consume those files instead of asking the
|
||||
renderer to do per-artifact scanner work.
|
||||
|
||||
The first downstream worker is a post-capture processor that reads
|
||||
`capture-jobs.jsonl` and writes both `scan-results.json` and
|
||||
`processing-report.json`. `scan-results.json` is the durable result contract:
|
||||
one entry per captured artifact card with extraction status, artifact identity
|
||||
when parsed, review state, and deferred value status. `processing-report.json`
|
||||
remains the diagnostic OCR/parse report. The worker OCRs/parses after capture
|
||||
and does not persist by default; DB writes require explicit opt-in after native
|
||||
crop OCR is validated with live evidence.
|
||||
|
||||
### Consequences
|
||||
|
||||
- Capture throughput can be optimized without renderer roundtrips per artifact.
|
||||
- The scanner data source now matches IK's artifact names and set/piece keys.
|
||||
Other vendored IK lists are data only until those values are actually scanned.
|
||||
- Evaluation can remain deferred without blocking capture speed.
|
||||
- Live safety stays read-only: no memory reads, hooks, injection, game-file
|
||||
modification, deleting, feeding, enhancing, or spending resources.
|
||||
- Non-16:9 layouts currently block in native preflight until layout support is
|
||||
expanded.
|
||||
|
||||
Reference in New Issue
Block a user