feat(scanner): add native artifact pipeline

Add native IK-style capture processing, Artifact Inventory, explicit promotion and single-result review. Confirm the three live OCR corrections in the eval corpus and preserve extraction/value separation.
This commit is contained in:
AzuTear
2026-07-09 23:30:42 +02:00
parent 28d60eb915
commit 639b0b7f59
92 changed files with 13606 additions and 1703 deletions
+81 -37
View File
@@ -3,8 +3,7 @@
This document is the source of truth for project intent, scope, runtime facts, and operational expectations.
For implementation structure, see [ARCHITECTURE.md](ARCHITECTURE.md). For engineering standards, see [CONVENTIONS.md](CONVENTIONS.md).
For the latest Inventory-Kamera comparison work, see
[scanner-ik-progress-report.md](scanner-ik-progress-report.md).
For the current app status, see [CURRENT_STATUS.md](CURRENT_STATUS.md).
For the next scan-result and artifact-inventory product phase, see
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md).
For the 2026-07-09 scanner merge evidence, see [MERGE_READINESS.md](MERGE_READINESS.md).
@@ -15,7 +14,7 @@ For Gitea push/authentication setup, see [GITEA_AUTH.md](GITEA_AUTH.md).
| Field | Value |
| --- | --- |
| Project name | Genshin Artifact Assistant |
| Status | Scanner baseline merged to `main`; scan result rail, artifact inventory, extraction quality, and corpus growth are next |
| Status | Artifact-first scanner baseline with native IK-style 20/50/100 scale evidence, explicit selected promotion, review/edit/approve, scan result rail, and Artifact-only Inventory pipeline; value scoring and later-session repeatability are next |
| Platform | Windows desktop |
| Target users | Genshin Impact players who want artifact decisions without complex optimizer setup |
| Runtime | Electron app with React UI and TypeScript |
@@ -35,7 +34,8 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
- Present finished scan results as a compact artifact list instead of a debug-heavy live stats surface.
- Keep extraction confidence separate from artifact value so uncertain OCR becomes review, not a misleading low score.
- Provide a browsable local artifact inventory with detail views before promoting broader recommendations.
- Keep the app offline-first and usable without Genshin Optimizer, Inventory Kamera, Enka, or HoYoLAB.
- Keep the app offline-first and usable without optimizer imports, Enka, HoYoLAB,
or any external scanner as a core dependency.
- Re-introduce recommendations only after the scanner base is trustworthy.
## Non-Goals
@@ -60,8 +60,8 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
| FR-009 | Provide account-level artifact triage after scanner trust is acceptable. | Should | Pending |
| FR-010 | Provide 1-3 simple build suggestions per character from owned artifacts after scanner trust is acceptable. | Should | Pending |
| FR-011 | Farming overlay for reward scans. | Later | Prototype shell |
| FR-012 | Show active scan results as a minimal right-side rail with artifact number, name or compact fallback, value score, and status pill. | Should | Planned |
| FR-013 | Provide a scanned artifact inventory view with compact score pills, filters, sorting, and click-through detail. | Should | Planned |
| FR-012 | Show active scan results as a minimal right-side rail with artifact number, name or compact fallback, value score, and status pill. | Should | Partial foundation |
| FR-013 | Provide a scanned artifact inventory view with compact score pills, filters, sorting, and click-through detail. | Should | Partial foundation |
| FR-014 | Provide artifact detail evaluation with screenshot/crops, parsed fields, OCR confidence, value reasons, and optional upgrade projection. | Should | Planned |
## Non-Functional Requirements
@@ -70,7 +70,7 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
| --- | --- | --- |
| Safety | Never perform irreversible in-game actions. | Code review and manual test |
| Performance | Single artifact read should feel interactive and batch scan should not stall on false progress. | Capture latency monitored manually; auto-scan stops on blocked verification |
| IK target | First 100 artifacts should scan with accuracy at least as good as Inventory Kamera and equal or better speed. | `npm run scan:goal:compare:validated` or `npm run scan:goal:compare:validated:wait` quality-gated report |
| Performance | A 100-artifact visible-inventory run should finish cleanly with low review/miss rates and report timing evidence. | `npm run scan:goal:validated` or `npm run scan:goal:validated:wait` quality-gated report |
| Privacy | Captures and parsed data stay local by default. | No remote upload in scanner path |
| Reliability | Uncertain OCR must be visible to the user. | Confidence and details view |
| Score integrity | Extraction confidence and artifact value are separate concepts. | Review state can block or qualify a value score |
@@ -108,10 +108,12 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
- A local canonical data package already exists in `src/data/genshinGameData.json`, generated from `genshin-db`.
- The parser already uses known sets, pieces, slots, stat aliases, set aliases, character aliases, and derived slot/set mapping.
- Review samples, learned replacements, parser notes, and stored artifacts already persist locally.
- The scan surface has an initial recent-results rail backed by the newest
stored artifacts.
- The auto-scan loop is no longer a naive click spammer: it has preflight, verification, miss handling, page fingerprinting, and stop conditions.
- The scanner now has an Inventory-Kamera comparison path: 32 safe artifact
targets per page, lookup-derived fields, fast OCR crop profile, current vs.
IK-traineddata benchmark endpoint, and a quality-gated live soak runner.
- The scanner now has a validated visible-inventory path: 32 safe artifact
targets per page, lookup-derived fields, fast OCR crop profile, and a
quality-gated live soak runner.
- Elevated live automation is validated in the current dev environment:
`/automation/probe-click?index=1` changed the selected artifact and
`/scanner/start?limit=2` completed with 2/2 verified reads and 0 misses.
@@ -128,42 +130,71 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
- Review samples can now be exported with `npm run eval:review-candidates` into
a Git-ignored human-labeling worklist. This is the next quality phase before
adding more OCR corpus cases or trusting review queue data as labels.
- Native IK-style Artifact capture is wired through the C# helper. It writes
card crops and run artifacts for downstream OCR/parse processing instead of
making React do per-artifact work in the hot capture loop.
- The native post-capture processor can write `scan-results.json` and
`processing-report.json`, preserve parser field confidence, match Artifact
results against IK artifact set/piece/slot data, and keep store persistence
opt-in.
- The Inventory view now has an Artifact-only pipeline surface for scope,
native capture, OCR queue, review gate, promotion, and evidence. Native rows
expose crop previews, IK/GOOD metadata, dry-run promotion state, and a
`Naechster Schritt` card.
- The active UI intentionally hides weapon, material, and character-detail IK
catalog coverage until those values are actually scanned.
### What is still structurally weak
- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
- Broader scan soak testing has reached clean 20-, 45-, and 100-artifact runs
with 0 misses on the current engine. The current-vs-IK-traineddata comparison
is now captured; `current` won the qualified 100-artifact comparison on
2026-07-08.
with 0 misses on the current engine. The current 2026-07-09 100-artifact run
completed `100/100` verified and parsed with `0` review and `0` misses.
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
- The scan page is cleaner than before, but it still needs the next minimalist
result-rail pass so the main flow shows preview plus completed artifact
outcomes instead of live diagnostic/stat content.
- The scan page now has a minimalist recent-results rail, and the Inventory view
can inspect native Artifact results, crop previews, IK match state, promotion
dry-runs, and pipeline risk/status. Value scoring remains incomplete;
single-result review/edit/approve is implemented and batch review is
intentionally unavailable.
- The new native IK-style path has same-session 20/50/100 live scale evidence
with complete capture/result counts and safe Review gating. Later-session
repeatability and packaged behavior remain open.
- Store promotion from native `scan-results.json` now supports one selected,
confirmed result at a time with main-process revalidation and a durable log.
Batch promotion intentionally remains unavailable.
- Native Review results can be corrected and approved or rejected one at a
time. Approval revalidates IK identity, canonical main values, and legal
substat rolls, then feeds the existing review-to-eval candidate pipeline.
- Recommendations and build logic exist, but artifact inventory, detail review,
and value scoring should land first so recommendations have trustworthy inputs.
- The latest source has completed the final current-vs-IK-traineddata live
comparison for this environment. Repeatability and 3 artifacts/second are
still open.
- Repeatability and 3 artifacts/second are still open; speed work should not
outrank result clarity, inventory UX, or corpus growth while the current path
is stable.
### Current product conclusion
The app has crossed from OCR-demo/prototype into a validated scanner baseline on
`main`. The current merge-ready path is the visible-inventory scan flow: the
The app has crossed from OCR-demo/prototype into an Artifact-first scanner app.
The broadest proven live path is still the visible-inventory scan flow: the
operator opens Artifact inventory with a visible detail card, the app verifies
the state, scans read-only, persists parsed artifacts, and keeps uncertain data
reviewable. The next product phase is not another broad speed rewrite; current
speed is acceptable for now. The priority is better content extraction, a
minimal scan-result rail, a browsable artifact inventory, detail evaluation, and
corpus growth before recommendations become the core product surface.
reviewable. The newer native IK-style path is the intended high-speed direction:
the helper captures Artifact card crops quickly, while OCR, parsing, review,
promotion, and value evaluation run downstream.
The next product phase is not broad category expansion. Weapons, materials, and
character details stay out of active scope. The priority is later-session
native repeatability, value
evaluation, and detail explanations before recommendations become the core
product surface.
## Product Direction
- Artifact scanning is the first-class feature.
- Character optimization returns only after scan quality is trustworthy.
- Team building stays out of the critical path until artifact ingestion is stable.
- Inventory Kamera remains a reference for scan choreography and page movement, not a runtime dependency.
- External scanners are not runtime dependencies. The app owns its scan
choreography, OCR, and quality gates.
- Self-learning stays deterministic and local first: review samples, aliases, crop offsets, and UI profile tuning before any ML retraining discussion.
- The scan workspace should be an operator surface: preview, live result rail,
Stop, status, and review access. Detailed stats and debug evidence belong in
@@ -302,7 +333,17 @@ Outcome:
and click-through details.
Status:
- Planned. See
- Started with a scan result rail in the scan surface and an `Inventory` view.
The rail can show native `scan-results.json` entries after post-processing
and falls back to recent stored artifacts. The inventory browser can filter,
sort, inspect native/store/snapshot rows, and preview native card crops from
the selected run directory. It also exposes the current IK inventorylist
version, active Artifact-only scope, and pipeline status for capture,
post-processing, review, promotion, and evidence. Native Artifact rows carry
IK/GOOD match status from the post-capture processor and a dry-run promotion
decision that shows whether a native result is speicherbar, already stored,
review-only, or blocked without writing to the store. Detail review exists;
deterministic value score and value reasons are still pending. See
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md)
for the implementation phases and acceptance criteria.
@@ -332,24 +373,27 @@ Status:
## Immediate Next Implementation Order
1. Keep the visible-inventory scanner path as the production baseline and avoid
1. Add deterministic Artifact value evaluation with explainable reasons.
2. Keep the visible-inventory scanner path as the production baseline and avoid
promoting `auto-entry`, `direct-inventory`, or `paimon-menu` until they pass
their own low-limit live validations.
2. Implement the scan result and inventory data contracts from
3. Keep the active native scope artifact-only until weapons, materials, and
character details are actually scanned.
4. Finish the scan result and inventory detail contracts from
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md),
preserving separate extraction confidence and artifact value.
3. Rework the scan page into preview plus minimal result rail; keep diagnostic
stats out of the primary scan surface.
4. Add the artifact inventory menu and detail view before expanding broad build
recommendations.
5. Grow the confirmed OCR corpus from review samples exported by
5. Finish value reasons in the existing Artifact inventory detail view before
expanding broad build recommendations.
6. Keep the existing scan preview/result rail and Inventory workflow compact;
move new diagnostics behind the dedicated diagnostics surface.
7. Continue growing the confirmed OCR corpus from review samples exported by
`npm run eval:review-candidates` and prepared through
`npm run eval:prepare-confirmed`.
6. Repeat live scanner runs in later sessions to prove repeatability across
8. Repeat live scanner runs in later sessions to prove repeatability across
pages, locked/unlocked artifacts, equipped footers, and duplicate handling.
7. Continue the optional `3 artifacts/second` work only if the next change can
9. Continue the optional `3 artifacts/second` work only if the next change can
reduce OCR/capture transport time without weakening quality gates.
8. Start recommendation/product UX work only after inventory/detail evaluation,
10. Start recommendation/product UX work only after inventory/detail evaluation,
repeat scan quality, and confirmed corpus coverage are strong enough to trust
stored artifacts.