feat(scanner): add native artifact pipeline
Add native IK-style capture processing, Artifact Inventory, explicit promotion and single-result review. Confirm the three live OCR corrections in the eval corpus and preserve extraction/value separation.
This commit is contained in:
+81
-37
@@ -3,8 +3,7 @@
|
||||
This document is the source of truth for project intent, scope, runtime facts, and operational expectations.
|
||||
|
||||
For implementation structure, see [ARCHITECTURE.md](ARCHITECTURE.md). For engineering standards, see [CONVENTIONS.md](CONVENTIONS.md).
|
||||
For the latest Inventory-Kamera comparison work, see
|
||||
[scanner-ik-progress-report.md](scanner-ik-progress-report.md).
|
||||
For the current app status, see [CURRENT_STATUS.md](CURRENT_STATUS.md).
|
||||
For the next scan-result and artifact-inventory product phase, see
|
||||
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md).
|
||||
For the 2026-07-09 scanner merge evidence, see [MERGE_READINESS.md](MERGE_READINESS.md).
|
||||
@@ -15,7 +14,7 @@ For Gitea push/authentication setup, see [GITEA_AUTH.md](GITEA_AUTH.md).
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| Project name | Genshin Artifact Assistant |
|
||||
| Status | Scanner baseline merged to `main`; scan result rail, artifact inventory, extraction quality, and corpus growth are next |
|
||||
| Status | Artifact-first scanner baseline with native IK-style 20/50/100 scale evidence, explicit selected promotion, review/edit/approve, scan result rail, and Artifact-only Inventory pipeline; value scoring and later-session repeatability are next |
|
||||
| Platform | Windows desktop |
|
||||
| Target users | Genshin Impact players who want artifact decisions without complex optimizer setup |
|
||||
| Runtime | Electron app with React UI and TypeScript |
|
||||
@@ -35,7 +34,8 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
- Present finished scan results as a compact artifact list instead of a debug-heavy live stats surface.
|
||||
- Keep extraction confidence separate from artifact value so uncertain OCR becomes review, not a misleading low score.
|
||||
- Provide a browsable local artifact inventory with detail views before promoting broader recommendations.
|
||||
- Keep the app offline-first and usable without Genshin Optimizer, Inventory Kamera, Enka, or HoYoLAB.
|
||||
- Keep the app offline-first and usable without optimizer imports, Enka, HoYoLAB,
|
||||
or any external scanner as a core dependency.
|
||||
- Re-introduce recommendations only after the scanner base is trustworthy.
|
||||
|
||||
## Non-Goals
|
||||
@@ -60,8 +60,8 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
| FR-009 | Provide account-level artifact triage after scanner trust is acceptable. | Should | Pending |
|
||||
| FR-010 | Provide 1-3 simple build suggestions per character from owned artifacts after scanner trust is acceptable. | Should | Pending |
|
||||
| FR-011 | Farming overlay for reward scans. | Later | Prototype shell |
|
||||
| FR-012 | Show active scan results as a minimal right-side rail with artifact number, name or compact fallback, value score, and status pill. | Should | Planned |
|
||||
| FR-013 | Provide a scanned artifact inventory view with compact score pills, filters, sorting, and click-through detail. | Should | Planned |
|
||||
| FR-012 | Show active scan results as a minimal right-side rail with artifact number, name or compact fallback, value score, and status pill. | Should | Partial foundation |
|
||||
| FR-013 | Provide a scanned artifact inventory view with compact score pills, filters, sorting, and click-through detail. | Should | Partial foundation |
|
||||
| FR-014 | Provide artifact detail evaluation with screenshot/crops, parsed fields, OCR confidence, value reasons, and optional upgrade projection. | Should | Planned |
|
||||
|
||||
## Non-Functional Requirements
|
||||
@@ -70,7 +70,7 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
| --- | --- | --- |
|
||||
| Safety | Never perform irreversible in-game actions. | Code review and manual test |
|
||||
| Performance | Single artifact read should feel interactive and batch scan should not stall on false progress. | Capture latency monitored manually; auto-scan stops on blocked verification |
|
||||
| IK target | First 100 artifacts should scan with accuracy at least as good as Inventory Kamera and equal or better speed. | `npm run scan:goal:compare:validated` or `npm run scan:goal:compare:validated:wait` quality-gated report |
|
||||
| Performance | A 100-artifact visible-inventory run should finish cleanly with low review/miss rates and report timing evidence. | `npm run scan:goal:validated` or `npm run scan:goal:validated:wait` quality-gated report |
|
||||
| Privacy | Captures and parsed data stay local by default. | No remote upload in scanner path |
|
||||
| Reliability | Uncertain OCR must be visible to the user. | Confidence and details view |
|
||||
| Score integrity | Extraction confidence and artifact value are separate concepts. | Review state can block or qualify a value score |
|
||||
@@ -108,10 +108,12 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
- A local canonical data package already exists in `src/data/genshinGameData.json`, generated from `genshin-db`.
|
||||
- The parser already uses known sets, pieces, slots, stat aliases, set aliases, character aliases, and derived slot/set mapping.
|
||||
- Review samples, learned replacements, parser notes, and stored artifacts already persist locally.
|
||||
- The scan surface has an initial recent-results rail backed by the newest
|
||||
stored artifacts.
|
||||
- The auto-scan loop is no longer a naive click spammer: it has preflight, verification, miss handling, page fingerprinting, and stop conditions.
|
||||
- The scanner now has an Inventory-Kamera comparison path: 32 safe artifact
|
||||
targets per page, lookup-derived fields, fast OCR crop profile, current vs.
|
||||
IK-traineddata benchmark endpoint, and a quality-gated live soak runner.
|
||||
- The scanner now has a validated visible-inventory path: 32 safe artifact
|
||||
targets per page, lookup-derived fields, fast OCR crop profile, and a
|
||||
quality-gated live soak runner.
|
||||
- Elevated live automation is validated in the current dev environment:
|
||||
`/automation/probe-click?index=1` changed the selected artifact and
|
||||
`/scanner/start?limit=2` completed with 2/2 verified reads and 0 misses.
|
||||
@@ -128,42 +130,71 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
- Review samples can now be exported with `npm run eval:review-candidates` into
|
||||
a Git-ignored human-labeling worklist. This is the next quality phase before
|
||||
adding more OCR corpus cases or trusting review queue data as labels.
|
||||
- Native IK-style Artifact capture is wired through the C# helper. It writes
|
||||
card crops and run artifacts for downstream OCR/parse processing instead of
|
||||
making React do per-artifact work in the hot capture loop.
|
||||
- The native post-capture processor can write `scan-results.json` and
|
||||
`processing-report.json`, preserve parser field confidence, match Artifact
|
||||
results against IK artifact set/piece/slot data, and keep store persistence
|
||||
opt-in.
|
||||
- The Inventory view now has an Artifact-only pipeline surface for scope,
|
||||
native capture, OCR queue, review gate, promotion, and evidence. Native rows
|
||||
expose crop previews, IK/GOOD metadata, dry-run promotion state, and a
|
||||
`Naechster Schritt` card.
|
||||
- The active UI intentionally hides weapon, material, and character-detail IK
|
||||
catalog coverage until those values are actually scanned.
|
||||
|
||||
### What is still structurally weak
|
||||
|
||||
- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
|
||||
- Broader scan soak testing has reached clean 20-, 45-, and 100-artifact runs
|
||||
with 0 misses on the current engine. The current-vs-IK-traineddata comparison
|
||||
is now captured; `current` won the qualified 100-artifact comparison on
|
||||
2026-07-08.
|
||||
with 0 misses on the current engine. The current 2026-07-09 100-artifact run
|
||||
completed `100/100` verified and parsed with `0` review and `0` misses.
|
||||
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
|
||||
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
|
||||
- The scan page is cleaner than before, but it still needs the next minimalist
|
||||
result-rail pass so the main flow shows preview plus completed artifact
|
||||
outcomes instead of live diagnostic/stat content.
|
||||
- The scan page now has a minimalist recent-results rail, and the Inventory view
|
||||
can inspect native Artifact results, crop previews, IK match state, promotion
|
||||
dry-runs, and pipeline risk/status. Value scoring remains incomplete;
|
||||
single-result review/edit/approve is implemented and batch review is
|
||||
intentionally unavailable.
|
||||
- The new native IK-style path has same-session 20/50/100 live scale evidence
|
||||
with complete capture/result counts and safe Review gating. Later-session
|
||||
repeatability and packaged behavior remain open.
|
||||
- Store promotion from native `scan-results.json` now supports one selected,
|
||||
confirmed result at a time with main-process revalidation and a durable log.
|
||||
Batch promotion intentionally remains unavailable.
|
||||
- Native Review results can be corrected and approved or rejected one at a
|
||||
time. Approval revalidates IK identity, canonical main values, and legal
|
||||
substat rolls, then feeds the existing review-to-eval candidate pipeline.
|
||||
- Recommendations and build logic exist, but artifact inventory, detail review,
|
||||
and value scoring should land first so recommendations have trustworthy inputs.
|
||||
- The latest source has completed the final current-vs-IK-traineddata live
|
||||
comparison for this environment. Repeatability and 3 artifacts/second are
|
||||
still open.
|
||||
- Repeatability and 3 artifacts/second are still open; speed work should not
|
||||
outrank result clarity, inventory UX, or corpus growth while the current path
|
||||
is stable.
|
||||
|
||||
### Current product conclusion
|
||||
|
||||
The app has crossed from OCR-demo/prototype into a validated scanner baseline on
|
||||
`main`. The current merge-ready path is the visible-inventory scan flow: the
|
||||
The app has crossed from OCR-demo/prototype into an Artifact-first scanner app.
|
||||
The broadest proven live path is still the visible-inventory scan flow: the
|
||||
operator opens Artifact inventory with a visible detail card, the app verifies
|
||||
the state, scans read-only, persists parsed artifacts, and keeps uncertain data
|
||||
reviewable. The next product phase is not another broad speed rewrite; current
|
||||
speed is acceptable for now. The priority is better content extraction, a
|
||||
minimal scan-result rail, a browsable artifact inventory, detail evaluation, and
|
||||
corpus growth before recommendations become the core product surface.
|
||||
reviewable. The newer native IK-style path is the intended high-speed direction:
|
||||
the helper captures Artifact card crops quickly, while OCR, parsing, review,
|
||||
promotion, and value evaluation run downstream.
|
||||
|
||||
The next product phase is not broad category expansion. Weapons, materials, and
|
||||
character details stay out of active scope. The priority is later-session
|
||||
native repeatability, value
|
||||
evaluation, and detail explanations before recommendations become the core
|
||||
product surface.
|
||||
|
||||
## Product Direction
|
||||
|
||||
- Artifact scanning is the first-class feature.
|
||||
- Character optimization returns only after scan quality is trustworthy.
|
||||
- Team building stays out of the critical path until artifact ingestion is stable.
|
||||
- Inventory Kamera remains a reference for scan choreography and page movement, not a runtime dependency.
|
||||
- External scanners are not runtime dependencies. The app owns its scan
|
||||
choreography, OCR, and quality gates.
|
||||
- Self-learning stays deterministic and local first: review samples, aliases, crop offsets, and UI profile tuning before any ML retraining discussion.
|
||||
- The scan workspace should be an operator surface: preview, live result rail,
|
||||
Stop, status, and review access. Detailed stats and debug evidence belong in
|
||||
@@ -302,7 +333,17 @@ Outcome:
|
||||
and click-through details.
|
||||
|
||||
Status:
|
||||
- Planned. See
|
||||
- Started with a scan result rail in the scan surface and an `Inventory` view.
|
||||
The rail can show native `scan-results.json` entries after post-processing
|
||||
and falls back to recent stored artifacts. The inventory browser can filter,
|
||||
sort, inspect native/store/snapshot rows, and preview native card crops from
|
||||
the selected run directory. It also exposes the current IK inventorylist
|
||||
version, active Artifact-only scope, and pipeline status for capture,
|
||||
post-processing, review, promotion, and evidence. Native Artifact rows carry
|
||||
IK/GOOD match status from the post-capture processor and a dry-run promotion
|
||||
decision that shows whether a native result is speicherbar, already stored,
|
||||
review-only, or blocked without writing to the store. Detail review exists;
|
||||
deterministic value score and value reasons are still pending. See
|
||||
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md)
|
||||
for the implementation phases and acceptance criteria.
|
||||
|
||||
@@ -332,24 +373,27 @@ Status:
|
||||
|
||||
## Immediate Next Implementation Order
|
||||
|
||||
1. Keep the visible-inventory scanner path as the production baseline and avoid
|
||||
1. Add deterministic Artifact value evaluation with explainable reasons.
|
||||
2. Keep the visible-inventory scanner path as the production baseline and avoid
|
||||
promoting `auto-entry`, `direct-inventory`, or `paimon-menu` until they pass
|
||||
their own low-limit live validations.
|
||||
2. Implement the scan result and inventory data contracts from
|
||||
3. Keep the active native scope artifact-only until weapons, materials, and
|
||||
character details are actually scanned.
|
||||
4. Finish the scan result and inventory detail contracts from
|
||||
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md),
|
||||
preserving separate extraction confidence and artifact value.
|
||||
3. Rework the scan page into preview plus minimal result rail; keep diagnostic
|
||||
stats out of the primary scan surface.
|
||||
4. Add the artifact inventory menu and detail view before expanding broad build
|
||||
recommendations.
|
||||
5. Grow the confirmed OCR corpus from review samples exported by
|
||||
5. Finish value reasons in the existing Artifact inventory detail view before
|
||||
expanding broad build recommendations.
|
||||
6. Keep the existing scan preview/result rail and Inventory workflow compact;
|
||||
move new diagnostics behind the dedicated diagnostics surface.
|
||||
7. Continue growing the confirmed OCR corpus from review samples exported by
|
||||
`npm run eval:review-candidates` and prepared through
|
||||
`npm run eval:prepare-confirmed`.
|
||||
6. Repeat live scanner runs in later sessions to prove repeatability across
|
||||
8. Repeat live scanner runs in later sessions to prove repeatability across
|
||||
pages, locked/unlocked artifacts, equipped footers, and duplicate handling.
|
||||
7. Continue the optional `3 artifacts/second` work only if the next change can
|
||||
9. Continue the optional `3 artifacts/second` work only if the next change can
|
||||
reduce OCR/capture transport time without weakening quality gates.
|
||||
8. Start recommendation/product UX work only after inventory/detail evaluation,
|
||||
10. Start recommendation/product UX work only after inventory/detail evaluation,
|
||||
repeat scan quality, and confirmed corpus coverage are strong enough to trust
|
||||
stored artifacts.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user