genshin-assistant/docs/scanner-results-inventory-roadmap.md

Scanner Results And Artifact Inventory Roadmap

This document defines the next product phase after the validated visible-inventory scanner baseline and the new native IK-style capture direction. The next capture milestone optimizes speed by moving game-control and card-crop capture into the C# helper. Artifact extraction, review safety, and inventory UX stay separate downstream work so the Electron app remains a visual/status surface instead of the worker.

For the short current status, see CURRENT_STATUS.md. For the current localization, local-deletion, timing, and reviewed-corpus checkpoint, see LOCALIZATION_LOCAL_DELETION_TIMING_CHECKPOINT_2026-07-11.md.

Product Stance

• Keep the scan workspace focused on operation, not analysis.
• Keep the live preview as the dominant surface.
• Keep Electron/React out of per-artifact scanner work in the fast path.
• Use IK inventorylists 1:1 as the scanner dictionary source for artifacts. Other IK lists may stay vendored, but weapons, materials, and character details are not active product scope until their values are actually scanned.
• Move debug metrics, OCR internals, and detailed evaluation behind details, diagnostics, or the inventory view.
• Do not merge scan confidence and artifact value into one ambiguous score.
• Treat uncertain OCR as review, not as a low-quality artifact.
• Treat direct visual star evidence as an evaluation gate: explicitly confirmed 1–4★ results remain visible but are excluded from roll evaluation; missing or conflicting star evidence fails closed to Review.
• The Nicht 5★ inventory filter is a read-only manual-inspection view, not a salvage decision. It must never expose automatic salvage or Quick Select.
• The app renderer defaults to English; German is a Settings preference only. The setting must not be presented as German Genshin OCR support.
• A wrong local artifact may be removed only through a confirmed app-local workflow. It must preserve raw scan evidence, restrict crop cleanup to the selected run directory, and never send game input.
• Prioritize extracting correct artifact content over another speed pass unless live timings regress materially.

Current 2026-07-11 Checkpoint

The implementation now has an English-default/Deutsch-selectable renderer, confirmed local artifact removal, a shared-clock native run timing record, and new visually reviewed OCR corpus cases. These are implementation and offline validation results, not a new live-release claim. The remaining release gates include the non-persisting 50-artifact native smoke that produces a complete run-timing.json. The built-renderer Builds/UI acceptance already passed; an elevated packaged-executable runtime remains a separate packaging boundary.

Target User Flow

1. The user opens Artifact inventory in Genshin with a visible detail card.
2. The scanner runs the existing read-only visible-inventory flow.
3. The scan view shows the latest screenshot/preview on the left.
4. A compact live result rail on the right receives one row per finished safe artifact result.
5. Each evaluated 5★ row shows only:
   • scan number,
   • artifact name or compact slot/set fallback,
   • artifact value score from 0 to 100,
   • a colored result pill.
6. An explicitly recognized 1–4★ row shows Nicht bewertet; an unconfirmed or conflicting star row stays Review rather than assuming 5★.
7. After the scan, the user opens the inventory menu to browse all scanned artifacts, including the separate read-only Nicht 5★ filter.
8. Clicking an artifact opens a detail view with screenshot, parsed fields, OCR confidence, scoring reasons, and optional upgrade projection.

Score Contract

The UI must keep two concepts separate:

Concept	Meaning	UI behavior
Extraction confidence	How reliable the scan/OCR/parser result is.	Drives Review, warnings, and detail confidence rows.
Visual star evidence	Direct starCount, confidence, and source from the native card crop.	Confirmed 1–4★ is excluded; missing, invalid, or conflicting evidence is Review; only confirmed 5★ proceeds to evaluation.
Artifact value score	How useful the artifact appears for builds.	Drives the 0-100 value and good/mid/weak pill.

Rules:

• If extraction confidence is too low, show Review instead of a normal value decision, even when a tentative value score exists.
• An explicit 1–4★ result gets no value score, projection, or promotion. It is visible only as an excluded manual-review item with its crop and details.
• Never infer a 5★ result from an absent, invalid, or conflicting direct visual star field in a current native job; it must stay Review.
• If the artifact is a duplicate, show duplicate state separately from value.
• The live rail may show one compact pill, but the data model should preserve separate extractionStatus and valueStatus fields.
• Score labels should be stable and simple:

Value score	Label
90-100	Strong
70-89	Good
45-69	Mid
0-44	Weak
unknown or unsafe	Review

The exact formula can start simple and deterministic. It should explain its reasons in the detail view before it becomes a recommendation source.

Planned Pipeline Shape

The current TypeScript scan loop can keep shipping as fallback while the native pipeline takes over high-speed capture. The target producer/consumer pipeline is:

flowchart LR
    Native["C# native click, scroll, capture worker"]
    Queue["Bounded card-crop queue"]
    OCR["OCR and parse workers"]
    Match["IK inventorylists matching"]
    Eval["Artifact evaluation (deferred)"]
    Aggregate["Aggregator and store"]
    UI["Live rail and inventory"]

    Native --> Queue
    Queue --> OCR
    OCR --> Match
    Match --> Eval
    Eval --> Aggregate
    Aggregate --> UI

Constraints:

• Only the native helper may control Genshin input, focus, click, scroll, card capture, or failsafe polling in the fast path.
• OCR/parse/evaluation workers may run concurrently on already captured screenshot/crop jobs.
• The queue must be bounded, initially around 4-8 jobs, so the scanner does not outrun retries, review decisions, or stop requests.
• The pipeline must preserve current safety rules: no memory reads, hooks, injection, game-file changes, deleting, feeding, enhancing, locking/unlocking, or spending resources.
• Evaluation may be omitted until after capture speed and crop quality are validated.

Implementation Phases

Phase 0 - Documentation and contracts

Status: prepared by this document and ADR-013.

Outcome:

• Product direction is documented.
• Main docs point to this roadmap.
• Acceptance criteria and checklists exist before code changes.
• Native run artifacts are part of the scanner contract: manifest.json, capture-jobs.jsonl, status.json, scan-results.json, and processing-report.json.

Phase 1 - Result data model

Status: foundation implemented for native post-capture processing. The native scanner reports run status and writes crop jobs; the post-capture processor now writes durable per-artifact scan-results.json entries next to the diagnostic processing-report.json. Native artifact results can now carry IK inventorylist match metadata, and IK set/piece/slot conflicts force review instead of clean extraction. The post-capture OCR/parse worker runs as a bounded queue while preserving result order. scan-results.json also stores parser field confidence metadata for native artifact details. Evaluation remains explicitly deferred.

Outcome:

• Add a durable scan result entry model with sequence number, capture metadata, parsed artifact identity, extraction status, artifact value score, value status, direct visual rarity evidence, duplicate/review flags, and timestamps. Initial native entries use valueStatus: "deferred" for clean parses, valueStatus: "review" for uncertain extraction, and valueStatus: "excluded" for explicit direct 1–4★ evidence.
• Preserve starCount, starConfidence, and starSource for current native jobs so the downstream gate can distinguish explicit 1–4★ exclusion from an ambiguous result that must remain Review.
• Keep existing stored artifact records compatible.
• Add tests for status derivation so low-confidence OCR cannot become a normal Good or Strong result.

Likely files:

• src/types/domain.ts
• src/types/storage.ts
• src/lib/scannerSession.ts
• src/lib/storedArtifactAdapter.ts
• src/lib/scanReviewUtils.ts

Phase 2 - Minimal live result rail

Status: implemented for current-session native post-capture results. The rail starts empty on app launch and receives only results loaded for a scan started in the current app session. Older stored/native history stays in the Inventory surface. Result rail rows can open that surface for crop/IK/detail inspection. A real Settings-UI Artifact-limit-5 run completed with 5 captured, processed, and parsed results, 0 review/errors, persist=false, and five rail rows. A real row-limit run completed at 8/8. The packaged full-owned-inventory baseline then captured 2,211/2,211 over 70 pages and showed all 2,211 live-session rows; current-package reprocessing passed at 136 Review (6.15%), 0 errors, and 0 store writes.

Outcome:

• Rework the scan main section into a left preview/progress work surface plus a right-side current-session result rail.
• Keep phase and outcome counts in that left work surface during a run; remove the duplicate standalone progress card and noisy debug stats from the primary scan area.
• Append rows only after an artifact has finished parse/evaluation.
• Keep Stop, scan status, and review access available.
• Keep debug stats in diagnostics or summary modals.

Likely files:

• src/features/scan/components/ScanMainSection.tsx
• src/features/scan/components/ScanResultCards.tsx
• src/features/scan/components/hooks/useScanMainSectionModel.ts
• src/features/scan/components/hooks/useScanResultCardsModel.ts
• src/styles/base.css

Phase 3 - Artifact inventory view

Status: foundation implemented and now scoped to active Artifact scanning only. The app has an Inventory navigation item with a compact browser for native scan-results.json entries, stored artifacts, and snapshot fallback rows. Filtering, sorting, a detail panel, and secure native crop preview loading are present. The view now surfaces the vendored IK Artifact version/counts, active Artifact-only scope, compact pipeline state for native capture, OCR queue, review, promotion, and evidence, plus per-result IK/GOOD match status for native artifacts. The inventory view also computes a dry-run promotion summary from scan-results.json plus the local artifact store, separating speicherbar, already stored, review, and blocked native results. One selected clean result can now be promoted after a second UI confirmation; the main process revalidates the run result, writes the store, updates scan-results.json, and appends promotion-log.jsonl. Weapons, materials, and character details remain hidden from the active feature UI while they are not scanned. The native helper still reports the category distinction in supportedCategories via catalogAvailable, nativeCaptureSupported, and scanStatus, so dev-control evidence cannot accidentally claim that every IK catalog has an implemented scanner. Native scan start, status, manifest, and capture jobs now carry an explicit scan category; only artifacts can currently produce native capture jobs. Deterministic roll-efficiency evaluation and single-result review/edit/approve are implemented in the detail phases below. The current checkpoint also adds a confirmed local-delete path. A native result is hidden by an append-only tombstone rather than an in-place result rewrite; its PNG crop is removed only after run-directory containment succeeds. A Store-only row, or the exact Store record linked to a native row, can be removed locally after confirmation. Active scans/processors block the action, and no delete path sends Genshin input.

Support code for simple IK weapon, character, and material name/GOOD-key matching exists, but those categories still need their own capture flows before they can be claimed as scanned inventory.

Outcome:

• Add a menu item for scanned artifact inventory.
• Show a compact, minimal list or dense grid of stored artifacts.
• Each entry shows the same score/pill language as the live rail.
• Provide filters and sorting for review, score, set, slot, equipped, locked, and newest scan.
• Provide a separate read-only Nicht 5★ filter for explicitly recognized 1–4★ native results. It must retain screenshot/crop and parsed detail for manual inspection, use no "safe to salvage" language, and offer no automatic salvage or Quick Select action.
• Provide a small confirmed local-delete action in artifact detail. It must describe its app-local scope, preserve raw native result evidence, limit crop removal to a contained PNG, and keep Store removal exact and opt-in.
• Avoid a marketing/landing layout; the first screen is the actual inventory.

Likely files:

• src/features/inventory/*
• src/features/layout/navigation.ts
• src/pages/app/*
• src/lib/artifactStore.ts
• repository bridge/storage files as needed

Phase 4 - Artifact detail view

Status: implemented baseline for native scan results. Inventory detail shows the native card crop from the run directory through the Electron bridge, constrained to PNG files inside the active native run folder. It also shows stored parser field-confidence rows from native scan-results.json, IK/GOOD metadata, dry-run promotion state, and a Naechster Schritt card. Native Review results now have an inline field editor with approve/reject, authoritative validation, run logs, and review-to-eval export. It now also shows roll-efficiency reasons, per-substat legal roll breakdown, build-fit deferral, and under-level projection when safe.

Outcome:

• Clicking a live row or inventory item opens detail.
• Detail shows screenshot or detail crop when available.
• Detail lists parsed fields, OCR confidence, parser notes, extraction status, value score, and scoring reasons.
• Review-required items make the uncertainty explicit and do not present their score as final.

Likely files:

• src/features/inventory/components/*
• src/features/scan/components/modals/*
• src/lib/artifactOcrParser.ts
• src/lib/scoring.ts

Phase 5 - Artifact value evaluation

Status: implemented as roll-efficiency-v1. Clean legal Artifacts are evaluated; Review and unknown data receive no confident score. For current native jobs, only explicitly confirmed 5★ evidence enters evaluation; explicit 1–4★ evidence becomes excluded with no score or projection, while ambiguous direct star evidence remains Review. Existing saved results are enriched on load and replayed offline without rescanning.

Outcome:

• Add a deterministic artifact value evaluator before promoting build recommendations.
• Explain the score through legal roll-tier quality and total roll counts.
• Keep set/main-stat desirability and character/build context separate.
• Keep locked/equipped state out of intrinsic roll quality.
• The evaluator must accept incomplete data and return review/unknown instead of confident nonsense.
• Do not turn an excluded 1–4★ result into an evaluation, store-promotion, projection, or automatic in-game action.

Likely files:

• src/lib/artifactEvaluation.ts
• src/lib/scoring.ts
• src/lib/substatRolls.ts
• src/lib/genshinLookup.ts
• targeted unit tests under src/lib/*.test.ts

Phase 6 - Upgrade projection

Status: implemented as detail-only roll-efficiency-projection-v1 for under-leveled, unambiguously identified 5-star Artifacts with four known legal substats. Packaged visual/live acceptance passed on 2026-07-10; the new live 100er contained only +20 Artifacts, so under-level behavior remains covered by the confirmed saved-run corpus.

Outcome:

• For unambiguously identified 5-star artifacts below max level, show optional projection only in detail.
• Provide worst, middle, and best projected value scores.
• Label projection as probabilistic and not a guaranteed result.
• Use known Genshin upgrade constraints and current substats; unknown or partially read data must disable or soften the projection.

Likely files:

• src/lib/artifactEvaluation.ts
• src/lib/substatRolls.ts
• src/features/inventory/components/*
• parser/scoring tests

Phase 7 - Queue-based analysis pipeline

Outcome:

• Introduce a bounded screenshot/crop job queue only after the UI/data contract is stable.
• Keep one game-control worker.
• Allow OCR/parse/evaluation workers to process queued jobs.
• Preserve stop/failsafe behavior and review decisions.
• Compare throughput against the current baseline without weakening accuracy.

Status: implemented and live accepted. Native capture writes crop jobs without waiting for OCR. The downstream processor tails those jobs with bounded parallelism, streams stable ordered deltas into the result rail, and writes durable reports. Packaged 20-result and 2,211-result runs proved live capture/ evaluation overlap; the full corpus also passed five-repeat deterministic replay and saved-run validation after current-package reprocessing. The current build also writes immutable first-observed lifecycle markers to run-timing.json. It derives end-to-end time directly from request to result reconciliation; capture and processing durations remain separate and must not be added.

Likely files:

• src/lib/autoScanLoop.ts
• src/lib/scannerSession.ts
• Electron capture/OCR boundary in electron/main.ts or extracted services
• scan-loop tests and live soak scripts

Phase 8 - Recommendation promotion

Status: offline implementation checkpoint completed. The Builds surface reads the newest complete native result run, validates source-bound profiles and five-piece evidence, and exposes up to three read-only, non-overlapping suggestions. The current package acceptance runner covers the fresh English/German locale flow, Builds context/focus, reduced motion, and overflow, and its built-renderer execution passed together with the local-delete confirmation probe. It is not evidence of an elevated packaged-executable launch, UAC, preload bridge, or native runtime; those remain separate from the renderer result and the required current-build scanner smoke.

Outcome:

• Promote only source-bound profile suggestions after scan result quality, inventory browsing, detail review, and value scoring are trustworthy.
• Read Build-Fit inputs from complete native results, not metadata-poor store records, because canonical IK Set keys and field-confidence evidence are required.
• Keep roll-efficiency-v1 and OCR confidence out of ranking. The visible rule-coverage value may use only documented Set/Main-Stat/Substat evidence; required full-build context must be explicit.
• Keep suggestions read-only, explain uncertainty/conflicts, and never infer character ownership, team, weapon, rotation, or an in-game action.

Acceptance Criteria

• The scan page still fits the primary workflow without page-level scrolling.
• The preview remains visible during active scan.
• The right rail shows finished artifact evaluations, not noisy intermediate parser/debug state.
• Review items are visibly different from weak artifacts.
• Artifact value score and extraction confidence remain separate in data.
• Inventory view can browse stored scan results without opening diagnostics.
• Detail view explains why an artifact received its score.
• Explicit 1–4★ results are visibly excluded rather than scored; ambiguous visual rarity is visibly Review.
• The Nicht 5★ inventory filter presents only manual review evidence (screenshot/crop and details), never a salvage or Quick Select control.
• A freshly initialized renderer is English; the Settings locale control can select German and restore English without implying German Genshin OCR support.
• Artifact detail can remove a wrong local row only after confirmation. Native removal preserves raw result evidence through a tombstone, constrains crop cleanup to the run directory, and never sends game input.
• Streaming/end-to-end reports use a complete, ordered shared-clock run-timing.json rather than summed capture and processing durations.
• Upgrade projection never implies a guaranteed future roll.
• Build suggestions use only complete native evidence, source-valid profiles, complete Set shapes, and explicit required aggregate context.
• Builds explains profile, sources, chosen five Artifacts, rule coverage, conflicts, and deferrals without presenting an optimizer or DPS score.
• Existing safety constraints and scan quality gates remain intact.

Validation Plan

• npm run lint
• npm test
• npm run build
• git diff --check
• npm run scan:native:replay against accepted bounded runs and the 2,211 corpus
• npm run package:offline-check
• Preserve the passing built-renderer npm run package:live:builds evidence for locale/Builds/delete UI behavior. Separately launch an elevated packaged executable before making a package-runtime/UAC claim.
• npm audit
• npm run scan:native:smoke before claiming native capture plus post-capture processing on live Genshin data.
• For this checkpoint, run the smoke at exactly 50 artifacts with scripts/native-live-smoke.ps1 -Limit 50; require reconciliation, zero processing errors, the review-rate gate, and a complete valid run-timing.json before cleanup, commit, or push.
• Test local deletion offline for active-run blocking, raw-result preservation, tombstone filtering, crop containment, and exact Store scope. The built renderer already passed the visible confirmation probe; repeat it only for a separate elevated packaged-executable runtime claim.
• Test fresh English/default, German selection, and English restoration through the app Settings modal; do not use translated text as the primary automation selector.
• Preserve the accepted 20/50/100 and 2,211 evidence before changing capture, parser, Review, or evaluation contracts.
• Add unit tests for score/status derivation and upgrade projection.
• Add direct visual 5★/4★/3★ and ambiguous-star fixtures. Verify that 1–4★ is stable excluded, ambiguity is Review, and neither path can be promoted or expose an in-game mutation.
• For scanner-facing changes, run a low-limit visible-inventory live scan before broad validation, then use the saved full corpus whenever new input is not required. The original execution plan is retained in TOMORROW_LIVE_TEST_PLAN_2026-07-10.md.
• Keep npm run scan:repeatability:wait for later regression checks, not for every UI-only pass.