genshin-assistant/docs/scanner-results-inventory-roadmap.md

Scanner Results And Artifact Inventory Roadmap

This document defines the next product phase after the validated visible-inventory scanner baseline and the new native IK-style capture direction. The next capture milestone optimizes speed by moving game-control and card-crop capture into the C# helper. Artifact extraction, review safety, and inventory UX stay separate downstream work so the Electron app remains a visual/status surface instead of the worker.

For the short current status, see CURRENT_STATUS.md.

Product Stance

• Keep the scan workspace focused on operation, not analysis.
• Keep the live preview as the dominant surface.
• Keep Electron/React out of per-artifact scanner work in the fast path.
• Use IK inventorylists 1:1 as the scanner dictionary source for artifacts. Other IK lists may stay vendored, but weapons, materials, and character details are not active product scope until their values are actually scanned.
• Move debug metrics, OCR internals, and detailed evaluation behind details, diagnostics, or the inventory view.
• Do not merge scan confidence and artifact value into one ambiguous score.
• Treat uncertain OCR as review, not as a low-quality artifact.
• Prioritize extracting correct artifact content over another speed pass unless live timings regress materially.

Target User Flow

1. The user opens Artifact inventory in Genshin with a visible detail card.
2. The scanner runs the existing read-only visible-inventory flow.
3. The scan view shows the latest screenshot/preview on the left.
4. A compact live result rail on the right receives one row per finished artifact evaluation.
5. Each row shows only:
   • scan number,
   • artifact name or compact slot/set fallback,
   • artifact value score from 0 to 100,
   • a colored result pill.
6. After the scan, the user opens the inventory menu to browse all scanned artifacts.
7. Clicking an artifact opens a detail view with screenshot, parsed fields, OCR confidence, scoring reasons, and optional upgrade projection.

Score Contract

The UI must keep two concepts separate:

Concept	Meaning	UI behavior
Extraction confidence	How reliable the scan/OCR/parser result is.	Drives Review, warnings, and detail confidence rows.
Artifact value score	How useful the artifact appears for builds.	Drives the 0-100 value and good/mid/weak pill.

Rules:

• If extraction confidence is too low, show Review instead of a normal value decision, even when a tentative value score exists.
• If the artifact is a duplicate, show duplicate state separately from value.
• The live rail may show one compact pill, but the data model should preserve separate extractionStatus and valueStatus fields.
• Score labels should be stable and simple:

Value score	Label
90-100	Strong
70-89	Good
45-69	Mid
0-44	Weak
unknown or unsafe	Review

The exact formula can start simple and deterministic. It should explain its reasons in the detail view before it becomes a recommendation source.

Planned Pipeline Shape

The current TypeScript scan loop can keep shipping as fallback while the native pipeline takes over high-speed capture. The target producer/consumer pipeline is:

flowchart LR
    Native["C# native click, scroll, capture worker"]
    Queue["Bounded card-crop queue"]
    OCR["OCR and parse workers"]
    Match["IK inventorylists matching"]
    Eval["Artifact evaluation (deferred)"]
    Aggregate["Aggregator and store"]
    UI["Live rail and inventory"]

    Native --> Queue
    Queue --> OCR
    OCR --> Match
    Match --> Eval
    Eval --> Aggregate
    Aggregate --> UI

Constraints:

• Only the native helper may control Genshin input, focus, click, scroll, card capture, or failsafe polling in the fast path.
• OCR/parse/evaluation workers may run concurrently on already captured screenshot/crop jobs.
• The queue must be bounded, initially around 4-8 jobs, so the scanner does not outrun retries, review decisions, or stop requests.
• The pipeline must preserve current safety rules: no memory reads, hooks, injection, game-file changes, deleting, feeding, enhancing, locking/unlocking, or spending resources.
• Evaluation may be omitted until after capture speed and crop quality are validated.

Implementation Phases

Phase 0 - Documentation and contracts

Status: prepared by this document and ADR-013.

Outcome:

• Product direction is documented.
• Main docs point to this roadmap.
• Acceptance criteria and checklists exist before code changes.
• Native run artifacts are part of the scanner contract: manifest.json, capture-jobs.jsonl, status.json, scan-results.json, and processing-report.json.

Phase 1 - Result data model

Status: foundation implemented for native post-capture processing. The native scanner reports run status and writes crop jobs; the post-capture processor now writes durable per-artifact scan-results.json entries next to the diagnostic processing-report.json. Native artifact results can now carry IK inventorylist match metadata, and IK set/piece/slot conflicts force review instead of clean extraction. The post-capture OCR/parse worker runs as a bounded queue while preserving result order. scan-results.json also stores parser field confidence metadata for native artifact details. Evaluation remains explicitly deferred.

Outcome:

• Add a durable scan result entry model with sequence number, capture metadata, parsed artifact identity, extraction status, artifact value score, value status, duplicate/review flags, and timestamps. Initial native entries use valueStatus: "deferred" for clean parses and valueStatus: "review" for uncertain extraction.
• Keep existing stored artifact records compatible.
• Add tests for status derivation so low-confidence OCR cannot become a normal Good or Strong result.

Likely files:

• src/types/domain.ts
• src/types/storage.ts
• src/lib/scannerSession.ts
• src/lib/storedArtifactAdapter.ts
• src/lib/scanReviewUtils.ts

Phase 2 - Minimal live result rail

Status: foundation implemented for native post-capture results. The scan main section shows newest stored artifacts as fallback and can display the latest native scan-results.json entries after post-processing. Result rail rows can open the Inventory surface for crop/IK/detail inspection. Value scores are still pending.

Outcome:

• Rework the scan main section into preview plus right-side result rail.
• Remove live evaluation cards and noisy stats from the primary scan area.
• Append rows only after an artifact has finished parse/evaluation.
• Keep Stop, scan status, and review access available.
• Keep debug stats in diagnostics or summary modals.

Likely files:

• src/features/scan/components/ScanMainSection.tsx
• src/features/scan/components/ScanResultCards.tsx
• src/features/scan/components/hooks/useScanMainSectionModel.ts
• src/features/scan/components/hooks/useScanResultCardsModel.ts
• src/styles/base.css

Phase 3 - Artifact inventory view

Status: foundation implemented and now scoped to active Artifact scanning only. The app has an Inventory navigation item with a compact browser for native scan-results.json entries, stored artifacts, and snapshot fallback rows. Filtering, sorting, a detail panel, and secure native crop preview loading are present. The view now surfaces the vendored IK Artifact version/counts, active Artifact-only scope, compact pipeline state for native capture, OCR queue, review, promotion, and evidence, plus per-result IK/GOOD match status for native artifacts. The inventory view also computes a dry-run promotion summary from scan-results.json plus the local artifact store, separating speicherbar, already stored, review, and blocked native results. One selected clean result can now be promoted after a second UI confirmation; the main process revalidates the run result, writes the store, updates scan-results.json, and appends promotion-log.jsonl. Weapons, materials, and character details remain hidden from the active feature UI while they are not scanned. The native helper still reports the category distinction in supportedCategories via catalogAvailable, nativeCaptureSupported, and scanStatus, so dev-control evidence cannot accidentally claim that every IK catalog has an implemented scanner. Native scan start, status, manifest, and capture jobs now carry an explicit scan category; only artifacts can currently produce native capture jobs. Value scoring is still pending; single-result review/edit/approve is implemented in the detail phase below.

Support code for simple IK weapon, character, and material name/GOOD-key matching exists, but those categories still need their own capture flows before they can be claimed as scanned inventory.

Outcome:

• Add a menu item for scanned artifact inventory.
• Show a compact, minimal list or dense grid of stored artifacts.
• Each entry shows the same score/pill language as the live rail.
• Provide filters and sorting for review, score, set, slot, equipped, locked, and newest scan.
• Avoid a marketing/landing layout; the first screen is the actual inventory.

Likely files:

• src/features/inventory/*
• src/features/layout/navigation.ts
• src/pages/app/*
• src/lib/artifactStore.ts
• repository bridge/storage files as needed

Phase 4 - Artifact detail view

Status: started for native scan results. Inventory detail can show the native card crop from the run directory through the Electron bridge, constrained to PNG files inside the active native run folder. It also shows stored parser field-confidence rows from native scan-results.json, IK/GOOD metadata, dry-run promotion state, and a Naechster Schritt card. Native Review results now have an inline field editor with approve/reject, authoritative validation, run logs, and review-to-eval export. Value reasons are still pending.

Outcome:

• Clicking a live row or inventory item opens detail.
• Detail shows screenshot or detail crop when available.
• Detail lists parsed fields, OCR confidence, parser notes, extraction status, value score, and scoring reasons.
• Review-required items make the uncertainty explicit and do not present their score as final.

Likely files:

• src/features/inventory/components/*
• src/features/scan/components/modals/*
• src/lib/artifactOcrParser.ts
• src/lib/scoring.ts

Phase 5 - Artifact value evaluation

Status: next product feature after the now-completed same-session native scale, selected promotion, and review/edit/approve workflow.

Outcome:

• Add a deterministic artifact value evaluator before promoting build recommendations.
• Explain the score through factors such as set, slot, main stat, substat quality, level, locked/equipped state, and available character/build context.
• The evaluator must accept incomplete data and return review/unknown instead of confident nonsense.

Likely files:

• src/lib/artifactEvaluation.ts
• src/lib/scoring.ts
• src/lib/substatRolls.ts
• src/lib/genshinLookup.ts
• targeted unit tests under src/lib/*.test.ts

Phase 6 - Upgrade projection

Status: later detail-level feature after artifact value evaluation.

Outcome:

• For artifacts below max level, show optional projection only in detail.
• Provide worst, middle, and best projected value scores.
• Label projection as probabilistic and not a guaranteed result.
• Use known Genshin upgrade constraints and current substats; unknown or partially read data must disable or soften the projection.

Likely files:

• src/lib/upgradeProjection.ts
• src/lib/substatRolls.ts
• src/features/inventory/components/*
• parser/scoring tests

Phase 7 - Queue-based analysis pipeline

Outcome:

• Introduce a bounded screenshot/crop job queue only after the UI/data contract is stable.
• Keep one game-control worker.
• Allow OCR/parse/evaluation workers to process queued jobs.
• Preserve stop/failsafe behavior and review decisions.
• Compare throughput against the current baseline without weakening accuracy.

Status: started for post-capture processing. Native capture already writes crop jobs without waiting for OCR. The downstream processor now consumes those jobs with bounded parallelism and writes stable ordered reports. Live throughput comparison remains a final validation item.

Likely files:

• src/lib/autoScanLoop.ts
• src/lib/scannerSession.ts
• Electron capture/OCR boundary in electron/main.ts or extracted services
• scan-loop tests and live soak scripts

Phase 8 - Recommendation promotion

Status: intentionally delayed until native artifact ingestion, review, promotion, value scoring, and repeatability are trustworthy.

Outcome:

• Promote account-level recommendations only after scan result quality, inventory browsing, detail review, and value scoring are trustworthy.
• Recommendations must reference stored artifact quality and uncertainty.

Acceptance Criteria

• The scan page still fits the primary workflow without page-level scrolling.
• The preview remains visible during active scan.
• The right rail shows finished artifact evaluations, not noisy intermediate parser/debug state.
• Review items are visibly different from weak artifacts.
• Artifact value score and extraction confidence remain separate in data.
• Inventory view can browse stored scan results without opening diagnostics.
• Detail view explains why an artifact received its score.
• Upgrade projection never implies a guaranteed future roll.
• Existing safety constraints and scan quality gates remain intact.

Validation Plan

• npm run lint
• npm test
• npm run build
• git diff --check
• npm run scan:native:smoke before claiming native capture plus post-capture processing on live Genshin data.
• Broader native Artifact runs with 20/50/100 items before claiming IK-style speed or stability.
• Add unit tests for score/status derivation and upgrade projection.
• For scanner-facing changes, run a low-limit visible-inventory live scan before wider validation.
• Keep npm run scan:repeatability:wait for later regression checks, not for every UI-only pass.