genshin-assistant/docs/scanner-results-inventory-roadmap.md

# Scanner Results And Artifact Inventory Roadmap

This document defines the next product phase after the validated visible-inventory
scanner baseline and the new native IK-style capture direction. The next capture
milestone optimizes speed by moving game-control and card-crop capture into the
C# helper. Artifact extraction, review safety, and inventory UX stay separate
downstream work so the Electron app remains a visual/status surface instead of
the worker.

For the short current status, see [CURRENT_STATUS.md](CURRENT_STATUS.md).
For the current localization, local-deletion, timing, and reviewed-corpus
checkpoint, see
[LOCALIZATION_LOCAL_DELETION_TIMING_CHECKPOINT_2026-07-11.md](LOCALIZATION_LOCAL_DELETION_TIMING_CHECKPOINT_2026-07-11.md).

## Product Stance

- Keep the scan workspace focused on operation, not analysis.
- Keep the live preview as the dominant surface.
- Keep Electron/React out of per-artifact scanner work in the fast path.
- Use IK `inventorylists` 1:1 as the scanner dictionary source for artifacts.
  Other IK lists may stay vendored, but weapons, materials, and character
  details are not active product scope until their values are actually scanned.
- Move debug metrics, OCR internals, and detailed evaluation behind details,
  diagnostics, or the inventory view.
- Do not merge scan confidence and artifact value into one ambiguous score.
- Treat uncertain OCR as review, not as a low-quality artifact.
- Treat direct visual star evidence as an evaluation gate: explicitly confirmed
  1–4★ results remain visible but are excluded from roll evaluation; missing or
  conflicting star evidence fails closed to `Review`.
- The `Nicht 5★` inventory filter is a read-only manual-inspection view, not a
  salvage decision. It must never expose automatic salvage or Quick Select.
- The app renderer defaults to English; German is a Settings preference only.
  The setting must not be presented as German Genshin OCR support.
- A wrong local artifact may be removed only through a confirmed app-local
  workflow. It must preserve raw scan evidence, restrict crop cleanup to the
  selected run directory, and never send game input.
- Prioritize extracting correct artifact content over another speed pass unless
  live timings regress materially.

## Current 2026-07-11 Checkpoint

The implementation now has an English-default/Deutsch-selectable renderer,
confirmed local artifact removal, a shared-clock native run timing record, and
new visually reviewed OCR corpus cases. These are implementation and offline
validation results, not a new live-release claim. The remaining release gates
include the non-persisting 50-artifact native smoke that produces a complete
`run-timing.json`. The built-renderer Builds/UI acceptance already passed; an
elevated packaged-executable runtime remains a separate packaging boundary.

## Target User Flow

1. The user opens Artifact inventory in Genshin with a visible detail card.
2. The scanner runs the existing read-only visible-inventory flow.
3. The scan view shows the latest screenshot/preview on the left.
4. A compact live result rail on the right receives one row per finished safe
   artifact result.
5. Each evaluated 5★ row shows only:
   - scan number,
   - artifact name or compact slot/set fallback,
   - artifact value score from `0` to `100`,
   - a colored result pill.
6. An explicitly recognized 1–4★ row shows `Nicht bewertet`; an unconfirmed or
   conflicting star row stays `Review` rather than assuming 5★.
7. After the scan, the user opens the inventory menu to browse all scanned
   artifacts, including the separate read-only `Nicht 5★` filter.
8. Clicking an artifact opens a detail view with screenshot, parsed fields,
   OCR confidence, scoring reasons, and optional upgrade projection.

## Score Contract

The UI must keep two concepts separate:

| Concept | Meaning | UI behavior |
| --- | --- | --- |
| Extraction confidence | How reliable the scan/OCR/parser result is. | Drives `Review`, warnings, and detail confidence rows. |
| Visual star evidence | Direct `starCount`, confidence, and source from the native card crop. | Confirmed 1–4★ is `excluded`; missing, invalid, or conflicting evidence is `Review`; only confirmed 5★ proceeds to evaluation. |
| Artifact value score | How useful the artifact appears for builds. | Drives the `0-100` value and good/mid/weak pill. |

Rules:

- If extraction confidence is too low, show `Review` instead of a normal value
  decision, even when a tentative value score exists.
- An explicit 1–4★ result gets no value score, projection, or promotion. It is
  visible only as an `excluded` manual-review item with its crop and details.
- Never infer a 5★ result from an absent, invalid, or conflicting direct visual
  star field in a current native job; it must stay `Review`.
- If the artifact is a duplicate, show duplicate state separately from value.
- The live rail may show one compact pill, but the data model should preserve
  separate `extractionStatus` and `valueStatus` fields.
- Score labels should be stable and simple:

| Value score | Label |
| --- | --- |
| `90-100` | Strong |
| `70-89` | Good |
| `45-69` | Mid |
| `0-44` | Weak |
| unknown or unsafe | Review |

The exact formula can start simple and deterministic. It should explain its
reasons in the detail view before it becomes a recommendation source.

## Planned Pipeline Shape

The current TypeScript scan loop can keep shipping as fallback while the native
pipeline takes over high-speed capture. The target producer/consumer pipeline is:

```mermaid
flowchart LR
    Native["C# native click, scroll, capture worker"]
    Queue["Bounded card-crop queue"]
    OCR["OCR and parse workers"]
    Match["IK inventorylists matching"]
    Eval["Artifact evaluation (deferred)"]
    Aggregate["Aggregator and store"]
    UI["Live rail and inventory"]

    Native --> Queue
    Queue --> OCR
    OCR --> Match
    Match --> Eval
    Eval --> Aggregate
    Aggregate --> UI
```

Constraints:

- Only the native helper may control Genshin input, focus, click, scroll, card
  capture, or failsafe polling in the fast path.
- OCR/parse/evaluation workers may run concurrently on already captured
  screenshot/crop jobs.
- The queue must be bounded, initially around `4-8` jobs, so the scanner does
  not outrun retries, review decisions, or stop requests.
- The pipeline must preserve current safety rules: no memory reads, hooks,
  injection, game-file changes, deleting, feeding, enhancing, locking/unlocking,
  or spending resources.
- Evaluation may be omitted until after capture speed and crop quality are
  validated.

## Implementation Phases

### Phase 0 - Documentation and contracts

Status: prepared by this document and ADR-013.

Outcome:

- Product direction is documented.
- Main docs point to this roadmap.
- Acceptance criteria and checklists exist before code changes.
- Native run artifacts are part of the scanner contract:
  `manifest.json`, `capture-jobs.jsonl`, `status.json`,
  `scan-results.json`, and `processing-report.json`.

### Phase 1 - Result data model

Status: foundation implemented for native post-capture processing. The native
scanner reports run status and writes crop jobs; the post-capture processor now
writes durable per-artifact `scan-results.json` entries next to the diagnostic
`processing-report.json`. Native artifact results can now carry IK inventorylist
match metadata, and IK set/piece/slot conflicts force review instead of clean
extraction. The post-capture OCR/parse worker runs as a bounded queue while
preserving result order. `scan-results.json` also stores parser field confidence
metadata for native artifact details. Evaluation remains explicitly deferred.

Outcome:

- Add a durable scan result entry model with sequence number, capture metadata,
  parsed artifact identity, extraction status, artifact value score, value
  status, direct visual rarity evidence, duplicate/review flags, and timestamps. Initial native entries use
  `valueStatus: "deferred"` for clean parses, `valueStatus: "review"` for
  uncertain extraction, and `valueStatus: "excluded"` for explicit direct
  1–4★ evidence.
- Preserve `starCount`, `starConfidence`, and `starSource` for current native
  jobs so the downstream gate can distinguish explicit 1–4★ exclusion from an
  ambiguous result that must remain `Review`.
- Keep existing stored artifact records compatible.
- Add tests for status derivation so low-confidence OCR cannot become a normal
  `Good` or `Strong` result.

Likely files:

- `src/types/domain.ts`
- `src/types/storage.ts`
- `src/lib/scannerSession.ts`
- `src/lib/storedArtifactAdapter.ts`
- `src/lib/scanReviewUtils.ts`

### Phase 2 - Minimal live result rail

Status: implemented for current-session native post-capture results. The rail
starts empty on app launch and receives only results loaded for a scan started
in the current app session. Older stored/native history stays in the Inventory
surface. Result rail rows can open that surface for crop/IK/detail inspection.
A real Settings-UI Artifact-limit-5 run completed with 5 captured, processed,
and parsed results, 0 review/errors, `persist=false`, and five rail rows. A
real row-limit run completed at 8/8. The packaged full-owned-inventory baseline
then captured 2,211/2,211 over 70 pages and showed all 2,211 live-session rows;
current-package reprocessing passed at 136 Review (6.15%), 0 errors, and 0
store writes.

Outcome:

- Rework the scan main section into a left preview/progress work surface plus a
  right-side current-session result rail.
- Keep phase and outcome counts in that left work surface during a run; remove
  the duplicate standalone progress card and noisy debug stats from the primary
  scan area.
- Append rows only after an artifact has finished parse/evaluation.
- Keep Stop, scan status, and review access available.
- Keep debug stats in diagnostics or summary modals.

Likely files:

- `src/features/scan/components/ScanMainSection.tsx`
- `src/features/scan/components/ScanResultCards.tsx`
- `src/features/scan/components/hooks/useScanMainSectionModel.ts`
- `src/features/scan/components/hooks/useScanResultCardsModel.ts`
- `src/styles/base.css`

### Phase 3 - Artifact inventory view

Status: foundation implemented and now scoped to active Artifact scanning only.
The app has an `Inventory` navigation item with
a compact browser for native `scan-results.json` entries, stored artifacts, and
snapshot fallback rows. Filtering, sorting, a detail panel, and secure native
crop preview loading are present. The view now surfaces the vendored IK
Artifact version/counts, active Artifact-only scope, compact pipeline state for
native capture, OCR queue, review, promotion, and evidence, plus per-result
IK/GOOD match status for native artifacts. The inventory view also computes a
dry-run promotion summary from `scan-results.json` plus the local artifact
store, separating `speicherbar`, already stored, review, and blocked native
results. One selected clean result can now be promoted after a second UI
confirmation; the main process revalidates the run result, writes the store,
updates `scan-results.json`, and appends `promotion-log.jsonl`. Weapons,
materials, and character details
remain hidden from the active feature UI while they are not scanned. The native
helper still reports the category distinction in
`supportedCategories` via `catalogAvailable`, `nativeCaptureSupported`, and
`scanStatus`, so dev-control evidence cannot accidentally claim that every IK
catalog has an implemented scanner. Native scan start, status, manifest, and
capture jobs now carry an explicit scan category; only `artifacts` can currently
  produce native capture jobs. Deterministic roll-efficiency evaluation and
  single-result review/edit/approve are implemented in the detail phases below.
  The current checkpoint also adds a confirmed local-delete path. A native
  result is hidden by an append-only tombstone rather than an in-place result
  rewrite; its PNG crop is removed only after run-directory containment
  succeeds. A Store-only row, or the exact Store record linked to a native row,
  can be removed locally after confirmation. Active scans/processors block the
  action, and no delete path sends Genshin input.

Support code for simple IK weapon, character, and material name/GOOD-key
matching exists, but those categories still need their own capture flows before
they can be claimed as scanned inventory.

Outcome:

- Add a menu item for scanned artifact inventory.
- Show a compact, minimal list or dense grid of stored artifacts.
- Each entry shows the same score/pill language as the live rail.
- Provide filters and sorting for review, score, set, slot, equipped, locked,
  and newest scan.
- Provide a separate read-only `Nicht 5★` filter for explicitly recognized
  1–4★ native results. It must retain screenshot/crop and parsed detail for
  manual inspection, use no "safe to salvage" language, and offer no automatic
  salvage or Quick Select action.
- Provide a small confirmed local-delete action in artifact detail. It must
  describe its app-local scope, preserve raw native result evidence, limit crop
  removal to a contained PNG, and keep Store removal exact and opt-in.
- Avoid a marketing/landing layout; the first screen is the actual inventory.

Likely files:

- `src/features/inventory/*`
- `src/features/layout/navigation.ts`
- `src/pages/app/*`
- `src/lib/artifactStore.ts`
- repository bridge/storage files as needed

### Phase 4 - Artifact detail view

Status: implemented baseline for native scan results. Inventory detail shows the native
card crop from the run directory through the Electron bridge, constrained to
PNG files inside the active native run folder. It also shows stored parser
field-confidence rows from native `scan-results.json`, IK/GOOD metadata, dry-run
promotion state, and a `Naechster Schritt` card. Native Review results now have
an inline field editor with approve/reject, authoritative validation, run logs,
and review-to-eval export. It now also shows roll-efficiency reasons,
per-substat legal roll breakdown, build-fit deferral, and under-level
projection when safe.

Outcome:

- Clicking a live row or inventory item opens detail.
- Detail shows screenshot or detail crop when available.
- Detail lists parsed fields, OCR confidence, parser notes, extraction status,
  value score, and scoring reasons.
- Review-required items make the uncertainty explicit and do not present their
  score as final.

Likely files:

- `src/features/inventory/components/*`
- `src/features/scan/components/modals/*`
- `src/lib/artifactOcrParser.ts`
- `src/lib/scoring.ts`

### Phase 5 - Artifact value evaluation

Status: implemented as `roll-efficiency-v1`. Clean legal Artifacts are
evaluated; Review and unknown data receive no confident score. For current
native jobs, only explicitly confirmed 5★ evidence enters evaluation;
explicit 1–4★ evidence becomes `excluded` with no score or projection, while
ambiguous direct star evidence remains `Review`. Existing saved results are
enriched on load and replayed offline without rescanning.

Outcome:

- Add a deterministic artifact value evaluator before promoting build
  recommendations.
- Explain the score through legal roll-tier quality and total roll counts.
- Keep set/main-stat desirability and character/build context separate.
- Keep locked/equipped state out of intrinsic roll quality.
- The evaluator must accept incomplete data and return review/unknown instead
  of confident nonsense.
- Do not turn an `excluded` 1–4★ result into an evaluation, store-promotion,
  projection, or automatic in-game action.

Likely files:

- `src/lib/artifactEvaluation.ts`
- `src/lib/scoring.ts`
- `src/lib/substatRolls.ts`
- `src/lib/genshinLookup.ts`
- targeted unit tests under `src/lib/*.test.ts`

### Phase 6 - Upgrade projection

Status: implemented as detail-only `roll-efficiency-projection-v1` for
under-leveled, unambiguously identified 5-star Artifacts with four known legal
substats. Packaged visual/live acceptance passed on 2026-07-10; the new live
100er contained only +20 Artifacts, so under-level behavior remains covered by
the confirmed saved-run corpus.

Outcome:

- For unambiguously identified 5-star artifacts below max level, show optional
  projection only in detail.
- Provide `worst`, `middle`, and `best` projected value scores.
- Label projection as probabilistic and not a guaranteed result.
- Use known Genshin upgrade constraints and current substats; unknown or
  partially read data must disable or soften the projection.

Likely files:

- `src/lib/artifactEvaluation.ts`
- `src/lib/substatRolls.ts`
- `src/features/inventory/components/*`
- parser/scoring tests

### Phase 7 - Queue-based analysis pipeline

Outcome:

- Introduce a bounded screenshot/crop job queue only after the UI/data contract
  is stable.
- Keep one game-control worker.
- Allow OCR/parse/evaluation workers to process queued jobs.
- Preserve stop/failsafe behavior and review decisions.
- Compare throughput against the current baseline without weakening accuracy.

Status: implemented and live accepted. Native capture writes crop jobs without
waiting for OCR. The downstream processor tails those jobs with bounded
parallelism, streams stable ordered deltas into the result rail, and writes
durable reports. Packaged 20-result and 2,211-result runs proved live capture/
evaluation overlap; the full corpus also passed five-repeat deterministic replay
and saved-run validation after current-package reprocessing. The current build
also writes immutable first-observed lifecycle markers to `run-timing.json`.
It derives end-to-end time directly from request to result reconciliation;
capture and processing durations remain separate and must not be added.

Likely files:

- `src/lib/autoScanLoop.ts`
- `src/lib/scannerSession.ts`
- Electron capture/OCR boundary in `electron/main.ts` or extracted services
- scan-loop tests and live soak scripts

### Phase 8 - Recommendation promotion

Status: offline implementation checkpoint completed. The Builds surface reads
the newest complete native result run, validates source-bound profiles and
five-piece evidence, and exposes up to three read-only, non-overlapping
suggestions. The current package acceptance runner covers the fresh
English/German locale flow, Builds context/focus, reduced motion, and overflow,
and its built-renderer execution passed together with the local-delete
confirmation probe. It is not evidence of an elevated packaged-executable
launch, UAC, preload bridge, or native runtime; those remain separate from the
renderer result and the required current-build scanner smoke.

Outcome:

- Promote only source-bound profile suggestions after scan result quality,
  inventory browsing, detail review, and value scoring are trustworthy.
- Read Build-Fit inputs from complete native results, not metadata-poor store
  records, because canonical IK Set keys and field-confidence evidence are
  required.
- Keep `roll-efficiency-v1` and OCR confidence out of ranking. The visible
  rule-coverage value may use only documented Set/Main-Stat/Substat evidence;
  required full-build context must be explicit.
- Keep suggestions read-only, explain uncertainty/conflicts, and never infer
  character ownership, team, weapon, rotation, or an in-game action.

## Acceptance Criteria

- The scan page still fits the primary workflow without page-level scrolling.
- The preview remains visible during active scan.
- The right rail shows finished artifact evaluations, not noisy intermediate
  parser/debug state.
- Review items are visibly different from weak artifacts.
- Artifact value score and extraction confidence remain separate in data.
- Inventory view can browse stored scan results without opening diagnostics.
- Detail view explains why an artifact received its score.
- Explicit 1–4★ results are visibly excluded rather than scored; ambiguous
  visual rarity is visibly `Review`.
- The `Nicht 5★` inventory filter presents only manual review evidence
  (screenshot/crop and details), never a salvage or Quick Select control.
- A freshly initialized renderer is English; the Settings locale control can
  select German and restore English without implying German Genshin OCR support.
- Artifact detail can remove a wrong local row only after confirmation. Native
  removal preserves raw result evidence through a tombstone, constrains crop
  cleanup to the run directory, and never sends game input.
- Streaming/end-to-end reports use a complete, ordered shared-clock
  `run-timing.json` rather than summed capture and processing durations.
- Upgrade projection never implies a guaranteed future roll.
- Build suggestions use only complete native evidence, source-valid profiles,
  complete Set shapes, and explicit required aggregate context.
- Builds explains profile, sources, chosen five Artifacts, rule coverage,
  conflicts, and deferrals without presenting an optimizer or DPS score.
- Existing safety constraints and scan quality gates remain intact.

## Validation Plan

- `npm run lint`
- `npm test`
- `npm run build`
- `git diff --check`
- `npm run scan:native:replay` against accepted bounded runs and the 2,211 corpus
- `npm run package:offline-check`
- Preserve the passing built-renderer `npm run package:live:builds` evidence
  for locale/Builds/delete UI behavior. Separately launch an elevated packaged
  executable before making a package-runtime/UAC claim.
- `npm audit`
- `npm run scan:native:smoke` before claiming native capture plus
  post-capture processing on live Genshin data.
- For this checkpoint, run the smoke at exactly 50 artifacts with
  `scripts/native-live-smoke.ps1 -Limit 50`; require reconciliation, zero
  processing errors, the review-rate gate, and a complete valid
  `run-timing.json` before cleanup, commit, or push.
- Test local deletion offline for active-run blocking, raw-result preservation,
  tombstone filtering, crop containment, and exact Store scope. The built
  renderer already passed the visible confirmation probe; repeat it only for a
  separate elevated packaged-executable runtime claim.
- Test fresh English/default, German selection, and English restoration through
  the app Settings modal; do not use translated text as the primary automation
  selector.
- Preserve the accepted 20/50/100 and 2,211 evidence before changing capture,
  parser, Review, or evaluation contracts.
- Add unit tests for score/status derivation and upgrade projection.
- Add direct visual 5★/4★/3★ and ambiguous-star fixtures. Verify that 1–4★ is
  stable `excluded`, ambiguity is `Review`, and neither path can be promoted or
  expose an in-game mutation.
- For scanner-facing changes, run a low-limit visible-inventory live scan before
  broad validation, then use the saved full corpus whenever new input is not
  required. The original execution plan is retained in
  [TOMORROW_LIVE_TEST_PLAN_2026-07-10.md](TOMORROW_LIVE_TEST_PLAN_2026-07-10.md).
- Keep `npm run scan:repeatability:wait` for later regression checks, not for
  every UI-only pass.