genshin-assistant/docs/PROJECT.md

# Project

This document is the source of truth for project intent, scope, runtime facts, and operational expectations.

For implementation structure, see [ARCHITECTURE.md](ARCHITECTURE.md). For engineering standards, see [CONVENTIONS.md](CONVENTIONS.md).
For the current app status, see [CURRENT_STATUS.md](CURRENT_STATUS.md).
For the next scan-result and artifact-inventory product phase, see
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md).
For the 2026-07-09 scanner merge evidence, see [MERGE_READINESS.md](MERGE_READINESS.md).
For Gitea push/authentication setup, see [GITEA_AUTH.md](GITEA_AUTH.md).

## Project Identity

| Field | Value |
| --- | --- |
| Project name | Genshin Artifact Assistant |
| Status | Artifact-first scanner baseline with native IK-style 20/50/100 scale evidence, explicit selected promotion, review/edit/approve, scan result rail, and Artifact-only Inventory pipeline; value scoring and later-session repeatability are next |
| Platform | Windows desktop |
| Target users | Genshin Impact players who want artifact decisions without complex optimizer setup |
| Runtime | Electron app with React UI and TypeScript |

## Vision

Genshin Artifact Assistant should make artifact management feel like a no-brainer. The user opens Genshin, runs a scan, and receives simple decisions: what is good, who can use it, what can probably be marked as trash, and which builds are currently available.

The app is not intended to replace deep min-max tools. It prioritizes time savings, confidence, and understandable recommendations over perfect theorycrafting.

## Goals

- Make artifact scanning stable enough that a normal user can trust it without babysitting every click.
- Build one local canonical Genshin data package for artifact sets, pieces, slots, stats, and characters.
- Parse artifact name, slot, main stat, substats, set, equipped state, and confidence deterministically against that package.
- Save weak or failed reads automatically as review samples and turn corrections into reusable local fixes.
- Present finished scan results as a compact artifact list instead of a debug-heavy live stats surface.
- Keep extraction confidence separate from artifact value so uncertain OCR becomes review, not a misleading low score.
- Provide a browsable local artifact inventory with detail views before promoting broader recommendations.
- Keep the app offline-first and usable without optimizer imports, Enka, HoYoLAB,
  or any external scanner as a core dependency.
- Re-introduce recommendations only after the scanner base is trustworthy.

## Non-Goals

- No memory reads, process hooks, game modification, packet inspection, or anti-cheat bypassing.
- No automatic deleting, feeding, enhancing, or spending resources.
- No advanced formula editor or full power-user optimizer in the MVP.
- No cloud sync by default.

## Functional Requirements

| ID | Requirement | Priority | Status |
| --- | --- | --- | --- |
| FR-001 | List capture sources and automatically prefer the detected Genshin window when available. | Must | Implemented |
| FR-002 | Read one currently opened artifact reliably from the local screen and show its parsed result. | Must | Implemented |
| FR-003 | Generate and maintain a local canonical Genshin data package for sets, pieces, slots, stats, characters, aliases, and UI profiles. | Must | Implemented baseline |
| FR-004 | Parse artifact fields only through deterministic matching, validation, and derivation against the canonical package. | Must | Implemented baseline |
| FR-005 | Run a stable automatic inventory scan: detect grid, click tile, verify detail change, parse, store, continue, scroll, resume. | Must | Implemented for visible-inventory baseline |
| FR-006 | Save low-confidence, failed, conflicting, or stale scans automatically as review samples with reason codes. | Must | Implemented baseline |
| FR-007 | Apply local learned fixes from review corrections before every new parse. | Must | Implemented baseline |
| FR-008 | Keep the scan UI operator-friendly: main preview first, debug in modals or drawers, completion summary after scan. | Must | Implemented baseline |
| FR-009 | Provide account-level artifact triage after scanner trust is acceptable. | Should | Pending |
| FR-010 | Provide 1-3 simple build suggestions per character from owned artifacts after scanner trust is acceptable. | Should | Pending |
| FR-011 | Farming overlay for reward scans. | Later | Prototype shell |
| FR-012 | Show active scan results as a minimal right-side rail with artifact number, name or compact fallback, value score, and status pill. | Should | Partial foundation |
| FR-013 | Provide a scanned artifact inventory view with compact score pills, filters, sorting, and click-through detail. | Should | Partial foundation |
| FR-014 | Provide artifact detail evaluation with screenshot/crops, parsed fields, OCR confidence, value reasons, and optional upgrade projection. | Should | Planned |

## Non-Functional Requirements

| Category | Requirement | Measurement |
| --- | --- | --- |
| Safety | Never perform irreversible in-game actions. | Code review and manual test |
| Performance | Single artifact read should feel interactive and batch scan should not stall on false progress. | Capture latency monitored manually; auto-scan stops on blocked verification |
| Performance | A 100-artifact visible-inventory run should finish cleanly with low review/miss rates and report timing evidence. | `npm run scan:goal:validated` or `npm run scan:goal:validated:wait` quality-gated report |
| Privacy | Captures and parsed data stay local by default. | No remote upload in scanner path |
| Reliability | Uncertain OCR must be visible to the user. | Confidence and details view |
| Score integrity | Extraction confidence and artifact value are separate concepts. | Review state can block or qualify a value score |
| Learning loop | Scanner mistakes should become reusable local review samples. | `review-samples.jsonl` |
| Maintainability | Scanner heuristics must be isolated and documented. | Parser tests, scan-loop tests, data generator, review sample pipeline |

## Tech Stack

| Area | Choice | Notes |
| --- | --- | --- |
| Desktop shell | Electron | Windows local app and overlay windows |
| Frontend | React + TypeScript + Vite | UI and client state |
| Styling | CSS with dark purple glassmorphism system | Premium fintech-inspired visual direction |
| OCR | Tesseract.js prototype plus deterministic normalization/derivation | OCR alone is not trusted as the decision source |
| Capture | Electron desktopCapturer plus Windows GDI Smart Capture | GDI path is used for Genshin Smart Capture reliability |
| Input automation | C# sidecar with elevated dev runner when needed | Live-validated for read-only inventory selection clicks; see `docs/AUTOMATION_LIVE_SCAN.md` |
| Tests | Vitest + TypeScript checks | Current validation baseline; regression samples must expand |
| Packaging | electron-builder | Configured in `package.json` |

## Runtime

| Environment | Entry Point | Notes |
| --- | --- | --- |
| Local dev | `npm run dev` | Starts Vite and Electron |
| Local dev with automation | `npm run dev:admin` | Required when `GenshinImpact.exe` is elevated; Windows blocks lower-integrity cursor/click input |
| Production build | `npm run build` | Builds React and Electron main process |
| Preview | `npm run preview` | Browser preview only; capture bridge is unavailable |

## Current State Review

### What already works

- The app can enumerate capture sources and often identify the Genshin window automatically.
- Single-artifact capture is no longer blind full-screen OCR; it produces detail crops, OCR blocks, parsed fields, confidence, and notes.
- A local canonical data package already exists in `src/data/genshinGameData.json`, generated from `genshin-db`.
- The parser already uses known sets, pieces, slots, stat aliases, set aliases, character aliases, and derived slot/set mapping.
- Review samples, learned replacements, parser notes, and stored artifacts already persist locally.
- The scan surface has an initial recent-results rail backed by the newest
  stored artifacts.
- The auto-scan loop is no longer a naive click spammer: it has preflight, verification, miss handling, page fingerprinting, and stop conditions.
- The scanner now has a validated visible-inventory path: 32 safe artifact
  targets per page, lookup-derived fields, fast OCR crop profile, and a
  quality-gated live soak runner.
- Elevated live automation is validated in the current dev environment:
  `/automation/probe-click?index=1` changed the selected artifact and
  `/scanner/start?limit=2` completed with 2/2 verified reads and 0 misses.
- A 2026-07-08 visible-inventory 50-artifact run completed cleanly with
  50/50 parsed and stored, 0 review, 0 duplicates, and 0 misses. It is stable
  but still too slow for the 2-3 artifacts/second target.
- Later 2026-07-08 direct-GDI hot-path runs completed 20/20 parsed with
  0 misses and 0 review. The best clean 20-artifact iteration reached
  7285 ms, or roughly 2.75 artifacts/second; the final stable
  `2026-07-08-direct-gdi-reviewfix` run completed in 7973 ms. The
  3 artifacts/second target remains unproven.
- The same direct-GDI path completed a 100-artifact run with 100/100 parsed,
  0 review, 0 misses, and 42064 ms elapsed across 4 pages.
- Review samples can now be exported with `npm run eval:review-candidates` into
  a Git-ignored human-labeling worklist. This is the next quality phase before
  adding more OCR corpus cases or trusting review queue data as labels.
- Native IK-style Artifact capture is wired through the C# helper. It writes
  card crops and run artifacts for downstream OCR/parse processing instead of
  making React do per-artifact work in the hot capture loop.
- The native post-capture processor can write `scan-results.json` and
  `processing-report.json`, preserve parser field confidence, match Artifact
  results against IK artifact set/piece/slot data, and keep store persistence
  opt-in.
- The Inventory view now has an Artifact-only pipeline surface for scope,
  native capture, OCR queue, review gate, promotion, and evidence. Native rows
  expose crop previews, IK/GOOD metadata, dry-run promotion state, and a
  `Naechster Schritt` card.
- The active UI intentionally hides weapon, material, and character-detail IK
  catalog coverage until those values are actually scanned.

### What is still structurally weak

- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
- Broader scan soak testing has reached clean 20-, 45-, and 100-artifact runs
  with 0 misses on the current engine. The current 2026-07-09 100-artifact run
  completed `100/100` verified and parsed with `0` review and `0` misses.
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
- The scan page now has a minimalist recent-results rail, and the Inventory view
  can inspect native Artifact results, crop previews, IK match state, promotion
  dry-runs, and pipeline risk/status. Value scoring remains incomplete;
  single-result review/edit/approve is implemented and batch review is
  intentionally unavailable.
- The new native IK-style path has same-session 20/50/100 live scale evidence
  with complete capture/result counts and safe Review gating. Later-session
  repeatability and packaged behavior remain open.
- Store promotion from native `scan-results.json` now supports one selected,
  confirmed result at a time with main-process revalidation and a durable log.
  Batch promotion intentionally remains unavailable.
- Native Review results can be corrected and approved or rejected one at a
  time. Approval revalidates IK identity, canonical main values, and legal
  substat rolls, then feeds the existing review-to-eval candidate pipeline.
- Recommendations and build logic exist, but artifact inventory, detail review,
  and value scoring should land first so recommendations have trustworthy inputs.
- Repeatability and 3 artifacts/second are still open; speed work should not
  outrank result clarity, inventory UX, or corpus growth while the current path
  is stable.

### Current product conclusion

The app has crossed from OCR-demo/prototype into an Artifact-first scanner app.
The broadest proven live path is still the visible-inventory scan flow: the
operator opens Artifact inventory with a visible detail card, the app verifies
the state, scans read-only, persists parsed artifacts, and keeps uncertain data
reviewable. The newer native IK-style path is the intended high-speed direction:
the helper captures Artifact card crops quickly, while OCR, parsing, review,
promotion, and value evaluation run downstream.

The next product phase is not broad category expansion. Weapons, materials, and
character details stay out of active scope. The priority is later-session
native repeatability, value
evaluation, and detail explanations before recommendations become the core
product surface.

## Product Direction

- Artifact scanning is the first-class feature.
- Character optimization returns only after scan quality is trustworthy.
- Team building stays out of the critical path until artifact ingestion is stable.
- External scanners are not runtime dependencies. The app owns its scan
  choreography, OCR, and quality gates.
- Self-learning stays deterministic and local first: review samples, aliases, crop offsets, and UI profile tuning before any ML retraining discussion.
- The scan workspace should be an operator surface: preview, live result rail,
  Stop, status, and review access. Detailed stats and debug evidence belong in
  diagnostics, summaries, or artifact detail.
- Artifact value scoring must not hide OCR uncertainty. `Review` is a distinct
  outcome, not just a weak artifact score.

## Execution Plan

### Phase 0 - Stabilize the operator surface

Outcome:
- Scan page reduced to source, main preview, result panel, primary scan actions, and compact status.
- Diagnostics, logs, crops, confidence breakdown, review queue, and learning internals moved behind modal or drawer entry points.
- Scan completion popup summarizes scanned, stored, duplicates, review samples, blocked reason, and elapsed time.

Status:
- Mostly done for the scanner baseline. The Diagnose/dev surface is separated
  and scrollable, scan summaries are compact, and the normal Auto-Scan path is
  guarded. Further UI polish remains useful but no longer blocks scanner merge.

### Phase 1 - Canonical game data package

Outcome:
- `scripts/generate-genshin-data.cjs` emits one stricter package contract for:
  - artifact sets
  - artifact pieces
  - slot-by-piece mapping
  - stats and allowed mains by slot
  - characters
  - aliases
  - UI profiles
  - source version metadata
- Parser regression tests run against saved review samples and known bad cases.
- Parser stops "free guessing" outside the canonical package.

Status:
- Implemented as a generated lookup package in `src/data/genshinGameData.json`
  with validation and parser integration. Continue regenerating and expanding
  aliases deliberately when Genshin data or OCR samples require it.

### Phase 2 - Deterministic parser hardening

Outcome:
- Name, slot, set, main stat, and equipped fields are parsed through layered validation:
  1. direct OCR cleanup
  2. alias normalization
  3. exact package match
  4. constrained fuzzy match
  5. safe derivation from piece/slot/value references
- Main stat/value inference is tightened with slot constraints and reference tables.
- Bad parses automatically generate structured review reasons.
- Equipped-character parsing is canonical-data constrained: noisy known names can
  match through aliases/fuzzy lookup, but unknown footer fragments stay
  `Not detected` instead of being stored as invented character names.

Status:
- Implemented for the merge baseline. Parser tests cover canonical set/slot/stat
  matching, equipped-character footer noise, known aliases, and unsafe one-letter
  fragments. Continue growing the confirmed review corpus before tightening
  thresholds further.

### Phase 3 - Scanner core rebuild

Outcome:
- Auto-scan becomes a dedicated engine with explicit states:
  - preflight
  - grid detection
  - click target
  - wait stable
  - detail verify
  - parse
  - store or review
  - next tile
  - row scroll
  - resume or stop
- Progress counts only when a new verified artifact or duplicate signature is confirmed.
- Repeated pages, unchanged detail cards, blocked cursor movement, and scroll failures stop the scan with diagnosis instead of producing fake progress.
- Fast artifact-read captures include the equipped footer when an equipped
  marker is visible; preflight and polling captures still skip expensive OCR.

Status:
- Implemented and merged for the visible-inventory path. Live validation on
  2026-07-09 covered `20/20` verified/parsed with `0` review and `0` misses,
  equipped-character persistence, unlocked lock state, positive locked state,
  and locked persistence. Explicit entry-mode experiments remain separate from
  the normal merge-ready path.

### Phase 4 - Input automation replacement

Outcome:
- Replace the production automation path with a persistent Windows sidecar dedicated to:
  - focus
  - move
  - click
  - scroll
  - capture
  - probe
- Session probe decides which input mode works before auto-scan is unlocked.
- Auto-scan never starts on a session that cannot prove one successful detail-card change.

Status:
- Read-only C# helper path and elevated dev startup are validated for the
  visible-inventory scanner baseline. Direct inventory, Paimon-menu, and
  auto-entry modes remain Dev-Control experiments and should be tested with low
  limits before being promoted.

### Phase 5 - Learning loop that actually compounds

Outcome:
- Weak scans save themselves as review samples automatically.
- User corrections update local:
  - text replacements
  - alias maps
  - crop offsets
  - UI profile adjustments
  - constrained set/piece/slot fixes
- "Apply learned fixes" runs before every parse.
- Review samples become both parser regression fixtures and learning inputs.

Status:
- Prepared in code for text replacements, field aliases, constrained fixes,
  crop adjustment proposals, and UI-profile adjustment proposals. Crop/profile
  changes still require live review before being auto-applied.

### Phase 6 - Minimal scan result rail and artifact inventory

Outcome:
- The scan page shows a large screenshot/preview and a compact right-side rail
  of finished artifact evaluations.
- Each row shows scan number, artifact name or compact fallback, value score,
  and a colored status pill.
- Debug stats, confidence breakdowns, and OCR internals move out of the main
  scan surface.
- A new artifact inventory menu provides compact browsing, filters, sorting,
  and click-through details.

Status:
- Started with a scan result rail in the scan surface and an `Inventory` view.
  The rail can show native `scan-results.json` entries after post-processing
  and falls back to recent stored artifacts. The inventory browser can filter,
  sort, inspect native/store/snapshot rows, and preview native card crops from
  the selected run directory. It also exposes the current IK inventorylist
  version, active Artifact-only scope, and pipeline status for capture,
  post-processing, review, promotion, and evidence. Native Artifact rows carry
  IK/GOOD match status from the post-capture processor and a dry-run promotion
  decision that shows whether a native result is speicherbar, already stored,
  review-only, or blocked without writing to the store. Detail review exists;
  deterministic value score and value reasons are still pending. See
  [scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md)
  for the implementation phases and acceptance criteria.

### Phase 7 - Artifact detail evaluation and upgrade projection

Outcome:
- Artifact detail explains parsed fields, OCR confidence, scoring reasons, and
  review needs.
- Upgrade projection is available only when enough data is known and is labeled
  as probabilistic, with worst/middle/best projected value scores.
- Low-confidence OCR disables or qualifies value conclusions instead of showing
  false certainty.

Status:
- Planned after the scan result rail and inventory data contracts.

### Phase 8 - Recommendations come back on top of a trusted scanner

Outcome:
- Account snapshot and build suggestions are only promoted once scan quality is high enough to trust owned artifacts.
- Recommendations explain uncertainty and surface conflicts instead of pretending perfect certainty.

Status:
- Next major product area after inventory, detail evaluation, repeatability, and
  corpus work. Do not promote recommendation UX until stored artifact quality is
  backed by more confirmed review samples and repeat live scan runs.

## Immediate Next Implementation Order

1. Add deterministic Artifact value evaluation with explainable reasons.
2. Keep the visible-inventory scanner path as the production baseline and avoid
   promoting `auto-entry`, `direct-inventory`, or `paimon-menu` until they pass
   their own low-limit live validations.
3. Keep the active native scope artifact-only until weapons, materials, and
   character details are actually scanned.
4. Finish the scan result and inventory detail contracts from
   [scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md),
   preserving separate extraction confidence and artifact value.
5. Finish value reasons in the existing Artifact inventory detail view before
   expanding broad build recommendations.
6. Keep the existing scan preview/result rail and Inventory workflow compact;
   move new diagnostics behind the dedicated diagnostics surface.
7. Continue growing the confirmed OCR corpus from review samples exported by
   `npm run eval:review-candidates` and prepared through
   `npm run eval:prepare-confirmed`.
8. Repeat live scanner runs in later sessions to prove repeatability across
   pages, locked/unlocked artifacts, equipped footers, and duplicate handling.
9. Continue the optional `3 artifacts/second` work only if the next change can
   reduce OCR/capture transport time without weakening quality gates.
10. Start recommendation/product UX work only after inventory/detail evaluation,
   repeat scan quality, and confirmed corpus coverage are strong enough to trust
   stored artifacts.

## Open Questions

| Question | Status |
| --- | --- |
| Is the current C# helper sufficient for production packaging, or does a later Rust/C++ sidecar still materially reduce latency or packaging risk? | Open |
| When should UI-profile learning be allowed to change crop geometry automatically versus requiring review approval? | Open |
| What scan-quality threshold is high enough before recommendations should be considered user-facing again? | Open |
| What exact first-pass value formula should drive the `0-100` artifact score before build-aware recommendations exist? | Open |
| Which upgrade projection model is honest enough for early UX: deterministic roll buckets, probability-weighted outcomes, or a deliberately simple best/middle/worst estimate? | Open |
| Which Genshin UI languages should be supported after English once the scanner contract is stable? | Open |