genshin-assistant/docs/PROJECT.md

# Project

This document is the source of truth for project intent, scope, runtime facts, and operational expectations.

For implementation structure, see [ARCHITECTURE.md](ARCHITECTURE.md). For engineering standards, see [CONVENTIONS.md](CONVENTIONS.md).

## Project Identity

| Field | Value |
| --- | --- |
| Project name | Genshin Artifact Assistant |
| Status | Scanner rebuild in progress |
| Platform | Windows desktop |
| Target users | Genshin Impact players who want artifact decisions without complex optimizer setup |
| Runtime | Electron app with React UI and TypeScript |

## Vision

Genshin Artifact Assistant should make artifact management feel like a no-brainer. The user opens Genshin, runs a scan, and receives simple decisions: what is good, who can use it, what can probably be marked as trash, and which builds are currently available.

The app is not intended to replace deep min-max tools. It prioritizes time savings, confidence, and understandable recommendations over perfect theorycrafting.

## Goals

- Make artifact scanning stable enough that a normal user can trust it without babysitting every click.
- Build one local canonical Genshin data package for artifact sets, pieces, slots, stats, and characters.
- Parse artifact name, slot, main stat, substats, set, equipped state, and confidence deterministically against that package.
- Save weak or failed reads automatically as review samples and turn corrections into reusable local fixes.
- Keep the app offline-first and usable without Genshin Optimizer, Inventory Kamera, Enka, or HoYoLAB.
- Re-introduce recommendations only after the scanner base is trustworthy.

## Non-Goals

- No memory reads, process hooks, game modification, packet inspection, or anti-cheat bypassing.
- No automatic deleting, feeding, enhancing, or spending resources.
- No advanced formula editor or full power-user optimizer in the MVP.
- No cloud sync by default.

## Functional Requirements

| ID | Requirement | Priority | Status |
| --- | --- | --- | --- |
| FR-001 | List capture sources and automatically prefer the detected Genshin window when available. | Must | Prototype |
| FR-002 | Read one currently opened artifact reliably from the local screen and show its parsed result. | Must | Prototype |
| FR-003 | Generate and maintain a local canonical Genshin data package for sets, pieces, slots, stats, characters, aliases, and UI profiles. | Must | In progress |
| FR-004 | Parse artifact fields only through deterministic matching, validation, and derivation against the canonical package. | Must | In progress |
| FR-005 | Run a stable automatic inventory scan: detect grid, click tile, verify detail change, parse, store, continue, scroll, resume. | Must | Prototype |
| FR-006 | Save low-confidence, failed, conflicting, or stale scans automatically as review samples with reason codes. | Must | Prototype |
| FR-007 | Apply local learned fixes from review corrections before every new parse. | Must | Prototype |
| FR-008 | Keep the scan UI operator-friendly: main preview first, debug in modals or drawers, completion summary after scan. | Must | In progress |
| FR-009 | Provide account-level artifact triage after scanner trust is acceptable. | Should | Pending |
| FR-010 | Provide 1-3 simple build suggestions per character from owned artifacts after scanner trust is acceptable. | Should | Pending |
| FR-011 | Farming overlay for reward scans. | Later | Prototype shell |

## Non-Functional Requirements

| Category | Requirement | Measurement |
| --- | --- | --- |
| Safety | Never perform irreversible in-game actions. | Code review and manual test |
| Performance | Single artifact read should feel interactive and batch scan should not stall on false progress. | Capture latency monitored manually; auto-scan stops on blocked verification |
| Privacy | Captures and parsed data stay local by default. | No remote upload in scanner path |
| Reliability | Uncertain OCR must be visible to the user. | Confidence and details view |
| Learning loop | Scanner mistakes should become reusable local review samples. | `review-samples.jsonl` |
| Maintainability | Scanner heuristics must be isolated and documented. | Parser tests, scan-loop tests, data generator, review sample pipeline |

## Tech Stack

| Area | Choice | Notes |
| --- | --- | --- |
| Desktop shell | Electron | Windows local app and overlay windows |
| Frontend | React + TypeScript + Vite | UI and client state |
| Styling | CSS with dark purple glassmorphism system | Premium fintech-inspired visual direction |
| OCR | Tesseract.js prototype plus deterministic normalization/derivation | OCR alone is not trusted as the decision source |
| Capture | Electron desktopCapturer plus Windows GDI Smart Capture | GDI path is used for Genshin Smart Capture reliability |
| Input automation | PowerShell sidecar prototype now, native sidecar planned | Current sidecar is good for proving behavior, not the final production path |
| Tests | Vitest + TypeScript checks | Current validation baseline; regression samples must expand |
| Packaging | electron-builder | Configured in `package.json` |

## Runtime

| Environment | Entry Point | Notes |
| --- | --- | --- |
| Local dev | `npm run dev` | Starts Vite and Electron |
| Local dev with automation | `npm run dev:admin` | Required when `GenshinImpact.exe` is elevated; Windows blocks lower-integrity cursor/click input |
| Production build | `npm run build` | Builds React and Electron main process |
| Preview | `npm run preview` | Browser preview only; capture bridge is unavailable |

## Current State Review

### What already works

- The app can enumerate capture sources and often identify the Genshin window automatically.
- Single-artifact capture is no longer blind full-screen OCR; it produces detail crops, OCR blocks, parsed fields, confidence, and notes.
- A local canonical data package already exists in `src/data/genshinGameData.json`, generated from `genshin-db`.
- The parser already uses known sets, pieces, slots, stat aliases, set aliases, character aliases, and derived slot/set mapping.
- Review samples, learned replacements, parser notes, and stored artifacts already persist locally.
- The auto-scan loop is no longer a naive click spammer: it has preflight, verification, miss handling, page fingerprinting, and stop conditions.

### What is still structurally weak

- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
- The current PowerShell input sidecar is serviceable for experimentation but not a strong production base for long-running, low-jitter auto-scan.
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
- The scan page is cleaner than before, but it still exposes too much operator/debug state in the main flow.
- Recommendations and build logic exist, but the scanner is not yet reliable enough to make them the core focus.

### Current product conclusion

The app should stop behaving like an OCR demo with extra features around it. The next phase is a scanner product rebuild: canonical data first, scan engine second, learning loop third, recommendations later.

## Product Direction

- Artifact scanning is the first-class feature.
- Character optimization returns only after scan quality is trustworthy.
- Team building stays out of the critical path until artifact ingestion is stable.
- Inventory Kamera remains a reference for scan choreography and page movement, not a runtime dependency.
- Self-learning stays deterministic and local first: review samples, aliases, crop offsets, and UI profile tuning before any ML retraining discussion.

## Execution Plan

### Phase 0 - Stabilize the operator surface

Outcome:
- Scan page reduced to source, main preview, result panel, primary scan actions, and compact status.
- Diagnostics, logs, crops, confidence breakdown, review queue, and learning internals moved behind modal or drawer entry points.
- Scan completion popup summarizes scanned, stored, duplicates, review samples, blocked reason, and elapsed time.

Status:
- In progress

### Phase 1 - Canonical game data package

Outcome:
- `scripts/generate-genshin-data.cjs` emits one stricter package contract for:
  - artifact sets
  - artifact pieces
  - slot-by-piece mapping
  - stats and allowed mains by slot
  - characters
  - aliases
  - UI profiles
  - source version metadata
- Parser regression tests run against saved review samples and known bad cases.
- Parser stops "free guessing" outside the canonical package.

Status:
- In progress

### Phase 2 - Deterministic parser hardening

Outcome:
- Name, slot, set, main stat, and equipped fields are parsed through layered validation:
  1. direct OCR cleanup
  2. alias normalization
  3. exact package match
  4. constrained fuzzy match
  5. safe derivation from piece/slot/value references
- Main stat/value inference is tightened with slot constraints and reference tables.
- Bad parses automatically generate structured review reasons.

Status:
- In progress

### Phase 3 - Scanner core rebuild

Outcome:
- Auto-scan becomes a dedicated engine with explicit states:
  - preflight
  - grid detection
  - click target
  - wait stable
  - detail verify
  - parse
  - store or review
  - next tile
  - row scroll
  - resume or stop
- Progress counts only when a new verified artifact or duplicate signature is confirmed.
- Repeated pages, unchanged detail cards, blocked cursor movement, and scroll failures stop the scan with diagnosis instead of producing fake progress.

Status:
- In progress

### Phase 4 - Input automation replacement

Outcome:
- Replace the production automation path with a persistent Windows sidecar dedicated to:
  - focus
  - move
  - click
  - scroll
  - capture
  - probe
- Session probe decides which input mode works before auto-scan is unlocked.
- Auto-scan never starts on a session that cannot prove one successful detail-card change.

Status:
- Planned

### Phase 5 - Learning loop that actually compounds

Outcome:
- Weak scans save themselves as review samples automatically.
- User corrections update local:
  - text replacements
  - alias maps
  - crop offsets
  - UI profile adjustments
  - constrained set/piece/slot fixes
- "Apply learned fixes" runs before every parse.
- Review samples become both parser regression fixtures and learning inputs.

Status:
- Planned

### Phase 6 - Recommendations come back on top of a trusted scanner

Outcome:
- Account snapshot and build suggestions are only promoted once scan quality is high enough to trust owned artifacts.
- Recommendations explain uncertainty and surface conflicts instead of pretending perfect certainty.

Status:
- Deferred until scanner trust is acceptable

## Immediate Next Implementation Order

1. Finish scan-page cleanup so the main operator view is no longer noisy.
2. Tighten the game data generator and parser contract, then backfill regression tests from real bad samples.
3. Continue moving auto-scan behavior out of `App.tsx` and into isolated scanner modules.
4. Replace or wrap the current PowerShell sidecar with a more stable long-lived automation process.
5. Extend the learning system from text-only fixes into crop/UI profile tuning.
6. Resume recommendation work only when scan accuracy is consistently trustworthy.

## Open Questions

| Question | Status |
| --- | --- |
| Should the production input sidecar be Rust/C++ first, or a transitional Node native addon, for the next iteration? | Open |
| When should UI-profile learning be allowed to change crop geometry automatically versus requiring review approval? | Open |
| What scan-quality threshold is high enough before recommendations should be considered user-facing again? | Open |
| Which Genshin UI languages should be supported after English once the scanner contract is stable? | Open |