genshin-assistant/docs/ARCHITECTURE.md

Architecture

This document describes the structure, boundaries, flows, and technical rules of the system.

High-Level Overview

Genshin Artifact Assistant is a local desktop application. Electron owns OS integration, screen capture, IPC, and overlay windows. React owns the interactive UI. Domain logic for OCR parsing, scoring, scanner state, and recommendations lives in TypeScript modules under src/lib.

The scanner performance direction is now native-first: the C# input helper owns the fast capture/click/scroll loop and uses vendored Inventory Kamera inventorylists as the scanner dictionary source. Electron is the process, IPC, packaging, hotkey, and dev-control shell. React is only the visual control and status surface for this path.

flowchart LR
    User["User"]
    Genshin["Genshin Impact Window"]
    Electron["Electron Main Process"]
    Native["C# Input Helper / Native Scanner"]
    React["React Renderer"]
    Parser["OCR Parser and Scoring"]
    LocalData["Local Snapshot / Future SQLite"]

    User --> React
    React --> Electron
    Electron --> Native
    Native --> Genshin
    Native --> Electron
    Electron --> React
    React --> Parser
    Parser --> React
    React --> LocalData

Architecture Style

Field	Value
Style	Desktop modular monolith
Primary reason	One local app with tight UI, capture, and parsing feedback loops
Main trade-off	Faster iteration now, but scanner heuristics must stay isolated to avoid UI coupling

Folder Structure

/
  electron/          Electron main process and preload bridge
  src/               React app, domain types, parser/scoring logic
  data/              Seed or package data
  docs/              Project documentation and decisions
  dist/              Generated renderer build
  dist-electron/     Generated Electron build
  outputs/           Packaged app output

Module Responsibilities

Module	Responsibility
electron/main.ts	Electron app composition, dependency wiring, app lifecycle, hotkeys, and IPC/dev-control registration
electron/appWindowManager.ts	Main window and overlay window lifecycle, menu-bar removal, dashboard focus behavior, and renderer window command delivery
electron/services/inputHelper.ts	Stable JSON protocol client for the compiled C# input/capture sidecar plus PowerShell fallback startup
electron/services/inputHelperPowerShellFallback.ts	PowerShell fallback script body for environments where the compiled helper is unavailable
electron/services/nativeScannerProcessingService.ts	Post-capture processor for native IK runs; reads crop jobs, OCRs/parses card crops, matches IK metadata, writes processing outputs, and safely loads native crop previews
electron/services/nativeScannerResultWorkflowService.ts	Authoritative promotion and review workflow for native results; reloads run state, validates edits, updates durable results/logs, and only writes the artifact store after explicit confirmation
electron/services/goodFileService.ts	Local GOOD export file writing and GOOD import file dialog/read handling
electron/preload.cjs	Safe renderer bridge exposed as window.assistantApi
native/input-helper/IkInventoryLists.cs	Loads and validates the vendored IK inventorylists feature catalog
native/input-helper/NativeScannerFiles.cs	Writes native scanner manifest, status, and JSONL crop-job files
src/lib/artifactStore.ts	Pure signature/id/record helpers for the persistent artifact store
src/lib/ikArtifactMatcher.ts	Pure IK inventorylist matcher for native artifact set/piece/slot validation and GOOD key metadata
src/lib/ikCatalogMatcher.ts	Pure IK inventorylist matcher for simple weapon, character, and material names plus compact GOOD-key samples
src/lib/ikScanCapabilities.ts	Pure capability summary that separates IK catalog coverage from implemented native capture support
src/lib/scanResultEntry.ts	Pure durable scan-result entry/status helpers that keep extraction confidence separate from deferred artifact value evaluation
src/App.tsx	Thin React entry that renders the app page
src/pages/AppPage.tsx and src/pages/app/*	App page composition and high-level layout routing
src/features/scan/hooks/useScanViewController.ts	Scan feature state composition and view-controller assembly
src/features/scan/hooks/scanViewScanActions.ts	Manual scan and visible-grid scan orchestration
src/features/scan/hooks/scanViewEntryActions.ts	Guided auto-entry choreography for visible inventory, direct inventory, and ESC/B fallback paths
src/features/scan/hooks/useScanGoodInterop.ts	Scan-page GOOD import/export actions against renderer repository ports
src/features/inventory/*	Scanned-artifact inventory browser for native result entries, stored artifacts, filters, sorting, compact detail state, native crop preview display, and IK catalog status
src/lib/artifactOcrParser.ts	Converts OCR output into a parsed artifact candidate with confidence and notes
src/lib/fuzzyMatch.ts	Generic fuzzy string matching for OCR text against known game data
src/lib/genshinLookup.ts	Pure lookup and validation API for generated Genshin data
src/lib/autoScanEntry.ts	Pure entry-mode planning and auto-scan preflight validation
src/lib/cardReadyGate.ts	Detail/page fingerprint readiness gate for scan timing
src/lib/artifactEvaluation.ts	Planned deterministic artifact value evaluation with score reasons and review-safe output
src/lib/upgradeProjection.ts	Planned best/middle/worst upgrade projection for under-leveled artifacts
src/lib/scoring.ts	Recommendation and build scoring logic
src/lib/demoData.ts	Temporary local demo snapshot
data/ik-inventorylists/*	1:1 vendored Inventory Kamera inventory lists used by the native scanner data preflight and future matching
src/data/genshinGameData.json	Generated local dictionary of characters, artifact sets, slots, and stats
scripts/generate-genshin-data.cjs	Regenerates the local Genshin dictionary from genshin-db
src/types/*	Shared app, capture, and domain contracts
src/styles/global.css	Stylesheet entrypoint importing split style modules
src/styles/base.css	Shared application, layout, scanner workspace, modal, triage, build, and overlay styles
src/styles/diagnostics.css	Diagnose/dev-view specific styles

Dependency Rules

• Renderer code calls Electron only through the preload bridge.
• Electron main process must not import React renderer modules.
• Pure parsing and scoring modules must not depend on Electron APIs.
• OCR uncertainty must be represented in data, not hidden in UI only.
• Generated folders must not be treated as source of truth.

Smart Capture Flow

sequenceDiagram
    participant UI as React Scan UI
    participant Bridge as Preload Bridge
    participant Main as Electron Main
    participant Game as Genshin Window
    participant OCR as OCR Worker

    UI->>Bridge: captureSource(sourceId, 0, focusGenshin=true)
    Bridge->>Main: IPC capture:captureSource
    Main->>Main: Hide app window
    Main->>Game: Focus Genshin window
    Main->>Main: Capture primary screen via GDI
    Main->>Main: Detect artifact detail panel
    Main->>OCR: OCR focused crops
    OCR-->>Main: Text and confidence
    Main->>Main: Restore app window
    Main-->>UI: Full capture, detail preview, crops, OCR results
    UI->>UI: Parse candidate and show result

Data Flow

Data	Source	Owner	Consumers
Capture sources	Electron desktopCapturer	Electron main	Scan UI
Screenshot	Windows GDI / desktopCapturer	Electron main	Cropper, OCR, UI preview
OCR crops	Electron main	Electron main	Details modal, parser
Native card crops	C# input helper	Native scanner run directory under app userData	Electron status, inventory preview, later OCR/parse queue
IK inventorylists	data/ik-inventorylists copied from Inventory Kamera 1.4.4	Source data package	Native scanner data preflight and future matcher
IK simple catalog match	IK weapon/character/material inventorylist catalog	IkCatalogItemMatch	Future category scanners, inventory catalog evidence
Game dictionary	genshin-db generated JSON	src/data/genshinGameData.json	OCR parser
Parsed artifact candidate	OCR parser	Renderer domain logic	Result panel, post-capture report, future local DB
IK artifact match	IK artifact inventorylist catalog	ScanResultIkMatch	Native post-capture review gate, scan result entry, inventory detail
Parser field confidence	Native post-capture parser	ScanResultFieldConfidence	Scan result entry, inventory detail review signal
Scan result entry	Scan loop or native post-capture processor	StoredScanResultEntry / native run scan-results.json	Live scan rail, artifact inventory, summary
Artifact evaluation	Deterministic evaluator	src/lib	Result pills, inventory sort/filter, detail reasons
Upgrade projection	Projection helper	src/lib	Artifact detail view only
Review samples	User action in Scan UI	Electron userData review-samples.jsonl	Future regression tests and OCR training
Stored artifacts	Manual/automatic scans	Electron userData artifact-store.json (dedupe by content signature)	Future triage, recommendations, SQLite migration
Recommendations	Scoring module	Renderer domain logic	Triage and builds views

Scan Modes

Manual scan is read-only: the user clicks artifacts in Genshin, the app repeatedly runs Smart Capture, deduplicates by content signature, persists new artifacts to the local store, and saves review samples when crops/OCR are missing, total confidence is low, field confidence is low, or parser notes indicate incomplete data.

Automatic grid scan is user-triggered input automation limited to clicking detected inventory tiles and wheel-scrolling the inventory. Safety and reliability rules:

• All input goes through the helper service boundary (currently a C# sidecar with fallback support behind the same JSON protocol). The helper owns focus, cursor movement, click, scroll, guard-state polling, elevation detection, and GDI capture. Mouse movement is sent as iterated relative input deltas instead of relying on a single absolute cursor jump. The helper verifies the cursor reached the target and refuses to click otherwise.
• npm run dev:admin is the validated dev path for automation when elevated input is required. The elevated PowerShell startup is handled by scripts/dev-admin.ps1 and logged to outputs/admin-start/admin-dev.log. The user must approve UAC manually; the app cannot approve the Secure Desktop prompt itself.
• Failsafe: before every click and scroll the renderer polls cursor position and ESC state. Holding ESC or moving the mouse away from the last automated position aborts the scan immediately; the Stop button also aborts. Only the GetAsyncKeyState held-down bit (0x8000) is used - the "pressed since last call" bit fires for stale ESC presses from normal Genshin menu navigation and caused false aborts.
• SendInput's return value is checked: zero injected events (UIPI, e.g. elevated Genshin vs. non-elevated app) aborts with an explicit hint instead of silently clicking into nothing.
• Dev-only probes under http://127.0.0.1:17317 are used for live validation: /automation/probe-click?index=N tests one read-only tile selection, and /scanner/start?limit=N starts the native capture scan with a temporary limit payload from an already visible artifact detail view. The live known-good result on 2026-07-07 is documented in AUTOMATION_LIVE_SCAN.md.
• Click verification: after each click the parsed detail-panel signature should change. An unchanged signature is a soft miss (it can also mean two OCR-identical neighbor pieces, common among +0 artifacts), so it is retried once with a small offset, logged with the stuck artifact name, and then skipped - never fatal on its own. The scan aborts only when the first ~6 clicks of page 1 produce nothing new (diagnosis hint: elevated Genshin blocks SendInput via UIPI, or grid coordinates are wrong) or a later page yields zero new artifacts.
• Scan stats separate clicked (click attempts), parsed (readable captures), stored (persisted), review (review samples), duplicates, and misses, so "scanned" cannot be mistaken for "successfully read".
• Artifact grid automation uses 32 safe click targets per full page (8 x 4). The apparent lower fifth row sits in the bottom control band on 16:9 captures and is not clicked automatically.
• Guided auto-entry is state gated. The normal scan button first performs a lightweight no-OCR preflight; OCR/store/review work starts only after the artifact inventory grid and artifact detail card are visually confirmed.
• The normal scan button does not navigate into inventory when that preflight fails; it blocks and asks the operator to open the Artifact inventory with a visible detail card. Explicit Dev-Control entry modes can still test direct inventory or Paimon-menu choreography, but they are not the merge-ready default path.
• Item verification uses the artifact OCR capture's own detail fingerprint, so the loop no longer performs a separate card-ready capture before OCR. Page waits remain fingerprint based and can proceed as soon as the inventory pane changes and stabilizes.
• Scrolling sends one wheel notch per grid row with the cursor anchored over the inventory (assumption: roughly one row per notch; overlap is absorbed by dedupe, and a page without new artifacts stops the scan).
• The scan never deletes, enhances, feeds, locks, or spends anything; it only selects tiles to read them.

Native IK capture scan is the new high-speed path. It runs inside the C# sidecar, computes a fixed 16:9 8x4 visible-inventory grid, clicks and scrolls natively, captures the artifact detail card as PNG crops, and reports progress through IPC/dev-control. The renderer only starts, stops, and displays status. OCR, parsing, GOOD persistence, and artifact value evaluation are deliberately separate follow-up stages so capture speed is not blocked by UI work.

Each native run writes a self-contained run directory under app userData:

• manifest.json: run schema, IK data version/categories, Genshin bounds, grid, detail crop rectangle, explicit scan category, and downstream queue contract.
• capture-jobs.jsonl: one job per captured card crop with page/row/column, client/screen coordinates, click event count, image path, and downstream ocr-parse-store marker. Jobs include the scan category; today only artifacts produces jobs. Native preflight and post-capture processing also guard this category boundary so catalog-only weapon, character, and material entries cannot be accidentally parsed as artifacts.
• status.json: latest scanner status snapshot for recovery and dev tooling. Its supportedCategories block separates IK catalog availability from native capture support: artifacts are the only native-capture category today, while weapons, characters, and materials are catalog-only until their own capture flows have evidence.
• scan-results.json: durable per-artifact result entries with capture metadata, parsed artifact identity when available, extraction status, and value status. Native capture currently writes deferred value status for clean extraction and review for uncertain extraction.
• processing-report.json: optional post-capture OCR/parse report generated from capture-jobs.jsonl. It records queueConcurrency for the bounded post-capture OCR/parse worker and is non-persisting by default until native crop OCR has live validation evidence.

Parsed artifacts from manual and renderer auto-scan modes are persisted into artifact-store.json keyed by a content signature that excludes the equipped character, so re-equipping updates a record instead of duplicating it. Leveling an artifact currently creates a new record (documented limitation until rescan-merge exists). Native IK capture currently writes card crops first; a downstream OCR/parse processor can report parsed artifacts from those crops without slowing capture. Store promotion remains opt-in until native crop OCR is validated. The Inventory surface derives a dry-run promotion summary from native scan-results.json entries and the local artifact store. It can show which native artifacts are ready for explicit promotion, already stored, review-only, or blocked, but it does not write store records by itself. During auto-scan, artifact store writes can be batched and flushed after the click/capture/OCR loop to avoid per-artifact save/reload churn in the hot path. Auto-scan artifact captures also bypass Electron source-list enumeration and use the GDI capture helper directly once the selected source/Genshin state has been preflighted. Manual captures and source refresh still use desktopCapturer.

Scan Results And Inventory UX

The next product surface is documented in scanner-results-inventory-roadmap.md. Architecture rules:

• The active scan view is an operator surface, not the full evaluator. It should show the screenshot/preview, the right-side live result rail, Stop/status controls, and review access.
• The live result rail receives completed artifact results only. Intermediate OCR/debug stats stay in diagnostics or detail.
• A scan result row preserves extraction status and artifact value status as separate data even when the UI shows one compact pill.
• The artifact inventory view owns browsing, filtering, sorting, opening detail, and showing the current artifact-only native pipeline state.
• Weapons, materials, and character details stay out of the active Inventory UI until their values are actually scanned; loaded IK catalog data alone is not a user-facing scanner capability.
• The artifact detail view owns screenshot/crop inspection, OCR confidence, parser notes, value score reasons, and upgrade projection.
• Native crop previews are served through the Electron bridge only for PNG paths inside the selected native scanner run directory.
• Native artifact post-processing uses IK artifact set/piece/slot matching as a review gate. A conflict is extraction uncertainty, not a weak artifact value.
• Native artifact post-processing stores per-field parser confidence so detail review can explain which OCR/parser fields are trustworthy.
• Upgrade projection is a local deterministic/probabilistic helper, never a claim that an artifact will roll a specific way.

Planned result flow:

flowchart LR
    ScanLoop["Scan loop"]
    Parsed["Parsed artifact"]
    ReviewGate["Extraction confidence / review gate"]
    Value["Artifact value evaluator"]
    Store["Artifact store"]
    Rail["Live result rail"]
    Inventory["Artifact inventory"]
    Detail["Artifact detail"]

    ScanLoop --> Parsed
    Parsed --> ReviewGate
    ReviewGate --> Value
    Value --> Store
    Value --> Rail
    Store --> Inventory
    Rail --> Detail
    Inventory --> Detail

Future queue refactor:

• One capture/game-control worker may click, scroll, focus, and poll failsafes.
• OCR/parse/evaluation may process bounded queued screenshot/crop jobs.
• Queueing must preserve stop behavior, duplicate handling, review decisions, and the existing read-only safety boundary.
• The native post-capture processor already consumes crop jobs with bounded parallelism while preserving report order. Further queue work is secondary to live throughput evidence and result quality.

Security And Safety

Area	Policy
Game access	Screen capture only; no memory reads, hooks, or process injection
Automation	Future feature only, opt-in and reversible
Data privacy	Local-first; no upload path in scanner MVP
Secrets	No cookies or API tokens required for scanner MVP
Unsafe actions	Never delete, feed, enhance, or spend resources

Error Handling

• Capture failures should surface in the scan status row.
• Missing Electron bridge should explain that browser preview cannot capture Genshin.
• OCR failures should leave the capture available and mark parsed fields as unknown.
• Low-confidence data should go to review instead of silent acceptance.
• Parsed artifact fields carry individual confidence and source metadata so the UI can show uncertainty per field.

Performance

Current OCR is measured through the eval harness and live scan assessments rather than assumed good. The app keeps a Tesseract.js worker pool and reports capture, OCR, card-ready, scroll-ready, active-scan, and projected-100 timings. For the next product phase, performance work should not displace extraction quality, result clarity, or review safety unless evidence shows a regression.