576 lines
39 KiB
Markdown
576 lines
39 KiB
Markdown
# Architecture
|
|
|
|
This document describes the structure, boundaries, flows, and technical rules of the system.
|
|
|
|
## High-Level Overview
|
|
|
|
Genshin Artifact Assistant is a local desktop application. Electron owns OS integration, screen capture, IPC, and overlay windows. React owns the interactive UI. Domain logic for OCR parsing, scoring, scanner state, and recommendations lives in TypeScript modules under `src/lib`.
|
|
|
|
The scanner performance direction is now native-first: the C# input helper owns
|
|
the fast capture/click/scroll loop and uses vendored Inventory Kamera
|
|
`inventorylists` as the scanner dictionary source. Electron is the process,
|
|
IPC, packaging, hotkey, and dev-control shell. React is only the visual control
|
|
and status surface for this path.
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
User["User"]
|
|
Genshin["Genshin Impact Window"]
|
|
Electron["Electron Main Process"]
|
|
Native["C# Input Helper / Native Scanner"]
|
|
React["React Renderer"]
|
|
Parser["OCR Parser and Scoring"]
|
|
LocalData["Local Snapshot / Future SQLite"]
|
|
|
|
User --> React
|
|
React --> Electron
|
|
Electron --> Native
|
|
Native --> Genshin
|
|
Native --> Electron
|
|
Electron --> React
|
|
React --> Parser
|
|
Parser --> React
|
|
React --> LocalData
|
|
```
|
|
|
|
## Architecture Style
|
|
|
|
| Field | Value |
|
|
| --- | --- |
|
|
| Style | Desktop modular monolith |
|
|
| Primary reason | One local app with tight UI, capture, and parsing feedback loops |
|
|
| Main trade-off | Faster iteration now, but scanner heuristics must stay isolated to avoid UI coupling |
|
|
|
|
## Folder Structure
|
|
|
|
```text
|
|
/
|
|
electron/ Electron main process and preload bridge
|
|
src/ React app, domain types, parser/scoring logic
|
|
data/ Seed or package data
|
|
docs/ Project documentation and decisions
|
|
dist/ Generated renderer build
|
|
dist-electron/ Generated Electron build
|
|
outputs/ Packaged app output
|
|
```
|
|
|
|
## Module Responsibilities
|
|
|
|
| Module | Responsibility |
|
|
| --- | --- |
|
|
| `electron/main.ts` | Electron app composition, dependency wiring, app lifecycle, hotkeys, and IPC/dev-control registration |
|
|
| `electron/appWindowManager.ts` | Main window and overlay window lifecycle, menu-bar removal, dashboard focus behavior, and renderer window command delivery |
|
|
| `electron/services/inputHelper.ts` | Stable JSON protocol client for the compiled C# input/capture sidecar plus PowerShell fallback startup |
|
|
| `electron/services/inputHelperPowerShellFallback.ts` | PowerShell fallback script body for environments where the compiled helper is unavailable |
|
|
| `electron/services/nativeScannerProcessingService.ts` | Streaming native-result processor; tails crop jobs, runs bounded ordered OCR/parse/evaluation, publishes result deltas, writes durable processing outputs, and safely loads native crop previews |
|
|
| `electron/services/nativeScannerResultWorkflowService.ts` | Authoritative promotion, review, and local-deletion workflow for native results; reloads run state, validates edits, appends local tombstones, safely removes contained crops, and writes/removes the artifact store only after explicit confirmation |
|
|
| `electron/services/nativeScannerResultTombstones.ts` | Append-only local tombstone contract that hides removed native rows without rewriting their raw scan evidence |
|
|
| `electron/services/nativeScannerRunTiming.ts` | Atomic, first-observation timing record for request, native scanner, capture, processing, durable-result, and reconciliation milestones |
|
|
| `electron/repositories/artifactStoreRepository.ts` | Local JSON artifact-store persistence including exact single-record removal through the main-process boundary |
|
|
| `electron/services/goodFileService.ts` | Local GOOD export file writing and GOOD import file dialog/read handling |
|
|
| `electron/preload.cjs` | Safe renderer bridge exposed as `window.assistantApi` |
|
|
| `native/input-helper/IkInventoryLists.cs` | Loads and validates the vendored IK `inventorylists` feature catalog |
|
|
| `native/input-helper/NativeScannerFiles.cs` | Writes native scanner manifest, status, and JSONL crop-job files |
|
|
| `src/lib/artifactStore.ts` | Pure signature/id/record helpers for the persistent artifact store |
|
|
| `src/lib/ikArtifactMatcher.ts` | Pure IK inventorylist matcher for native artifact set/piece/slot validation and GOOD key metadata |
|
|
| `src/lib/ikCatalogMatcher.ts` | Pure IK inventorylist matcher for simple weapon, character, and material names plus compact GOOD-key samples |
|
|
| `src/lib/ikScanCapabilities.ts` | Pure capability summary that separates IK catalog coverage from implemented native capture support |
|
|
| `src/lib/scanResultEntry.ts` | Pure durable scan-result entry/status helpers that keep extraction confidence separate from versioned Artifact evaluation |
|
|
| `src/App.tsx` | Thin React entry that renders the app page |
|
|
| `src/pages/AppPage.tsx` and `src/pages/app/*` | App page composition and high-level layout routing |
|
|
| `src/features/layout/*` | Scanner-first navigation, grouped workspace/tool items, shell, topbar readiness, and view-level metadata |
|
|
| `src/features/feedback/*` | Renderer-only feedback primitives: queued mutation toasts, inline spinner, and content skeletons |
|
|
| `src/features/scan/hooks/useScanViewController.ts` | Scan feature state composition and view-controller assembly |
|
|
| `src/features/scan/hooks/scanViewScanActions.ts` | Manual scan and visible-grid scan orchestration |
|
|
| `src/features/scan/hooks/scanViewEntryActions.ts` | Guided auto-entry choreography for visible inventory, direct inventory, and ESC/B fallback paths |
|
|
| `src/features/scan/hooks/useScanGoodInterop.ts` | Scan-page GOOD import/export actions against renderer repository ports |
|
|
| `src/features/inventory/*` | Scanned-artifact inventory browser for native result entries, stored artifacts, filters, sorting, compact detail state, native crop preview display, and IK catalog status |
|
|
| `src/features/settings/*` | Application settings modal, including persisted renderer-language selection |
|
|
| `src/i18n/*` | Renderer locale contract, English/German message catalogs, local persistence, document-language synchronization, and translation hook |
|
|
| `src/lib/artifactOcrParser.ts` | Converts OCR output into a parsed artifact candidate with confidence and notes |
|
|
| `src/lib/fuzzyMatch.ts` | Generic fuzzy string matching for OCR text against known game data |
|
|
| `src/lib/genshinLookup.ts` | Pure lookup and validation API for generated Genshin data |
|
|
| `src/lib/autoScanEntry.ts` | Pure entry-mode planning and auto-scan preflight validation |
|
|
| `src/lib/cardReadyGate.ts` | Detail/page fingerprint readiness gate for scan timing |
|
|
| `src/lib/artifactEvaluation.ts` | Deterministic legal-roll efficiency, review/unknown safety, reason codes, stored-result enrichment, and conservative 5-star Worst/Middle/Best under-level projection |
|
|
| `src/lib/buildFitContract*.ts`, `src/lib/buildFitEligibility.ts`, and `src/types/buildFit.ts` | Versioned Build-Fit facade, contract validation including source expiry, native-result input mapping, hard extraction/Review eligibility gates, conflict evidence, and score-free V1 assessment output |
|
|
| `src/lib/buildFitCombination.ts` | Deterministic score-free five-Artifact verifier for unique slots, Set-plan shapes, sourced full-build aggregate context, nested eligibility, and assignment/equipment conflicts |
|
|
| `src/lib/buildFitEvidenceRanking.ts` | Bounded, deterministic, read-only selection of up to three source-bound complete combinations; it uses Set/Main-Stat/Substat evidence only and never Roll-Efficiency or OCR confidence as a rank weight |
|
|
| `src/lib/buildFitExplicitContext.ts` | Converts a user-confirmed, session-only full-build value into a source-bound aggregate context without inferring weapon/team/rotation values |
|
|
| `src/data/buildFitProfiles.ts` | Versioned curated profile corpus with explicit target assumptions, direct source references, expiry, safe evidence thresholds, and no authority outside its documented scope |
|
|
| `src/eval/nativeScanReplay.ts` | Node-only deterministic replay/report contract for repeatedly evaluating existing native scan results without capture or input |
|
|
| `scripts/replay-native-results.ts` | CLI wrapper that locates saved scan results and writes ignored offline replay evidence |
|
|
| `scripts/validate-saved-native-runs.ts` | Read-only accepted-target validator (including 2,211) for run contracts, PNG containment, non-persistence, review provenance, deterministic evaluation, and projection boundaries |
|
|
| `scripts/analyze-native-review.ts` | Read-only reparse and reason-distribution analyzer for a saved native run, including current Review-rate gating without capture or persistence |
|
|
| `scripts/run-offline-acceptance.mjs` | Sequential non-live acceptance runner that records lint, tests, OCR/assessment, saved-run validation, package build/verification, audits, and diff-check outcomes |
|
|
| `scripts/run-packaged-builds-acceptance.mjs` | Built-renderer acceptance runner for fresh English locale, German switching, Builds state, focus, reduced motion, overflow, screenshots, diagnostics, and local-delete confirmation probing |
|
|
| `scripts/native-live-smoke.ps1` | Non-persisting native smoke runner that validates result reconciliation and complete shared-clock timing evidence |
|
|
| `scripts/verify-packaged-app.mjs` | Static and unpacked-app verifier for executable, ASAR entries, relative renderer assets, current runtime signature, preload, C# helper, IK resources, and version consistency |
|
|
| `electron/runtimePaths.ts` | Pure runtime-resource policy: packaged builds use `process.resourcesPath`; repository fallbacks exist only in development unless an explicit environment override is set |
|
|
| `src/lib/scoring.ts` | Legacy demo-only recommendation/build scorer; real local account snapshots never call it, while Build-Fit reads source-safe native evidence through its separate ranker |
|
|
| `src/lib/demoData.ts` | Temporary local demo snapshot |
|
|
| `data/ik-inventorylists/*` | 1:1 vendored Inventory Kamera inventory lists used by the native scanner data preflight and future matching |
|
|
| `src/data/genshinGameData.json` | Generated local dictionary of characters, artifact sets, slots, and stats |
|
|
| `scripts/generate-genshin-data.cjs` | Regenerates the local Genshin dictionary from `genshin-db` |
|
|
| `src/types/*` | Shared app, capture, and domain contracts |
|
|
| `src/styles/global.css` | Legacy stylesheet entrypoint importing the base and diagnostics modules |
|
|
| `src/styles/tokens.css` | Galaxy color, surface, typography, spacing, depth, motion, and layer tokens |
|
|
| `src/styles/motion.css` and `src/styles/feedback.css` | Focus-visible, reduced-motion, page/micro-motion, toast, spinner, and skeleton presentation |
|
|
| `src/styles/*-galaxy.css` and `src/features/inventory/inventory-galaxy.css` | Feature-scoped Galaxy shell, scanner, secondary-view, and inventory presentation |
|
|
| `src/styles/base.css` | Shared application, layout, scanner workspace, modal, triage, build, and overlay compatibility styles |
|
|
| `src/styles/diagnostics.css` | Diagnose/dev-view specific styles |
|
|
|
|
## Dependency Rules
|
|
|
|
- Renderer code calls Electron only through the preload bridge.
|
|
- Electron main process must not import React renderer modules.
|
|
- Pure parsing and scoring modules must not depend on Electron APIs.
|
|
- OCR uncertainty must be represented in data, not hidden in UI only.
|
|
- Feedback primitives may reflect an operation but must not become the source of
|
|
truth for scan, review, promotion, or persistence state.
|
|
- Generated folders must not be treated as source of truth.
|
|
- Application copy is rendered through the locale provider. New, missing, or
|
|
invalid persisted locale values resolve to English; German is a user-selected
|
|
renderer preference. This does not expand the English-only Genshin OCR/crop
|
|
profile.
|
|
- Local-delete flows must remain behind the Electron main-process boundary.
|
|
They may remove app-local records and contained crop files only; they must not
|
|
call the input helper or dispatch game input.
|
|
|
|
## Smart Capture Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant UI as React Scan UI
|
|
participant Bridge as Preload Bridge
|
|
participant Main as Electron Main
|
|
participant Game as Genshin Window
|
|
participant OCR as OCR Worker
|
|
|
|
UI->>Bridge: captureSource(sourceId, 0, focusGenshin=true)
|
|
Bridge->>Main: IPC capture:captureSource
|
|
Main->>Main: Hide app window
|
|
Main->>Game: Focus Genshin window
|
|
Main->>Main: Capture primary screen via GDI
|
|
Main->>Main: Detect artifact detail panel
|
|
Main->>OCR: OCR focused crops
|
|
OCR-->>Main: Text and confidence
|
|
Main->>Main: Restore app window
|
|
Main-->>UI: Full capture, detail preview, crops, OCR results
|
|
UI->>UI: Parse candidate and show result
|
|
```
|
|
|
|
## Data Flow
|
|
|
|
| Data | Source | Owner | Consumers |
|
|
| --- | --- | --- | --- |
|
|
| Capture sources | Electron desktopCapturer | Electron main | Scan UI |
|
|
| Screenshot | Windows GDI / desktopCapturer | Electron main | Cropper, OCR, UI preview |
|
|
| OCR crops | Electron main | Electron main | Details modal, parser |
|
|
| Native card crops | C# input helper | Native scanner run directory under app userData | Electron status, inventory preview, later OCR/parse queue |
|
|
| IK inventorylists | `data/ik-inventorylists` copied from Inventory Kamera 1.4.4 | Source data package | Native scanner data preflight and future matcher |
|
|
| IK simple catalog match | IK weapon/character/material inventorylist catalog | `IkCatalogItemMatch` | Future category scanners, inventory catalog evidence |
|
|
| Game dictionary | `genshin-db` generated JSON | `src/data/genshinGameData.json` | OCR parser |
|
|
| Application locale | Local renderer storage key `gaa-ui-locale` | `src/i18n` | App shell and feature copy; defaults to English |
|
|
| Parsed artifact candidate | OCR parser | Renderer domain logic | Result panel, post-capture report, future local DB |
|
|
| IK artifact match | IK artifact inventorylist catalog | `ScanResultIkMatch` | Native post-capture review gate, scan result entry, inventory detail |
|
|
| Parser field confidence | Native post-capture parser | `ScanResultFieldConfidence` | Scan result entry, inventory detail review signal |
|
|
| Scan result entry | Scan loop or native post-capture processor | `StoredScanResultEntry` / native run `scan-results.json` | Live scan rail, artifact inventory, summary |
|
|
| Native-result tombstone | Confirmed local deletion | Native run `deleted-results.jsonl` | Native-result loaders; raw `scan-results.json` remains replay evidence |
|
|
| Native run timing | First observed lifecycle events | Native run `run-timing.json` | Live-smoke validator, diagnostics, timing reports |
|
|
| Artifact evaluation | Deterministic evaluator | `src/lib` | Result pills, inventory sort/filter, detail reasons |
|
|
| Upgrade projection | Projection helper | `src/lib` | Artifact detail view only |
|
|
| Build-Fit suggestions | Latest complete native `scan-results.json` plus explicit session context | Renderer-only Build-Fit model | Builds surface; read-only profile evidence, never legacy demo scoring |
|
|
| Review samples | User action in Scan UI | Electron userData `review-samples.jsonl` | Future regression tests and OCR training |
|
|
| Stored artifacts | Manual/automatic scans | Electron userData `artifact-store.json` (dedupe by content signature) | Future triage, recommendations, SQLite migration |
|
|
| Recommendations | Scoring module | Renderer domain logic | Triage and builds views |
|
|
|
|
## Scan Modes
|
|
|
|
**Manual scan** is read-only: the user clicks artifacts in Genshin, the app repeatedly runs Smart Capture, deduplicates by content signature, persists new artifacts to the local store, and saves review samples when crops/OCR are missing, total confidence is low, field confidence is low, or parser notes indicate incomplete data.
|
|
|
|
**Automatic grid scan** is user-triggered input automation limited to clicking detected inventory tiles and wheel-scrolling the inventory. Safety and reliability rules:
|
|
|
|
- All input goes through the helper service boundary (currently a C# sidecar with
|
|
fallback support behind the same JSON protocol). The helper owns focus, cursor
|
|
movement, click, scroll, guard-state polling, elevation detection, and GDI
|
|
capture. Mouse movement is sent as iterated relative input deltas instead of
|
|
relying on a single absolute cursor jump. The helper verifies the cursor
|
|
reached the target and refuses to click otherwise.
|
|
- Native and PowerShell helper discovery accepts only the exact supported game
|
|
process names `GenshinImpact` and `YuanShen`. Cached HWNDs are revalidated
|
|
against that whitelist before reuse, so the assistant cannot become its own
|
|
capture/input target merely because its title contains `Genshin`.
|
|
- `npm run dev:admin` is the validated dev path for automation when elevated
|
|
input is required. The elevated PowerShell startup is handled by
|
|
`scripts/dev-admin.ps1` and logged to `outputs/admin-start/admin-dev.log`.
|
|
The user must approve UAC manually; the app cannot approve the Secure Desktop
|
|
prompt itself.
|
|
- Failsafe: before every click and scroll the renderer polls cursor position and ESC state. Holding ESC or moving the mouse away from the last automated position aborts the scan immediately; the Stop button also aborts. Only the `GetAsyncKeyState` held-down bit (0x8000) is used - the "pressed since last call" bit fires for stale ESC presses from normal Genshin menu navigation and caused false aborts.
|
|
- SendInput's return value is checked: zero injected events (UIPI, e.g. elevated Genshin vs. non-elevated app) aborts with an explicit hint instead of silently clicking into nothing.
|
|
- Dev-only probes under `http://127.0.0.1:17317` are used for live validation:
|
|
`/automation/probe-click?index=N` tests one read-only tile selection, and
|
|
`/scanner/start?limit=N` starts the native capture scan with a temporary
|
|
limit payload from an already visible artifact detail view.
|
|
The live known-good result on 2026-07-07 is documented in
|
|
[AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
|
|
- Click verification: after each click the parsed detail-panel signature should change. An unchanged signature is a soft miss (it can also mean two OCR-identical neighbor pieces, common among +0 artifacts), so it is retried once with a small offset, logged with the stuck artifact name, and then skipped - never fatal on its own. The scan aborts only when the first ~6 clicks of page 1 produce nothing new (diagnosis hint: elevated Genshin blocks SendInput via UIPI, or grid coordinates are wrong) or a later page yields zero new artifacts.
|
|
- Scan stats separate clicked (click attempts), parsed (readable captures), stored (persisted), review (review samples), duplicates, and misses, so "scanned" cannot be mistaken for "successfully read".
|
|
- Artifact grid automation uses 32 safe click targets per full page
|
|
(`8 x 4`). The apparent lower fifth row sits in the bottom control band on
|
|
16:9 captures and is not clicked automatically.
|
|
- Guided auto-entry is state gated. The normal scan button first performs a
|
|
lightweight no-OCR preflight; OCR/store/review work starts only after the
|
|
artifact inventory grid and artifact detail card are visually confirmed.
|
|
- The normal scan button does not navigate into inventory when that preflight
|
|
fails; it blocks and asks the operator to open the Artifact inventory with a
|
|
visible detail card. Explicit Dev-Control entry modes can still test direct
|
|
inventory or Paimon-menu choreography, but they are not the merge-ready
|
|
default path.
|
|
- Item verification uses the artifact OCR capture's own detail fingerprint, so
|
|
the loop no longer performs a separate card-ready capture before OCR. Page
|
|
waits remain fingerprint based and can proceed as soon as the inventory pane
|
|
changes and stabilizes.
|
|
- Scrolling sends one wheel notch per grid row with the cursor anchored over the inventory (assumption: roughly one row per notch; overlap is absorbed by dedupe, and a page without new artifacts stops the scan).
|
|
- The scan never deletes, enhances, feeds, locks, or spends anything; it only selects tiles to read them.
|
|
- Inventory cleanup is separate from game interaction: a confirmed local removal may tombstone one native scan result, delete only its contained PNG crop, and optionally remove its explicitly linked local Store record. It never invokes the input helper or changes Genshin data.
|
|
|
|
**Native IK capture scan** is the new high-speed path. It runs inside the C#
|
|
sidecar, computes a fixed 16:9 8x4 visible-inventory grid, clicks and scrolls
|
|
natively, captures the artifact detail card as PNG crops, and reports progress
|
|
through IPC/dev-control. The renderer starts/stops capture and starts one
|
|
downstream processor when the run directory becomes available. OCR, parsing,
|
|
review gating, and artifact value evaluation run concurrently on bounded
|
|
workers so capture speed is not blocked by UI work; GOOD persistence remains a
|
|
separate explicit action.
|
|
|
|
Before the first native Artifact click, the helper moves the cursor to the grid
|
|
anchor, sends a fixed bounded upward wheel sequence that covers the supported
|
|
2,400-item inventory capacity, and waits for the inventory to stabilize. Total,
|
|
top-reset, and active capture timings remain separate so reset cost does not
|
|
distort capture throughput. For streaming end-to-end timing, the app must use
|
|
one wall-clock span from native start through final result reconciliation;
|
|
capture and processing durations may overlap and must never be added. Genshin may be
|
|
focused once during scan startup; after that, the helper compares the actual
|
|
foreground HWND with the original Genshin HWND immediately before every click
|
|
and every wheel event. Focus loss blocks the run without refocusing or sending
|
|
further input. Stop, ESC, Enter, and F9 use the same guard path.
|
|
|
|
The bounded wheel reset remains the production start strategy. Closing or
|
|
toggling the inventory with `B` and then selecting the Artifact tab is only a
|
|
candidate experiment: it depends on the current game/menu state and an
|
|
additional coordinate click, and it has not been live benchmarked against the
|
|
accepted wheel reset. It must not replace that reset without a guarded A/B live
|
|
test that proves both reliability and a meaningful timing benefit.
|
|
|
|
### Scan Scope Resolution
|
|
|
|
The normal renderer scan starts with `{ mode: "all" }`. Before starting the
|
|
native helper it reads the inventory counter and resolves the target from the
|
|
currently owned Artifact count (`current`), never from the slot capacity
|
|
(`total`). Capacity is only a plausibility bound. A missing, invalid, or
|
|
out-of-range owned count blocks a full-inventory run instead of guessing a
|
|
target, because the native helper does not yet have a reliable end-of-list
|
|
detector.
|
|
|
|
The optional bounded scopes are normalized before they reach the helper:
|
|
|
|
- an Artifact limit becomes `min(requested artifacts, owned artifacts)`;
|
|
- a row limit becomes `min(requested rows * detected grid columns, owned
|
|
artifacts)`; the accepted 16:9 profile currently has eight columns;
|
|
- every resolved target stays within the supported 2,400-Artifact safety bound.
|
|
|
|
Scope selection is session state. A new app session returns to the full owned
|
|
inventory default rather than silently reusing an earlier partial limit.
|
|
|
|
The Artifact-limit branch has one real Settings-UI acceptance run at limit 5:
|
|
Genshin/detail/grid preflight passed; target, captured, processed, and parsed
|
|
all reached 5; review and processing errors stayed at 0; persistence remained
|
|
disabled; and the session rail showed five results. A later packaged Settings-
|
|
UI run resolved one row to eight Artifacts and completed 8/8. The packaged
|
|
full-owned-inventory run then resolved 2,211 from the OCR-owned count, captured
|
|
2,211 crops over 70 pages, and populated 2,211 session rows with live capture/
|
|
evaluation overlap. The initial 490-Review result correctly failed the 15% gate;
|
|
current-package reprocessing of the same complete corpus passed at 136 Review
|
|
(6.15%), zero errors, and zero persistence writes. Saved-run validation passed
|
|
with zero issues.
|
|
|
|
Each native run writes a self-contained run directory under app userData:
|
|
|
|
- `manifest.json`: run schema, IK data version/categories, Genshin bounds,
|
|
grid, detail crop rectangle, explicit scan category, and downstream queue
|
|
contract.
|
|
- `capture-jobs.jsonl`: one job per captured card crop with page/row/column,
|
|
client/screen coordinates, click event count, image path, and downstream
|
|
`ocr-parse-store` marker. Jobs include the scan category; today only
|
|
`artifacts` produces jobs. Native preflight and post-capture processing also
|
|
guard this category boundary so catalog-only weapon, character, and material
|
|
entries cannot be accidentally parsed as artifacts.
|
|
- `status.json`: latest scanner status snapshot for recovery and dev tooling.
|
|
Its `supportedCategories` block separates IK catalog availability from native
|
|
capture support: artifacts are the only native-capture category today, while
|
|
weapons, characters, and materials are catalog-only until their own capture
|
|
flows have evidence.
|
|
- `scan-results.json`: durable per-artifact result entries with capture
|
|
metadata, parsed artifact identity, extraction status, versioned value
|
|
evaluation, reason codes, roll breakdown, and score. Clean legal Artifact
|
|
extraction writes `evaluated`; uncertain extraction writes `review`;
|
|
inconsistent/incomplete value data writes `unknown`. `deferred` remains a
|
|
compatibility state and for non-evaluated category/build context boundaries.
|
|
- `processing-report.json`: incremental and final OCR/parse report generated
|
|
from the tailed `capture-jobs.jsonl`. It records `queueConcurrency` for the
|
|
bounded worker pool and is non-persisting by default until native crop OCR
|
|
has live validation evidence.
|
|
- `run-timing.json`: atomic `native-scanner-run-timing-v1` evidence. It records
|
|
first-observed request, scanner, capture, processing, durable-result, and
|
|
result-reconciliation timestamps, then derives request-to-durable and
|
|
request-to-reconciled durations directly from those endpoints.
|
|
- `deleted-results.jsonl`: append-only tombstones for confirmed local removals.
|
|
Loaders hide tombstoned rows while preserving the original result array for
|
|
replay/audit. A tombstone can name one safely contained crop and an optional
|
|
exact linked Store record; it is not a game action.
|
|
|
|
Parsed artifacts from manual and renderer auto-scan modes are persisted into
|
|
`artifact-store.json` keyed by a content signature that excludes the equipped
|
|
character, so re-equipping updates a record instead of duplicating it. Leveling
|
|
an artifact currently creates a new record (documented limitation until
|
|
rescan-merge exists). Native IK capture flushes each crop job immediately; the
|
|
downstream processor tails, OCRs, parses, evaluates, and publishes that job
|
|
without waiting for the full capture target. Store promotion remains opt-in
|
|
until native crop OCR is validated.
|
|
The Inventory surface derives a dry-run promotion summary from native
|
|
`scan-results.json` entries and the local artifact store. It can show which
|
|
native artifacts are ready for explicit promotion, already stored, review-only,
|
|
or blocked, but it does not write store records by itself. It evaluates legacy
|
|
saved results on load, displays roll-efficiency reasons, and computes
|
|
under-level projection without requiring a rescan.
|
|
During auto-scan, artifact store writes can be batched and flushed after the
|
|
click/capture/OCR loop to avoid per-artifact save/reload churn in the hot path.
|
|
Auto-scan artifact captures also bypass Electron source-list enumeration and use
|
|
the GDI capture helper directly once the selected source/Genshin state has been
|
|
preflighted. Manual captures and source refresh still use `desktopCapturer`.
|
|
|
|
### Run Timing Evidence
|
|
|
|
The timing contract is intentionally separate from scanner status polling. The
|
|
first valid timestamp for each milestone is immutable, so retries and later
|
|
renderer polls cannot rewrite a measured interval. `requestToResultsReconciledMs`
|
|
is calculated only from `requestStartedAt` to `resultsReconciledAt`; capture and
|
|
processing can overlap and are never summed to create a synthetic end-to-end
|
|
duration. A live timing claim requires all terminal markers, a valid ordering,
|
|
and final result reconciliation.
|
|
|
|
### Local Artifact Removal
|
|
|
|
The detail delete control is an app-data correction tool, not an in-game action.
|
|
It is unavailable while the relevant native scan or processor is active and
|
|
requires a user confirmation. For a native result, the authoritative workflow
|
|
first appends a tombstone, filters the row from the local view, then removes a
|
|
PNG only after verifying that its resolved path stays within the selected run
|
|
directory. The raw `scan-results.json` evidence is never rewritten. A linked
|
|
Store record is removable only when it exactly matches the selected local
|
|
record and the user explicitly includes it; a Store-only row uses the same
|
|
main-process-scoped local removal. None of these paths invoke the input helper,
|
|
focus Genshin, or modify game data.
|
|
|
|
## Scanner-First Information Architecture
|
|
|
|
The renderer follows the user's task sequence instead of exposing every
|
|
technical subsystem at the same level:
|
|
|
|
1. **Scanner** is the default workspace and owns readiness, start/stop,
|
|
progress, preview, the completed-result rail, and access to review.
|
|
2. **Artifacts** owns collection browsing, filters, selected detail, explicit
|
|
single-result promotion/review, and secondary technical evidence. Collection-
|
|
level pipeline evidence opens in a modal; selected-result OCR evidence stays
|
|
in a local disclosure.
|
|
3. **Review** summarizes extraction uncertainty and routes the user back to the
|
|
relevant Artifact detail. It does not make trash, feed, or deletion claims.
|
|
4. **Builds** reads only the newest complete native result run because it
|
|
retains canonical IK Set keys and per-field confidence. It can show at most
|
|
three source-bound, read-only profile suggestions from the separate evidence
|
|
ranker. Profile expiry, Review gates, complete Set shape, required aggregate
|
|
context, conflict evidence, and non-overlap remain explicit. Current roll
|
|
efficiency is never presented as build fit, and character ownership/team/
|
|
weapon/rotation are not inferred from Artifact data.
|
|
5. **Overlay** and **Diagnostics** are secondary tools. They remain available
|
|
without competing with the primary scan-to-review journey.
|
|
6. **Settings** is a shell-level secondary modal. It currently owns the
|
|
persisted app-language preference; changing it updates renderer copy only
|
|
and does not imply support for another Genshin OCR language.
|
|
|
|
On the Scanner surface, app/Genshin readiness and one primary start/stop action
|
|
come before source selection or manual capture. Advanced source, scope, and
|
|
manual-read controls stay in one focus-trapped modal so opening them never
|
|
reflows the scanner workspace. This is a presentation rule; it does not weaken
|
|
preflight, focus, review, or persistence gates.
|
|
|
|
## Scan Results And Inventory UX
|
|
|
|
The next product surface is documented in
|
|
[scanner-results-inventory-roadmap.md](scanner-results-inventory-roadmap.md).
|
|
Architecture rules:
|
|
|
|
- The active scan view is an operator surface, not the full evaluator. It should
|
|
show readiness, one primary start/stop action, the left preview/work surface,
|
|
the right-side live result rail, progress/status, and review access. While a
|
|
scan is active, the left work surface owns capture phase and target progress;
|
|
the right result surface owns the independent OCR/parse/evaluation progress
|
|
and completed-result stream. A second standalone progress card would
|
|
duplicate that state and is not part of the layout.
|
|
- The live result rail receives fully evaluated artifacts from the current app
|
|
scan session only, in ascending capture sequence, and starts empty on app
|
|
launch. Intermediate OCR/debug payloads stay in diagnostics or detail, while
|
|
older persisted records remain available in the Artifacts view.
|
|
- A scan result row preserves extraction status and artifact value status as
|
|
separate data even when the UI shows one compact pill.
|
|
- The artifact inventory view owns browsing, filtering, sorting, opening detail,
|
|
and showing the current artifact-only native pipeline state. Pipeline and IK
|
|
evidence should be collapsed unless the user asks for it.
|
|
- Weapons, materials, and character details stay out of the active Inventory UI
|
|
until their values are actually scanned; loaded IK catalog data alone is not a
|
|
user-facing scanner capability.
|
|
- The artifact detail view owns screenshot/crop inspection, OCR confidence,
|
|
parser notes, value score reasons, and upgrade projection.
|
|
- Native crop previews are served through the Electron bridge only for PNG paths
|
|
inside the selected native scanner run directory.
|
|
- Native artifact post-processing uses IK artifact set/piece/slot matching as a
|
|
review gate. A conflict is extraction uncertainty, not a weak artifact value.
|
|
- Native artifact post-processing stores per-field parser confidence so detail
|
|
review can explain which OCR/parser fields are trustworthy.
|
|
- Upgrade projection is a local deterministic/probabilistic helper, never a
|
|
claim that an artifact will roll a specific way.
|
|
- `build-fit-evidence-ranking-v1` is not a damage formula or optimizer. It
|
|
ranks only documented Set/Main-Stat/Substat rule coverage after the scanner
|
|
confidence gate. It never consumes Roll-Efficiency, roll magnitude, or OCR
|
|
confidence as a ranking component.
|
|
- A profile with an expired source review fails closed. Required aggregate
|
|
targets remain deferred unless an explicit, user-confirmed session context
|
|
references a declared local/user contract source.
|
|
|
|
Implemented result flow:
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
ScanLoop["Native capture producer"]
|
|
Queue["capture-jobs.jsonl"]
|
|
Workers["Bounded OCR/parse workers"]
|
|
Parsed["Parsed artifact"]
|
|
ReviewGate["Extraction confidence / review gate"]
|
|
Value["Artifact value evaluator"]
|
|
Store["Artifact store"]
|
|
Rail["Live result rail"]
|
|
Inventory["Artifact inventory"]
|
|
Detail["Artifact detail"]
|
|
|
|
ScanLoop --> Queue
|
|
Queue --> Workers
|
|
Workers --> Parsed
|
|
Parsed --> ReviewGate
|
|
ReviewGate --> Value
|
|
Value --> Store
|
|
Value --> Rail
|
|
Store --> Inventory
|
|
Rail --> Detail
|
|
Inventory --> Detail
|
|
```
|
|
|
|
Queue and replay state:
|
|
|
|
- One capture/game-control producer may click, scroll, focus, poll failsafes,
|
|
and append flushed crop jobs.
|
|
- Up to four OCR/parse/evaluation workers tail complete queue records while the
|
|
producer runs. Temporary EOF is non-terminal; producer terminal plus a fully
|
|
drained queue is terminal.
|
|
- Queueing must preserve stop behavior, duplicate handling, review decisions,
|
|
and the existing read-only safety boundary.
|
|
- Results are sequence-deduplicated, reordered to capture order, evaluated
|
|
before publication, exposed to the renderer as monotonic deltas, and written
|
|
incrementally to `scan-results.json` before final reconciliation.
|
|
- Saved `scan-results.json` can be replayed repeatedly through the same value
|
|
and projection code. Replay hashes the complete derived payload and fails
|
|
when a clean parsed Artifact remains unevaluated; it never touches Genshin.
|
|
|
|
## UI Feedback And Motion
|
|
|
|
- Scan progress is durable view state and remains visible while a scan runs;
|
|
toast messages never replace progress, blockers, or final result state.
|
|
- Scan progress is measured against the resolved owned-inventory target or the
|
|
user-selected Artifact/row limit and distinguishes capture/read, parse,
|
|
persistence, review, duplicate, and error outcomes.
|
|
- Skeletons reserve the Inventory list/detail layout during initial reads.
|
|
Inline spinners indicate bounded refresh, preview, promotion, and review work.
|
|
- Toasts report the outcome of explicit renderer mutations such as promotion,
|
|
review approval/rejection, and GOOD import/export. Errors use alert semantics;
|
|
informational and success messages use polite live regions.
|
|
- Shared Galaxy design tokens own semantic colors, surfaces, spacing, depth,
|
|
focus, and motion timing. Feature styles consume those tokens instead of
|
|
inventing scanner-state colors independently.
|
|
- Micro-motion may clarify entry, progress, and state changes, but functionality
|
|
must remain complete with `prefers-reduced-motion: reduce`; focus-visible
|
|
treatment and text/icons must carry meaning without motion or color alone.
|
|
|
|
## Evaluation Boundaries
|
|
|
|
- `roll-efficiency-v1` measures legal substat roll-tier quality only.
|
|
- Set preference, main-stat desirability, character role, and substat usefulness
|
|
are build context and remain `deferred`.
|
|
- Locked/equipped state affects workflow, not intrinsic roll quality.
|
|
- Evaluation uses rarity-specific roll tables and legal total roll counts.
|
|
- Review or incomplete data receives no confident normal score.
|
|
- `roll-efficiency-projection-v1` is available only for unambiguously identified
|
|
5-star Artifacts with four known legal substats below +20. It models legal
|
|
tier bounds, not guaranteed future rolls; 4-star or rarity-ambiguous data is
|
|
deliberately unavailable.
|
|
|
|
## Packaging Boundary
|
|
|
|
- Production packaging uses Electron 43 and electron-builder.
|
|
- Renderer/main/preload live inside `app.asar`; the native helper and IK lists
|
|
are explicit unpacked `extraResources`.
|
|
- Vite uses the relative base `./`, because production windows load
|
|
`dist/index.html` with `BrowserWindow.loadFile` instead of an HTTP origin.
|
|
- In packaged mode helper and IK lookup use `process.resourcesPath` and do not
|
|
search `process.cwd()` or the repository implicitly. Explicit
|
|
`INPUT_HELPER_EXE` and `IK_INVENTORYLISTS_DIR` overrides remain available.
|
|
- `npm run package:offline-check` builds and verifies those paths without
|
|
launching the app or sending game input. The verifier rejects absolute
|
|
renderer asset references and a packaged runtime signature that differs from
|
|
current source.
|
|
- The built-renderer execution of `npm run package:live:builds` passed the
|
|
current English/German locale flow, Builds loading/context/focus, reduced
|
|
motion, overflow, and local-delete confirmation probe. It is renderer
|
|
evidence only: it did not launch an elevated packaged executable or prove its
|
|
UAC/bridge/native-runtime path, and it is not a substitute for native Genshin
|
|
smoke evidence.
|
|
- Packaged runtime, UAC, preload calls, native capture, result loading, and
|
|
Inventory rendering passed live acceptance from `%TEMP%` on 2026-07-10.
|
|
The packaged 5er remained non-persistent; see
|
|
[NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md](NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md).
|
|
|
|
## Security And Safety
|
|
|
|
| Area | Policy |
|
|
| --- | --- |
|
|
| Game access | Screen capture only; no memory reads, hooks, or process injection |
|
|
| Automation | Future feature only, opt-in and reversible |
|
|
| Data privacy | Local-first; no upload path in scanner MVP |
|
|
| Secrets | No cookies or API tokens required for scanner MVP |
|
|
| Unsafe actions | Never delete, feed, enhance, or spend resources |
|
|
|
|
## Error Handling
|
|
|
|
- Capture failures should surface in the scan status row.
|
|
- Missing Electron bridge should explain that browser preview cannot capture Genshin.
|
|
- OCR failures should leave the capture available and mark parsed fields as unknown.
|
|
- Low-confidence data should go to review instead of silent acceptance.
|
|
- Parsed artifact fields carry individual confidence and source metadata so the UI can show uncertainty per field.
|
|
|
|
## Performance
|
|
|
|
Current OCR is measured through the eval harness and live scan assessments
|
|
rather than assumed good. The app keeps a Tesseract.js worker pool and reports
|
|
capture, OCR, card-ready, scroll-ready, active-scan, and projected-100 timings.
|
|
For the next product phase, performance work should not displace extraction
|
|
quality, result clarity, or review safety unless evidence shows a regression.
|