bao/genshin-assistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: initialize repository baseline

Browse Source 
Import the existing Electron + React + TypeScript app as the version-control
baseline before the scanner rework (C# input/capture sidecar, resolution-anchored
layout profiles, OCR preprocessing, eval harness, rescan-merge, GOOD interop).

Housekeeping in this commit:
- Remove orphaned temp_inputhelper_block.ts (duplicate of the input-helper script).
- Ignore .claude/scheduled_tasks.lock local session state.
- Add .gitattributes to normalize line endings (LF in repo).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
AzuTear
2026-07-05 20:31:01 +02:00
commit e76d88e0c7
147 changed files with 26220 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+144
View File
@@ -0,0 +1,144 @@
# Architecture
This document describes the structure, boundaries, flows, and technical rules of the system.
## High-Level Overview
Genshin Artifact Assistant is a local desktop application. Electron owns OS integration, screen capture, IPC, and overlay windows. React owns the interactive UI. Domain logic for OCR parsing, scoring, scanner state, and recommendations lives in TypeScript modules under `src/lib`.
```mermaid
flowchart LR
User["User"]
Genshin["Genshin Impact Window"]
Electron["Electron Main Process"]
React["React Renderer"]
Parser["OCR Parser and Scoring"]
LocalData["Local Snapshot / Future SQLite"]
User --> React
React --> Electron
Electron --> Genshin
Electron --> React
React --> Parser
Parser --> React
React --> LocalData
```
## Architecture Style
| Field | Value |
| --- | --- |
| Style | Desktop modular monolith |
| Primary reason | One local app with tight UI, capture, and parsing feedback loops |
| Main trade-off | Faster iteration now, but scanner heuristics must stay isolated to avoid UI coupling |
## Folder Structure
```text
/
electron/ Electron main process and preload bridge
src/ React app, domain types, parser/scoring logic
data/ Seed or package data
docs/ Project documentation and decisions
dist/ Generated renderer build
dist-electron/ Generated Electron build
outputs/ Packaged app output
```
## Module Responsibilities
| Module | Responsibility |
| --- | --- |
| `electron/main.ts` | Window lifecycle, capture source listing, Smart Capture, OCR crop generation, overlay window IPC, persistent PowerShell input/capture helper, JSON artifact store |
| `electron/preload.cjs` | Safe renderer bridge exposed as `window.assistantApi` |
| `src/lib/artifactStore.ts` | Pure signature/id/record helpers for the persistent artifact store |
| `src/App.tsx` | Main app shell, scan view, triage view, build view, overlay preview |
| `src/lib/artifactOcrParser.ts` | Converts OCR output into a parsed artifact candidate with confidence and notes |
| `src/lib/fuzzyMatch.ts` | Generic fuzzy string matching for OCR text against known game data |
| `src/lib/scoring.ts` | Recommendation and build scoring logic |
| `src/lib/demoData.ts` | Temporary local demo snapshot |
| `src/data/genshinGameData.json` | Generated local dictionary of characters, artifact sets, slots, and stats |
| `scripts/generate-genshin-data.cjs` | Regenerates the local Genshin dictionary from `genshin-db` |
| `src/types/*` | Shared app, capture, and domain contracts |
## Dependency Rules
- Renderer code calls Electron only through the preload bridge.
- Electron main process must not import React renderer modules.
- Pure parsing and scoring modules must not depend on Electron APIs.
- OCR uncertainty must be represented in data, not hidden in UI only.
- Generated folders must not be treated as source of truth.
## Smart Capture Flow
```mermaid
sequenceDiagram
participant UI as React Scan UI
participant Bridge as Preload Bridge
participant Main as Electron Main
participant Game as Genshin Window
participant OCR as OCR Worker
UI->>Bridge: captureSource(sourceId, 0, focusGenshin=true)
Bridge->>Main: IPC capture:captureSource
Main->>Main: Hide app window
Main->>Game: Focus Genshin window
Main->>Main: Capture primary screen via GDI
Main->>Main: Detect artifact detail panel
Main->>OCR: OCR focused crops
OCR-->>Main: Text and confidence
Main->>Main: Restore app window
Main-->>UI: Full capture, detail preview, crops, OCR results
UI->>UI: Parse candidate and show result
```
## Data Flow
| Data | Source | Owner | Consumers |
| --- | --- | --- | --- |
| Capture sources | Electron desktopCapturer | Electron main | Scan UI |
| Screenshot | Windows GDI / desktopCapturer | Electron main | Cropper, OCR, UI preview |
| OCR crops | Electron main | Electron main | Details modal, parser |
| Game dictionary | `genshin-db` generated JSON | `src/data/genshinGameData.json` | OCR parser |
| Parsed artifact candidate | OCR parser | Renderer domain logic | Result panel, future local DB |
| Review samples | User action in Scan UI | Electron userData `review-samples.jsonl` | Future regression tests and OCR training |
| Stored artifacts | Manual/automatic scans | Electron userData `artifact-store.json` (dedupe by content signature) | Future triage, recommendations, SQLite migration |
| Recommendations | Scoring module | Renderer domain logic | Triage and builds views |
## Scan Modes
**Manual scan** is read-only: the user clicks artifacts in Genshin, the app repeatedly runs Smart Capture, deduplicates by content signature, persists new artifacts to the local store, and saves review samples when crops/OCR are missing, total confidence is low, field confidence is low, or parser notes indicate incomplete data.
**Automatic grid scan** is user-triggered input automation limited to clicking detected inventory tiles and wheel-scrolling the inventory. Safety and reliability rules:
- All input goes through one persistent PowerShell helper process (`input-helper.ps1` in userData) that compiles the Win32 interop once and speaks JSON over stdin/stdout (ops: ping, focus, cursor, click, scroll, capture). Mouse movement is sent as iterated relative SendInput deltas (what a real mouse produces): Genshin tracks the cursor via raw input and snaps the OS cursor back to its own position every frame, so SetCursorPos/absolute moves silently stop working once the game owns the cursor. The helper verifies the cursor reached the target and refuses to click otherwise.
- Failsafe: before every click and scroll the renderer polls cursor position and ESC state. Holding ESC or moving the mouse away from the last automated position aborts the scan immediately; the Stop button also aborts. Only the `GetAsyncKeyState` held-down bit (0x8000) is used - the "pressed since last call" bit fires for stale ESC presses from normal Genshin menu navigation and caused false aborts.
- SendInput's return value is checked: zero injected events (UIPI, e.g. elevated Genshin vs. non-elevated app) aborts with an explicit hint instead of silently clicking into nothing.
- Click verification: after each click the parsed detail-panel signature should change. An unchanged signature is a soft miss (it can also mean two OCR-identical neighbor pieces, common among +0 artifacts), so it is retried once with a small offset, logged with the stuck artifact name, and then skipped - never fatal on its own. The scan aborts only when the first ~6 clicks of page 1 produce nothing new (diagnosis hint: elevated Genshin blocks SendInput via UIPI, or grid coordinates are wrong) or a later page yields zero new artifacts.
- Scan stats separate clicked (click attempts), parsed (readable captures), stored (persisted), review (review samples), duplicates, and misses, so "scanned" cannot be mistaken for "successfully read".
- Scrolling sends one wheel notch per grid row with the cursor anchored over the inventory (assumption: roughly one row per notch; overlap is absorbed by dedupe, and a page without new artifacts stops the scan).
- The scan never deletes, enhances, feeds, locks, or spends anything; it only selects tiles to read them.
Parsed artifacts from both modes are persisted into `artifact-store.json` keyed by a content signature that excludes the equipped character, so re-equipping updates a record instead of duplicating it. Leveling an artifact currently creates a new record (documented limitation until rescan-merge exists).
## Security And Safety
| Area | Policy |
| --- | --- |
| Game access | Screen capture only; no memory reads, hooks, or process injection |
| Automation | Future feature only, opt-in and reversible |
| Data privacy | Local-first; no upload path in scanner MVP |
| Secrets | No cookies or API tokens required for scanner MVP |
| Unsafe actions | Never delete, feed, enhance, or spend resources |
## Error Handling
- Capture failures should surface in the scan status row.
- Missing Electron bridge should explain that browser preview cannot capture Genshin.
- OCR failures should leave the capture available and mark parsed fields as unknown.
- Low-confidence data should go to review instead of silent acceptance.
- Parsed artifact fields carry individual confidence and source metadata so the UI can show uncertainty per field.
## Performance
Current OCR is prototype-grade and may be slower than the target scanner. Two batch-scan bottlenecks were removed: input/capture no longer spawn a PowerShell process (and recompile Win32 interop) per action, and the Tesseract worker is created once and reused across captures. The eventual batch scanner should still move expensive capture/OCR/build work into workers or a native sidecar.
docs/CHECKLISTS.md
+37
View File
@@ -0,0 +1,37 @@
# Checklists
## Scanner Change
- [ ] The expected crop or capture behavior is clear.
- [ ] Genshin is not accessed through memory reads, hooks, injection, or game files.
- [ ] Capture failures are shown to the user.
- [ ] OCR uncertainty remains inspectable in Details.
- [ ] Parser output does not silently trust low-confidence text.
- [ ] `npm run lint` passes.
- [ ] `npm test` passes when parser/scoring logic changed.
- [ ] `npm run build` passes.
- [ ] Manual Smart Capture is tested when possible.
## UI Change
- [ ] The main workflow remains visible without unnecessary scrolling.
- [ ] Debug or secondary information is moved behind buttons/modals where appropriate.
- [ ] Disabled states prevent actions without required data.
- [ ] Text fits in controls and panels.
- [ ] Desktop viewport is checked manually.
- [ ] `npm run build` passes.
## Parser Or Scoring Change
- [ ] Known-good OCR samples still parse correctly.
- [ ] Ambiguous data becomes unknown or review, not false certainty.
- [ ] Character, set, stat, and slot dictionaries are updated deliberately.
- [ ] Tests cover changed behavior when practical.
## Release Or Packaging
- [ ] Production build succeeds.
- [ ] Electron preload bridge is copied to `dist-electron/preload.cjs`.
- [ ] App starts outside browser preview.
- [ ] Smart Capture bridge is available.
- [ ] No generated debug artifacts are included accidentally.
docs/CONVENTIONS.md
+50
View File
@@ -0,0 +1,50 @@
# Conventions
This document defines project engineering standards.
## Naming
- Use Genshin domain language where it makes behavior clearer: artifact, slot, set, main stat, substat, triage, build.
- Name booleans as predicates such as `isScanning`, `bridgeReady`, or `isGenshinCandidate`.
- Prefer explicit scanner names such as `createArtifactCrops` over generic names such as `processImage`.
## File Organization
- Keep Electron OS integration in `electron/`.
- Keep React components in `src/`, with extraction when `App.tsx` becomes hard to review.
- Keep pure domain logic in `src/lib/`.
- Keep shared contracts in `src/types/`.
- Keep generated outputs in `dist/`, `dist-electron/`, and `outputs/`.
## UI Rules
- The scan page should prioritize the capture workspace over secondary status content.
- Details and debug information belong in modals or secondary panels.
- Avoid long, overfilled cards on scanner pages.
- The design direction is dark purple fintech glassmorphism with premium, focused controls.
- Disable buttons when their required data does not exist.
## Scanner Rules
- Prefer focused crops over full-screen OCR.
- Confidence and raw OCR details must remain inspectable.
- Heuristics should fail safely into unknown fields or review notes.
- Do not add irreversible game actions.
## TypeScript Rules
- Keep strict type checks passing.
- Do not use `any` for capture, OCR, artifact, or recommendation contracts unless a boundary genuinely requires it.
- Parser functions should be deterministic and testable.
## Testing
- Use unit tests for parser and scoring logic.
- Use build/type checks for Electron IPC contract changes.
- Use manual Smart Capture smoke tests for crop and capture changes.
## Documentation
- Update `docs/PROJECT.md` when product scope changes.
- Update `docs/ARCHITECTURE.md` when module boundaries or flows change.
- Add an ADR to `docs/DECISIONS.md` for durable technical trade-offs.
docs/DECISIONS.md
+135
View File
@@ -0,0 +1,135 @@
# Decisions
This document contains Architecture Decision Records.
## ADR Index
| ID | Title | Status | Date |
| --- | --- | --- | --- |
| ADR-001 | Build a local Electron app first | Accepted | 2026-07-04 |
| ADR-002 | Use screen capture as the primary scanner source | Accepted | 2026-07-04 |
| ADR-003 | Keep APIs and GOOD compatibility optional | Accepted | 2026-07-04 |
| ADR-004 | Treat in-game marking as a later opt-in feature | Accepted | 2026-07-04 |
| ADR-005 | Use a generated Genshin data package for OCR matching | Accepted | 2026-07-04 |
| ADR-006 | Persistent input helper and JSON artifact store before SQLite | Accepted | 2026-07-04 |
## ADR-001: Build A Local Electron App First
### Status
Accepted
### Context
The product needs a Windows desktop UI, local screen capture, possible overlay windows, and future optional input automation.
### Decision
Use Electron with React and TypeScript for the MVP.
### Consequences
- Fast UI iteration and easy local packaging.
- Electron main-process code must be treated as a separate boundary from renderer code.
- Native or Rust sidecars can be added later for high-performance capture/OCR work.
## ADR-002: Use Screen Capture As The Primary Scanner Source
### Status
Accepted
### Context
The user wants an app that works without Inventory Kamera, Genshin Optimizer, Enka, or HoYoLAB as core dependencies.
### Decision
Use local screen capture as the primary source. Current Smart Capture focuses Genshin, hides the app, captures the primary screen through Windows GDI, detects the artifact detail panel, then OCRs focused crops.
### Consequences
- The app remains offline-first.
- OCR and crop reliability are core product risks.
- UI language, resolution, HDR, and game layout changes need explicit test coverage.
## ADR-003: Keep APIs And GOOD Compatibility Optional
### Status
Accepted
### Context
External APIs and existing optimizer formats can speed up setup, but should not define the main user workflow.
### Decision
Keep Enka, HoYoLAB, Akasha, Genshin Optimizer, and GOOD import/export as optional future compatibility layers.
### Consequences
- The app can work without external accounts or cookies.
- Data package and scanner quality become more important.
- Compatibility can be added when it helps testing, migration, or export.
## ADR-004: Treat In-Game Marking As A Later Opt-In Feature
### Status
Accepted
### Context
Locking or marking artifacts in game may save time, but input automation increases ToS and misclick risk.
### Decision
Do not ship in-game marking in the scanner MVP. If implemented later, it must be off by default, reversible, whitelisted, previewed before execution, and stoppable with ESC or user mouse movement.
### Consequences
- Early scanner work stays lower risk.
- App-internal triage remains the first decision layer.
- No delete, feed, enhance, or resource-spending automation is allowed.
## ADR-005: Use A Generated Genshin Data Package For OCR Matching
### Status
Accepted
### Context
Hardcoded arrays for characters and artifact sets caused repeated scanner failures whenever the user tested a newer character, set, or artifact name.
### Decision
Generate `src/data/genshinGameData.json` from `genshin-db` and use it as the local matching dictionary for artifact sets, artifact piece names, characters, slots, main stats, and substats.
### Consequences
- The scanner can recognize new characters and sets as soon as the local data package is regenerated from an updated `genshin-db`.
- Parser logic stays generic and testable instead of growing one-off fixes.
- OCR still needs good crops and text quality; the data package improves recognition but cannot solve unreadable screenshots by itself.
## ADR-006: Persistent Input Helper And JSON Artifact Store Before SQLite
### Status
Accepted
### Context
Per-action PowerShell scripts recompiled the Win32 interop for every click, scroll, and capture (1-2s each) and one `Marshal::SizeOf` call was broken in Windows PowerShell 5.1, so SendInput clicks silently never executed. Scan results were also not persisted anywhere; only review samples reached disk. Adding `better-sqlite3` (native module) was considered too heavy for this step.
### Decision
Run one persistent PowerShell helper process (compiled once, JSON protocol over stdin/stdout) for focus, cursor/ESC state, click, scroll, and GDI capture. Persist parsed artifacts into `artifact-store.json` in userData, deduplicated by a content signature that excludes the equipped character. Keep SQLite as the planned future store; the JSON store is the migration source.
### Consequences
- Batch scans become fast enough to be testable and the failsafe (ESC or user mouse movement aborts) can poll cheaply between actions.
- Automated clicks are verified by checking that the parsed detail signature changed; repeated failures abort with a diagnosis hint instead of clicking blindly.
- Leveling an artifact changes its signature and creates a new record; rescan-merge is an open follow-up.
- If the helper process dies it is respawned on the next request; pending requests fail loudly instead of hanging.
docs/PROJECT.md
+242
View File
@@ -0,0 +1,242 @@
# Project
This document is the source of truth for project intent, scope, runtime facts, and operational expectations.
For implementation structure, see [ARCHITECTURE.md](ARCHITECTURE.md). For engineering standards, see [CONVENTIONS.md](CONVENTIONS.md).
## Project Identity
| Field | Value |
| --- | --- |
| Project name | Genshin Artifact Assistant |
| Status | Scanner rebuild in progress |
| Platform | Windows desktop |
| Target users | Genshin Impact players who want artifact decisions without complex optimizer setup |
| Runtime | Electron app with React UI and TypeScript |
## Vision
Genshin Artifact Assistant should make artifact management feel like a no-brainer. The user opens Genshin, runs a scan, and receives simple decisions: what is good, who can use it, what can probably be marked as trash, and which builds are currently available.
The app is not intended to replace deep min-max tools. It prioritizes time savings, confidence, and understandable recommendations over perfect theorycrafting.
## Goals
- Make artifact scanning stable enough that a normal user can trust it without babysitting every click.
- Build one local canonical Genshin data package for artifact sets, pieces, slots, stats, and characters.
- Parse artifact name, slot, main stat, substats, set, equipped state, and confidence deterministically against that package.
- Save weak or failed reads automatically as review samples and turn corrections into reusable local fixes.
- Keep the app offline-first and usable without Genshin Optimizer, Inventory Kamera, Enka, or HoYoLAB.
- Re-introduce recommendations only after the scanner base is trustworthy.
## Non-Goals
- No memory reads, process hooks, game modification, packet inspection, or anti-cheat bypassing.
- No automatic deleting, feeding, enhancing, or spending resources.
- No advanced formula editor or full power-user optimizer in the MVP.
- No cloud sync by default.
## Functional Requirements
| ID | Requirement | Priority | Status |
| --- | --- | --- | --- |
| FR-001 | List capture sources and automatically prefer the detected Genshin window when available. | Must | Prototype |
| FR-002 | Read one currently opened artifact reliably from the local screen and show its parsed result. | Must | Prototype |
| FR-003 | Generate and maintain a local canonical Genshin data package for sets, pieces, slots, stats, characters, aliases, and UI profiles. | Must | In progress |
| FR-004 | Parse artifact fields only through deterministic matching, validation, and derivation against the canonical package. | Must | In progress |
| FR-005 | Run a stable automatic inventory scan: detect grid, click tile, verify detail change, parse, store, continue, scroll, resume. | Must | Prototype |
| FR-006 | Save low-confidence, failed, conflicting, or stale scans automatically as review samples with reason codes. | Must | Prototype |
| FR-007 | Apply local learned fixes from review corrections before every new parse. | Must | Prototype |
| FR-008 | Keep the scan UI operator-friendly: main preview first, debug in modals or drawers, completion summary after scan. | Must | In progress |
| FR-009 | Provide account-level artifact triage after scanner trust is acceptable. | Should | Pending |
| FR-010 | Provide 1-3 simple build suggestions per character from owned artifacts after scanner trust is acceptable. | Should | Pending |
| FR-011 | Farming overlay for reward scans. | Later | Prototype shell |
## Non-Functional Requirements
| Category | Requirement | Measurement |
| --- | --- | --- |
| Safety | Never perform irreversible in-game actions. | Code review and manual test |
| Performance | Single artifact read should feel interactive and batch scan should not stall on false progress. | Capture latency monitored manually; auto-scan stops on blocked verification |
| Privacy | Captures and parsed data stay local by default. | No remote upload in scanner path |
| Reliability | Uncertain OCR must be visible to the user. | Confidence and details view |
| Learning loop | Scanner mistakes should become reusable local review samples. | `review-samples.jsonl` |
| Maintainability | Scanner heuristics must be isolated and documented. | Parser tests, scan-loop tests, data generator, review sample pipeline |
## Tech Stack
| Area | Choice | Notes |
| --- | --- | --- |
| Desktop shell | Electron | Windows local app and overlay windows |
| Frontend | React + TypeScript + Vite | UI and client state |
| Styling | CSS with dark purple glassmorphism system | Premium fintech-inspired visual direction |
| OCR | Tesseract.js prototype plus deterministic normalization/derivation | OCR alone is not trusted as the decision source |
| Capture | Electron desktopCapturer plus Windows GDI Smart Capture | GDI path is used for Genshin Smart Capture reliability |
| Input automation | PowerShell sidecar prototype now, native sidecar planned | Current sidecar is good for proving behavior, not the final production path |
| Tests | Vitest + TypeScript checks | Current validation baseline; regression samples must expand |
| Packaging | electron-builder | Configured in `package.json` |
## Runtime
| Environment | Entry Point | Notes |
| --- | --- | --- |
| Local dev | `npm run dev` | Starts Vite and Electron |
| Local dev with automation | `npm run dev:admin` | Required when `GenshinImpact.exe` is elevated; Windows blocks lower-integrity cursor/click input |
| Production build | `npm run build` | Builds React and Electron main process |
| Preview | `npm run preview` | Browser preview only; capture bridge is unavailable |
## Current State Review
### What already works
- The app can enumerate capture sources and often identify the Genshin window automatically.
- Single-artifact capture is no longer blind full-screen OCR; it produces detail crops, OCR blocks, parsed fields, confidence, and notes.
- A local canonical data package already exists in `src/data/genshinGameData.json`, generated from `genshin-db`.
- The parser already uses known sets, pieces, slots, stat aliases, set aliases, character aliases, and derived slot/set mapping.
- Review samples, learned replacements, parser notes, and stored artifacts already persist locally.
- The auto-scan loop is no longer a naive click spammer: it has preflight, verification, miss handling, page fingerprinting, and stop conditions.
### What is still structurally weak
- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
- The current PowerShell input sidecar is serviceable for experimentation but not a strong production base for long-running, low-jitter auto-scan.
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
- The scan page is cleaner than before, but it still exposes too much operator/debug state in the main flow.
- Recommendations and build logic exist, but the scanner is not yet reliable enough to make them the core focus.
### Current product conclusion
The app should stop behaving like an OCR demo with extra features around it. The next phase is a scanner product rebuild: canonical data first, scan engine second, learning loop third, recommendations later.
## Product Direction
- Artifact scanning is the first-class feature.
- Character optimization returns only after scan quality is trustworthy.
- Team building stays out of the critical path until artifact ingestion is stable.
- Inventory Kamera remains a reference for scan choreography and page movement, not a runtime dependency.
- Self-learning stays deterministic and local first: review samples, aliases, crop offsets, and UI profile tuning before any ML retraining discussion.
## Execution Plan
### Phase 0 - Stabilize the operator surface
Outcome:
- Scan page reduced to source, main preview, result panel, primary scan actions, and compact status.
- Diagnostics, logs, crops, confidence breakdown, review queue, and learning internals moved behind modal or drawer entry points.
- Scan completion popup summarizes scanned, stored, duplicates, review samples, blocked reason, and elapsed time.
Status:
- In progress
### Phase 1 - Canonical game data package
Outcome:
- `scripts/generate-genshin-data.cjs` emits one stricter package contract for:
- artifact sets
- artifact pieces
- slot-by-piece mapping
- stats and allowed mains by slot
- characters
- aliases
- UI profiles
- source version metadata
- Parser regression tests run against saved review samples and known bad cases.
- Parser stops "free guessing" outside the canonical package.
Status:
- In progress
### Phase 2 - Deterministic parser hardening
Outcome:
- Name, slot, set, main stat, and equipped fields are parsed through layered validation:
1. direct OCR cleanup
2. alias normalization
3. exact package match
4. constrained fuzzy match
5. safe derivation from piece/slot/value references
- Main stat/value inference is tightened with slot constraints and reference tables.
- Bad parses automatically generate structured review reasons.
Status:
- In progress
### Phase 3 - Scanner core rebuild
Outcome:
- Auto-scan becomes a dedicated engine with explicit states:
- preflight
- grid detection
- click target
- wait stable
- detail verify
- parse
- store or review
- next tile
- row scroll
- resume or stop
- Progress counts only when a new verified artifact or duplicate signature is confirmed.
- Repeated pages, unchanged detail cards, blocked cursor movement, and scroll failures stop the scan with diagnosis instead of producing fake progress.
Status:
- In progress
### Phase 4 - Input automation replacement
Outcome:
- Replace the production automation path with a persistent Windows sidecar dedicated to:
- focus
- move
- click
- scroll
- capture
- probe
- Session probe decides which input mode works before auto-scan is unlocked.
- Auto-scan never starts on a session that cannot prove one successful detail-card change.
Status:
- Planned
### Phase 5 - Learning loop that actually compounds
Outcome:
- Weak scans save themselves as review samples automatically.
- User corrections update local:
- text replacements
- alias maps
- crop offsets
- UI profile adjustments
- constrained set/piece/slot fixes
- "Apply learned fixes" runs before every parse.
- Review samples become both parser regression fixtures and learning inputs.
Status:
- Planned
### Phase 6 - Recommendations come back on top of a trusted scanner
Outcome:
- Account snapshot and build suggestions are only promoted once scan quality is high enough to trust owned artifacts.
- Recommendations explain uncertainty and surface conflicts instead of pretending perfect certainty.
Status:
- Deferred until scanner trust is acceptable
## Immediate Next Implementation Order
1. Finish scan-page cleanup so the main operator view is no longer noisy.
2. Tighten the game data generator and parser contract, then backfill regression tests from real bad samples.
3. Continue moving auto-scan behavior out of `App.tsx` and into isolated scanner modules.
4. Replace or wrap the current PowerShell sidecar with a more stable long-lived automation process.
5. Extend the learning system from text-only fixes into crop/UI profile tuning.
6. Resume recommendation work only when scan accuracy is consistently trustworthy.
## Open Questions
| Question | Status |
| --- | --- |
| Should the production input sidecar be Rust/C++ first, or a transitional Node native addon, for the next iteration? | Open |
| When should UI-profile learning be allowed to change crop geometry automatically versus requiring review approval? | Open |
| What scan-quality threshold is high enough before recommendations should be considered user-facing again? | Open |
| Which Genshin UI languages should be supported after English once the scanner contract is stable? | Open |
docs/PROMPTS.md
+37
View File
@@ -0,0 +1,37 @@
# Prompts
## Scanner Bug Report
Use this when reporting a capture/OCR issue:
```text
Genshin resolution:
Display scaling:
Selected source:
Expected crop:
Actual crop:
Screenshot of scan page:
Screenshot of Details modal:
OCR candidates text:
```
## Feature Request
```text
Goal:
User workflow:
Must-have behavior:
Nice-to-have behavior:
Safety constraints:
Validation idea:
```
## Review Request
```text
Please review this change for:
- scanner safety
- parser correctness
- UI clarity
- validation gaps
```
docs/branching.md
+10
View File
@@ -0,0 +1,10 @@
# Branching
This project is currently developed locally. When it becomes a Git repository, use focused branches:
- `feature/<short-name>` for new user-facing features.
- `fix/<short-name>` for bug fixes.
- `docs/<short-name>` for documentation-only work.
- `scanner/<short-name>` for capture, OCR, crop, or parser work.
Keep branches small enough to review and validate quickly.
docs/release-process.md
+25
View File
@@ -0,0 +1,25 @@
# Release Process
## Local Release Readiness
Before packaging:
1. Run `npm run lint`.
2. Run `npm test`.
3. Run `npm run build`.
4. Start the Electron app.
5. Run a manual Smart Capture smoke test.
## Packaging
The package configuration lives in `package.json` under `build`.
Future packaging command:
```powershell
npx electron-builder --win
```
## Rollback
Until releases are formalized, rollback means returning to the last known working project folder or Git commit. Do not overwrite user data or local snapshots during rollback.
docs/repository-setup.md
+29
View File
@@ -0,0 +1,29 @@
# Repository Setup
## Install
```powershell
npm install
```
## Development
```powershell
npm run dev
```
Use the Electron window for scanner work. Browser preview does not expose the capture bridge.
## Validation
```powershell
npm run lint
npm test
npm run build
```
## Notes
- Genshin should be visible on the primary display for the current Smart Capture path.
- The app is Windows-first.
- Scanner OCR is still prototype-grade.
docs/workflow.md
+47
View File
@@ -0,0 +1,47 @@
# Development Workflow
## 1. Clarify Scope
Define whether the task affects scanner capture, OCR parsing, recommendations, UI, documentation, or packaging. Scanner and automation work require extra safety review.
## 2. Inspect Existing Code
Use `rg` and read the relevant files before editing. For this project, likely starting points are:
- `electron/main.ts`
- `electron/preload.cjs`
- `src/App.tsx`
- `src/lib/artifactOcrParser.ts`
- `src/lib/scoring.ts`
- `src/styles/global.css`
## 3. Implement Small Changes
Keep edits focused. Avoid mixing UI redesign, scanner logic, parser logic, and documentation unless the request explicitly spans them.
## 4. Validate
Default validation:
```powershell
npm run lint
npm test
npm run build
```
For scanner changes, restart Electron after build or after changing `electron/main.ts`, because the main process does not hot reload reliably.
## 5. Manual Smoke Test
For Smart Capture:
1. Open Genshin in borderless/windowed mode.
2. Open the artifact inventory detail view.
3. Click `Sources` and confirm Genshin is selected or available.
4. Click `Smart Capture`.
5. Confirm the preview shows the artifact detail panel.
6. Open `Details` and inspect crops/OCR.
## 6. Review Outcome
Document important scanner heuristics, accepted limitations, and future fixes in `docs/DECISIONS.md` or this workflow when they become durable.