13 Commits

Author SHA1 Message Date
AzuTear ef65c3e6a0 feat(scanner): validate elevated live automation 2026-07-07 07:49:22 +02:00
AzuTear 7930e369a7 feat(ocr): substat-roll validation + rarity inference for GOOD export
Adds the accuracy check yas / Genshin Optimizer use: a substat value is only
legitimate if it equals round(sum of 1..6 rolls) from that stat's roll table.
Values that fit no combination at either rarity are guaranteed OCR misreads.

- src/lib/substatRolls.ts: 5-star roll tables (+ 4-star %/crit tables to tell
  rarities apart), pure isPlausibleSubstat/implausibleSubstats, and inferRarity
  (level > 16 or roll-table fit; conservative, defaults to 5). Validates against
  the union of rarities so valid 4-star pieces are not false-flagged.
- Wired in: shouldFlagArtifactForReview routes implausible substats to review;
  the parser adds an explanatory note; goodInterop export replaces the hardcoded
  rarity:5 with inferRarity (fixes wrong 4-star exports to GO/IK).
- 10 new unit tests; 131 total green; eval still 100%.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:21:28 +02:00
AzuTear b8309af377 fix(input): inject a no-op input event so force-foreground actually works
Follow-up to the AttachThreadInput change: verified against an isolated repro
(a foreground-stealing window + the helper spawned exactly like the app) that
AttachThreadInput + clearing the foreground-lock timeout was NOT sufficient on
this Windows build - SetForegroundWindow still returned false and Genshin stayed
in the background.

The missing condition is "the calling process received the last input event".
Injecting a benign no-op input (a 0,0 relative mouse move, no cursor movement, no
menu-mnemonic side effect) right before SetForegroundWindow satisfies it. With the
nudge the repro now returns focused:true / setForegroundResult:true from a
background process while another app holds the foreground - the exact auto-scan
start scenario. Applied to both the C# sidecar and the PowerShell fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:13:25 +02:00
AzuTear c8ae0dd7bf fix(input): force Genshin foreground via AttachThreadInput (auto-scan no longer aborts)
Auto-scan aborted immediately with "Genshin konnte nicht in den Vordergrund
geholt werden". Root cause: the focus call runs in the background input/capture
helper process, and Windows' foreground lock silently refuses SetForegroundWindow
from a process that is neither foreground nor the last input source. When the user
clicks "Auto-Scan starten" the Electron window is foreground, so the helper's plain
SetForegroundWindow is dropped and focus stays false.

Fix (both the C# sidecar and the PowerShell fallback): before SetForegroundWindow,
attach our thread's input queue to the target (and current-foreground) window
thread with AttachThreadInput and clear SPI_..FOREGROUNDLOCKTIMEOUT, then restore.
This is the same technique Inventory Kamera and other reliable automators use; it
is what our helper was missing after the old ALT-tap workaround was removed on the
wrong assumption that equal integrity level is sufficient (that only covers UIPI
input injection, not foreground changes).

Sidecar recompiled + republished; electron build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 16:07:30 +02:00
AzuTear 113601f031 fix(electron): point main + preload + index paths at the real build layout
npm run dev failed with ERR_MODULE_NOT_FOUND for dist-electron/services/inputHelper.

Root cause: the electron program includes src runtime files (repositories,
layoutProfile, ocrPreprocess), so tsc's inferred rootDir is the project root and
it emits the entry at dist-electron/electron/main.js (with src at
dist-electron/src). package.json "main" still pointed at a stale flat
dist-electron/main.js fossil from an older build layout, whose extensionless
imports don't resolve under NodeNext ESM.

- package.json main -> dist-electron/electron/main.js.
- predev/build copy preload.cjs into dist-electron/electron/ (next to main.js,
  where main.ts resolves it via __dirname).
- main.ts loads ../../dist/index.html (one level deeper now) for the packaged
  window + overlay.

Verified: clean electron build emits only the nested layout; electron . loads the
main process past module resolution (only ERR_CONNECTION_REFUSED for the dev
server, expected standalone); npm run build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 08:11:34 +02:00
AzuTear 39691fd40b feat(good): wire GOOD import/export UI into the Diagnose view
Completes the GOOD interop point end-to-end (the conversion engine landed earlier
in goodInterop.ts). No new IPC needed - reuses the existing artifacts:load /
artifacts:saveMany / good:export bridge.

- Scan controller gains exportGoodFromStore (loadArtifacts -> storedArtifactsToGood
  -> exportGood) and importGoodArtifacts (saveMany + snapshot refresh), plus a
  canGoodInterop flag.
- DiagnosticsView adds a GOOD Interop card: export the scan store as GOOD, or
  import a GOOD file. The file is read in the renderer via a file input +
  goodDatabaseToStoredArtifacts, so no file-dialog IPC is required.

Verified in the browser preview after a clean restart: the Diagnose view renders
all cards (Status, Last scan, GOOD Interop, Automation log, Crops/OCR), the
Scan<->Diagnose switch works with no console errors (the earlier hook-order
warnings were stale-HMR artifacts from deleting files mid-session). 120 tests +
build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 07:56:31 +02:00
AzuTear 8d4d24f0bb feat(ui): separate dev info into a Diagnose view, declutter the scan workspace
Reworks the UI so the Scan tab shows only core actions and all developer /
diagnostic surfaces live in one dedicated view.

- New "Diagnose" nav view. ScanView stays mounted for scan|diagnose (the scan
  controller state persists across the switch) and renders either the clean
  workspace or the new DiagnosticsView by mode.
- DiagnosticsView consolidates runtime/rights, grid detection, learning + data
  staleness, fingerprint, auto-scan counters, the automation log, the dev-output
  toggle, the Demo-Daten action, and the raw crops/OCR/confidence dump. Reuses
  the existing diagnostics/details model hooks.
- Scan workspace decluttered: removed the Scanner Diagnose button and the Details
  button, dropped the rules/grid/mode dev fields from the result brief and capture
  meta, shortened the topbar headline, and moved Demo-Daten out of the topbar.
- Removed the now-dead ScanDiagnosticsModal / ScanDetailsModal components (their
  content moved into the view); metrics grid hidden on the Diagnose view.
- Added dev:web script + .claude/launch.json for browser preview.

Verified in the browser preview (both views render, no console errors); 120
tests + build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 07:46:31 +02:00
AzuTear b8a38ba547 docs: scanner rework status (done vs remaining live-calibration items)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 21:57:23 +02:00
AzuTear dcac155887 feat(store): GOOD interop, rescan-merge, data staleness, lock detection
Task #5 building blocks, each a pure + unit-tested module:

- goodInterop.ts: GOOD (Genshin Optimizer / Inventory Kamera / Akasha) import and
  export for scanned StoredArtifactRecords - slot/stat/set key maps both ways,
  substat string <-> { key, value }, main-value reconstruction on import
  (ADR-003). Export is lossless; import is best-effort (GOOD lacks piece names).
- artifactMerge.ts: rescan-merge (ADR-006 follow-up). Level-independent identity
  (set + slot + main + substat NAME set) collapses leveled re-scan duplicates,
  keeping the higher-level/stronger record and summing timesSeen. Conservative:
  differing substat lineups never merge.
- dataPackageStatus.ts: warns when the genshin-db package is older than ~45 days
  (a patch cycle) so new sets/characters aren't silently missed; surfaced in the
  Scanner Diagnose data-package line. Adds dataGeneratedAt to genshinData.
- lockDetection.ts: EXPERIMENTAL read-only lock-status heuristic (gold-pixel
  ratio in a top-right icon crop). Pure + tested but not wired into capture; crop
  position and threshold need calibration against a reference 16:9 screenshot.

120 tests + build green. Remaining wiring (needs UI / live calibration): GOOD
import/export buttons and live lock detection.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 21:55:20 +02:00
AzuTear 6ea3d9e714 feat(scan): card-ready gating replaces the fixed settle delay
Adds src/lib/cardReadyGate.ts: after a tile click, poll the detail fingerprint
until it has both changed from the previous artifact and stabilized across
consecutive samples, instead of waiting a hardcoded 280ms and hoping.

- Faster on quick machines (proceeds as soon as the card is stable), correct on
  slow ones (waits up to the budget).
- Robust to particle/hover-glow animation: requiring two consecutive equal
  samples ignores single-frame noise, and if the card never fully stabilizes it
  still proceeds once the content has changed rather than looping on an animated
  frame.
- ESC/stop abort is honored between polls via checkAbort.

autoScanLoop now uses waitForCardReady for both the initial read and the one
retry; CLICK_SETTLE_MS removed. 6 new unit tests; 94 total green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 21:47:03 +02:00
AzuTear 2c6c1a8b31 feat(ocr): resolution-anchored layout module + crop preprocessing
Implements ADR-009 (structure + preprocessing; exact IK fixed coordinates still
need calibration against a reference 16:9 screenshot).

- src/lib/layoutProfile.ts: pure, unit-tested geometry for the artifact screen -
  detail rect, the four detail crops, inventory rect/count crop, 5-col grid,
  16:9 detection, aspect label, and an off-16:9 support warning. Single source of
  truth; electron/main.ts now delegates all crop/grid geometry to it and keeps
  colour detection only as the detail-rect fallback.
- src/lib/ocrPreprocess.ts: pure, unit-tested Otsu binarization with inversion
  (artifact text is the bright foreground) over a BGRA bitmap.
- main.ts: OCR now reads an upscaled + binarized copy of each crop; the original
  crop is retained for the diagnostics UI. CaptureResult carries layout info
  { aspect, isSixteenNine, warning }.

NOTE: image preprocessing changes the OCR input and cannot be validated by the
text-level eval harness; it needs a live Genshin 16:9 capture to confirm/tune
(threshold, invert, upscale factor). 88 tests + build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 21:42:16 +02:00
AzuTear c7138b541d feat(native): C# input/capture sidecar replacing the PowerShell helper
Implements ADR-008. native/input-helper is a self-contained .NET 9 console exe
speaking the identical JSON-over-stdin/stdout protocol as the old PowerShell
helper (ping/cursor/runtime/focus/click/scroll/bounds/capture), so the
InputHelperService interface is unchanged.

- Win32 interop compiled once (native exe), not per call.
- PerMonitorV2 DPI via manifest so click/capture coordinates stay correct on
  mixed-DPI multi-monitor setups.
- capture returns base64 PNG bytes inline (imageBase64) instead of writing a
  temp file per frame; the client handles both base64 and the PowerShell path.
- InputHelperClient prefers the exe and falls back to the embedded PowerShell
  helper when the exe is absent, so the app still runs without the .NET build.
- main.ts resolves the exe (INPUT_HELPER_EXE env -> packaged resources/input-helper
  -> native/input-helper/bin/publish). electron-builder ships it via extraResources.
- npm run helper:build; README documents the build + fallback.

Verified end-to-end through the compiled client: sidecar spawns, runtime info
and a base64 primary-screen capture return correctly. Build stays green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 21:33:26 +02:00
AzuTear 92345ef51a fix(scan): repair pre-existing tsc errors so the build is green
The scan feature had 16 tsc errors from refactor drift; npm run build failed.

- modals/hooks/* are five levels deep but imported ../../../../lib (four);
  corrected to ../../../../../lib.
- useScanDiagnosticsModalModel / useScanReviewQueueModalModel picked
  saveReviewSample / canSaveReviewSample / loadReviewQueue from the modal Props,
  but the components wire those through from controller.*; source them from the
  controller type instead (fixes the downstream unknown-type errors).
- useScanResultCardModel let its field-row tuple array widen to
  (string | ParsedField)[][]; annotate it Array<[string, ParsedField]> like the
  sibling hook.
- ScanTopControlsModel was missing autoScanRunning (destructured by the
  component); add it. useScanTopControlsModel does not use refreshCaptureSources,
  so its input is Omit<...,"refreshCaptureSources">.

tsc, vite build, and the electron build all pass; 74 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 21:25:34 +02:00
77 changed files with 3659 additions and 641 deletions
+11
View File
@@ -0,0 +1,11 @@
{
"version": "0.0.1",
"configurations": [
{
"name": "vite",
"runtimeExecutable": "npm",
"runtimeArgs": ["run", "dev:web"],
"port": 5173
}
]
}
+14
View File
@@ -34,6 +34,20 @@ npm run dev
Use the Electron app window for scanner work. The browser preview does not expose the local capture bridge.
### Input/Capture helper (C# sidecar)
Input automation and screen capture run through a compiled C# sidecar
(`native/input-helper`, see ADR-008). Build it once:
```powershell
npm run helper:build # requires the .NET SDK; produces a self-contained exe
```
The app auto-detects the exe (`INPUT_HELPER_EXE` env override → packaged
`resources/input-helper``native/input-helper/bin/publish`). If the exe is not
present it falls back to the embedded PowerShell helper, so the app still runs
without the .NET build - just slower and with the old per-frame temp-file capture.
### Automatischer Scan: als Administrator starten
Genshin läuft erhöht (Administrator). Windows (UIPI) verwirft dann alle simulierten Maus-Eingaben aus einer nicht-erhöhten App - SendInput meldet dabei trotzdem Erfolg. Für den automatischen Scan muss die App deshalb ebenfalls erhöht laufen:
-23
View File
@@ -1,23 +0,0 @@
@echo off
rem Startet den Dev-Modus mit Administratorrechten (ein UAC-Prompt erscheint).
rem Noetig, weil Genshin erhoeht laeuft: Windows (UIPI) verwirft sonst alle
rem simulierten Maus-Eingaben an das Spiel - SendInput meldet trotzdem Erfolg.
rem Der Punkt hinter %~dp0 verhindert, dass der abschliessende Backslash das
rem schliessende Anfuehrungszeichen escaped.
rem -NoExit haelt das erhoehte (innere) Fenster offen, selbst wenn das Skript
rem einen Fehler wirft oder npm run dev sofort wieder beendet.
rem
rem Dieses AEUSSERE Fenster (das du beim Doppelklick oder ueber
rem "npm run dev:admin" siehst) schloss sich frueher sofort, sobald
rem Start-Process zurueckkehrte - auch wenn UAC abgelehnt wurde oder die
rem Elevation ganz fehlschlug, ohne dass davon irgendetwas sichtbar war.
rem try/catch + timeout zeigen jetzt den Fehler und halten das Fenster kurz offen.
"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -ExecutionPolicy Bypass -Command "$script='%~dp0scripts\dev-admin-start.ps1'; $project='%~dp0.'; try { Start-Process powershell -ArgumentList @('-NoProfile','-ExecutionPolicy','Bypass','-NoExit','-File',$script,'-ProjectRoot',$project) -Verb RunAs -ErrorAction Stop; Write-Host ''; Write-Host 'UAC-Abfrage gestartet. Bitte bestaetigen - danach oeffnet sich ein neues Administrator-Fenster mit npm run dev.' -ForegroundColor Green } catch { Write-Host ''; Write-Host 'Admin-Start fehlgeschlagen oder UAC-Abfrage abgelehnt:' -ForegroundColor Red; Write-Host $_.Exception.Message -ForegroundColor Red }"
echo.
echo Dieses Fenster kannst du jetzt schliessen (Taste druecken oder 15s warten). Das eigentliche Programm laeuft im neuen Administrator-Fenster.
rem timeout statt pause/choice: pause und choice warten unter umgeleiteter
rem Standardeingabe (z.B. beim Testen ueber ein Skript) fuer immer, weil sie
rem auf ein echtes Konsolen-Handle angewiesen sind. timeout erkennt eine
rem umgeleitete Eingabe explizit und bricht sofort ab statt zu haengen, waehrend
rem es bei einem echten Doppelklick normal 15s wartet oder bei Tastendruck endet.
timeout /t 15 >nul 2>&1
+17 -2
View File
@@ -49,7 +49,7 @@ flowchart LR
| Module | Responsibility |
| --- | --- |
| `electron/main.ts` | Window lifecycle, capture source listing, Smart Capture, OCR crop generation, overlay window IPC, persistent PowerShell input/capture helper, JSON artifact store |
| `electron/main.ts` | Window lifecycle, capture source listing, Smart Capture, OCR crop generation, overlay window IPC, input/capture sidecar orchestration, JSON artifact store, dev-only scanner control endpoints |
| `electron/preload.cjs` | Safe renderer bridge exposed as `window.assistantApi` |
| `src/lib/artifactStore.ts` | Pure signature/id/record helpers for the persistent artifact store |
| `src/App.tsx` | Main app shell, scan view, triage view, build view, overlay preview |
@@ -111,9 +111,24 @@ sequenceDiagram
**Automatic grid scan** is user-triggered input automation limited to clicking detected inventory tiles and wheel-scrolling the inventory. Safety and reliability rules:
- All input goes through one persistent PowerShell helper process (`input-helper.ps1` in userData) that compiles the Win32 interop once and speaks JSON over stdin/stdout (ops: ping, focus, cursor, click, scroll, capture). Mouse movement is sent as iterated relative SendInput deltas (what a real mouse produces): Genshin tracks the cursor via raw input and snaps the OS cursor back to its own position every frame, so SetCursorPos/absolute moves silently stop working once the game owns the cursor. The helper verifies the cursor reached the target and refuses to click otherwise.
- All input goes through the helper service boundary (currently a C# sidecar with
fallback support behind the same JSON protocol). The helper owns focus, cursor
movement, click, scroll, guard-state polling, elevation detection, and GDI
capture. Mouse movement is sent as iterated relative input deltas instead of
relying on a single absolute cursor jump. The helper verifies the cursor
reached the target and refuses to click otherwise.
- `npm run dev:admin` is the validated dev path for automation when elevated
input is required. The elevated PowerShell startup is handled by
`scripts/dev-admin.ps1` and logged to `outputs/admin-start/admin-dev.log`.
The user must approve UAC manually; the app cannot approve the Secure Desktop
prompt itself.
- Failsafe: before every click and scroll the renderer polls cursor position and ESC state. Holding ESC or moving the mouse away from the last automated position aborts the scan immediately; the Stop button also aborts. Only the `GetAsyncKeyState` held-down bit (0x8000) is used - the "pressed since last call" bit fires for stale ESC presses from normal Genshin menu navigation and caused false aborts.
- SendInput's return value is checked: zero injected events (UIPI, e.g. elevated Genshin vs. non-elevated app) aborts with an explicit hint instead of silently clicking into nothing.
- Dev-only probes under `http://127.0.0.1:17317` are used for live validation:
`/automation/probe-click?index=N` tests one read-only tile selection, and
`/scanner/start?limit=N` starts an auto-scan with a temporary limit payload.
The live known-good result on 2026-07-07 is documented in
[AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
- Click verification: after each click the parsed detail-panel signature should change. An unchanged signature is a soft miss (it can also mean two OCR-identical neighbor pieces, common among +0 artifacts), so it is retried once with a small offset, logged with the stuck artifact name, and then skipped - never fatal on its own. The scan aborts only when the first ~6 clicks of page 1 produce nothing new (diagnosis hint: elevated Genshin blocks SendInput via UIPI, or grid coordinates are wrong) or a later page yields zero new artifacts.
- Scan stats separate clicked (click attempts), parsed (readable captures), stored (persisted), review (review samples), duplicates, and misses, so "scanned" cannot be mistaken for "successfully read".
- Scrolling sends one wheel notch per grid row with the cursor anchored over the inventory (assumption: roughly one row per notch; overlap is absorbed by dedupe, and a page without new artifacts stops the scan).
+154
View File
@@ -0,0 +1,154 @@
# Automation Live Scan Runbook
This document is the durable reference for automatic artifact scanning, mouse
movement, click input, elevation, and live validation status.
## Current Known-Good State
Validated live on 2026-07-07 with Genshin open in the artifact inventory at
1920x1080, English UI:
- `npm run dev:admin` starts the app elevated after the user confirms UAC.
- Runtime status reported `isElevated: true`, `genshinFound: true`, and
`targetProcess: "GenshinImpact"`.
- The safe probe endpoint `/automation/probe-click?index=1` focused Genshin,
moved the cursor to the second visible inventory tile, clicked it, and changed
the artifact detail panel fingerprint.
- Probe result: `clicked: true`, `inputBlocked: false`,
`foregroundProcess: "GenshinImpact"`, and `changed: true`.
- A bounded live auto-scan via `/scanner/start?limit=2` completed with:
`clicked: 2`, `attempted: 2`, `verified: 2`, `parsed: 2`, `stored: 2`,
`review: 2`, `misses: 0`, `status: "done"`.
This proves that the current elevated app plus helper path can deliver mouse
movement and click input to the focused Genshin client in this environment.
## Elevation And UAC
Use:
```powershell
npm run dev:admin
```
The command runs `scripts/dev-admin.ps1`, which launches a new elevated
PowerShell window running `scripts/dev-admin-start.ps1`. The elevated start is
logged to:
```text
outputs/admin-start/admin-dev.log
```
The user must confirm the Windows UAC prompt. The app cannot and must not click
the Secure Desktop UAC prompt for itself. After confirmation, the app can verify
its own runtime through the dev status endpoint.
Useful checks:
```powershell
Invoke-RestMethod http://127.0.0.1:17317/health
Invoke-RestMethod http://127.0.0.1:17317/scanner/status
```
Expected runtime facts before automatic scan:
- `isElevated: true`
- `genshinFound: true`
- `targetProcess: "GenshinImpact"`
- hotkeys registered
## Mouse And Click Validation
Use the probe before broad auto-scan work:
```powershell
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1" |
ConvertTo-Json -Depth 12
```
The probe performs one read-only inventory selection click. It does not delete,
feed, enhance, lock, unlock, spend, or modify game resources.
Interpretation:
- `click.ok: true`, `clicked: true`, `inputBlocked: false` means Windows did not
block SendInput/UIPI in the current configuration.
- `focused: true` and `foregroundProcess: "GenshinImpact"` means the click was
sent while Genshin was foreground.
- `changed: true` means the detail panel changed after the click.
- `changed: false` can be benign if the target tile was already selected or two
neighboring artifacts render identically; retry with another `index`, `row`,
or `col`.
Examples:
```powershell
# Second visible tile
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1"
# Specific grid cell
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?row=0&col=3"
```
## Bounded Live Auto-Scan
For live validation, prefer a bounded scan first:
```powershell
Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?limit=2"
```
Then poll:
```powershell
Invoke-RestMethod "http://127.0.0.1:17317/scanner/status" |
ConvertTo-Json -Depth 12
```
The `/scanner/start?limit=N` endpoint sends a renderer command payload with a
temporary scan limit. It does not change the normal UI setting. The normal
hotkeys and buttons still use the UI's configured scan limit.
## Anti-Cheat And Safety Boundary
Do not describe the current implementation as bypassing anti-cheat. The app
does not read memory, hook the process, inject code, modify game files, inspect
packets, or interact with kernel drivers. It uses normal Windows screen capture,
focus, cursor movement, wheel, and click input.
The practical finding is narrower:
- A non-elevated app can be blocked by Windows integrity/UIPI when the target
process is elevated or protected.
- Running the app elevated fixed input delivery in the tested environment.
- Genshin's anti-cheat may still affect behavior on other machines, game modes,
overlays, or future versions. Re-run the probe before trusting broad scans.
Never add automation that deletes, feeds, enhances, locks/unlocks, spends
resources, reads memory, hooks, injects, or modifies Genshin.
## Live Layout Facts
The current 16:9 layout profile is calibrated from a 1920x1080 English
artifact-inventory capture:
- detail rect approximately `x=1308`, `y=120`, `width=492`, `height=838`
- inventory grid: `8 x 5`
- first tile center: `x=179`, `y=254`, `row=0`, `col=0`
- second tile center: `x=325`, `y=254`, `row=0`, `col=1`
- inventory count crop successfully read `2059/2400` in the live session
The profile is resolution-scaled for 16:9. Off-profile setups should be treated
as higher risk and validated with Smart Capture plus the probe.
## Validation Checklist
Before marking an automation change done:
1. Run `npm run lint`.
2. Run `npx tsc -p tsconfig.electron.json` when Electron/preload/main changed.
3. Run `npm test`.
4. Run `npm run build`.
5. If Genshin is available, run `/automation/probe-click?index=1`.
6. For scan-loop changes, run `/scanner/start?limit=2` before any broader scan.
7. Record new live findings in this file and in `docs/scanner-rework-status.md`.
+51
View File
@@ -15,6 +15,7 @@ This document contains Architecture Decision Records.
| ADR-007 | Measure OCR accuracy with a labeled eval harness before reworking the scanner | Accepted | 2026-07-05 |
| ADR-008 | Replace the PowerShell input/capture helper with a C# sidecar | Accepted | 2026-07-05 |
| ADR-009 | Resolution-anchored layout profiles and OCR preprocessing over color detection | Accepted | 2026-07-05 |
| ADR-010 | Elevated dev runner and bounded live automation probes | Accepted | 2026-07-07 |
## ADR-001: Build A Local Electron App First
@@ -235,3 +236,53 @@ validated against the ADR-007 eval harness.
insufficient.
- Non-16:9 or non-borderless setups are explicitly unsupported for the auto
scanner; the app should detect and warn rather than silently misread.
## ADR-010: Elevated Dev Runner And Bounded Live Automation Probes
### Status
Accepted
### Context
Automatic grid scanning needs read-only mouse movement, click, and wheel input
to reach the focused Genshin window. A lower-integrity app can fail to deliver
input to an elevated or protected target because of Windows UIPI/integrity
boundaries. During live testing, `npm run dev:admin` originally printed that a
new Administrator window was started, but the elevated PowerShell received no
arguments, so the intended dev process did not reliably start.
The project also needed a smaller live validation path than a full inventory
scan. A full scan is too risky as the first proof of input delivery because it
can click many tiles before a bad coordinate, focus issue, or blocked input is
understood.
### Decision
Keep automatic scan input automation read-only and require an elevated runtime
when Windows reports that automation would otherwise be blocked. Replace the
old `dev-admin.cmd` entry with `scripts/dev-admin.ps1`, quote the elevated
PowerShell arguments explicitly, and log elevated startup to
`outputs/admin-start/admin-dev.log`.
Add dev-only HTTP checks:
- `/automation/probe-click?index=N` or `?row=R&col=C` performs one safe
inventory selection click and verifies whether the detail panel changed.
- `/scanner/start?limit=N` sends a temporary scan-limit payload to the renderer,
so live auto-scan validation can start with two items instead of the UI
default.
Document the workflow in [AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
### Consequences
- The user still has to approve Windows UAC manually; the app must not try to
click the Secure Desktop prompt.
- We can distinguish input delivery from OCR/parser quality with a one-click
probe before running any broader scan.
- Live validation now has a low-risk path: check elevation and Genshin
detection, run a single probe click, then run a bounded `limit=2` scan.
- The implementation remains inside the allowed safety boundary: no memory
reads, hooks, injection, game-file modification, deleting, feeding, enhancing,
locking/unlocking, or spending resources.
+9 -5
View File
@@ -72,7 +72,7 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
| Styling | CSS with dark purple glassmorphism system | Premium fintech-inspired visual direction |
| OCR | Tesseract.js prototype plus deterministic normalization/derivation | OCR alone is not trusted as the decision source |
| Capture | Electron desktopCapturer plus Windows GDI Smart Capture | GDI path is used for Genshin Smart Capture reliability |
| Input automation | PowerShell sidecar prototype now, native sidecar planned | Current sidecar is good for proving behavior, not the final production path |
| Input automation | C# sidecar with elevated dev runner when needed | Live-validated for read-only inventory selection clicks; see `docs/AUTOMATION_LIVE_SCAN.md` |
| Tests | Vitest + TypeScript checks | Current validation baseline; regression samples must expand |
| Packaging | electron-builder | Configured in `package.json` |
@@ -95,11 +95,15 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
- The parser already uses known sets, pieces, slots, stat aliases, set aliases, character aliases, and derived slot/set mapping.
- Review samples, learned replacements, parser notes, and stored artifacts already persist locally.
- The auto-scan loop is no longer a naive click spammer: it has preflight, verification, miss handling, page fingerprinting, and stop conditions.
- Elevated live automation is validated in the current dev environment:
`/automation/probe-click?index=1` changed the selected artifact and
`/scanner/start?limit=2` completed with 2/2 verified reads and 0 misses.
### What is still structurally weak
- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
- The current PowerShell input sidecar is serviceable for experimentation but not a strong production base for long-running, low-jitter auto-scan.
- Broader scan soak testing still needs to increase the live limit gradually and
validate scroll/page transitions beyond the first visible row.
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
- The scan page is cleaner than before, but it still exposes too much operator/debug state in the main flow.
@@ -196,7 +200,7 @@ Outcome:
- Auto-scan never starts on a session that cannot prove one successful detail-card change.
Status:
- Planned
- First live path validated; broader soak testing still needed
### Phase 5 - Learning loop that actually compounds
@@ -228,7 +232,7 @@ Status:
1. Finish scan-page cleanup so the main operator view is no longer noisy.
2. Tighten the game data generator and parser contract, then backfill regression tests from real bad samples.
3. Continue moving auto-scan behavior out of `App.tsx` and into isolated scanner modules.
4. Replace or wrap the current PowerShell sidecar with a more stable long-lived automation process.
4. Soak-test the elevated C# helper automation path with gradually larger scan limits and page scroll transitions.
5. Extend the learning system from text-only fixes into crop/UI profile tuning.
6. Resume recommendation work only when scan accuracy is consistently trustworthy.
@@ -236,7 +240,7 @@ Status:
| Question | Status |
| --- | --- |
| Should the production input sidecar be Rust/C++ first, or a transitional Node native addon, for the next iteration? | Open |
| Is the current C# helper sufficient for production packaging, or does a later Rust/C++ sidecar still materially reduce latency or packaging risk? | Open |
| When should UI-profile learning be allowed to change crop geometry automatically versus requiring review approval? | Open |
| What scan-quality threshold is high enough before recommendations should be considered user-facing again? | Open |
| Which Genshin UI languages should be supported after English once the scanner contract is stable? | Open |
+61
View File
@@ -0,0 +1,61 @@
# Scanner rework status
Progress on the approved scanner/OCR rework. See ADR-007/008/009/010 in
[DECISIONS.md](DECISIONS.md) for the decisions behind these. For the current
live automation runbook, see
[AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
## Done (implemented, unit-tested, build green)
- **OCR eval harness** — `src/eval/`, `npm run eval`, gate in `npm test`. See
[ocr-eval.md](ocr-eval.md).
- **C# input/capture sidecar** — `native/input-helper/`, `npm run helper:build`.
Replaces the PowerShell helper on the same JSON protocol; PowerShell remains a
fallback. Verified end-to-end (spawn, runtime, base64 capture).
- **Layout profiles + OCR preprocessing** — `src/lib/layoutProfile.ts` (pure
geometry, 16:9 detection), `src/lib/ocrPreprocess.ts` (grayscale + Otsu
binarize). main.ts now uses calibrated 16:9 detail/count/grid coordinates
first and OCRs an upscaled + binarized copy.
- **Card-ready gating** — `src/lib/cardReadyGate.ts` replaces the fixed 280 ms
settle with change+stability polling; robust to animation.
- **GOOD interop** — `src/lib/goodInterop.ts` (export + best-effort import for
scanned records), Electron file-picker import/export, and store merge.
- **Rescan-merge** — `src/lib/artifactMerge.ts` collapses leveled re-scan
duplicates by a level-independent identity.
- **Data staleness warning** — `src/lib/dataPackageStatus.ts`, surfaced in the
Scanner Diagnose data-package line.
- **Lock detection (experimental)** — `src/lib/lockDetection.ts`, wired into
live capture as a read-only `locked` flag and persisted with scanned records.
- **Elevated live automation path** — `npm run dev:admin` now starts through
`scripts/dev-admin.ps1` and logs to `outputs/admin-start/admin-dev.log`.
Live status confirmed `isElevated: true`, `genshinFound: true`, and
`targetProcess: "GenshinImpact"`.
- **Read-only click probe** — `/automation/probe-click?index=1` verified that
the app can focus Genshin, move to a visible inventory tile, click it, and
observe a changed detail panel fingerprint (`clicked: true`,
`inputBlocked: false`, `changed: true`).
- **Bounded auto-scan validation** — `/scanner/start?limit=2` completed live
with 2 clicks, 2 verified detail views, 2 parsed artifacts, 2 stored records,
2 review samples, and 0 misses.
## Remaining — needs the live environment or a UI pass
These cannot be finished/validated without Genshin running at the user's
resolution or without UI work best tested live:
1. **Validate/tune OCR preprocessing** on more real captures — confirm invert +
threshold + upscale factor help (not hurt) actual Tesseract reads. The
text-level eval harness cannot measure image preprocessing.
2. **Validate locked=true** against a known locked artifact — unlocked/grey lock
was live-checked; a gold locked icon still needs a positive sample.
3. **Broader scan soak test** — after the bounded two-item live scan passed,
the next automation validation should increase the limit gradually and watch
for repeated pages, scroll behavior, duplicate handling, and OCR review rate.
## Grow the eval corpus
Every low-confidence review sample already stores its crops + OCR. Confirm/correct
those via `reviewSampleToEvalCase` and commit them into `src/eval/corpus/` so the
harness keeps measuring real-world accuracy across patches. See
[ocr-eval.md](ocr-eval.md).
+3
View File
@@ -17,6 +17,7 @@ import type {
SaveResultWithPath,
SaveSnapshotResult,
GoodDatabase,
GoodImportFileResult,
ScannerStatusPayload,
} from "../../src/types/global.js";
import type {
@@ -51,6 +52,7 @@ interface PersistenceHandlersDependencies {
loadScannerLearningRules: () => Promise<LoadScannerLearningRulesResult>;
writeScannerLearningRules: (rules: ScannerLearningRulePayload) => Promise<SaveScannerLearningRulesResult>;
exportGood: (payload: GoodDatabase) => Promise<SaveResultWithPath>;
importGoodFile: () => Promise<GoodImportFileResult>;
}
interface CaptureHandlersDependencies {
@@ -86,6 +88,7 @@ export function registerIpcHandlers(dependencies: IpcBootstrapDependencies) {
loadScannerLearningRules: dependencies.loadScannerLearningRules,
writeScannerLearningRules: dependencies.writeScannerLearningRules,
exportGood: dependencies.exportGood,
importGoodFile: dependencies.importGoodFile,
});
registerCaptureHandlers({
+256
View File
@@ -0,0 +1,256 @@
import fs from "node:fs/promises";
import http, { type Server } from "node:http";
import path from "node:path";
import type {
CaptureOptions,
CaptureResult,
CaptureSourceInfo,
ClickResult,
ReviewSampleListResult,
ScannerCommand,
ScannerStatusPayload,
} from "../src/types/global.js";
interface DevControlServerDependencies {
registeredHotkeys: Record<string, boolean>;
hasMainWindow: () => boolean;
sendScannerCommand: (command: ScannerCommand | "probe-click") => void;
clickScreen: (x: number, y: number) => Promise<ClickResult>;
scannerStatus: () => ScannerStatusPayload;
loadReviewSamples: (limit?: number) => Promise<ReviewSampleListResult>;
listCaptureSources: () => Promise<CaptureSourceInfo[]>;
captureSource: (
id: string,
delayMs?: number,
focusGenshin?: boolean,
options?: CaptureOptions,
) => Promise<CaptureResult>;
}
function writeDevJson(res: http.ServerResponse, statusCode: number, payload: unknown) {
res.writeHead(statusCode, {
"content-type": "application/json; charset=utf-8",
"cache-control": "no-store",
});
res.end(JSON.stringify(payload));
}
function dataUrlBase64(dataUrl: string) {
return dataUrl.replace(/^data:image\/png;base64,/, "");
}
function devCaptureOutputDir() {
return path.join(process.cwd(), "outputs", "live-capture");
}
function safeDebugFilePart(value: string) {
return value.replace(/[^A-Za-z0-9._-]+/g, "-").replace(/^-+|-+$/g, "").slice(0, 80) || "capture";
}
function dataUrlFingerprint(dataUrl: string | undefined) {
if (!dataUrl) return "";
let hash = 2166136261;
const stride = Math.max(1, Math.floor(dataUrl.length / 4096));
for (let index = 0; index < dataUrl.length; index += stride) {
hash ^= dataUrl.charCodeAt(index);
hash = Math.imul(hash, 16777619);
}
return `${dataUrl.length.toString(16)}:${(hash >>> 0).toString(16)}`;
}
async function writeDevCaptureImage(filePath: string, dataUrl: string | undefined) {
if (!dataUrl) return null;
await fs.writeFile(filePath, Buffer.from(dataUrlBase64(dataUrl), "base64"));
return filePath;
}
async function writeDevCaptureSnapshot(capture: CaptureResult) {
const outputDir = devCaptureOutputDir();
await fs.mkdir(outputDir, { recursive: true });
const stamp = new Date().toISOString().replace(/[\\/:]/g, "-").replace(/\..+?$/, "");
const prefix = safeDebugFilePart(`${stamp}-${capture.name}`);
const files = {
full: await writeDevCaptureImage(path.join(outputDir, `${prefix}-full.png`), capture.dataUrl),
detail: await writeDevCaptureImage(path.join(outputDir, `${prefix}-detail.png`), capture.detailDataUrl),
inventory: await writeDevCaptureImage(path.join(outputDir, `${prefix}-inventory.png`), capture.inventoryDataUrl),
crops: [] as Array<{ id: string; label: string; path: string; rect: { x: number; y: number; width: number; height: number } }>,
};
for (const crop of capture.crops ?? []) {
const cropPath = path.join(outputDir, `${prefix}-${safeDebugFilePart(crop.id)}.png`);
const written = await writeDevCaptureImage(cropPath, crop.dataUrl);
if (written) files.crops.push({ id: crop.id, label: crop.label, path: written, rect: crop.rect });
}
const summary = {
id: capture.id,
name: capture.name,
width: capture.width,
height: capture.height,
capturedAt: capture.capturedAt,
captureTarget: capture.captureTarget,
ocrSkipped: capture.ocrSkipped,
ocrTimedOut: capture.ocrTimedOut,
layout: capture.layout,
inventoryGrid: capture.inventoryGrid
? {
rows: capture.inventoryGrid.rows,
cols: capture.inventoryGrid.cols,
confidence: capture.inventoryGrid.confidence,
source: capture.inventoryGrid.source,
firstCenter: capture.inventoryGrid.centers[0] ?? null,
lastCenter: capture.inventoryGrid.centers.at(-1) ?? null,
}
: null,
inventoryCount: capture.inventoryCount ?? null,
locked: capture.locked,
crops: (capture.crops ?? []).map((crop) => ({ id: crop.id, label: crop.label, rect: crop.rect })),
ocr: capture.ocr ?? [],
files,
};
const summaryPath = path.join(outputDir, `${prefix}-summary.json`);
await fs.writeFile(summaryPath, JSON.stringify(summary, null, 2), "utf8");
return { ...summary, summaryPath };
}
function wait(ms: number) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function findGenshinSource(sources: CaptureSourceInfo[], sourceId: string | null) {
return sourceId
? sources.find((entry) => entry.id === sourceId)
: sources.find((entry) => entry.isGenshinCandidate);
}
function sourceListForError(sources: CaptureSourceInfo[]) {
return sources.map(({ id, name, isGenshinCandidate }) => ({ id, name, isGenshinCandidate }));
}
export function createDevControlServer(deps: DevControlServerDependencies): Server {
const server = http.createServer((req, res) => {
if (req.socket.remoteAddress && !["127.0.0.1", "::1", "::ffff:127.0.0.1"].includes(req.socket.remoteAddress)) {
writeDevJson(res, 403, { ok: false, error: "local only" });
return;
}
const url = new URL(req.url ?? "/", "http://127.0.0.1");
if (url.pathname === "/health") {
writeDevJson(res, 200, { ok: true, hotkeys: deps.registeredHotkeys, hasWindow: deps.hasMainWindow() });
return;
}
if (url.pathname === "/scanner/start") {
const limit = Number(url.searchParams.get("limit") ?? Number.NaN);
const command: ScannerCommand = Number.isFinite(limit) && limit > 0
? { type: "start-auto", scanLimit: limit }
: "start-auto";
deps.sendScannerCommand(command);
writeDevJson(res, 200, { ok: true, command });
return;
}
if (url.pathname === "/scanner/stop") {
deps.sendScannerCommand("stop");
writeDevJson(res, 200, { ok: true, command: "stop" });
return;
}
if (url.pathname === "/scanner/probe") {
deps.sendScannerCommand("probe-click");
writeDevJson(res, 200, { ok: true, command: "probe-click" });
return;
}
if (url.pathname === "/automation/click") {
const x = Number(url.searchParams.get("x"));
const y = Number(url.searchParams.get("y"));
if (!Number.isFinite(x) || !Number.isFinite(y)) {
writeDevJson(res, 400, { ok: false, error: "x and y query params are required" });
return;
}
deps.clickScreen(Math.round(x), Math.round(y))
.then((payload: unknown) => writeDevJson(res, 200, { ok: true, payload }))
.catch((error: unknown) => writeDevJson(res, 500, { ok: false, error: error instanceof Error ? error.message : String(error) }));
return;
}
if (url.pathname === "/scanner/status") {
writeDevJson(res, 200, { ok: true, status: deps.scannerStatus() });
return;
}
if (url.pathname === "/review/samples") {
deps.loadReviewSamples(Number(url.searchParams.get("limit") ?? 20))
.then((payload: unknown) => writeDevJson(res, 200, payload))
.catch((error: unknown) => writeDevJson(res, 500, { ok: false, error: error instanceof Error ? error.message : String(error) }));
return;
}
if (url.pathname === "/capture/smart") {
const sourceId = url.searchParams.get("sourceId");
const focus = url.searchParams.get("focus") !== "0";
const skipOcr = url.searchParams.get("skipOcr") === "1";
deps.listCaptureSources()
.then(async (sources) => {
const source = findGenshinSource(sources, sourceId);
if (!source) {
writeDevJson(res, 404, { ok: false, error: "No Genshin capture source found.", sources: sourceListForError(sources) });
return;
}
const capture = await deps.captureSource(source.id, 250, focus, { skipOcr });
const summary = await writeDevCaptureSnapshot(capture);
writeDevJson(res, 200, { ok: true, source: { id: source.id, name: source.name }, summary });
})
.catch((error: unknown) => writeDevJson(res, 500, { ok: false, error: error instanceof Error ? error.message : String(error) }));
return;
}
if (url.pathname === "/automation/probe-click") {
const sourceId = url.searchParams.get("sourceId");
const requestedIndex = Number(url.searchParams.get("index") ?? "1");
const requestedRow = Number(url.searchParams.get("row") ?? Number.NaN);
const requestedCol = Number(url.searchParams.get("col") ?? Number.NaN);
deps.listCaptureSources()
.then(async (sources) => {
const source = findGenshinSource(sources, sourceId);
if (!source) {
writeDevJson(res, 404, { ok: false, error: "No Genshin capture source found.", sources: sourceListForError(sources) });
return;
}
const before = await deps.captureSource(source.id, 150, true, { skipOcr: true });
const centers = before.inventoryGrid?.centers ?? [];
const target = Number.isFinite(requestedRow) && Number.isFinite(requestedCol)
? centers.find((center) => center.row === requestedRow && center.col === requestedCol)
: centers[Math.max(0, Math.min(centers.length - 1, Number.isFinite(requestedIndex) ? requestedIndex : 1))];
if (!target) {
writeDevJson(res, 409, { ok: false, error: "No inventory grid target available.", grid: before.inventoryGrid ?? null });
return;
}
const beforeFingerprint = dataUrlFingerprint(before.detailDataUrl);
const click = await deps.clickScreen(target.x, target.y);
await wait(650);
const after = await deps.captureSource(source.id, 0, true, { skipOcr: true });
const afterFingerprint = dataUrlFingerprint(after.detailDataUrl);
const changed = Boolean(beforeFingerprint && afterFingerprint && beforeFingerprint !== afterFingerprint);
writeDevJson(res, 200, {
ok: Boolean(click.ok && click.clicked && changed),
changed,
target,
click,
before: {
captureTarget: before.captureTarget,
grid: before.inventoryGrid ? { rows: before.inventoryGrid.rows, cols: before.inventoryGrid.cols, source: before.inventoryGrid.source, confidence: before.inventoryGrid.confidence } : null,
detailFingerprint: beforeFingerprint,
},
after: {
captureTarget: after.captureTarget,
grid: after.inventoryGrid ? { rows: after.inventoryGrid.rows, cols: after.inventoryGrid.cols, source: after.inventoryGrid.source, confidence: after.inventoryGrid.confidence } : null,
detailFingerprint: afterFingerprint,
},
});
})
.catch((error: unknown) => writeDevJson(res, 500, { ok: false, error: error instanceof Error ? error.message : String(error) }));
return;
}
writeDevJson(res, 404, { ok: false, error: "unknown endpoint" });
});
server.listen(17317, "127.0.0.1");
return server;
}
+7
View File
@@ -13,6 +13,7 @@ import type {
SaveScannerLearningRulesResult,
ScannerLearningRulePayload,
SaveResultWithPath,
GoodImportFileResult,
} from "../../src/types/global.js";
import type { StoredArtifactRecord } from "../../src/types/storage.js";
@@ -28,6 +29,7 @@ interface PersistenceDependencies {
loadScannerLearningRules: () => Promise<LoadScannerLearningRulesResult>;
writeScannerLearningRules: (rules: ScannerLearningRulePayload) => Promise<SaveScannerLearningRulesResult>;
exportGood: (payload: GoodDatabase) => Promise<SaveResultWithPath>;
importGoodFile: () => Promise<GoodImportFileResult>;
}
export function registerPersistenceHandlers({
@@ -39,6 +41,7 @@ export function registerPersistenceHandlers({
loadScannerLearningRules,
writeScannerLearningRules,
exportGood,
importGoodFile,
}: PersistenceDependencies) {
ipcMain.handle("review:saveSample", async (_event, sample: ReviewSamplePayload) => {
try {
@@ -81,4 +84,8 @@ export function registerPersistenceHandlers({
ipcMain.handle("good:export", async (_event, payload: GoodDatabase) => {
return exportGood(payload);
});
ipcMain.handle("good:importFile", async () => {
return importGoodFile();
});
}
+130 -199
View File
@@ -1,18 +1,22 @@
import { app, BrowserWindow, Menu, desktopCapturer, globalShortcut, nativeImage, screen, type NativeImage } from "electron";
import { app, BrowserWindow, Menu, desktopCapturer, dialog, globalShortcut, nativeImage, screen, type NativeImage } from "electron";
import fs from "node:fs/promises";
import http, { type Server } from "node:http";
import { existsSync } from "node:fs";
import type { Server } from "node:http";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { createWorker } from "tesseract.js";
import { createInputHelperService, type InputHelperService } from "./services/inputHelper.js";
import { createRepositoryContext, type RepositoryContext } from "./bootstrap/repositoryContext.js";
import { registerIpcHandlers } from "./bootstrap/ipcBootstrap.js";
import { createDevControlServer } from "./devControlServer.js";
import type { AppSnapshot } from "../src/types/domain.js";
import type {
CaptureOptions,
CaptureResult,
GoodDatabase,
GoodImportFileResult,
SaveResultWithPath,
ScannerCommand,
ScannerLearningRulePayload,
ScannerStatusPayload,
} from "../src/types/global.js";
@@ -21,6 +25,18 @@ import type {
ReviewSamplesRepositoryPort,
ScannerLearningRepositoryPort,
} from "./repositories/index.js";
import {
aspectRatioLabel,
detailCropRects,
inventoryCountCropRect,
inventoryGrid as layoutInventoryGrid,
inventoryRect as layoutInventoryRect,
isSixteenNine,
layoutSupportWarning,
profileDetailRect,
} from "../src/lib/layoutProfile.js";
import { binarizeForOcr } from "../src/lib/ocrPreprocess.js";
import { detectLockState, lockIconCropRect } from "../src/lib/lockDetection.js";
// Chromium's renderer sandbox can refuse to fully initialize (or silently
// crash the GPU/renderer process) when the hosting process runs with a full
@@ -70,6 +86,25 @@ function getInputHelperService() {
return inputHelperService;
}
// Locate the compiled C# input/capture sidecar (ADR-008). Falls back to null so
// the service uses the embedded PowerShell helper when the exe was never built.
function resolveInputHelperExePath(): string | null {
const candidates = [
process.env.INPUT_HELPER_EXE,
path.join(process.resourcesPath, "input-helper", "InputHelper.exe"),
path.join(app.getAppPath(), "native", "input-helper", "bin", "publish", "InputHelper.exe"),
].filter((candidate): candidate is string => Boolean(candidate));
for (const candidate of candidates) {
try {
if (existsSync(candidate)) return candidate;
} catch {
// Unreadable path; try the next candidate.
}
}
return null;
}
function getRepositoryContext() {
if (!repositoryContext) {
throw new Error("Repository context has not been initialized.");
@@ -363,7 +398,7 @@ function createMainWindow() {
if (isDev) {
mainWindow.loadURL(process.env.VITE_DEV_SERVER_URL!);
} else {
mainWindow.loadFile(path.join(__dirname, "../dist/index.html"));
mainWindow.loadFile(path.join(__dirname, "../../dist/index.html"));
}
}
@@ -385,7 +420,7 @@ function focusMainWindow() {
return { ok: true };
}
function sendScannerCommand(command: "start-auto" | "stop" | "probe-click") {
function sendScannerCommand(command: ScannerCommand | "probe-click") {
if (!mainWindow || mainWindow.isDestroyed()) return;
mainWindow.webContents.send("scanner:command", command);
}
@@ -400,71 +435,18 @@ function registerScannerHotkeys() {
};
}
function writeDevJson(res: http.ServerResponse, statusCode: number, payload: unknown) {
res.writeHead(statusCode, {
"content-type": "application/json; charset=utf-8",
"cache-control": "no-store",
});
res.end(JSON.stringify(payload));
}
function startDevControlServer() {
if (!isDev || devControlServer) return;
devControlServer = http.createServer((req, res) => {
if (req.socket.remoteAddress && !["127.0.0.1", "::1", "::ffff:127.0.0.1"].includes(req.socket.remoteAddress)) {
writeDevJson(res, 403, { ok: false, error: "local only" });
return;
}
const url = new URL(req.url ?? "/", "http://127.0.0.1");
if (url.pathname === "/health") {
writeDevJson(res, 200, { ok: true, hotkeys: registeredHotkeys, hasWindow: Boolean(mainWindow && !mainWindow.isDestroyed()) });
return;
}
if (url.pathname === "/scanner/start") {
sendScannerCommand("start-auto");
writeDevJson(res, 200, { ok: true, command: "start-auto" });
return;
}
if (url.pathname === "/scanner/stop") {
sendScannerCommand("stop");
writeDevJson(res, 200, { ok: true, command: "stop" });
return;
}
if (url.pathname === "/scanner/probe") {
sendScannerCommand("probe-click");
writeDevJson(res, 200, { ok: true, command: "probe-click" });
return;
}
if (url.pathname === "/automation/click") {
const x = Number(url.searchParams.get("x"));
const y = Number(url.searchParams.get("y"));
if (!Number.isFinite(x) || !Number.isFinite(y)) {
writeDevJson(res, 400, { ok: false, error: "x and y query params are required" });
return;
}
getInputHelperService()
.clickScreen(Math.round(x), Math.round(y))
.then((payload: unknown) => writeDevJson(res, 200, { ok: true, payload }))
.catch((error: unknown) => writeDevJson(res, 500, { ok: false, error: error instanceof Error ? error.message : String(error) }));
return;
}
if (url.pathname === "/scanner/status") {
writeDevJson(res, 200, { ok: true, status: scannerDevStatus });
return;
}
if (url.pathname === "/review/samples") {
loadReviewSamples(Number(url.searchParams.get("limit") ?? 20))
.then((payload: unknown) => writeDevJson(res, 200, payload))
.catch((error: unknown) => writeDevJson(res, 500, { ok: false, error: error instanceof Error ? error.message : String(error) }));
return;
}
writeDevJson(res, 404, { ok: false, error: "unknown endpoint" });
devControlServer = createDevControlServer({
registeredHotkeys,
hasMainWindow: () => Boolean(mainWindow && !mainWindow.isDestroyed()),
sendScannerCommand,
clickScreen: clickScreenCommand,
scannerStatus: () => scannerDevStatus,
loadReviewSamples,
listCaptureSources,
captureSource,
});
devControlServer.listen(17317, "127.0.0.1");
}
function createOverlayWindow() {
@@ -497,7 +479,7 @@ function createOverlayWindow() {
if (isDev) {
overlayWindow.loadURL(`${process.env.VITE_DEV_SERVER_URL!}?overlay=1`);
} else {
overlayWindow.loadFile(path.join(__dirname, "../dist/index.html"), {
overlayWindow.loadFile(path.join(__dirname, "../../dist/index.html"), {
query: { overlay: "1" },
});
}
@@ -726,87 +708,55 @@ function imageCropDataUrl(sourceImage: NativeImage, rect: Electron.Rectangle, im
return sourceImage.crop(safeRect).toDataURL();
}
// Preprocessed copy of a crop for OCR (ADR-009): upscale for more pixels, then
// grayscale + Otsu-binarize with inversion (artifact text is the bright
// foreground). The original crop is kept separately for the diagnostics UI.
function preprocessedCropDataUrl(sourceImage: NativeImage, rect: Electron.Rectangle, imageSize: { width: number; height: number }) {
const safeRect = clampCaptureRect(rect, imageSize);
const upscaled = sourceImage.crop(safeRect).resize({ width: Math.max(1, safeRect.width * 2), quality: "best" });
const size = upscaled.getSize();
if (!size.width || !size.height) return upscaled.toDataURL();
const binarized = binarizeForOcr({ data: upscaled.getBitmap(), width: size.width, height: size.height });
return nativeImage
.createFromBitmap(Buffer.from(binarized.data), { width: binarized.width, height: binarized.height })
.toDataURL();
}
function createCrops(
sourceImage: NativeImage,
imageSize: { width: number; height: number },
detailRect: Electron.Rectangle,
inventoryRect: Electron.Rectangle,
) {
const templates: CropTemplate[] = [
{
id: "artifact-title",
label: "Artifact title",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.05),
width: Math.round(detailRect.width * 0.82),
height: Math.round(detailRect.height * 0.16),
},
},
{
id: "artifact-main-stat",
label: "Main stat",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.20),
width: Math.round(detailRect.width * 0.82),
height: Math.round(detailRect.height * 0.18),
},
},
{
id: "artifact-substats",
label: "Substats",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.41),
width: Math.round(detailRect.width * 0.82),
height: Math.round(detailRect.height * 0.25),
},
},
{
id: "artifact-footer",
label: "Footer",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.78),
width: Math.round(detailRect.width * 0.82),
height: Math.round(detailRect.height * 0.16),
},
},
];
const templates: CropTemplate[] = detailCropRects(detailRect, imageSize);
if (inventoryRect.width > 120 && inventoryRect.height > 80) {
templates.push({
id: "inventory-count",
label: "Inventory count",
rect: {
x: Math.round(inventoryRect.x + inventoryRect.width * 0.62),
y: Math.round(inventoryRect.y + inventoryRect.height * 0.02),
width: Math.round(inventoryRect.width * 0.34),
height: Math.round(inventoryRect.height * 0.09),
},
rect: inventoryCountCropRect(inventoryRect, imageSize),
});
}
return templates
.map((template) => ({
...template,
rect: clampCaptureRect(template.rect, imageSize),
dataUrl: imageCropDataUrl(sourceImage, template.rect, imageSize),
}))
.filter((crop) => crop.rect.width > 0 && crop.rect.height > 0)
.map((crop) => ({
...crop,
rect: {
x: crop.rect.x,
y: crop.rect.y,
width: crop.rect.width,
height: crop.rect.height,
},
}));
.map((template) => {
const rect = clampCaptureRect(template.rect, imageSize);
return {
id: template.id,
label: template.label,
rect,
dataUrl: imageCropDataUrl(sourceImage, rect, imageSize),
ocrDataUrl: preprocessedCropDataUrl(sourceImage, rect, imageSize),
};
})
.filter((crop) => crop.rect.width > 0 && crop.rect.height > 0);
}
function inferDetailRect(bitmap: Buffer, imageSize: { width: number; height: number }) {
if (isSixteenNine(imageSize)) {
return profileDetailRect(imageSize);
}
const { width, height } = imageSize;
const sampleStrideX = width > 2200 ? 4 : 3;
const sampleStrideY = height > 1400 ? 4 : 3;
@@ -845,79 +795,17 @@ function inferDetailRect(bitmap: Buffer, imageSize: { width: number; height: num
return clampCaptureRect({ x: left, y: top, width: widthGuess, height: heightGuess }, imageSize);
}
if (width > 0 && height > 0) {
return clampCaptureRect(
{
x: Math.round(width * 0.50),
y: Math.round(height * 0.08),
width: Math.round(width * 0.46),
height: Math.round(height * 0.74),
},
imageSize,
);
}
return { x: 0, y: 0, width, height };
// Colour detection found nothing usable; fall back to the resolution-anchored
// profile rect (single source of truth in layoutProfile).
return profileDetailRect(imageSize);
}
function inferInventoryRect(imageSize: { width: number; height: number }, detailRect: Electron.Rectangle) {
const { width, height } = imageSize;
const preferredWidth = Math.max(140, Math.round(width * 0.48));
const x = Math.round(width * 0.03);
const y = Math.round(detailRect.y + detailRect.height * 0.09);
const availableWidth = Math.max(100, detailRect.x - Math.round(width * 0.04));
const panelWidth = Math.max(100, Math.min(preferredWidth, availableWidth));
const safeWidth = panelWidth > width * 0.85 ? Math.round(width * 0.55) : panelWidth;
return clampCaptureRect(
{
x,
y,
width: Math.min(safeWidth, Math.max(width - x - Math.round(width * 0.02), 100)),
height: Math.max(140, Math.round(height * 0.70)),
},
imageSize,
);
return layoutInventoryRect(imageSize, detailRect);
}
function inferInventoryGrid(imageSize: { width: number; height: number }, detailRect: Electron.Rectangle) {
const inventoryRect = inferInventoryRect(imageSize, detailRect);
const cols = 5;
if (inventoryRect.width < 160 || inventoryRect.height < 140) {
return {
centers: [],
rows: 0,
cols: 0,
confidence: 0,
source: "missing" as const,
};
}
const cellWidth = Math.max(56, Math.round(inventoryRect.width / cols));
const stepX = Math.round(cellWidth * 0.96);
const stepY = Math.round(cellWidth * 1.03);
const visibleRows = Math.max(2, Math.min(6, Math.round(inventoryRect.height / Math.max(stepY, 1))));
const startX = inventoryRect.x + Math.max(6, Math.round(stepX * 0.45));
const startY = inventoryRect.y + Math.max(6, Math.round(stepY * 0.45));
const centers = [];
for (let row = 0; row < visibleRows; row++) {
for (let col = 0; col < cols; col++) {
const x = startX + col * stepX;
const y = startY + row * stepY;
if (x < imageSize.width && y < imageSize.height) {
centers.push({ x, y, row, col });
}
}
}
const trimmed = centers.filter((center) => center.x > 0 && center.y > 0);
return {
centers: trimmed,
rows: visibleRows,
cols,
confidence: trimmed.length >= cols * 2 ? 76 : trimmed.length >= cols ? 58 : 36,
source: "detected" as const,
};
return layoutInventoryGrid(imageSize, detailRect);
}
async function buildCaptureResult(
@@ -936,10 +824,18 @@ async function buildCaptureResult(
const detailRect = inferDetailRect(bitmap, size);
const inventoryRect = inferInventoryRect(size, detailRect);
const crops = createCrops(sourceImage, size, detailRect, inventoryRect);
const lockRect = clampCaptureRect(lockIconCropRect(detailRect, size), size);
const lockImage = sourceImage.crop(lockRect);
const lockSize = lockImage.getSize();
const locked = lockSize.width > 0 && lockSize.height > 0
? detectLockState({ data: lockImage.getBitmap(), width: lockSize.width, height: lockSize.height })
: undefined;
const croppedPayload = crops.map((crop) => ({
id: crop.id,
label: crop.label,
dataUrl: crop.dataUrl,
// OCR reads the preprocessed (upscaled + binarized) crop; the original is
// kept below for the diagnostics UI.
dataUrl: crop.ocrDataUrl ?? crop.dataUrl,
}));
const recognized = options.skipOcr ? { ocr: [], timedOut: false } : await runOcrOnCropsWithTimeout(croppedPayload);
@@ -970,6 +866,12 @@ async function buildCaptureResult(
})),
inventoryGrid: inferInventoryGrid(size, detailRect),
inventoryCount: count,
locked,
layout: {
aspect: aspectRatioLabel(size),
isSixteenNine: isSixteenNine(size),
warning: layoutSupportWarning(size),
},
};
}
@@ -1025,6 +927,34 @@ async function exportGood(payload: GoodDatabase): Promise<SaveResultWithPath> {
}
}
async function importGoodFile(): Promise<GoodImportFileResult> {
const dialogOptions = {
title: "GOOD-Datei importieren",
properties: ["openFile"],
filters: [{ name: "GOOD JSON", extensions: ["json"] }],
} satisfies Electron.OpenDialogOptions;
const dialogResult = mainWindow && !mainWindow.isDestroyed()
? await dialog.showOpenDialog(mainWindow, dialogOptions)
: await dialog.showOpenDialog(dialogOptions);
if (dialogResult.canceled || dialogResult.filePaths.length === 0) {
return { ok: false, canceled: true, path: "" };
}
const filePath = dialogResult.filePaths[0];
try {
const text = await fs.readFile(filePath, "utf8");
return { ok: true, canceled: false, path: filePath, database: JSON.parse(text) };
} catch (error) {
return {
ok: false,
canceled: false,
path: filePath,
error: error instanceof Error ? error.message : String(error),
};
}
}
function initializeAppLifecycle() {
app.whenReady().then(() => {
const userDataPath = app.getPath("userData");
@@ -1032,7 +962,7 @@ function initializeAppLifecycle() {
artifactStoreRepository = repositoryContext.artifactStoreRepository;
reviewSamplesRepository = repositoryContext.reviewSamplesRepository;
scannerLearningRepository = repositoryContext.scannerLearningRepository;
inputHelperService = createInputHelperService({ userDataPath });
inputHelperService = createInputHelperService({ userDataPath, exePath: resolveInputHelperExePath() });
registerIpcHandlers({
focusMainWindow: () => focusMainWindow(),
@@ -1053,6 +983,7 @@ function initializeAppLifecycle() {
loadScannerLearningRules: () => loadScannerLearningRules(),
writeScannerLearningRules: (rules: ScannerLearningRulePayload) => writeScannerLearningRules(rules),
exportGood: (exportPayload: GoodDatabase) => exportGood(exportPayload),
importGoodFile: () => importGoodFile(),
listSources: () => listCaptureSources(),
captureSource: (
id: string,
+2
View File
@@ -11,6 +11,7 @@ contextBridge.exposeInMainWorld("assistantApi", {
getAutomationGuard: () => ipcRenderer.invoke("automation:getGuard"),
focusMainWindow: () => ipcRenderer.invoke("app:focusMainWindow"),
focusGenshin: () => ipcRenderer.invoke("automation:focusGenshin"),
focusGenshinForScanStart: () => ipcRenderer.invoke("automation:focusGenshin"),
getRuntimeInfo: () => ipcRenderer.invoke("app:getRuntimeInfo"),
saveReviewSample: (sample) => ipcRenderer.invoke("review:saveSample", sample),
loadReviewSamples: (limit = 50) => ipcRenderer.invoke("review:loadSamples", limit),
@@ -19,6 +20,7 @@ contextBridge.exposeInMainWorld("assistantApi", {
loadArtifacts: () => ipcRenderer.invoke("artifacts:load"),
saveArtifacts: (records) => ipcRenderer.invoke("artifacts:saveMany", records),
exportGood: (payload) => ipcRenderer.invoke("good:export", payload),
importGoodFile: () => ipcRenderer.invoke("good:importFile"),
publishScannerStatus: (status) => ipcRenderer.invoke("scanner:publishStatus", status),
showOverlay: () => ipcRenderer.invoke("overlay:show"),
hideOverlay: () => ipcRenderer.invoke("overlay:hide"),
+5 -3
View File
@@ -1,5 +1,5 @@
import { contextBridge, ipcRenderer } from "electron";
import type { CaptureOptions, GoodDatabase, ReviewSamplePayload, ScannerStatusPayload, ScannerLearningRulePayload } from "../src/types/global.js";
import type { CaptureOptions, GoodDatabase, ReviewSamplePayload, ScannerCommand, ScannerStatusPayload, ScannerLearningRulePayload } from "../src/types/global.js";
import type { StoredArtifactRecord } from "../src/types/storage.js";
import type { AppSnapshot } from "../src/types/domain.js";
@@ -14,6 +14,7 @@ contextBridge.exposeInMainWorld("assistantApi", {
getAutomationGuard: () => ipcRenderer.invoke("automation:getGuard"),
focusMainWindow: () => ipcRenderer.invoke("app:focusMainWindow"),
focusGenshin: () => ipcRenderer.invoke("automation:focusGenshin"),
focusGenshinForScanStart: () => ipcRenderer.invoke("automation:focusGenshin"),
getRuntimeInfo: () => ipcRenderer.invoke("app:getRuntimeInfo"),
saveReviewSample: (sample: ReviewSamplePayload) => ipcRenderer.invoke("review:saveSample", sample),
loadReviewSamples: (limit = 50) => ipcRenderer.invoke("review:loadSamples", limit),
@@ -22,11 +23,12 @@ contextBridge.exposeInMainWorld("assistantApi", {
loadArtifacts: () => ipcRenderer.invoke("artifacts:load"),
saveArtifacts: (records: StoredArtifactRecord[]) => ipcRenderer.invoke("artifacts:saveMany", records),
exportGood: (payload: GoodDatabase) => ipcRenderer.invoke("good:export", payload),
importGoodFile: () => ipcRenderer.invoke("good:importFile"),
publishScannerStatus: (status: ScannerStatusPayload) => ipcRenderer.invoke("scanner:publishStatus", status),
showOverlay: () => ipcRenderer.invoke("overlay:show"),
hideOverlay: () => ipcRenderer.invoke("overlay:hide"),
onScannerCommand: (callback: (command: "start-auto" | "stop") => void) => {
const listener = (_event: Electron.IpcRendererEvent, command: "start-auto" | "stop") => callback(command);
onScannerCommand: (callback: (command: ScannerCommand) => void) => {
const listener = (_event: Electron.IpcRendererEvent, command: ScannerCommand) => callback(command);
ipcRenderer.on("scanner:command", listener);
return () => ipcRenderer.removeListener("scanner:command", listener);
},
@@ -54,6 +54,7 @@ export class JsonArtifactStoreRepository implements ArtifactStoreRepositoryPort
lastSeenAt: now,
timesSeen: (existing.timesSeen ?? 1) + 1,
confidence: Math.max(existing.confidence ?? 0, record.confidence ?? 0),
locked: typeof record.locked === "boolean" ? record.locked : existing.locked,
// A later confident scan clears the review flag; an uncertain rescan
// must not downgrade an already confirmed artifact.
needsReview: Boolean(existing.needsReview) && Boolean(record.needsReview),
@@ -136,6 +137,7 @@ function normalizeStoredArtifactRecordForLoad(record: StoredArtifactRecord) {
...record,
timesSeen: reviewOnly ? 1 : normalizedTimesSeen,
firstSeenAt: record.firstSeenAt ?? record.lastSeenAt,
locked: typeof record.locked === "boolean" ? record.locked : undefined,
};
}
@@ -183,6 +185,7 @@ function mergeArtifactRecords(existing: StoredArtifactRecord, incoming: StoredAr
substats: [...(preferredSubstats ?? [])],
equipped: preferred.equipped && preferred.equipped !== "Not detected" ? preferred.equipped : secondary.equipped,
confidence: Math.max(existing.confidence ?? 0, incoming.confidence ?? 0),
locked: typeof incoming.locked === "boolean" ? incoming.locked : existing.locked,
needsReview: Boolean(existing.needsReview) && Boolean(incoming.needsReview),
source: resolveStoredArtifactSource(existing.source, incoming.source),
firstSeenAt: existing.firstSeenAt ?? now,
+97 -24
View File
@@ -39,6 +39,16 @@ public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern bool ShowWindowAsync(IntPtr hWnd, int nCmdShow);
[DllImport("user32.dll")]
public static extern bool BringWindowToTop(IntPtr hWnd);
[DllImport("user32.dll", SetLastError=true)]
public static extern bool AttachThreadInput(uint idAttach, uint idAttachTo, bool fAttach);
[DllImport("kernel32.dll")]
public static extern uint GetCurrentThreadId();
[DllImport("user32.dll", SetLastError=true, EntryPoint="SystemParametersInfoW")]
public static extern bool SystemParametersInfoGet(uint uiAction, uint uiParam, ref uint pvParam, uint fWinIni);
[DllImport("user32.dll", SetLastError=true, EntryPoint="SystemParametersInfoW")]
public static extern bool SystemParametersInfoSet(uint uiAction, uint uiParam, IntPtr pvParam, uint fWinIni);
[DllImport("user32.dll")]
public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll")]
public static extern bool IsWindow(IntPtr hWnd);
@@ -181,6 +191,44 @@ function Find-GenshinWindow {
return $script:genshinHwnd
}
# Plain SetForegroundWindow from this background helper process is silently
# refused by Windows' foreground lock. Attach our thread's input queue to the
# target (and current foreground) window thread and clear the lock timeout, so
# the foreground change is honored - the same technique Inventory Kamera uses.
function Force-Foreground {
param([IntPtr]$hwnd)
$current = [Native.InputHelper]::GetCurrentThreadId()
$targetPid = [uint32]0
$target = [Native.InputHelper]::GetWindowThreadProcessId($hwnd, [ref]$targetPid)
$fgWindow = [Native.InputHelper]::GetForegroundWindow()
$foreground = [uint32]0
if ($fgWindow -ne [IntPtr]::Zero) {
$fgPid = [uint32]0
$foreground = [Native.InputHelper]::GetWindowThreadProcessId($fgWindow, [ref]$fgPid)
}
$attachedTarget = $false
$attachedForeground = $false
$oldTimeout = [uint32]0
$timeoutRead = $false
try {
if ($target -ne 0 -and $target -ne $current) { $attachedTarget = [Native.InputHelper]::AttachThreadInput($current, $target, $true) }
if ($foreground -ne 0 -and $foreground -ne $current -and $foreground -ne $target) { $attachedForeground = [Native.InputHelper]::AttachThreadInput($current, $foreground, $true) }
$timeoutRead = [Native.InputHelper]::SystemParametersInfoGet(0x2000, 0, [ref]$oldTimeout, 0)
[Native.InputHelper]::SystemParametersInfoSet(0x2001, 0, [IntPtr]::Zero, 0x0002) | Out-Null
# Inject a no-op input (0,0 mouse move) so this process is the last input
# source, which Windows requires before it will honor a foreground change.
Send-MouseInput -flags 0x0001 | Out-Null
[Native.InputHelper]::ShowWindowAsync($hwnd, 9) | Out-Null
[Native.InputHelper]::BringWindowToTop($hwnd) | Out-Null
return [Native.InputHelper]::SetForegroundWindow($hwnd)
} finally {
if ($timeoutRead) { [Native.InputHelper]::SystemParametersInfoSet(0x2001, 0, [IntPtr]::new([int64]$oldTimeout), 0x0002) | Out-Null }
if ($attachedForeground) { [Native.InputHelper]::AttachThreadInput($current, $foreground, $false) | Out-Null }
if ($attachedTarget) { [Native.InputHelper]::AttachThreadInput($current, $target, $false) | Out-Null }
}
}
function Focus-GenshinWindow {
$hwnd = Find-GenshinWindow
$info = @{
@@ -194,19 +242,7 @@ function Focus-GenshinWindow {
$info.alreadyForeground = ([Native.InputHelper]::GetForegroundWindow() -eq $hwnd)
if (-not $info.alreadyForeground) {
[Native.InputHelper]::ShowWindowAsync($hwnd, 9) | Out-Null
# A previous version tapped ALT (keybd_event) right before this call to
# satisfy Windows' "who's allowed to change the foreground window"
# eligibility check. That tap has a side effect in most Win32 apps: a
# bare ALT press/release toggles menu-mnemonic navigation mode (verified
# live - it left a real app's menu bar highlighted after just this call),
# which then swallows the next several keyboard/mouse events as menu
# navigation instead of routing them to the app - looking exactly like
# "clicks/keys report success but do nothing". This app and Genshin run
# at the same (elevated) integrity level, so plain SetForegroundWindow
# already succeeds without the ALT tap - confirmed with a standalone
# compiled test against a live target window.
$info.setForegroundResult = [Native.InputHelper]::SetForegroundWindow($hwnd)
$info.setForegroundResult = Force-Foreground -hwnd $hwnd
Start-Sleep -Milliseconds 140
}
@@ -382,7 +418,7 @@ class InputHelperClient {
private starting: Promise<void> | null = null;
private disposed = false;
constructor(private readonly scriptUserDataPath: string) {}
constructor(private readonly options: { scriptUserDataPath: string; exePath?: string | null }) {}
private async ensureStarted() {
if (this.child) return;
@@ -396,14 +432,31 @@ class InputHelperClient {
}
private async start() {
const scriptPath = path.join(this.scriptUserDataPath, "input-helper.ps1");
// Prefer the compiled C# sidecar (ADR-008). If it is missing or fails to
// start, fall back to the embedded PowerShell helper so the app keeps working
// on machines where the native exe was never built.
if (this.options.exePath) {
try {
await this.startWith(spawn(this.options.exePath, [], { windowsHide: true, stdio: ["pipe", "pipe", "pipe"] }));
return;
} catch {
this.teardownChild();
}
}
await this.startWith(await this.spawnPowershell());
}
private async spawnPowershell() {
const scriptPath = path.join(this.options.scriptUserDataPath, "input-helper.ps1");
await fs.mkdir(path.dirname(scriptPath), { recursive: true });
await fs.writeFile(scriptPath, INPUT_HELPER_SCRIPT, "utf8");
const child = spawn("powershell.exe", ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", scriptPath], {
return spawn("powershell.exe", ["-NoProfile", "-ExecutionPolicy", "Bypass", "-File", scriptPath], {
windowsHide: true,
stdio: ["pipe", "pipe", "pipe"],
});
}
private async startWith(child: ChildProcessWithoutNullStreams) {
child.stdout.setEncoding("utf8");
child.stdout.on("data", (chunk: string) => this.handleStdout(chunk));
child.stderr.setEncoding("utf8");
@@ -419,10 +472,22 @@ class InputHelperClient {
});
this.child = child;
// First request compiles the Win32 interop; give it extra time.
// The C# sidecar answers ping immediately; the PowerShell fallback compiles
// Win32 interop on the first request, so give it extra time.
await this.send("ping", {}, 20000);
}
private teardownChild() {
const child = this.child;
this.child = null;
this.buffer = "";
try {
child?.kill();
} catch {
// Child was never spawned or already gone.
}
}
private handleStdout(chunk: string) {
this.buffer += chunk;
let newlineIndex = this.buffer.indexOf("\n");
@@ -485,8 +550,8 @@ export interface InputHelperService {
dispose(): void;
}
export function createInputHelperService(options: { userDataPath: string }): InputHelperService {
const inputHelper = new InputHelperClient(options.userDataPath);
export function createInputHelperService(options: { userDataPath: string; exePath?: string | null }): InputHelperService {
const inputHelper = new InputHelperClient({ scriptUserDataPath: options.userDataPath, exePath: options.exePath ?? null });
async function request(op: string, params: Record<string, unknown> = {}, timeoutMs = 8000) {
return inputHelper.request(op, params, timeoutMs);
@@ -590,16 +655,24 @@ export function createInputHelperService(options: { userDataPath: string }): Inp
async function capturePrimaryScreenViaGdi() {
const result = await request("capture", {}, 15000);
const capturePath = String(result.path);
const buffer = await fs.readFile(capturePath);
await fs.unlink(capturePath).catch(() => undefined);
// The C# sidecar returns PNG bytes inline (no temp file). The PowerShell
// fallback writes a temp PNG and returns its path.
let base64: string;
if (typeof result.imageBase64 === "string" && result.imageBase64) {
base64 = result.imageBase64;
} else {
const capturePath = String(result.path);
const buffer = await fs.readFile(capturePath);
await fs.unlink(capturePath).catch(() => undefined);
base64 = buffer.toString("base64");
}
const captureTargetRaw = typeof result.captureTarget === "string" ? result.captureTarget : "";
const captureTarget: "primary-screen" | "genshin-client" = captureTargetRaw === "primary-screen" || captureTargetRaw === "genshin-client"
? captureTargetRaw
: "primary-screen";
return {
dataUrl: `data:image/png;base64,${buffer.toString("base64")}`,
dataUrl: `data:image/png;base64,${base64}`,
width: Number(result.width),
height: Number(result.height),
originX: Number(result.originX),
+4
View File
@@ -0,0 +1,4 @@
# .NET build outputs — the self-contained exe is built via `npm run helper:build`,
# not committed (it is ~100 MB).
bin/
obj/
+23
View File
@@ -0,0 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0-windows</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<AssemblyName>InputHelper</AssemblyName>
<RootNamespace>GenshinAssistant.InputHelper</RootNamespace>
<!-- Screen.PrimaryScreen (Forms) + Bitmap/Graphics.CopyFromScreen (Drawing). -->
<UseWindowsForms>true</UseWindowsForms>
<!-- Single self-contained exe: no .NET install needed on the user's machine. -->
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>true</PublishSingleFile>
<IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>
<InvariantGlobalization>true</InvariantGlobalization>
<!-- Per-monitor DPI v2 so SendInput/capture coordinates match a mixed-DPI
multi-monitor setup (same reason as the old PowerShell helper). -->
<ApplicationManifest>app.manifest</ApplicationManifest>
</PropertyGroup>
</Project>
+535
View File
@@ -0,0 +1,535 @@
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Security.Principal;
using System.Text;
using System.Text.Json;
using System.Windows.Forms;
// Long-lived input/capture sidecar for the Genshin Artifact Assistant.
// Drop-in replacement for the old PowerShell helper (see ADR-008): identical
// JSON-over-stdin/stdout protocol - one JSON request per line, one JSON response
// per line - so the Electron-side InputHelperService is unchanged. Win32 interop
// is compiled once (this is a native exe), and capture returns base64 PNG bytes
// directly instead of writing a temp file per frame.
namespace GenshinAssistant.InputHelper;
internal static class Program
{
private static IntPtr _genshinHwnd = IntPtr.Zero;
private static int Main()
{
// Manifest already declares PerMonitorV2; this is a belt-and-suspenders
// call for hosts that ignore the manifest.
try { Native.SetProcessDpiAwarenessContext(Native.DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2); }
catch { try { Native.SetProcessDpiAwareness(2); } catch { /* oldest fallback */ Native.SetProcessDPIAware(); } }
Console.OutputEncoding = Encoding.UTF8;
var stdout = Console.Out;
string? line;
while ((line = Console.In.ReadLine()) != null)
{
if (line.Trim().Length == 0) continue;
var response = new Dictionary<string, object?> { ["id"] = "", ["ok"] = true };
try
{
using var doc = JsonDocument.Parse(line);
var root = doc.RootElement;
response["id"] = GetString(root, "id");
var op = GetString(root, "op");
Handle(op, root, response);
}
catch (Exception ex)
{
response["ok"] = false;
response["error"] = ex.Message;
}
stdout.WriteLine(JsonSerializer.Serialize(response));
stdout.Flush();
}
return 0;
}
private static void Handle(string op, JsonElement root, Dictionary<string, object?> response)
{
switch (op)
{
case "ping":
response["pong"] = true;
break;
case "cursor":
{
var state = GetCursorState();
response["cursorX"] = state.X;
response["cursorY"] = state.Y;
response["escapePressed"] = state.Escape;
response["enterPressed"] = state.Enter;
response["f9Pressed"] = state.F9;
break;
}
case "runtime":
{
var hwnd = FindGenshinWindow();
var fgHwnd = Native.GetForegroundWindow();
response["isElevated"] = IsElevated();
response["genshinFound"] = hwnd != IntPtr.Zero;
response["genshinHwnd"] = hwnd.ToInt64();
response["targetProcess"] = ProcessNameFromHwnd(hwnd);
response["foregroundProcess"] = ProcessNameFromHwnd(fgHwnd);
response["foregroundHwnd"] = fgHwnd.ToInt64();
response["helperPid"] = Environment.ProcessId;
break;
}
case "focus":
{
var info = FocusGenshinWindow();
response["focused"] = info.Focused;
response["alreadyForeground"] = info.AlreadyForeground;
response["foregroundProcess"] = info.ForegroundProcess;
response["targetProcess"] = info.TargetProcess;
response["genshinFound"] = info.Hwnd != IntPtr.Zero;
response["setForegroundResult"] = info.SetForegroundResult;
break;
}
case "click":
{
var info = FocusGenshinWindow();
if (info.Focused && !info.AlreadyForeground) Thread.Sleep(120);
var targetX = GetInt(root, "x");
var targetY = GetInt(root, "y");
// Bare SetCursorPos then a batched down+up click, matching the
// verified Inventory Kamera sequence: no extra move event, no
// gap between move and click.
Native.SetCursorPos(targetX, targetY);
Native.GetCursorPos(out var pt);
var onTarget = Math.Abs(targetX - pt.X) <= 2 && Math.Abs(targetY - pt.Y) <= 2;
var clickEventsSent = onTarget ? SendMouseClickBatch() : 0u;
var state = GetCursorState();
response["cursorX"] = state.X;
response["cursorY"] = state.Y;
response["escapePressed"] = state.Escape;
response["enterPressed"] = state.Enter;
response["f9Pressed"] = state.F9;
response["moved"] = onTarget;
response["focused"] = info.Focused;
response["alreadyForeground"] = info.AlreadyForeground;
response["foregroundProcess"] = info.ForegroundProcess;
response["targetProcess"] = info.TargetProcess;
response["isElevated"] = IsElevated();
// Only report a click when the cursor is verifiably on target and
// SendInput injected both events; real acceptance is proven later
// by the detail-panel fingerprint.
response["clicked"] = onTarget && clickEventsSent >= 2;
response["inputBlocked"] = onTarget && clickEventsSent < 2;
break;
}
case "scroll":
{
var info = FocusGenshinWindow();
if (info.Focused && !info.AlreadyForeground) Thread.Sleep(120);
if (TryGetInt(root, "x", out var ax) && TryGetInt(root, "y", out var ay))
{
Native.SetCursorPos(ax, ay);
Thread.Sleep(30);
}
Native.GetCursorPos(out var pt);
response["cursorX"] = pt.X;
response["cursorY"] = pt.Y;
response["focused"] = info.Focused;
response["foregroundProcess"] = info.ForegroundProcess;
response["isElevated"] = IsElevated();
var notches = GetInt(root, "notches");
var stepDelta = notches < 0 ? -120 : 120;
var count = Math.Min(60, Math.Abs(notches));
uint sentTotal = 0;
for (var i = 0; i < count; i++)
{
sentTotal += SendMouseWheel(stepDelta);
Thread.Sleep(45);
}
response["notchesSent"] = sentTotal;
response["inputBlocked"] = count > 0 && sentTotal == 0;
break;
}
case "bounds":
{
var bounds = GetGenshinClientBounds();
if (bounds == null)
{
response["found"] = false;
}
else
{
response["found"] = true;
response["left"] = bounds.Value.Left;
response["top"] = bounds.Value.Top;
response["width"] = bounds.Value.Width;
response["height"] = bounds.Value.Height;
}
break;
}
case "capture":
{
var bounds = GetGenshinClientBounds();
string captureTarget;
Rect area;
if (bounds == null)
{
var screen = Screen.PrimaryScreen!.Bounds;
area = new Rect { Left = screen.Left, Top = screen.Top, Width = screen.Width, Height = screen.Height };
captureTarget = "primary-screen";
}
else
{
area = bounds.Value;
captureTarget = "genshin-client";
}
using var bitmap = new Bitmap(area.Width, area.Height, PixelFormat.Format32bppArgb);
using (var graphics = Graphics.FromImage(bitmap))
{
graphics.CopyFromScreen(area.Left, area.Top, 0, 0, bitmap.Size, CopyPixelOperation.SourceCopy);
}
using var stream = new MemoryStream();
bitmap.Save(stream, ImageFormat.Png);
response["imageBase64"] = Convert.ToBase64String(stream.ToArray());
response["width"] = area.Width;
response["height"] = area.Height;
response["originX"] = area.Left;
response["originY"] = area.Top;
response["captureTarget"] = captureTarget;
break;
}
default:
response["ok"] = false;
response["error"] = "unknown op";
break;
}
}
private readonly struct Rect
{
public int Left { get; init; }
public int Top { get; init; }
public int Width { get; init; }
public int Height { get; init; }
}
private struct CursorState
{
public int X;
public int Y;
public bool Escape;
public bool Enter;
public bool F9;
}
private struct FocusInfo
{
public IntPtr Hwnd;
public bool Focused;
public bool AlreadyForeground;
public string ForegroundProcess;
public string TargetProcess;
public bool SetForegroundResult;
}
private static CursorState GetCursorState()
{
Native.GetCursorPos(out var pt);
// Only 0x8000 (held right now). The 0x0001 "pressed since last call" bit
// is unreliable and fires for ESC presses used to navigate Genshin menus.
var esc = (Native.GetAsyncKeyState(0x1B) & 0x8000) != 0;
var enter = (Native.GetAsyncKeyState(0x0D) & 0x8000) != 0;
var f9 = (Native.GetAsyncKeyState(0x78) & 0x8000) != 0;
return new CursorState { X = pt.X, Y = pt.Y, Escape = esc, Enter = enter, F9 = f9 };
}
private static FocusInfo FocusGenshinWindow()
{
var hwnd = FindGenshinWindow();
var info = new FocusInfo
{
Hwnd = hwnd,
ForegroundProcess = "",
TargetProcess = ProcessNameFromHwnd(hwnd),
};
if (hwnd == IntPtr.Zero) return info;
info.AlreadyForeground = Native.GetForegroundWindow() == hwnd;
if (!info.AlreadyForeground)
{
info.SetForegroundResult = ForceForeground(hwnd);
Thread.Sleep(140);
}
var foreground = Native.GetForegroundWindow();
info.Focused = foreground == hwnd;
info.ForegroundProcess = ProcessNameFromHwnd(foreground);
return info;
}
// Plain SetForegroundWindow from a background process is silently refused by
// Windows' foreground lock. Inventory Kamera and other reliable automation
// tools bypass it by attaching the calling thread's input queue to the target
// (and current-foreground) window thread and clearing the lock timeout, so the
// foreground change is honored. Without this the auto-scan aborts with
// "Genshin konnte nicht in den Vordergrund geholt werden".
private static bool ForceForeground(IntPtr hwnd)
{
var current = Native.GetCurrentThreadId();
var target = Native.GetWindowThreadProcessId(hwnd, out _);
var foregroundHwnd = Native.GetForegroundWindow();
var foreground = foregroundHwnd != IntPtr.Zero ? Native.GetWindowThreadProcessId(foregroundHwnd, out _) : 0u;
var attachedTarget = false;
var attachedForeground = false;
uint oldTimeout = 0;
var timeoutRead = false;
try
{
if (target != 0 && target != current) attachedTarget = Native.AttachThreadInput(current, target, true);
if (foreground != 0 && foreground != current && foreground != target)
attachedForeground = Native.AttachThreadInput(current, foreground, true);
timeoutRead = Native.SystemParametersInfo(Native.SPI_GETFOREGROUNDLOCKTIMEOUT, 0, ref oldTimeout, 0);
Native.SystemParametersInfo(Native.SPI_SETFOREGROUNDLOCKTIMEOUT, 0, IntPtr.Zero, Native.SPIF_SENDCHANGE);
// Inject a no-op input (0,0 relative mouse move) so this process counts
// as the last input source - one of the conditions Windows requires to
// allow a foreground change. This is what the removed ALT tap did, but
// without the menu-mnemonic side effect.
NudgeInput();
Native.ShowWindowAsync(hwnd, 9); // SW_RESTORE
Native.BringWindowToTop(hwnd);
var ok = Native.SetForegroundWindow(hwnd);
return ok;
}
finally
{
if (timeoutRead)
Native.SystemParametersInfo(Native.SPI_SETFOREGROUNDLOCKTIMEOUT, 0, new IntPtr((long)oldTimeout), Native.SPIF_SENDCHANGE);
if (attachedForeground) Native.AttachThreadInput(current, foreground, false);
if (attachedTarget) Native.AttachThreadInput(current, target, false);
}
}
private static IntPtr FindGenshinWindow()
{
if (_genshinHwnd != IntPtr.Zero && Native.IsWindow(_genshinHwnd)) return _genshinHwnd;
foreach (var proc in Process.GetProcesses())
{
try
{
var name = proc.ProcessName;
if ((name.Contains("GenshinImpact", StringComparison.OrdinalIgnoreCase)
|| name.Contains("YuanShen", StringComparison.OrdinalIgnoreCase)
|| name.Contains("Genshin", StringComparison.OrdinalIgnoreCase))
&& proc.MainWindowHandle != IntPtr.Zero)
{
_genshinHwnd = proc.MainWindowHandle;
return _genshinHwnd;
}
}
catch
{
// Process exited between enumeration and inspection; ignore.
}
}
_genshinHwnd = IntPtr.Zero;
return _genshinHwnd;
}
private static Rect? GetGenshinClientBounds()
{
var hwnd = FindGenshinWindow();
if (hwnd == IntPtr.Zero) return null;
if (!Native.GetClientRect(hwnd, out var rect)) return null;
var topLeft = new Native.POINT { X = 0, Y = 0 };
if (!Native.ClientToScreen(hwnd, ref topLeft)) return null;
var width = rect.Right - rect.Left;
var height = rect.Bottom - rect.Top;
if (width <= 0 || height <= 0) return null;
return new Rect { Left = topLeft.X, Top = topLeft.Y, Width = width, Height = height };
}
private static string ProcessNameFromHwnd(IntPtr hwnd)
{
if (hwnd == IntPtr.Zero) return "";
Native.GetWindowThreadProcessId(hwnd, out var pid);
if (pid == 0) return "";
try { return Process.GetProcessById((int)pid).ProcessName; }
catch { return ""; }
}
private static bool IsElevated()
{
using var identity = WindowsIdentity.GetCurrent();
var principal = new WindowsPrincipal(identity);
return principal.IsInRole(WindowsBuiltInRole.Administrator);
}
private static void NudgeInput()
{
var move = new Native.INPUT[1];
move[0].type = 0; // INPUT_MOUSE
move[0].mi.dwFlags = Native.MOUSEEVENTF_MOVE; // dx=dy=0 -> no cursor movement
Native.SendInput(1, move, Marshal.SizeOf<Native.INPUT>());
}
private static uint SendMouseClickBatch()
{
var inputs = new Native.INPUT[2];
inputs[0].type = 0; // INPUT_MOUSE
inputs[0].mi.dwFlags = Native.MOUSEEVENTF_LEFTDOWN;
inputs[1].type = 0;
inputs[1].mi.dwFlags = Native.MOUSEEVENTF_LEFTUP;
return Native.SendInput(2, inputs, Marshal.SizeOf<Native.INPUT>());
}
private static uint SendMouseWheel(int wheelData)
{
var inputs = new Native.INPUT[1];
inputs[0].type = 0;
inputs[0].mi.mouseData = unchecked((uint)wheelData);
inputs[0].mi.dwFlags = Native.MOUSEEVENTF_WHEEL;
return Native.SendInput(1, inputs, Marshal.SizeOf<Native.INPUT>());
}
private static string GetString(JsonElement root, string name)
=> root.TryGetProperty(name, out var value) ? value.ToString() : "";
private static int GetInt(JsonElement root, string name)
=> TryGetInt(root, name, out var value) ? value : 0;
private static bool TryGetInt(JsonElement root, string name, out int value)
{
value = 0;
if (!root.TryGetProperty(name, out var element)) return false;
if (element.ValueKind == JsonValueKind.Number && element.TryGetInt32(out value)) return true;
if (element.ValueKind == JsonValueKind.String && int.TryParse(element.GetString(), out value)) return true;
return false;
}
}
internal static class Native
{
public const uint MOUSEEVENTF_MOVE = 0x0001;
public const uint MOUSEEVENTF_LEFTDOWN = 0x0002;
public const uint MOUSEEVENTF_LEFTUP = 0x0004;
public const uint MOUSEEVENTF_WHEEL = 0x0800;
public const uint SPI_GETFOREGROUNDLOCKTIMEOUT = 0x2000;
public const uint SPI_SETFOREGROUNDLOCKTIMEOUT = 0x2001;
public const uint SPIF_SENDCHANGE = 0x0002;
public static readonly IntPtr DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2 = new(-4);
[StructLayout(LayoutKind.Sequential)]
public struct POINT { public int X; public int Y; }
[StructLayout(LayoutKind.Sequential)]
public struct RECT { public int Left; public int Top; public int Right; public int Bottom; }
// Binary-compatible with the Win32 INPUT for mouse-only use on x64:
// type(4) + 4 pad + MOUSEINPUT(32) = 40 bytes = sizeof(INPUT).
[StructLayout(LayoutKind.Sequential)]
public struct MOUSEINPUT
{
public int dx;
public int dy;
public uint mouseData;
public uint dwFlags;
public uint time;
public UIntPtr dwExtraInfo;
}
[StructLayout(LayoutKind.Sequential)]
public struct INPUT
{
public int type;
public MOUSEINPUT mi;
}
[DllImport("user32.dll")]
public static extern bool SetProcessDPIAware();
[DllImport("shcore.dll")]
public static extern int SetProcessDpiAwareness(int value);
[DllImport("user32.dll")]
public static extern bool SetProcessDpiAwarenessContext(IntPtr value);
[DllImport("user32.dll")]
public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")]
public static extern bool GetCursorPos(out POINT lpPoint);
[DllImport("user32.dll", SetLastError = true)]
public static extern bool GetClientRect(IntPtr hWnd, out RECT lpRect);
[DllImport("user32.dll", SetLastError = true)]
public static extern bool ClientToScreen(IntPtr hWnd, ref POINT lpPoint);
[DllImport("user32.dll")]
public static extern short GetAsyncKeyState(int vKey);
[DllImport("user32.dll", SetLastError = true)]
public static extern uint SendInput(uint nInputs, INPUT[] pInputs, int cbSize);
[DllImport("user32.dll")]
public static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern bool ShowWindowAsync(IntPtr hWnd, int nCmdShow);
[DllImport("user32.dll")]
public static extern bool BringWindowToTop(IntPtr hWnd);
[DllImport("user32.dll", SetLastError = true)]
public static extern bool AttachThreadInput(uint idAttach, uint idAttachTo, bool fAttach);
[DllImport("kernel32.dll")]
public static extern uint GetCurrentThreadId();
[DllImport("user32.dll", SetLastError = true)]
public static extern bool SystemParametersInfo(uint uiAction, uint uiParam, ref uint pvParam, uint fWinIni);
[DllImport("user32.dll", SetLastError = true)]
public static extern bool SystemParametersInfo(uint uiAction, uint uiParam, IntPtr pvParam, uint fWinIni);
[DllImport("user32.dll")]
public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll")]
public static extern bool IsWindow(IntPtr hWnd);
[DllImport("user32.dll")]
public static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint lpdwProcessId);
}
+11
View File
@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="utf-8"?>
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
<assemblyIdentity version="1.0.0.0" name="GenshinAssistant.InputHelper" type="win32" />
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings>
<!-- PerMonitorV2: coordinates stay correct across mixed-DPI monitors. -->
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness>
<dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true/pm</dpiAware>
</windowsSettings>
</application>
</assembly>
+15 -4
View File
@@ -3,18 +3,20 @@
"version": "0.1.0",
"private": true,
"description": "Local Windows assistant for scanning Genshin artifacts and suggesting no-brainer builds.",
"main": "dist-electron/main.js",
"main": "dist-electron/electron/main.js",
"type": "module",
"scripts": {
"predev": "powershell -NoProfile -ExecutionPolicy Bypass -File scripts\\kill-stale-instances.ps1 && tsc -p tsconfig.electron.json && copy electron\\preload.cjs dist-electron\\preload.cjs",
"predev": "powershell -NoProfile -ExecutionPolicy Bypass -File scripts\\kill-stale-instances.ps1 && tsc -p tsconfig.electron.json && copy electron\\preload.cjs dist-electron\\electron\\preload.cjs",
"dev": "concurrently -k \"vite --host 127.0.0.1\" \"wait-on tcp:5173 && cross-env VITE_DEV_SERVER_URL=http://127.0.0.1:5173 electron .\"",
"dev:admin": ".\\dev-admin.cmd",
"build": "tsc && vite build && tsc -p tsconfig.electron.json && copy electron\\\\preload.cjs dist-electron\\\\preload.cjs",
"dev:web": "vite --host 127.0.0.1",
"dev:admin": "powershell -NoProfile -ExecutionPolicy Bypass -File scripts\\dev-admin.ps1 -ProjectRoot .",
"build": "tsc && vite build && tsc -p tsconfig.electron.json && copy electron\\\\preload.cjs dist-electron\\\\electron\\\\preload.cjs",
"preview": "vite preview --host 127.0.0.1",
"start": "electron .",
"lint": "tsc --noEmit",
"test": "vitest run",
"eval": "vitest run src/eval/ocrEval.test.ts",
"helper:build": "dotnet publish native/input-helper/InputHelper.csproj -c Release -o native/input-helper/bin/publish",
"data:genshin": "node scripts/generate-genshin-data.cjs"
},
"dependencies": {
@@ -48,6 +50,15 @@
"dist-electron/**/*",
"package.json"
],
"extraResources": [
{
"from": "native/input-helper/bin/publish",
"to": "input-helper",
"filter": [
"**/*"
]
}
],
"win": {
"target": "nsis",
"requestedExecutionLevel": "requireAdministrator"
+9
View File
@@ -9,8 +9,17 @@ $ErrorActionPreference = "Stop"
try {
$project = (Resolve-Path -LiteralPath $ProjectRoot).Path
$logDir = Join-Path $project "outputs\admin-start"
New-Item -ItemType Directory -Force -Path $logDir | Out-Null
$logPath = Join-Path $logDir "admin-dev.log"
try {
Start-Transcript -Path $logPath -Append | Out-Null
} catch {
Write-Host "WARNUNG: Admin-Start-Log konnte nicht geschrieben werden: $($_.Exception.Message)" -ForegroundColor Yellow
}
Write-Host "Projekt: $project"
Write-Host "Admin-Log: $logPath"
# A UAC-elevated process gets its environment rebuilt fresh from the
# registry; it does NOT inherit PATH edits that only exist in the calling
+33
View File
@@ -0,0 +1,33 @@
param(
[string]$ProjectRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
)
$ErrorActionPreference = "Stop"
function Quote-ProcessArgument([string]$Value) {
return '"' + $Value.Replace('"', '\"') + '"'
}
try {
$project = (Resolve-Path -LiteralPath $ProjectRoot).Path
$script = Join-Path $PSScriptRoot "dev-admin-start.ps1"
$powershellExe = Join-Path $env:SystemRoot "System32\WindowsPowerShell\v1.0\powershell.exe"
$arguments = @(
"-NoProfile",
"-ExecutionPolicy Bypass",
"-NoExit",
"-File $(Quote-ProcessArgument $script)",
"-ProjectRoot $(Quote-ProcessArgument $project)"
) -join " "
Start-Process -FilePath $powershellExe -ArgumentList $arguments -WorkingDirectory $project -Verb RunAs -WindowStyle Normal -ErrorAction Stop
Write-Host ""
Write-Host "UAC-Abfrage gestartet. Bitte bestaetigen - danach oeffnet sich ein neues Administrator-Fenster mit npm run dev." -ForegroundColor Green
Write-Host "Dieses Fenster kann geschlossen werden; das eigentliche Programm laeuft im neuen Administrator-Fenster."
} catch {
Write-Host ""
Write-Host "Admin-Start fehlgeschlagen oder UAC-Abfrage abgelehnt:" -ForegroundColor Red
Write-Host $_.Exception.Message -ForegroundColor Red
exit 1
}
+1 -10
View File
@@ -90,7 +90,7 @@ export function AppTopbar({
<header className="topbar">
<div>
<p className="eyebrow">Lokaler Windows-Assistent</p>
<h1>Scanne dein Inventar, triff einfache Artifact-Entscheidungen.</h1>
<h1>Inventar scannen, Artifacts entscheiden.</h1>
</div>
<div className="topbar-actions">
{topbarStatus && <span className="topbar-status">{topbarStatus}</span>}
@@ -107,15 +107,6 @@ export function AppTopbar({
{overlayIcon}
<span>{overlayButtonLabel}</span>
</button>
<button
className="ghost-button"
onClick={handleDemoScan}
disabled={isDemoDisabled}
title={demoButtonTitle}
>
{demoIcon}
{demoButtonLabel}
</button>
</div>
</header>
);
+2 -1
View File
@@ -1,4 +1,4 @@
import { Layers3, Radar, Wand2, Eye } from "lucide-react";
import { Layers3, Radar, Wand2, Eye, Wrench } from "lucide-react";
import type { AppNavigationItem } from "./types";
export const appNavigationItems: AppNavigationItem[] = [
@@ -6,5 +6,6 @@ export const appNavigationItems: AppNavigationItem[] = [
{ id: "triage", label: "Triage", icon: Layers3 },
{ id: "builds", label: "Builds", icon: Wand2 },
{ id: "overlay", label: "Overlay", icon: Eye },
{ id: "diagnose", label: "Diagnose", icon: Wrench },
];
+1 -1
View File
@@ -1,7 +1,7 @@
import { type ComponentType, type ReactNode } from "react";
import type { LucideProps } from "lucide-react";
export type NavigationId = "scan" | "triage" | "builds" | "overlay";
export type NavigationId = "scan" | "triage" | "builds" | "overlay" | "diagnose";
export interface AppNavigationItem {
id: NavigationId;
+22 -2
View File
@@ -1,9 +1,29 @@
import { ScanViewLayout } from "./components/ScanViewLayout";
import { ScanViewLayout } from "./components/ScanViewLayout";
import { DiagnosticsView } from "./components/DiagnosticsView";
import { useScanViewController } from "./hooks/useScanViewController";
import type { ScanViewProps } from "./types";
export function ScanView(props: ScanViewProps) {
interface ScanViewExtraProps {
/** "workspace" shows the clean scan surface; "diagnose" shows all dev info. */
mode?: "workspace" | "diagnose";
onDemoScan?: () => void;
canDemoScan?: boolean;
}
export function ScanView({ mode = "workspace", onDemoScan, canDemoScan, ...props }: ScanViewProps & ScanViewExtraProps) {
const controller = useScanViewController(props);
if (mode === "diagnose") {
return (
<DiagnosticsView
controller={controller}
latestCapture={props.latestCapture}
captureStatus={props.captureStatus}
onDemoScan={onDemoScan}
canDemoScan={canDemoScan}
/>
);
}
return <ScanViewLayout {...props} controller={controller} />;
}
@@ -0,0 +1,252 @@
import { useState } from "react";
import { AlertTriangle, Download, Play, Upload, Wrench } from "lucide-react";
import type { CaptureResult } from "../../../types/global";
import type { ScanViewControllerResult } from "../types";
import { FieldConfidenceList } from "./ScanResultCards";
import { useScanDiagnosticsModalModel } from "./modals/hooks/useScanDiagnosticsModalModel";
import { useScanDetailsModalModel } from "./modals/hooks/useScanDetailsModalModel";
interface DiagnosticsViewProps {
controller: ScanViewControllerResult;
latestCapture: CaptureResult | null;
captureStatus: string;
onDemoScan?: () => void;
canDemoScan?: boolean;
}
// All developer / diagnostic surfaces live here, separated from the Scan
// workspace: runtime + rights, grid detection, learning + data-package status,
// fingerprint, auto-scan counters, the automation log, and the raw crop/OCR/
// confidence dump. Read-only; it drives no scan action except the demo snapshot.
export function DiagnosticsView({ controller, latestCapture, captureStatus, onDemoScan, canDemoScan }: DiagnosticsViewProps) {
const {
statusTitle,
rightsClassName,
rightsValue,
genshinClassName,
genshinValue,
shouldShowAdminBanner,
gridSourceClass,
gridMainValue,
gridMetaValue,
learningRulesText,
learningRulesSubtext,
autoScanModeLabel,
fingerprintText,
runtimeRows,
scanLimitText,
autoScanStatsLines,
playerProgress,
reviewStatus,
automationLogLines,
canSaveReviewSample,
handleSaveReviewSample,
} = useScanDiagnosticsModalModel({
setDetailsOpen: controller.setDetailsOpen,
setDiagnosticsOpen: controller.setDiagnosticsOpen,
saveReviewSample: controller.saveReviewSample,
canSaveReviewSample: controller.canSaveReviewSample,
latestCapture,
controller,
captureStatus,
});
const { parsedNotes, showParsedNotes, cropRows, ocrRows, debugText, showCrops, showOcr } = useScanDetailsModalModel({
setDetailsOpen: controller.setDetailsOpen,
parsedArtifact: controller.parsedArtifact,
latestCapture,
});
const [interopStatus, setInteropStatus] = useState("");
const handleExportGood = async () => {
setInteropStatus("Exportiere GOOD...");
const result = await controller.exportGoodFromStore();
setInteropStatus(
result.ok
? `GOOD exportiert: ${result.count} Artifacts${result.path ? ` -> ${result.path}` : ""}`
: "GOOD-Export fehlgeschlagen (App im Electron-Fenster oeffnen).",
);
};
const handleImportGood = async () => {
setInteropStatus("Waehle GOOD-Datei...");
const result = await controller.importGoodFromFile();
if (result.canceled) {
setInteropStatus("GOOD-Import abgebrochen.");
return;
}
setInteropStatus(
result.ok
? `Importiert: ${result.added} neu, ${result.updated} aktualisiert (${result.count} gelesen).`
: result.error === "No valid GOOD artifacts found."
? "Keine gueltigen Artifacts in der Datei gefunden."
: "Import fehlgeschlagen (Datei ist kein gueltiges GOOD/JSON oder Bridge fehlt).",
);
};
return (
<section className="diagnose-view">
<div className="diagnose-header">
<div>
<p className="eyebrow">Diagnose &amp; Dev</p>
<h2>Laufzeit, Erkennung &amp; Rohdaten</h2>
</div>
<div className="diagnose-header-actions">
<button className="ghost-button" onClick={controller.toggleDevMode}>
<Wrench size={15} />
{statusTitle}
</button>
{onDemoScan && (
<button className="ghost-button" onClick={onDemoScan} disabled={!canDemoScan}>
<Play size={15} />
Demo-Daten laden
</button>
)}
</div>
</div>
<div className="diagnose-grid">
<div className="diagnose-card">
<p className="eyebrow">Status</p>
<div className="scanner-preflight diagnostics-preflight">
<div className={rightsClassName}>
<span>App-Rechte</span>
<strong>{rightsValue}</strong>
</div>
<div className={genshinClassName}>
<span>Genshin</span>
<strong>{genshinValue}</strong>
</div>
</div>
{shouldShowAdminBanner && (
<p className="scanner-subcopy">
App laeuft im Standard-Modus. Auto-Scan braucht Administrator-Rechte: App schliessen und als Administrator neu starten.
</p>
)}
<div className={`grid-detection-strip ${gridSourceClass}`}>
<span>Tile grid</span>
<strong>{gridMainValue}</strong>
<small>{gridMetaValue}</small>
</div>
<div className="learning-strip">
<span>Learning &amp; Daten</span>
<strong>{learningRulesText}</strong>
<small>{learningRulesSubtext}</small>
</div>
<div className="learning-strip">
<span>Fingerprint</span>
<strong>{fingerprintText}</strong>
<small>Aktiver Capture-Fingerprint fuer die Duplikat-Erkennung.</small>
</div>
</div>
<div className="diagnose-card">
<p className="eyebrow">{autoScanModeLabel}</p>
{playerProgress.show ? (
<div className="auto-scan-strip">
{autoScanStatsLines.map((entry) => (
<span className="auto-scan-stat" key={entry.label}>
<strong>{entry.value}</strong>
<small>{entry.label}</small>
</span>
))}
</div>
) : (
<p className="result-empty">Noch keine Scan-Aktivitaet.</p>
)}
<p className="scanner-result-caption">{scanLimitText}</p>
<div className="scanner-status-row diagnostics-status">
{runtimeRows.map((row) => (
<span key={row}>{row}</span>
))}
</div>
{reviewStatus && <p className="review-status">{reviewStatus}</p>}
</div>
</div>
<div className="diagnose-card">
<div className="diagnose-card-heading">
<p className="eyebrow">GOOD Interop</p>
<div className="diagnose-header-actions">
<button className="ghost-button" onClick={handleExportGood} disabled={!controller.canGoodInterop}>
<Download size={15} />
GOOD exportieren
</button>
<button className="ghost-button" onClick={handleImportGood} disabled={!controller.canGoodInterop}>
<Upload size={15} />
GOOD importieren
</button>
</div>
</div>
<p className="scanner-subcopy">
Exportiert den Scan-Store als GOOD (Genshin Optimizer / Inventory Kamera / Akasha) oder importiert eine GOOD-Datei in den Store.
</p>
{interopStatus && <p className="review-status">{interopStatus}</p>}
</div>
<div className="diagnose-card">
<p className="eyebrow">Automation log</p>
<div className="automation-log-lines">
{automationLogLines.length > 0 ? (
automationLogLines.map((line, index) => <span key={`${index}-${line}`}>{line}</span>)
) : (
<strong>Keine Scan-Aktivitaet.</strong>
)}
</div>
</div>
<div className="diagnose-card">
<div className="diagnose-card-heading">
<p className="eyebrow">Crops, OCR &amp; Confidence</p>
<button className="ghost-button" onClick={handleSaveReviewSample} disabled={!canSaveReviewSample}>
<AlertTriangle size={15} />
Review-Sample speichern
</button>
</div>
{controller.parsedArtifact ? (
<>
<FieldConfidenceList parsedArtifact={controller.parsedArtifact} />
{showParsedNotes && (
<div className="parsed-notes compact">
{parsedNotes.map((note) => (
<span key={note}>{note}</span>
))}
</div>
)}
</>
) : (
<p className="result-empty">Noch kein Artifact gelesen. Lies ein Artifact im Scan-Tab, um Crops und OCR zu sehen.</p>
)}
{showCrops && (
<div className="crop-grid details-grid">
{cropRows.map((crop) => (
<div className="crop-card" key={crop.id}>
<img src={crop.dataUrl} alt={crop.label} />
<div>
<strong>{crop.label}</strong>
<span>{crop.x},{crop.y} - {crop.width}x{crop.height}</span>
</div>
</div>
))}
</div>
)}
{showOcr && (
<div className="ocr-panel">
<strong>OCR candidates</strong>
{ocrRows.map((entry) => (
<div className="ocr-row" key={entry.id}>
<div>
<span>{entry.label}</span>
<small>{entry.confidence}% confidence</small>
</div>
<pre>{entry.text}</pre>
</div>
))}
</div>
)}
{latestCapture && <p className="capture-debug">{debugText}</p>}
</div>
</section>
);
}
@@ -1,4 +1,4 @@
import { AlertTriangle, Camera, Eye } from "lucide-react";
import { AlertTriangle, Camera } from "lucide-react";
import { ArtifactResultCard } from "./ScanResultCards";
import { useScanMainSectionModel } from "./hooks/useScanMainSectionModel";
import type { ScanMainSectionProps } from "./types";
@@ -63,9 +63,7 @@ export function ScanMainSection({
</div>
<div className="capture-stage-meta">
<div><span>Quelle</span><strong>{sourceLabel}</strong></div>
<div><span>Grid</span><strong>{gridLabel}</strong></div>
<div><span>Inventar</span><strong>{inventoryLabel}</strong></div>
<div><span>Modus</span><strong>{captureModeText}</strong></div>
</div>
</div>
@@ -81,17 +79,8 @@ export function ScanMainSection({
<span>{targetLabel}</span>
<span>{dbLabel}</span>
<span>{reviewLabel}</span>
<span>{rulesLabel}</span>
</div>
<div className="scanner-result-actions">
<button
className="ghost-button"
onClick={handleOpenDetails}
disabled={!canOpenDetails}
>
<Eye size={15} />
Details
</button>
<button className="ghost-button" onClick={handleOpenReviewQueue} disabled={autoScanRunning || !canOpenReviewQueue}>
<AlertTriangle size={15} />
Review Queue
@@ -1,46 +1,27 @@
import { ScanDiagnosticsModal } from "./modals/ScanDiagnosticsModal";
import { ScanSettingsModal } from "./modals/ScanSettingsModal";
import { ScanDetailsModal } from "./modals/ScanDetailsModal";
import { ScanReviewQueueModal } from "./modals/ScanReviewQueueModal";
import { ScanSummaryModal } from "./modals/ScanSummaryModal";
import type { ScanModalsSectionProps } from "./types";
// Diagnostics + crop/OCR details moved to the dedicated Diagnose view; only the
// core workspace modals live here.
export function ScanModalsSection({
captureStatus,
latestCapture,
diagnosticsOpen,
settingsOpen,
detailsOpen,
reviewQueueOpen,
controller,
setDiagnosticsOpen,
setSettingsOpen,
setDetailsOpen,
setReviewQueueOpen,
setScanSummary,
}: ScanModalsSectionProps) {
return (
<>
<ScanDiagnosticsModal
open={diagnosticsOpen}
captureStatus={captureStatus}
latestCapture={latestCapture}
controller={controller}
setDetailsOpen={setDetailsOpen}
setDiagnosticsOpen={setDiagnosticsOpen}
/>
<ScanSettingsModal
open={settingsOpen}
latestCapture={latestCapture}
controller={controller}
setSettingsOpen={setSettingsOpen}
/>
<ScanDetailsModal
open={detailsOpen}
latestCapture={latestCapture}
controller={controller}
setDetailsOpen={setDetailsOpen}
/>
<ScanReviewQueueModal
open={reviewQueueOpen}
controller={controller}
@@ -5,7 +5,6 @@ import {
Radar,
RefreshCw,
SlidersHorizontal,
Wrench,
} from "lucide-react";
import type { ScanTopControlsSectionProps } from "./types";
import { useScanTopControlsModel } from "./hooks/useScanTopControlsModel";
@@ -66,7 +65,7 @@ export function ScanTopControlsSection({
<div>
<p className="eyebrow">Scanner</p>
<h2>Artifact capture workspace</h2>
<p className="scanner-subcopy">Grosse Vorschau vorn, klare Aktionen oben, Diagnose und Review nur bei Bedarf.</p>
<p className="scanner-subcopy">Quelle waehlen, Auto-Scan starten, Ergebnis rechts pruefen. Dev-Details im Diagnose-Tab.</p>
</div>
<div className="scanner-header-pills">
<span className={`runtime-pill ${bridgePillClass}`}>{bridgeStatusText}</span>
@@ -118,15 +117,6 @@ export function ScanTopControlsSection({
<SlidersHorizontal size={15} />
Scan-Setup
</button>
<button
className="ghost-button dev-toggle"
onClick={openDiagnostics}
disabled={!bridgeReady}
title={diagnosticsButtonTitle}
>
<Wrench size={15} />
Scanner Diagnose
</button>
</div>
<div className="player-scan-actions">
@@ -60,19 +60,20 @@ export function useScanResultCardModel({
parsed,
}: Pick<ArtifactResultCardProps, "parsed">): ScanResultCardModel {
const substats = parsed.substats;
const rows: Array<[string, ParsedField]> = [
["Name", parsed.fields.name],
["Slot", parsed.fields.slot],
["Level", getLevelField(parsed)],
["Main", mergeField(parsed.fields.mainStat, parsed.fields.mainValue)],
["Set", parsed.fields.setName],
["Equipped", parsed.fields.equipped],
["Substats", parsed.fields.substats],
];
return {
levelField: getLevelField(parsed),
quality: resolveQuality(parsed.confidence),
fieldRows: [
["Name", parsed.fields.name],
["Slot", parsed.fields.slot],
["Level", getLevelField(parsed)],
["Main", mergeField(parsed.fields.mainStat, parsed.fields.mainValue)],
["Set", parsed.fields.setName],
["Equipped", parsed.fields.equipped],
["Substats", parsed.fields.substats],
].map(([label, field]) => ({
fieldRows: rows.map(([label, field]) => ({
label,
field,
confidenceClassName: resolveFieldConfidenceClass(field),
@@ -8,6 +8,7 @@ export interface ScanTopControlsModel {
canStartAutoScan: boolean;
canStartManualScan: boolean;
canCaptureSingle: boolean;
autoScanRunning: boolean;
handleSourceChange: (event: ChangeEvent<HTMLSelectElement>) => void;
selectGenshinSource: () => void;
openSettings: () => void;
@@ -41,7 +42,7 @@ export function useScanTopControlsModel({
bridgeReady,
isScanning,
controller,
}: ScanTopControlsSectionProps): ScanTopControlsModel {
}: Omit<ScanTopControlsSectionProps, "refreshCaptureSources">): ScanTopControlsModel {
const {
setSettingsOpen,
setDiagnosticsOpen,
@@ -116,6 +117,7 @@ export function useScanTopControlsModel({
canStartAutoScan: hasSourceSelected && !isScanning && !autoScanRunning && canAutoScan && !requiresAdminForAutoScan,
canStartManualScan: hasSourceSelected && !isScanning && !autoScanRunning && canCaptureSource,
canCaptureSingle: hasSourceSelected && !isScanning && !autoScanRunning && canCaptureSource,
autoScanRunning,
handleSourceChange,
selectGenshinSource,
openSettings,
@@ -1,83 +0,0 @@
import { FieldConfidenceList } from "../ScanResultCards";
import type { ScanDetailsModalProps } from "./types";
import { useScanDetailsModalModel } from "./hooks/useScanDetailsModalModel";
export function ScanDetailsModal({
open,
latestCapture,
controller,
setDetailsOpen,
}: ScanDetailsModalProps) {
const { parsedArtifact } = controller;
const {
closeDetails,
stopPropagation,
parsedNotes,
showParsedNotes,
cropRows,
ocrRows,
debugText,
showCrops,
showOcr,
} = useScanDetailsModalModel({
setDetailsOpen,
parsedArtifact,
latestCapture,
});
if (!open || !latestCapture) return null;
return (
<div className="modal-backdrop" role="presentation" onClick={closeDetails}>
<div className="modal-panel" role="dialog" aria-modal="true" onClick={stopPropagation}>
<div className="modal-header">
<div>
<p className="eyebrow">Dev</p>
<h2>Crops, OCR & Confidence</h2>
</div>
<button className="ghost-button" onClick={closeDetails}>Schliessen</button>
</div>
<div className="modal-body">
{parsedArtifact && (
<>
<FieldConfidenceList parsedArtifact={parsedArtifact} />
{showParsedNotes && (
<div className="parsed-notes compact">
{parsedNotes.map((note) => <span key={note}>{note}</span>)}
</div>
)}
</>
)}
{showCrops && (
<div className="crop-grid details-grid">
{cropRows.map((crop) => (
<div className="crop-card" key={crop.id}>
<img src={crop.dataUrl} alt={crop.label} />
<div>
<strong>{crop.label}</strong>
<span>{crop.x},{crop.y} - {crop.width}x{crop.height}</span>
</div>
</div>
))}
</div>
)}
{showOcr && (
<div className="ocr-panel">
<strong>OCR candidates</strong>
{ocrRows.map((entry) => (
<div className="ocr-row" key={entry.id}>
<div>
<span>{entry.label}</span>
<small>{entry.confidence}% confidence</small>
</div>
<pre>{entry.text}</pre>
</div>
))}
</div>
)}
<p className="capture-debug">{debugText}</p>
</div>
</div>
</div>
);
}
@@ -1,172 +0,0 @@
import { AlertTriangle, Eye, Wrench } from "lucide-react";
import type { ScanDiagnosticsModalProps } from "./types";
import { useScanDiagnosticsModalModel } from "./hooks/useScanDiagnosticsModalModel";
export function ScanDiagnosticsModal({
open,
captureStatus,
latestCapture,
controller,
setDetailsOpen,
setDiagnosticsOpen,
}: ScanDiagnosticsModalProps) {
const {
toggleDevMode,
} = controller;
const {
closeDiagnostics,
openDetails,
handleSaveReviewSample,
stopPropagation,
canOpenDetails,
statusTitle,
rightsClassName,
rightsValue,
genshinClassName,
genshinValue,
shouldShowAdminBanner,
gridSourceClass,
gridMainValue,
gridMetaValue,
learningRulesText,
learningRulesSubtext,
autoScanModeLabel,
autoScanRunning,
fingerprintText,
runtimeRows,
scanLimitText,
scanTipText,
autoScanStatsLines,
playerProgress,
showDevRows,
reviewStatus,
automationLogLines,
canSaveReviewSample,
} = useScanDiagnosticsModalModel({
setDetailsOpen,
setDiagnosticsOpen,
saveReviewSample: controller.saveReviewSample,
canSaveReviewSample: controller.canSaveReviewSample,
latestCapture,
controller,
captureStatus,
});
if (!open) return null;
return (
<div className="modal-backdrop" role="presentation" onClick={closeDiagnostics}>
<div className="modal-panel scanner-diagnostics-modal" role="dialog" aria-modal="true" onClick={stopPropagation}>
<div className="modal-header">
<div>
<p className="eyebrow">Scanner Diagnose</p>
<h2>Input, Grid & Lernstatus</h2>
</div>
<button className="ghost-button" onClick={closeDiagnostics}>Schliessen</button>
</div>
<div className="modal-body">
<div className="diagnostics-actions">
<button className="ghost-button" onClick={toggleDevMode}>
<Wrench size={15} />
{statusTitle}
</button>
</div>
<div className="scanner-preflight diagnostics-preflight">
<div className={rightsClassName}>
<span>App-Rechte</span>
<strong>{rightsValue}</strong>
</div>
<div className={genshinClassName}>
<span>Genshin</span>
<strong>{genshinValue}</strong>
</div>
</div>
{shouldShowAdminBanner && (
<p className="scanner-subcopy">
App laeuft im Standard-Modus. Auto-Scan braucht Administrator-Rechte: App schliessen und als Administrator neu starten (z.B. Terminal per Rechtsklick "Als Administrator ausfuehren" und darin "npm run dev").
</p>
)}
<div className={`grid-detection-strip ${gridSourceClass}`}>
<span>Tile grid</span>
<strong>{gridMainValue}</strong>
<small>{gridMetaValue}</small>
</div>
<div className="learning-strip">
<span>Learning</span>
<strong>{learningRulesText}</strong>
<small>{learningRulesSubtext}</small>
</div>
{playerProgress.show && (
<div className="auto-scan-strip">
<span>{autoScanModeLabel}</span>
{autoScanStatsLines.map((entry) => (
<span className="auto-scan-stat" key={entry.label}>
<strong>{entry.value}</strong>
<small>{entry.label}</small>
</span>
))}
</div>
)}
<div className="learning-strip">
<span>Fingerprint</span>
<strong>{fingerprintText}</strong>
<small>Active capture fingerprint used for deterministic duplicate guard checks.</small>
</div>
{showDevRows && (
<div className="scanner-status-row diagnostics-status">
{runtimeRows.map((row) => (
<span key={row}>{row}</span>
))}
</div>
)}
{autoScanRunning && playerProgress.show && (
<div className="player-progress">
<div className="player-progress-bar">
<div style={{ width: `${playerProgress.width}%` }} />
</div>
</div>
)}
{reviewStatus && (
<p className="review-status">{reviewStatus}</p>
)}
<div className="automation-log">
<span>Automation</span>
<div className="automation-log-lines">
{automationLogLines.length > 0 ? (
automationLogLines.map((line, index) => (
<span key={`${index}-${line}`}>{line}</span>
))
) : (
<strong>No scan activity yet.</strong>
)}
</div>
</div>
<div className="dev-section-actions">
<button className="ghost-button" onClick={openDetails} disabled={!canOpenDetails}>
<Eye size={15} />
Crops, OCR & Confidence
</button>
<button className="ghost-button" onClick={handleSaveReviewSample} disabled={!canSaveReviewSample}>
<AlertTriangle size={15} />
Review-Sample speichern
</button>
</div>
<p className="scanner-result-caption">{scanLimitText}</p>
<p className="scan-summary-copy">{scanTipText}</p>
</div>
</div>
</div>
);
}
@@ -1,6 +1,6 @@
import { useCallback, type MouseEvent } from "react";
import type { ScanDetailsModalProps } from "../types";
import type { ParsedArtifactCandidate } from "../../../../lib/artifactOcrParser";
import type { ParsedArtifactCandidate } from "../../../../../lib/artifactOcrParser";
export interface ScanDetailsModalModel {
closeDetails: () => void;
@@ -1,5 +1,6 @@
import { detailFingerprint } from "../../../../lib/autoScanLoop";
import { sourceVersion } from "../../../../lib/genshinData";
import { detailFingerprint } from "../../../../../lib/autoScanLoop";
import { dataGeneratedAt, sourceVersion } from "../../../../../lib/genshinData";
import { dataPackageStatus } from "../../../../../lib/dataPackageStatus";
import { useCallback, useMemo, type MouseEvent } from "react";
import type { ScanDiagnosticsModalProps } from "../types";
@@ -39,16 +40,20 @@ export interface ScanDiagnosticsModalModel {
canSaveReviewSample: boolean;
}
type ScanDiagnosticsController = ScanDiagnosticsModalProps["controller"];
interface UseScanDiagnosticsModalModelInput extends Pick<
ScanDiagnosticsModalProps,
| "setDetailsOpen"
| "setDiagnosticsOpen"
| "saveReviewSample"
| "canSaveReviewSample"
| "latestCapture"
| "controller"
> {
captureStatus: string;
// saveReviewSample / canSaveReviewSample live on the controller, not the modal
// props; the component wires them through from controller.* .
saveReviewSample: ScanDiagnosticsController["saveReviewSample"];
canSaveReviewSample: ScanDiagnosticsController["canSaveReviewSample"];
}
export function useScanDiagnosticsModalModel({
@@ -92,7 +97,8 @@ export function useScanDiagnosticsModalModel({
: "Run a capture once while the artifact inventory is visible.";
const learningRulesText = controller.learningRulesLoaded ? `${controller.learningRuleCount} local rules` : "loading";
const learningRulesSubtext = `Review samples feed deterministic OCR fixes before each new parse. Data package: ${sourceVersion}`;
const dataStaleness = dataPackageStatus(dataGeneratedAt, sourceVersion);
const learningRulesSubtext = `Review samples feed deterministic OCR fixes before each new parse. Data package: ${sourceVersion}${dataStaleness.warning ? ` - ${dataStaleness.warning}` : ""}`;
const playerProgress = useMemo(() => {
const width = Math.min(
@@ -1,5 +1,5 @@
import { useCallback, useMemo, type MouseEvent } from "react";
import type { ReviewSampleAnalysis } from "../../../../lib/reviewSampleAnalysis";
import type { ReviewSampleAnalysis } from "../../../../../lib/reviewSampleAnalysis";
import type { ScanReviewQueueModalProps } from "../types";
export interface ScanReviewQueueRow {
@@ -18,8 +18,10 @@ export interface ScanReviewQueueModalModel {
interface UseScanReviewQueueModalModelInput extends Pick<
ScanReviewQueueModalProps,
"setReviewQueueOpen" | "loadReviewQueue"
"setReviewQueueOpen"
> {
// loadReviewQueue is wired through from controller.* by the component.
loadReviewQueue: ScanReviewQueueModalProps["controller"]["loadReviewQueue"];
reviewAnalysis: ReviewSampleAnalysis;
reviewSampleTotal: number;
}
@@ -1,6 +1,6 @@
import { useCallback, type ChangeEvent, type MouseEvent } from "react";
import type { CaptureResult, RuntimeInfo } from "../../../../../types/global";
import { clampScanLimit, clampSkipRows } from "../../../../lib/scannerSession";
import { clampScanLimit, clampSkipRows } from "../../../../../lib/scannerSession";
export interface ScanSettingsModalModel {
closeSettings: () => void;
@@ -244,7 +244,7 @@ export async function persistParsedArtifact(
return false;
}
try {
const result = await artifactRepo.saveMany([toStoredArtifact(parsed, source, needsReview)]);
const result = await artifactRepo.saveMany([toStoredArtifact(parsed, source, needsReview, capture?.locked)]);
if (result?.ok) {
setStoredTotal(result.total);
void onStoredArtifactsChanged?.();
@@ -297,6 +297,7 @@ export async function saveReviewSample(
})),
inventoryGrid: capture.inventoryGrid,
inventoryCount: capture.inventoryCount,
locked: capture.locked,
ocr: capture.ocr,
},
parsed,
+34 -9
View File
@@ -4,7 +4,16 @@ import { runAutoScanLoop } from "../../../lib/autoScanLoop";
import { clampScanLimit, emptyAutoScanStats, resolveScanTargetCount, type AutoScanStats, type ScanSummary } from "../../../lib/scannerSession";
import { getAutoReviewReason, wait } from "../../../lib/scanReviewUtils";
import type { AutomationRepositoryPort, RuntimeRepositoryPort } from "../../../infrastructure/repositories/rendererBridgeRepositories";
import type { AutomationGuard, BooleanResult, CaptureOptions, CaptureResult, ClickResult, RuntimeInfo, ScrollResult } from "../../../types/global";
import type {
AutomationGuard,
BooleanResult,
CaptureOptions,
CaptureResult,
ClickResult,
FocusGenshinResult,
RuntimeInfo,
ScrollResult,
} from "../../../types/global";
import type { ParsedArtifactCandidate } from "../../../lib/artifactOcrParser";
import type { MutableRefObject } from "react";
import type { Dispatch, SetStateAction } from "react";
@@ -36,6 +45,10 @@ export interface ScanActionContext {
focusDashboard: () => Promise<void>;
}
export interface VisibleGridScanOptions {
scanLimit?: number;
}
function buildScanSignature(parsed: ParsedArtifactCandidate) {
return `${parsed.name}-${parsed.slot}-${parsed.mainStat}-${parsed.mainValue}-${parsed.level ?? 0}`;
}
@@ -137,7 +150,7 @@ export async function runAutoReviewScan(context: ScanActionContext): Promise<voi
appendAutomationLog(`manual scan finished: ${stats.parsed} parsed, ${stats.stored} stored, ${stats.review} review`);
}
export async function runVisibleGridScan(context: ScanActionContext): Promise<void> {
export async function runVisibleGridScan(context: ScanActionContext, options: VisibleGridScanOptions = {}): Promise<void> {
const {
autoScanRunning,
bridgeReady,
@@ -158,13 +171,14 @@ export async function runVisibleGridScan(context: ScanActionContext): Promise<vo
persistParsedArtifact,
saveReviewSample,
shouldFlagArtifactForReview,
scanLimit,
scanLimit: configuredScanLimit,
skipRows,
detectedInventoryCount,
focusDashboard,
} = context;
const scanLimit = typeof options.scanLimit === "number" ? clampScanLimit(options.scanLimit) : configuredScanLimit;
if (autoScanRunning || !bridgeReady || !selectedSourceId || !automationRepo?.clickScreen || !automationRepo?.scrollScreen) return;
if (autoScanRunning || !bridgeReady || !selectedSourceId || !automationRepo?.focusGenshinForScanStart || !automationRepo?.focusGenshin || !automationRepo?.clickScreen || !automationRepo?.scrollScreen) return;
const requiresAdminForAutoScan = requiresAdminForAutomation(runtimeInfo);
if (requiresAdminForAutoScan) {
@@ -208,13 +222,24 @@ export async function runVisibleGridScan(context: ScanActionContext): Promise<vo
appendAutomationLog(`runtime ping: elevated=${freshRuntime.isElevated} ${required}`);
}
const focusGenshinForScanStart = automationRepo.focusGenshinForScanStart ?? automationRepo.focusGenshin;
setReviewStatus("Genshin wird in den Vordergrund geholt...");
const focusResult = await automationRepo?.focusGenshin().catch(() => null);
if (focusResult) {
appendAutomationLog(
`focus: ${focusResult.focused ? "ok" : "fehlgeschlagen"} found:${focusResult.genshinFound ? "yes" : "no"} setForeground:${focusResult.setForegroundResult ?? "n/a"} target:${focusResult.targetProcess || "?"} fg:${focusResult.foregroundProcess || "?"}`,
);
let focusResult: FocusGenshinResult | null = null;
for (let attempt = 1; attempt <= 3; attempt += 1) {
const current = await focusGenshinForScanStart().catch(() => null);
if (!current) {
appendAutomationLog(`focus attempt ${attempt}/3: exception`);
} else {
focusResult = current;
appendAutomationLog(
`focus attempt ${attempt}/3: ${current.focused ? "ok" : "failed"} found:${current.genshinFound ? "yes" : "no"} setForeground:${current.setForegroundResult ?? "n/a"} target:${current.targetProcess || "?"} fg:${current.foregroundProcess || "?"}`,
);
if (current.focused) break;
if (!current.genshinFound) break;
}
if (attempt < 3) await wait(300);
}
if (!focusResult?.focused) {
setAutoScanRunning(false);
const reason = !focusResult?.genshinFound
@@ -1,5 +1,7 @@
import { useEffect } from "react";
import type { AutomationRepositoryPort } from "../../../infrastructure/repositories/rendererBridgeRepositoryTypes";
import type { ScannerCommand } from "../../../types/global";
import type { VisibleGridScanOptions } from "./scanViewScanActions";
interface ScanCommandListenerInput {
automationRepo?: AutomationRepositoryPort;
@@ -7,7 +9,7 @@ interface ScanCommandListenerInput {
isScanning: boolean;
selectedSourceId: string;
requestScanStop: (reason: string) => void;
runVisibleGridScan: () => Promise<void>;
runVisibleGridScan: (options?: VisibleGridScanOptions) => Promise<void>;
}
export function useScanCommandListener({
@@ -20,15 +22,16 @@ export function useScanCommandListener({
}: ScanCommandListenerInput) {
useEffect(() => {
if (!automationRepo?.onCommand) return;
return automationRepo.onCommand((command: "start-auto" | "stop") => {
return automationRepo.onCommand((command: ScannerCommand) => {
if (command === "stop") {
requestScanStop("Hotkey/Dev-Stop gedrueckt.");
return;
}
if (command === "start-auto" && !autoScanRunning && !isScanning && selectedSourceId) {
void runVisibleGridScan();
const commandType = typeof command === "string" ? command : command.type;
if (commandType === "start-auto" && !autoScanRunning && !isScanning && selectedSourceId) {
const options = typeof command === "string" ? undefined : { scanLimit: command.scanLimit };
void runVisibleGridScan(options);
}
});
}, [automationRepo, autoScanRunning, isScanning, selectedSourceId, requestScanStop, runVisibleGridScan]);
}
@@ -1,6 +1,6 @@
import { useCallback, useEffect, useMemo } from "react";
import type { Dispatch, MutableRefObject, SetStateAction } from "react";
import { runAutoReviewScan as runAutoReviewScanAction, runVisibleGridScan as runVisibleGridScanAction } from "./scanViewScanActions";
import { runAutoReviewScan as runAutoReviewScanAction, runVisibleGridScan as runVisibleGridScanAction, type VisibleGridScanOptions } from "./scanViewScanActions";
import {
initializeLearningState,
loadReviewQueue as loadReviewQueueFromRepo,
@@ -77,7 +77,7 @@ export interface ScanViewActionResult {
loadReviewQueue: () => Promise<void>;
openReviewQueue: () => Promise<void>;
runAutoReviewScan: () => Promise<void>;
runVisibleGridScan: () => Promise<void>;
runVisibleGridScan: (options?: VisibleGridScanOptions) => Promise<void>;
}
export function useScanViewActions(input: ScanViewActionInput): ScanViewActionResult {
@@ -262,11 +262,11 @@ export function useScanViewActions(input: ScanViewActionInput): ScanViewActionRe
await runAutoReviewScanAction(scanActionContext);
}, [autoScanRunning, canCaptureSource, selectedSourceId, scanActionContext]);
const runVisibleGridScan = useCallback(async () => {
const runVisibleGridScan = useCallback(async (options: VisibleGridScanOptions = {}) => {
if (autoScanRunning || !bridgeReady || !selectedSourceId || !automationRepo?.clickScreen || !automationRepo?.scrollScreen) {
return;
}
await runVisibleGridScanAction(scanActionContext);
await runVisibleGridScanAction(scanActionContext, options);
}, [
autoScanRunning,
bridgeReady,
@@ -16,6 +16,8 @@ import { useScanViewStateSync } from "./useScanViewStateSync";
import { emptyAutoScanStats, resolveScanTargetCount, type AutoScanStats, type ScanSummary } from "../../../lib/scannerSession";
import type { ScanViewProps, ScanViewControllerResult } from "../types";
import type { CaptureResult, ClickResult, ReviewSampleRecord } from "../../../types/global";
import type { StoredArtifactRecord } from "../../../types/storage";
import { goodDatabaseToStoredArtifacts, type GoodImportDatabase, storedArtifactsToGood } from "../../../lib/goodInterop";
import { createRendererRepositories } from "../../../infrastructure/repositories/rendererBridgeRepositories";
export function useScanViewController({
@@ -37,6 +39,7 @@ export function useScanViewController({
const snapshotRepo = repositories?.snapshot;
const automationRepo = repositories?.automation;
const captureRepo = repositories?.capture;
const exportRepo = repositories?.export;
const [detailsOpen, setDetailsOpen] = useState(false);
const [diagnosticsOpen, setDiagnosticsOpen] = useState(false);
@@ -152,6 +155,42 @@ export function useScanViewController({
setReviewQueueOpen,
});
const canGoodInterop = bridgeReady && Boolean(artifactRepo?.loadAll) && Boolean(artifactRepo?.saveMany);
const exportGoodFromStore = useCallback(async () => {
if (!artifactRepo?.loadAll || !exportRepo?.exportGood) return { ok: false, count: 0 };
const loaded = await artifactRepo.loadAll();
const records = loaded.artifacts ?? [];
const good = storedArtifactsToGood(records);
const result = await exportRepo.exportGood(good);
return { ok: Boolean(result.ok), path: result.path, count: good.artifacts.length };
}, [artifactRepo, exportRepo]);
const importGoodArtifacts = useCallback(async (records: StoredArtifactRecord[]) => {
if (!artifactRepo?.saveMany || records.length === 0) return { ok: false, added: 0, updated: 0 };
const result = await artifactRepo.saveMany(records);
if (typeof result.total === "number") setStoredTotal(result.total);
await onStoredArtifactsChanged?.();
return { ok: Boolean(result.ok), added: result.added ?? 0, updated: result.updated ?? 0 };
}, [artifactRepo, onStoredArtifactsChanged]);
const importGoodFromFile = useCallback(async () => {
if (!exportRepo?.importGoodFile || !artifactRepo?.saveMany) {
return { ok: false, added: 0, updated: 0, count: 0, error: "GOOD import is unavailable." };
}
const fileResult = await exportRepo.importGoodFile();
if (fileResult.canceled) return { ok: false, added: 0, updated: 0, count: 0, canceled: true };
if (!fileResult.ok) {
return { ok: false, added: 0, updated: 0, count: 0, path: fileResult.path, error: fileResult.error };
}
const records = goodDatabaseToStoredArtifacts(fileResult.database as GoodImportDatabase);
if (records.length === 0) {
return { ok: false, added: 0, updated: 0, count: 0, path: fileResult.path, error: "No valid GOOD artifacts found." };
}
const saved = await importGoodArtifacts(records);
return { ...saved, count: records.length, path: fileResult.path };
}, [artifactRepo, exportRepo, importGoodArtifacts]);
useScanViewStateSync({
artifactRepo,
latestCapture,
@@ -229,5 +268,9 @@ export function useScanViewController({
openReviewQueue: openReviewQueueModal,
runAutoReviewScan,
runVisibleGridScan,
canGoodInterop,
exportGoodFromStore,
importGoodFromFile,
importGoodArtifacts,
};
}
+5
View File
@@ -1,5 +1,6 @@
import type { AppSnapshot } from "../../types/domain";
import type { BooleanResult, CaptureOptions, CaptureResult, CaptureSourceInfo, RuntimeInfo, ReviewSampleRecord } from "../../types/global";
import type { StoredArtifactRecord } from "../../types/storage";
import type { AutoScanStats, ScanSummary } from "../../lib/scannerSession";
import type { ParsedArtifactCandidate } from "../../lib/artifactOcrParser";
import type { ScannerLearningRules } from "../../lib/scannerLearning";
@@ -79,4 +80,8 @@ export interface ScanViewControllerResult {
openReviewQueue: () => Promise<void>;
runAutoReviewScan: () => Promise<void>;
runVisibleGridScan: () => Promise<void>;
canGoodInterop: boolean;
exportGoodFromStore: () => Promise<{ ok: boolean; path?: string; count: number }>;
importGoodFromFile: () => Promise<{ ok: boolean; added: number; updated: number; count: number; canceled?: boolean; path?: string; error?: string }>;
importGoodArtifacts: (records: StoredArtifactRecord[]) => Promise<{ ok: boolean; added: number; updated: number }>;
}
@@ -26,6 +26,7 @@ import type {
ClickResult,
ReviewSampleListResult,
SaveScannerLearningRulesResult,
GoodImportFileResult,
} from "../../types/global";
const EMPTY_SNAPSHOT: AppSnapshot | null = null;
@@ -67,6 +68,12 @@ const EMPTY_SAVE_RULES_RESULT: SaveScannerLearningRulesResult = {
rules: {},
total: 0,
};
const EMPTY_GOOD_IMPORT_FILE_RESULT: GoodImportFileResult = {
ok: false,
canceled: false,
path: "",
error: "Electron bridge unavailable.",
};
async function createBridgeSafeCall<TResult>(
callback: () => Promise<TResult> | TResult | null | undefined,
@@ -179,6 +186,14 @@ export function createRendererRepositories(): RendererRepositories | null {
() => bridge.getAutomationGuard(),
emptyAutomationGuard(),
),
focusGenshinForScanStart: () =>
createBridgeSafeCall(
() =>
typeof bridge.focusGenshinForScanStart === "function"
? bridge.focusGenshinForScanStart()
: bridge.focusGenshin(),
emptyFocusGenshinResult(),
),
focusGenshin: () =>
createBridgeSafeCall(
() => bridge.focusGenshin(),
@@ -197,6 +212,7 @@ export function createRendererRepositories(): RendererRepositories | null {
const exportRepo: ScanExportPort = {
exportGood: (payload) => createBridgeSafeCall(() => bridge.exportGood(payload), EMPTY_SAVE_RESULT),
importGoodFile: () => createBridgeSafeCall(() => bridge.importGoodFile(), EMPTY_GOOD_IMPORT_FILE_RESULT),
};
return {
@@ -8,8 +8,10 @@ import type {
ScannerStatusPayload,
ReviewSamplePayload,
GoodDatabase,
GoodImportFileResult,
FocusGenshinResult,
RuntimeInfo,
ScannerCommand,
LoadScannerLearningRulesResult,
SaveScannerLearningRulesResult,
ArtifactStoreLoadResult,
@@ -58,10 +60,11 @@ export interface SnapshotRepositoryPort {
export interface AutomationRepositoryPort {
getAutomationGuard(): Promise<AutomationGuard>;
focusGenshin(): Promise<FocusGenshinResult>;
focusGenshinForScanStart: () => Promise<FocusGenshinResult>;
focusMainWindow(): Promise<BooleanResult>;
clickScreen(x: number, y: number): Promise<ClickResult>;
scrollScreen(notches: number, anchorX?: number, anchorY?: number): Promise<ScrollResult>;
onCommand(callback: (command: "start-auto" | "stop") => void): () => void;
onCommand(callback: (command: ScannerCommand) => void): () => void;
}
export interface OverlayRepositoryPort {
@@ -70,6 +73,7 @@ export interface OverlayRepositoryPort {
export interface ScanExportPort {
exportGood(payload: GoodDatabase): Promise<SaveResultWithPath>;
importGoodFile(): Promise<GoodImportFileResult>;
}
export interface RendererRepositories {
+74
View File
@@ -0,0 +1,74 @@
import { describe, expect, it } from "vitest";
import type { StoredArtifactRecord } from "../types/storage";
import { mergeIdentity, mergeRescannedArtifacts, substatName } from "./artifactMerge";
function record(overrides: Partial<StoredArtifactRecord> = {}): StoredArtifactRecord {
return {
id: "x",
name: "Gladiator's Nostalgia",
slot: "Flower of Life",
level: 0,
setName: "Gladiator's Finale",
mainStat: "HP",
mainValue: "717",
substats: ["CRIT DMG+5.4%", "ATK+19", "HP%+4.1%", "Energy Recharge+5.2%"],
equipped: "Not detected",
confidence: 80,
needsReview: false,
source: "auto-scan",
...overrides,
};
}
describe("artifactMerge", () => {
it("strips values to get the substat name", () => {
expect(substatName("CRIT DMG+13.2%")).toBe("CRIT DMG");
expect(substatName("ATK+19")).toBe("ATK");
});
it("identity ignores level and substat values", () => {
const low = record({ level: 0, substats: ["CRIT DMG+5.4%", "ATK+19"] });
const high = record({ level: 20, substats: ["CRIT DMG+13.2%", "ATK+37"] });
expect(mergeIdentity(low)).toBe(mergeIdentity(high));
});
it("collapses a leveled re-scan into one record, keeping the higher level", () => {
const low = record({ id: "a", level: 0, timesSeen: 1 });
const high = record({
id: "b",
level: 20,
timesSeen: 1,
substats: ["CRIT DMG+13.2%", "ATK+37", "HP%+15.7%", "Energy Recharge+11.7%"],
equipped: "Bennett",
});
const { merged, collapsed } = mergeRescannedArtifacts([low, high]);
expect(collapsed).toBe(1);
expect(merged).toHaveLength(1);
expect(merged[0].level).toBe(20);
expect(merged[0].equipped).toBe("Bennett");
expect(merged[0].timesSeen).toBe(2);
});
it("does not merge pieces with different substat lineups", () => {
const threeLine = record({ id: "a", substats: ["CRIT DMG+5.4%", "ATK+19", "HP%+4.1%"] });
const fourLine = record({ id: "b", substats: ["CRIT DMG+5.4%", "ATK+19", "HP%+4.1%", "DEF+16"] });
const { merged, collapsed } = mergeRescannedArtifacts([threeLine, fourLine]);
expect(collapsed).toBe(0);
expect(merged).toHaveLength(2);
});
it("keeps distinct sets/slots/mains apart", () => {
const flower = record({ slot: "Flower of Life", mainStat: "HP" });
const plume = record({ slot: "Plume of Death", mainStat: "ATK" });
const { merged } = mergeRescannedArtifacts([flower, plume]);
expect(merged).toHaveLength(2);
});
it("preserves the earliest firstSeenAt and latest lastSeenAt", () => {
const older = record({ id: "a", level: 0, firstSeenAt: "2026-01-01", lastSeenAt: "2026-01-02" });
const newer = record({ id: "b", level: 20, firstSeenAt: "2026-06-01", lastSeenAt: "2026-06-10" });
const { merged } = mergeRescannedArtifacts([older, newer]);
expect(merged[0].firstSeenAt).toBe("2026-01-01");
expect(merged[0].lastSeenAt).toBe("2026-06-10");
});
});
+84
View File
@@ -0,0 +1,84 @@
import type { StoredArtifactRecord } from "../types/storage";
import { storedArtifactStrength } from "./artifactStore";
// Rescan-merge (ADR-006 open follow-up): leveling an artifact changes its store
// signature (level + substat values), so re-scanning a leveled piece creates a
// duplicate record. This collapses those duplicates using a level-independent
// identity: set + slot + main stat + the SET OF SUBSTAT NAMES (values and level
// excluded). Substat names do not change with leveling, so two scans of the same
// 5-star piece at different levels share an identity and merge; two genuinely
// different pieces with an identical substat lineup can still be merged, which is
// an accepted, low-stakes risk for a triage helper (hence an explicit
// reconciliation pass, not a change to the per-save signature).
export function substatName(substat: string): string {
const plusIndex = substat.indexOf("+");
return (plusIndex >= 0 ? substat.slice(0, plusIndex) : substat).trim();
}
export function mergeIdentity(record: StoredArtifactRecord): string {
const substatNames = record.substats.map(substatName).filter(Boolean).sort().join(",");
return [record.setName, record.slot, record.mainStat, substatNames].join("::");
}
// The more-progressed / stronger record wins: higher level first, then strength.
function preferred(a: StoredArtifactRecord, b: StoredArtifactRecord): StoredArtifactRecord {
const levelA = a.level ?? 0;
const levelB = b.level ?? 0;
if (levelA !== levelB) return levelA > levelB ? a : b;
return storedArtifactStrength(a) >= storedArtifactStrength(b) ? a : b;
}
function minDate(a: string | undefined, b: string | undefined): string | undefined {
if (!a) return b;
if (!b) return a;
return a <= b ? a : b;
}
function maxDate(a: string | undefined, b: string | undefined): string | undefined {
if (!a) return b;
if (!b) return a;
return a >= b ? a : b;
}
function mergePair(winner: StoredArtifactRecord, other: StoredArtifactRecord): StoredArtifactRecord {
return {
...winner,
// The winner needs review only if it did on its own; a confident higher-level
// scan should clear a stale low-confidence duplicate.
needsReview: winner.needsReview,
timesSeen: (winner.timesSeen ?? 1) + (other.timesSeen ?? 1),
firstSeenAt: minDate(winner.firstSeenAt, other.firstSeenAt),
lastSeenAt: maxDate(winner.lastSeenAt, other.lastSeenAt),
// Keep an equipped character if either scan detected one.
equipped:
winner.equipped && !/not detected/i.test(winner.equipped)
? winner.equipped
: other.equipped,
};
}
export interface MergeResult {
merged: StoredArtifactRecord[];
collapsed: number;
}
export function mergeRescannedArtifacts(records: readonly StoredArtifactRecord[]): MergeResult {
const byIdentity = new Map<string, StoredArtifactRecord>();
let collapsed = 0;
for (const record of records) {
const identity = mergeIdentity(record);
const existing = byIdentity.get(identity);
if (!existing) {
byIdentity.set(identity, record);
continue;
}
const winner = preferred(existing, record);
const loser = winner === existing ? record : existing;
byIdentity.set(identity, mergePair(winner, loser));
collapsed++;
}
return { merged: [...byIdentity.values()], collapsed };
}
+21
View File
@@ -264,6 +264,27 @@ describe("parseArtifactCandidate", () => {
expect(parsed?.fields.substats.confidence).toBe(96);
});
it("parses the live calibrated 1080p Conductor circlet capture", () => {
const parsed = parseArtifactCandidate(captureFromOcr({
"artifact-title": "Conductor's Top Hat",
"artifact-main-stat": "Circlet of Logos\nHP\n7. 0 % i",
"artifact-substats": "a +\n+ Energy Recharge+4.5%\n+ ATK+14\n- Elemental Mastery+19\n- ATK+5.3% (unactivated)",
"artifact-footer": "",
}));
expect(parsed?.name).toBe("Conductor's Top Hat");
expect(parsed?.slot).toBe("Circlet of Logos");
expect(parsed?.setName).toBe("Wanderer's Troupe");
expect(parsed?.mainStat).toBe("HP%");
expect(parsed?.mainValue).toBe("7.0%");
expect(parsed?.substats).toEqual([
"Energy Recharge+4.5%",
"ATK+14",
"Elemental Mastery+19",
"ATK%+5.3%",
]);
});
it("keeps a percent main value even when OCR misses the main stat label", () => {
const parsed = parseArtifactCandidate(captureFromOcr({
"artifact-title": "Moonlit Offering's Final\nSands of Eon",
+9 -5
View File
@@ -19,6 +19,7 @@ import {
textReplacements,
} from "./genshinData.js";
import { fuzzyFindKnown, simplifyForMatch } from "./fuzzyMatch.js";
import { implausibleSubstats } from "./substatRolls.js";
type MainStatValueReference = { stat: string; base: number; max: number };
@@ -110,6 +111,8 @@ export function parseArtifactCandidate(capture: CaptureResult | null): ParsedArt
if (!mainStatField.value) notes.push("Main stat not confidently parsed.");
if (!mainValueField.value) notes.push("Main stat value not confidently parsed.");
if (substats.length < 3) notes.push("Substats look incomplete; crop or OCR needs tuning.");
const implausible = implausibleSubstats(substats);
if (implausible.length) notes.push(`Substat value has no valid roll combination (likely OCR misread): ${implausible.join(", ")}.`);
if (!setField.value) notes.push("Set name not confidently parsed.");
for (const [label, parsedField] of Object.entries({
@@ -273,14 +276,15 @@ function findMainValue(text: string, mainStat: string, slot: string, level: numb
}
function extractPercentValue(text: string) {
const percentPattern = /([0-9]{1,3}(?:\s*[.,:\u00B7]\s*[0-9])?)\s*%/;
const lineMatches = text
.split("\n")
.map((line) => line.match(/([0-9]{1,3}(?:[.,:\u00B7][0-9])?)\s*%/))
.map((line) => line.match(percentPattern))
.filter((match): match is RegExpMatchArray => Boolean(match));
const preferred = lineMatches[0] ?? text.match(/([0-9]{1,3}(?:[.,:\u00B7][0-9])?)\s*%/);
const preferred = lineMatches[0] ?? text.match(percentPattern);
if (!preferred?.[1]) return "";
return `${preferred[1].replace(/[:,\u00B7]/g, ".")}%`;
return `${normalizeMainValue(preferred[1])}%`;
}
function inferMainStat(slot: string, text: string): ParsedField {
@@ -299,7 +303,7 @@ function inferMainStat(slot: string, text: string): ParsedField {
function findDirectMainStat(text: string) {
const compact = simplifyForMatch(text);
const hasPercentValue = /[0-9]{1,3}(?:\.[0-9])?\s*%/.test(text);
const hasPercentValue = /[0-9]{1,3}(?:\s*[.,:\u00B7]\s*[0-9])?\s*%/.test(text);
const priority = [
"Physical DMG Bonus",
"Elemental Mastery",
@@ -478,7 +482,7 @@ function isPercentMainStat(stat: string) {
}
function promotePercentVariant(stat: string, text: string) {
if (["ATK", "HP", "DEF"].includes(stat) && /[0-9]{1,3}(?:\.[0-9])?\s*%/.test(text)) return `${stat}%`;
if (["ATK", "HP", "DEF"].includes(stat) && /[0-9]{1,3}(?:\s*[.,:\u00B7]\s*[0-9])?\s*%/.test(text)) return `${stat}%`;
return stat;
}
+2 -1
View File
@@ -45,7 +45,7 @@ export function hashId(value: string) {
return `${hash.toString(16).padStart(8, "0")}-${value.length.toString(16)}`;
}
export function toStoredArtifact(parsed: ParsedArtifactCandidate, source: string, needsReview: boolean): StoredArtifactRecord {
export function toStoredArtifact(parsed: ParsedArtifactCandidate, source: string, needsReview: boolean, locked?: boolean): StoredArtifactRecord {
return {
id: hashId(storeSignature(parsed)),
name: parsed.name,
@@ -58,6 +58,7 @@ export function toStoredArtifact(parsed: ParsedArtifactCandidate, source: string
equipped: parsed.equipped,
confidence: parsed.confidence,
needsReview,
locked,
source,
};
}
+98
View File
@@ -1,5 +1,58 @@
import { describe, expect, it } from "vitest";
import { detailFingerprint, fingerprintDataUrl, isRepeatedProcessedPageFingerprint, screenFingerprint } from "./autoScanLoop";
import { runAutoScanLoop } from "./autoScanLoop";
import type { AutoScanLoopDependencies } from "./autoScanLoop";
import type { CaptureResult } from "../types/global";
type ScanTestCapture = CaptureResult & { inventoryGrid: NonNullable<CaptureResult["inventoryGrid"]> };
const sampleGrid = {
centers: [{ x: 80, y: 90, row: 0, col: 0 }],
rows: 1,
cols: 1,
confidence: 86,
source: "detected" as const,
};
const sampleParse = {
name: "A Tiara of Torrents",
slot: "Flower of Life",
level: 16,
mainStat: "HP",
mainValue: "10.7%",
substats: ["ATK+29", "DEF+10"],
setName: "Tenacity of the Millelith",
equipped: "Traveler",
confidence: 84,
notes: [],
fields: {
name: { value: "A Tiara of Torrents", confidence: 84, source: "ocr" as const },
slot: { value: "Flower of Life", confidence: 84, source: "ocr" as const },
level: { value: "16", confidence: 84, source: "ocr" as const },
mainStat: { value: "HP", confidence: 84, source: "ocr" as const },
mainValue: { value: "10.7%", confidence: 84, source: "ocr" as const },
setName: { value: "Tenacity of the Millelith", confidence: 84, source: "ocr" as const },
equipped: { value: "Traveler", confidence: 84, source: "ocr" as const },
substats: { value: "ATK+29, DEF+10", confidence: 84, source: "ocr" as const },
},
};
function capture(overrides: Partial<ScanTestCapture> = {}): ScanTestCapture {
return {
id: "source",
name: "screen-capture",
width: 1920,
height: 1080,
dataUrl: "data:image/png;base64,AA",
capturedAt: "2026-01-01T00:00:00.000Z",
captureTarget: "desktop-source",
ocr: [],
detailDataUrl: "data:image/png;base64,DETAIL-AAA",
inventoryDataUrl: "data:image/png;base64,GRID-AAA",
inventoryGrid: sampleGrid,
...overrides,
};
}
describe("autoScanLoop fingerprints", () => {
it("distinguishes captures that share the same prefix but differ later", () => {
@@ -47,4 +100,49 @@ describe("autoScanLoop fingerprints", () => {
expect(isRepeatedProcessedPageFingerprint("abc", seen, 2)).toBe(true);
expect(isRepeatedProcessedPageFingerprint("", seen, 3)).toBe(false);
});
it("does not block before first click when start capture is from the primary screen", async () => {
const startCapture = capture({
captureTarget: "primary-screen",
detailDataUrl: `data:image/png;base64,${"D".repeat(600)}`,
});
let clicked = 0;
const deps: AutoScanLoopDependencies = {
api: {
clickScreen: async () => {
clicked += 1;
return {
ok: true,
x: 0,
y: 0,
cursorX: 0,
cursorY: 0,
clicked: true,
moved: true,
focused: true,
};
},
scrollScreen: async () => ({ ok: true, notchesSent: 0 }),
getAutomationGuard: async () => ({ ok: true, escapePressed: false, enterPressed: false, f9Pressed: false }),
},
captureSelectedSource: async () => capture({ detailDataUrl: `data:image/png;base64,${"E".repeat(600)}` }),
captureFastSelectedSource: async () => startCapture,
parseArtifact: () => sampleParse,
persistParsedArtifact: async () => false,
saveReviewSample: async () => ({ ok: true }),
getAutoReviewReason: () => "",
shouldFlagArtifactForReview: () => false,
appendAutomationLog: () => undefined,
appendClickDiagnostics: () => undefined,
setReviewStatus: () => undefined,
setAutoScanStats: () => undefined,
shouldStop: () => false,
};
const result = await runAutoScanLoop(deps, { scanLimit: 1, skipRows: 0, detectedInventoryCount: null });
expect(result.blockedReason).toBe("");
expect(result.status).toBe("done");
expect(clicked).toBe(1);
});
});
+38 -17
View File
@@ -3,6 +3,7 @@ import type { ParsedArtifactCandidate } from "./artifactOcrParser";
import { sessionSignature } from "./artifactStore";
import { buildGridModel, buildInventoryPagePlan, type GridTarget } from "./automationPlanner";
import { classifyAutoScanCapture, shouldAbortAfterConsecutiveMisses } from "./autoScanController";
import { waitForCardReady } from "./cardReadyGate";
import { captureRejectionReason, captureSourceRejectionReason, shouldPersistParsedArtifact } from "./scannerCaptureQuality";
import type { AutoScanStats, ScanSummary } from "./scannerSession";
import { clampSkipRows, emptyAutoScanStats, resolveScanTargetCount } from "./scannerSession";
@@ -59,7 +60,11 @@ export type AutoScanLoopResult = {
targetCount: number;
};
const CLICK_SETTLE_MS = 280;
// Card-ready gating replaces a fixed settle delay: poll the detail fingerprint
// until it has changed and stabilized (or the budget is spent). See cardReadyGate.
const CARD_READY_MAX_MS = 900;
const CARD_READY_POLL_MS = 90;
const CARD_READY_STABLE_SAMPLES = 2;
const MISS_ABORT_THRESHOLD = 3;
const UNREADABLE_ABORT_THRESHOLD = 5;
@@ -93,6 +98,8 @@ export async function runAutoScanLoop(
let aborted = false;
let consecutiveMisses = 0;
let rowsQueued = 0;
const primaryScreenStartWarning =
"Start-Capture ist vom Primary-Screen, kein spezifischer Genshin-Client-Marker vorhanden - Auto-Scan wird mit Vorsicht fortgesetzt.";
function updateStats() {
setAutoScanStats({ ...stats });
@@ -148,7 +155,8 @@ export async function runAutoScanLoop(
}
let currentCapture = await captureSelectedSource(0, true);
const initialCaptureRejection = captureSourceRejectionReason(currentCapture);
const isPrimaryCapture = currentCapture?.captureTarget === "primary-screen";
const initialCaptureRejection = isPrimaryCapture ? "" : captureSourceRejectionReason(currentCapture);
let gridModel = buildGridModel(currentCapture?.inventoryGrid);
if (initialCaptureRejection || !gridModel || gridModel.targets.length === 0) {
@@ -157,11 +165,28 @@ export async function runAutoScanLoop(
return { status: "blocked", stats, blockedReason: reason, pageCount: 0, gridLabel: reason, targetCount: maxTargets };
}
if (isPrimaryCapture) {
appendAutomationLog(primaryScreenStartWarning);
}
let lastDetailSignature = "";
const initialParsed = parseArtifact(currentCapture);
if (initialParsed) lastDetailSignature = sessionSignature(initialParsed);
let lastDetailViewFingerprint = detailFingerprint(currentCapture);
async function awaitCardReady() {
return waitForCardReady(
{
sampleFingerprint: async () => detailFingerprint(await captureFastSelectedSource(0, true)),
wait,
now: () => Date.now(),
checkAbort: checkGuard,
},
lastDetailViewFingerprint,
{ minStableSamples: CARD_READY_STABLE_SAMPLES, maxWaitMs: CARD_READY_MAX_MS, pollIntervalMs: CARD_READY_POLL_MS },
);
}
try {
while (!blockedReason && !shouldStop() && stats.clicked < maxTargets) {
page++;
@@ -224,16 +249,13 @@ export async function runAutoScanLoop(
break;
}
let waitStop = await waitDuringScan(CLICK_SETTLE_MS);
if (waitStop) {
blockedReason = waitStop;
let ready = await awaitCardReady();
if (ready.abortReason) {
blockedReason = ready.abortReason;
aborted = true;
break;
}
let previewCapture = await captureFastSelectedSource(0, true);
let previewFingerprint = detailFingerprint(previewCapture);
let changedDetail = Boolean(previewFingerprint && previewFingerprint !== lastDetailViewFingerprint);
let changedDetail = ready.changed;
if (!changedDetail) {
appendAutomationLog(`retry r${target.row} c${target.col}: Detailansicht unveraendert`);
@@ -244,15 +266,13 @@ export async function runAutoScanLoop(
aborted = true;
break;
}
waitStop = await waitDuringScan(CLICK_SETTLE_MS);
if (waitStop) {
blockedReason = waitStop;
ready = await awaitCardReady();
if (ready.abortReason) {
blockedReason = ready.abortReason;
aborted = true;
break;
}
previewCapture = await captureFastSelectedSource(0, true);
previewFingerprint = detailFingerprint(previewCapture);
changedDetail = Boolean(previewFingerprint && previewFingerprint !== lastDetailViewFingerprint);
changedDetail = ready.changed;
}
if (!changedDetail) {
@@ -482,6 +502,7 @@ export function fingerprintDataUrl(dataUrl: string) {
return `${dataUrl.length.toString(16)}:${(hash >>> 0).toString(16)}`;
}
function wait(ms: number) {
return new Promise((resolve) => window.setTimeout(resolve, ms));
function wait(ms: number): Promise<void> {
const schedule = typeof window !== "undefined" && window.setTimeout ? window.setTimeout : setTimeout;
return new Promise((resolve) => schedule(() => resolve(), ms));
}
+70
View File
@@ -0,0 +1,70 @@
import { describe, expect, it } from "vitest";
import { waitForCardReady, type CardReadyDeps } from "./cardReadyGate";
// Deterministic clock: each wait advances virtual time, each sample pops the
// next scripted fingerprint.
function harness(samples: string[], step = 90, checkAbort?: () => string) {
let clock = 0;
let index = 0;
const deps: CardReadyDeps = {
sampleFingerprint: async () => samples[Math.min(index++, samples.length - 1)],
wait: async (ms) => {
clock += ms;
},
now: () => clock,
checkAbort,
};
return { deps, sampleCount: () => index, step };
}
describe("waitForCardReady", () => {
it("returns ready once the card changed and stabilized", async () => {
const { deps } = harness(["old", "new", "new"]);
const result = await waitForCardReady(deps, "old", { minStableSamples: 2, maxWaitMs: 900, pollIntervalMs: 90 });
expect(result.ready).toBe(true);
expect(result.changed).toBe(true);
expect(result.stable).toBe(true);
expect(result.fingerprint).toBe("new");
expect(result.polls).toBe(3);
});
it("keeps polling while the detail still shows the previous artifact", async () => {
const { deps } = harness(["old", "old", "new", "new"]);
const result = await waitForCardReady(deps, "old", { minStableSamples: 2 });
expect(result.ready).toBe(true);
expect(result.fingerprint).toBe("new");
expect(result.polls).toBe(4);
});
it("proceeds after the budget when content changed but never stabilizes (animation)", async () => {
// Always different (animated glow): changed but never two-in-a-row equal.
const animated = ["a1", "a2", "a3", "a4", "a5", "a6", "a7", "a8", "a9", "a10"];
const { deps } = harness(animated, 90);
const result = await waitForCardReady(deps, "old", { minStableSamples: 2, maxWaitMs: 300, pollIntervalMs: 90 });
expect(result.changed).toBe(true);
expect(result.stable).toBe(false);
expect(result.ready).toBe(true); // budget spent, but content did change
});
it("reports not-ready when the detail never changes within budget", async () => {
const { deps } = harness(["old", "old", "old", "old", "old", "old"], 90);
const result = await waitForCardReady(deps, "old", { minStableSamples: 2, maxWaitMs: 200, pollIntervalMs: 90 });
expect(result.changed).toBe(false);
expect(result.ready).toBe(false);
});
it("aborts immediately when checkAbort returns a reason", async () => {
const { deps } = harness(["new", "new"], 90, () => "ESC gehalten");
const result = await waitForCardReady(deps, "old", {});
expect(result.abortReason).toBe("ESC gehalten");
expect(result.ready).toBe(false);
expect(result.polls).toBe(0);
});
it("ignores empty fingerprints for stability", async () => {
const { deps } = harness(["", "", "new", "new"], 90);
const result = await waitForCardReady(deps, "old", { minStableSamples: 2, maxWaitMs: 900 });
expect(result.ready).toBe(true);
expect(result.fingerprint).toBe("new");
});
});
+92
View File
@@ -0,0 +1,92 @@
// Card-ready gating for the auto-scan loop (ADR replaces the fixed 280ms settle
// delay). After clicking a tile, instead of waiting a hardcoded interval and
// hoping the detail card has rendered, poll a cheap detail fingerprint until it
// has both (a) changed from the previously-read artifact and (b) stabilized
// across consecutive samples. This is faster on quick machines and correct on
// slow ones, and it is robust to particle/hover-glow animation: if the card
// never fully stabilizes within the budget it still proceeds once the content
// has changed, rather than looping forever on an animated frame.
//
// Pure except for the injected async sampler/clock, so it is unit testable.
export interface CardReadyOptions {
/** Consecutive equal samples required to call the card stable. */
minStableSamples?: number;
/** Total time budget before giving up on full stability. */
maxWaitMs?: number;
/** Delay between samples. */
pollIntervalMs?: number;
}
export interface CardReadyDeps {
/** Fast capture -> detail fingerprint. */
sampleFingerprint: () => Promise<string>;
wait: (ms: number) => Promise<void>;
now: () => number;
/** Returns a non-empty reason to abort (ESC held, stop pressed, ...). */
checkAbort?: () => Promise<string> | string;
}
export interface CardReadyResult {
/** Safe to read the full artifact: content changed (and stabilized or budget spent). */
ready: boolean;
/** The detail differs from the previously-read artifact. */
changed: boolean;
/** Reached the required number of consecutive equal samples. */
stable: boolean;
/** Latest sampled fingerprint. */
fingerprint: string;
/** Non-empty when aborted via checkAbort. */
abortReason: string;
polls: number;
}
const DEFAULT_MIN_STABLE = 2;
const DEFAULT_MAX_WAIT_MS = 900;
const DEFAULT_POLL_MS = 90;
export async function waitForCardReady(
deps: CardReadyDeps,
previousFingerprint: string,
options: CardReadyOptions = {},
): Promise<CardReadyResult> {
const minStable = Math.max(1, options.minStableSamples ?? DEFAULT_MIN_STABLE);
const maxWaitMs = Math.max(0, options.maxWaitMs ?? DEFAULT_MAX_WAIT_MS);
const pollIntervalMs = Math.max(1, options.pollIntervalMs ?? DEFAULT_POLL_MS);
const start = deps.now();
let previousSample = "";
let stableCount = 0;
let latest = "";
let polls = 0;
for (;;) {
if (deps.checkAbort) {
const abortReason = await deps.checkAbort();
if (abortReason) {
return { ready: false, changed: false, stable: false, fingerprint: latest, abortReason, polls };
}
}
latest = await deps.sampleFingerprint();
polls++;
stableCount = latest && latest === previousSample ? stableCount + 1 : 1;
previousSample = latest;
const changed = Boolean(latest) && latest !== previousFingerprint;
const stable = stableCount >= minStable;
if (changed && stable) {
return { ready: true, changed: true, stable: true, fingerprint: latest, abortReason: "", polls };
}
if (deps.now() - start >= maxWaitMs) {
// Budget spent. Proceed if the content has at least changed, even if it is
// still animating (never fully stabilizes).
return { ready: changed, changed, stable, fingerprint: latest, abortReason: "", polls };
}
await deps.wait(pollIntervalMs);
}
}
+41
View File
@@ -0,0 +1,41 @@
import { describe, expect, it } from "vitest";
import { dataPackageAgeDays, dataPackageStatus } from "./dataPackageStatus";
const NOW = Date.parse("2026-07-05T00:00:00.000Z");
describe("dataPackageStatus", () => {
it("computes age in whole days", () => {
expect(dataPackageAgeDays("2026-07-01T00:00:00.000Z", NOW)).toBe(4);
expect(dataPackageAgeDays("2026-07-05T00:00:00.000Z", NOW)).toBe(0);
});
it("returns null for missing or invalid timestamps", () => {
expect(dataPackageAgeDays("", NOW)).toBeNull();
expect(dataPackageAgeDays("not-a-date", NOW)).toBeNull();
});
it("clamps future timestamps to zero", () => {
expect(dataPackageAgeDays("2026-08-01T00:00:00.000Z", NOW)).toBe(0);
});
it("does not warn for a fresh package", () => {
const status = dataPackageStatus("2026-06-20T00:00:00.000Z", "genshin-db@5.2.12", NOW);
expect(status.stale).toBe(false);
expect(status.warning).toBe("");
expect(status.ageDays).toBe(15);
});
it("warns for a package older than the max age", () => {
const status = dataPackageStatus("2026-04-01T00:00:00.000Z", "genshin-db@5.2.12", NOW, 45);
expect(status.stale).toBe(true);
expect(status.warning).toContain("Datenpaket");
expect(status.warning).toContain("aktualisieren");
});
it("stays quiet when the generation date is unknown", () => {
const status = dataPackageStatus("", "unknown", NOW);
expect(status.stale).toBe(false);
expect(status.warning).toBe("");
expect(status.ageDays).toBeNull();
});
});
+47
View File
@@ -0,0 +1,47 @@
// Data-package staleness (ADR-005 follow-up). The genshin-db data package is a
// local snapshot; when a new Genshin version ships new sets/characters, an old
// package silently fails to recognize them. We cannot query the live game version
// offline, so staleness is based on the package's generation age: Genshin patches
// land roughly every six weeks, so a package older than ~45 days likely predates
// a content patch and should be regenerated with `npm run data:genshin`.
const DAY_MS = 24 * 60 * 60 * 1000;
export const DEFAULT_MAX_AGE_DAYS = 45;
export function dataPackageAgeDays(generatedAt: string, now: number = Date.now()): number | null {
if (!generatedAt) return null;
const generated = Date.parse(generatedAt);
if (!Number.isFinite(generated)) return null;
const age = (now - generated) / DAY_MS;
return age < 0 ? 0 : Math.floor(age);
}
export interface DataPackageStatus {
ageDays: number | null;
stale: boolean;
warning: string;
}
export function dataPackageStatus(
generatedAt: string,
sourceVersion: string,
now: number = Date.now(),
maxAgeDays: number = DEFAULT_MAX_AGE_DAYS,
): DataPackageStatus {
const ageDays = dataPackageAgeDays(generatedAt, now);
if (ageDays === null) {
return {
ageDays: null,
stale: false,
warning: "",
};
}
const stale = ageDays > maxAgeDays;
return {
ageDays,
stale,
warning: stale
? `Datenpaket (${sourceVersion}) ist ${ageDays} Tage alt. Neue Sets/Charaktere fehlen evtl. - mit "npm run data:genshin" aktualisieren.`
: "",
};
}
+1
View File
@@ -44,6 +44,7 @@ export const characterAliases = genshinGameData.aliases?.characterAliases ?? {};
export const knownSets = genshinGameData.artifactSets.map((set) => set.name);
export const knownCharacters = (genshinGameData.characters ?? []).map((character) => character.name);
export const sourceVersion = genshinGameData.sourceVersion ?? "unknown";
export const dataGeneratedAt = (genshinGameData as { generatedAt?: string }).generatedAt ?? "";
export const fixedMainStatBySlot: Record<string, string> = {
"Flower of Life": "HP",
+111
View File
@@ -0,0 +1,111 @@
import { describe, expect, it } from "vitest";
import type { StoredArtifactRecord } from "../types/storage";
import {
goodDatabaseToStoredArtifacts,
goodSubstatToString,
goodToStoredArtifact,
setKeyToName,
setNameToKey,
storedArtifactToGood,
storedArtifactsToGood,
substatStringToGood,
} from "./goodInterop";
const record: StoredArtifactRecord = {
id: "x",
name: "Gladiator's Nostalgia",
slot: "Flower of Life",
level: 20,
setName: "Gladiator's Finale",
mainStat: "HP",
mainValue: "4,780",
substats: ["CRIT DMG+13.2%", "ATK+19", "HP%+15.7%", "Energy Recharge+5.2%"],
equipped: "Bennett",
confidence: 90,
needsReview: false,
source: "auto-scan",
};
describe("goodInterop set keys", () => {
it("converts set names to GOOD PascalCase keys", () => {
expect(setNameToKey("Gladiator's Finale")).toBe("GladiatorsFinale");
expect(setNameToKey("Viridescent Venerer")).toBe("ViridescentVenerer");
expect(setNameToKey("Emblem of Severed Fate")).toBe("EmblemOfSeveredFate");
});
it("round-trips known set keys back to names", () => {
for (const name of ["Gladiator's Finale", "Viridescent Venerer"]) {
expect(setKeyToName(setNameToKey(name))).toBe(name);
}
});
});
describe("goodInterop substats", () => {
it("parses percent and flat substat strings", () => {
expect(substatStringToGood("CRIT DMG+13.2%")).toEqual({ key: "critDMG_", value: 13.2 });
expect(substatStringToGood("ATK+19")).toEqual({ key: "atk", value: 19 });
expect(substatStringToGood("HP%+15.7%")).toEqual({ key: "hp_", value: 15.7 });
});
it("returns null for unparseable substats", () => {
expect(substatStringToGood("nonsense")).toBeNull();
expect(substatStringToGood("Unknown+5")).toBeNull();
});
it("round-trips substat strings", () => {
for (const entry of record.substats) {
const good = substatStringToGood(entry)!;
expect(goodSubstatToString(good)).toBe(entry);
}
});
});
describe("goodInterop export", () => {
it("exports a stored artifact to GOOD", () => {
const good = storedArtifactToGood(record);
expect(good.setKey).toBe("GladiatorsFinale");
expect(good.slotKey).toBe("flower");
expect(good.mainStatKey).toBe("hp");
expect(good.level).toBe(20);
expect(good.rarity).toBe(5);
expect(good.substats).toContainEqual({ key: "critDMG_", value: 13.2 });
});
it("wraps records in a GOOD database envelope", () => {
const db = storedArtifactsToGood([record]);
expect(db.format).toBe("GOOD");
expect(db.artifacts).toHaveLength(1);
});
});
describe("goodInterop import", () => {
it("imports a GOOD artifact back to a stored record", () => {
const good = storedArtifactToGood(record);
const back = goodToStoredArtifact({ ...good, location: "Bennett" })!;
expect(back.slot).toBe("Flower of Life");
expect(back.setName).toBe("Gladiator's Finale");
expect(back.mainStat).toBe("HP");
expect(back.equipped).toBe("Bennett");
expect(back.substats).toContain("CRIT DMG+13.2%");
expect(back.source).toBe("good-import");
});
it("computes a main value from slot + main stat + level on import", () => {
const good = storedArtifactToGood(record);
const back = goodToStoredArtifact(good)!;
// Flower HP main at +20 is the reference max; just assert it is populated.
expect(back.mainValue).not.toBe("");
});
it("skips artifacts with an unknown slot or main stat", () => {
expect(goodToStoredArtifact({ setKey: "GladiatorsFinale", slotKey: "bogus", mainStatKey: "hp" })).toBeNull();
expect(goodToStoredArtifact({ setKey: "GladiatorsFinale", slotKey: "flower", mainStatKey: "bogus" })).toBeNull();
});
it("maps a full GOOD database", () => {
const db = storedArtifactsToGood([record, { ...record, slot: "Plume of Death", mainStat: "ATK", mainValue: "311" }]);
const imported = goodDatabaseToStoredArtifacts(db);
expect(imported).toHaveLength(2);
expect(imported.map((entry) => entry.slot)).toEqual(["Flower of Life", "Plume of Death"]);
});
});
+202
View File
@@ -0,0 +1,202 @@
import type { GoodExportArtifact } from "../types/global";
import type { StoredArtifactRecord } from "../types/storage";
import { knownSets, mainStatValueReferences, pieceToSet, pieceToSlot } from "./genshinData";
import { simplifyForMatch } from "./fuzzyMatch";
import { inferRarity } from "./substatRolls";
// GOOD (Genshin Open Object Description) interop for scanned artifacts, so the
// local store can round-trip with Genshin Optimizer / Inventory Kamera / Akasha
// (ADR-003). Export is lossless for the fields GOOD carries; import is
// best-effort because GOOD does not store piece names or main-stat values.
export interface GoodImportArtifact {
setKey: string;
slotKey: string;
rarity?: number;
level?: number;
mainStatKey: string;
substats?: Array<{ key: string; value: number }>;
location?: string;
lock?: boolean;
}
export interface GoodImportDatabase {
format?: string;
version?: number;
source?: string;
artifacts?: GoodImportArtifact[];
}
const SLOT_TO_GOOD: Record<string, string> = {
"Flower of Life": "flower",
"Plume of Death": "plume",
"Sands of Eon": "sands",
"Goblet of Eonothem": "goblet",
"Circlet of Logos": "circlet",
};
const GOOD_TO_SLOT: Record<string, string> = Object.fromEntries(
Object.entries(SLOT_TO_GOOD).map(([display, key]) => [key, display]),
);
// display name (as used across the app, including the HP%/ATK%/DEF% variants) ->
// GOOD stat key + whether it is a percent stat.
interface StatEntry {
display: string;
key: string;
percent: boolean;
}
const STAT_ENTRIES: StatEntry[] = [
{ display: "HP", key: "hp", percent: false },
{ display: "HP%", key: "hp_", percent: true },
{ display: "ATK", key: "atk", percent: false },
{ display: "ATK%", key: "atk_", percent: true },
{ display: "DEF", key: "def", percent: false },
{ display: "DEF%", key: "def_", percent: true },
{ display: "Elemental Mastery", key: "eleMas", percent: false },
{ display: "Energy Recharge", key: "enerRech_", percent: true },
{ display: "CRIT Rate", key: "critRate_", percent: true },
{ display: "CRIT DMG", key: "critDMG_", percent: true },
{ display: "Healing Bonus", key: "heal_", percent: true },
{ display: "Physical DMG Bonus", key: "physical_dmg_", percent: true },
{ display: "Pyro DMG Bonus", key: "pyro_dmg_", percent: true },
{ display: "Hydro DMG Bonus", key: "hydro_dmg_", percent: true },
{ display: "Electro DMG Bonus", key: "electro_dmg_", percent: true },
{ display: "Cryo DMG Bonus", key: "cryo_dmg_", percent: true },
{ display: "Anemo DMG Bonus", key: "anemo_dmg_", percent: true },
{ display: "Geo DMG Bonus", key: "geo_dmg_", percent: true },
{ display: "Dendro DMG Bonus", key: "dendro_dmg_", percent: true },
];
const DISPLAY_TO_STAT = new Map(STAT_ENTRIES.map((entry) => [entry.display, entry]));
const KEY_TO_STAT = new Map(STAT_ENTRIES.map((entry) => [entry.key, entry]));
export function statDisplayToGoodKey(display: string): string {
return DISPLAY_TO_STAT.get(display)?.key ?? "";
}
export function goodKeyToStatDisplay(key: string): string {
return KEY_TO_STAT.get(key)?.display ?? "";
}
export function setNameToKey(name: string): string {
// GOOD removes apostrophes without re-capitalizing ("Gladiator's" ->
// "Gladiators"), then PascalCases the remaining whitespace/hyphen words.
return name
.replace(/[']/g, "")
.split(/[\s-]+/)
.map((word) => word.replace(/[^A-Za-z0-9]/g, ""))
.filter(Boolean)
.map((word) => word.charAt(0).toUpperCase() + word.slice(1))
.join("");
}
const SET_KEY_TO_NAME = new Map(knownSets.map((name) => [setNameToKey(name), name]));
export function setKeyToName(key: string): string {
const direct = SET_KEY_TO_NAME.get(key);
if (direct) return direct;
const simplifiedKey = simplifyForMatch(key);
const match = knownSets.find((name) => simplifyForMatch(setNameToKey(name)) === simplifiedKey);
return match ?? key;
}
// "CRIT DMG+13.2%" / "ATK+19" -> GOOD { key, value }.
export function substatStringToGood(entry: string): { key: string; value: number } | null {
const plusIndex = entry.indexOf("+");
if (plusIndex <= 0) return null;
const display = entry.slice(0, plusIndex).trim();
const key = statDisplayToGoodKey(display);
if (!key) return null;
const value = Number.parseFloat(entry.slice(plusIndex + 1).replace(/[%,\s]/g, ""));
if (!Number.isFinite(value)) return null;
return { key, value };
}
export function goodSubstatToString(substat: { key: string; value: number }): string {
const entry = KEY_TO_STAT.get(substat.key);
if (!entry) return "";
return entry.percent ? `${entry.display}+${substat.value}%` : `${entry.display}+${substat.value}`;
}
export function storedArtifactToGood(record: StoredArtifactRecord): GoodExportArtifact {
const substats = record.substats
.map((entry) => substatStringToGood(entry))
.filter((entry): entry is { key: string; value: number } => entry !== null);
return {
setKey: setNameToKey(record.setName),
slotKey: SLOT_TO_GOOD[record.slot] ?? "",
rarity: inferRarity(record.level ?? 0, record.substats),
level: record.level ?? 0,
mainStatKey: statDisplayToGoodKey(record.mainStat),
substats,
lock: Boolean((record as { locked?: boolean }).locked),
};
}
export function storedArtifactsToGood(records: readonly StoredArtifactRecord[], source = "Genshin Artifact Assistant") {
return {
format: "GOOD" as const,
version: 2,
source,
artifacts: records.map(storedArtifactToGood),
};
}
function reversePieceLookup(setName: string, slotDisplay: string): string {
for (const [piece, set] of pieceToSet.entries()) {
if (set === setName && pieceToSlot.get(piece) === slotDisplay) return piece;
}
return "";
}
function computeMainValue(slotDisplay: string, mainStatDisplay: string, level: number): string {
const references = mainStatValueReferences[slotDisplay];
const reference = Array.isArray(references)
? (references as Array<{ stat: string; base: number; max: number }>).find((entry) => entry.stat === mainStatDisplay)
: undefined;
if (!reference) return "";
const clamped = Math.max(0, Math.min(20, level));
const value = reference.base + (reference.max - reference.base) * (clamped / 20);
const isPercent = DISPLAY_TO_STAT.get(mainStatDisplay)?.percent ?? false;
return isPercent ? `${(Math.round(value * 10) / 10).toFixed(1)}%` : Math.round(value).toLocaleString("en-US");
}
export function goodToStoredArtifact(good: GoodImportArtifact, index = 0): StoredArtifactRecord | null {
const slot = GOOD_TO_SLOT[good.slotKey];
const mainStat = goodKeyToStatDisplay(good.mainStatKey);
if (!slot || !mainStat) return null;
const setName = setKeyToName(good.setKey);
const level = typeof good.level === "number" ? good.level : 0;
const substats = (good.substats ?? [])
.map((substat) => goodSubstatToString(substat))
.filter(Boolean);
const name = reversePieceLookup(setName, slot) || setName;
return {
id: `good-${good.setKey}-${good.slotKey}-${index}`,
name,
slot,
level,
setName,
mainStat,
mainValue: computeMainValue(slot, mainStat, level),
substats,
equipped: good.location || "Not detected",
confidence: 100,
needsReview: false,
source: "good-import",
};
}
export function goodDatabaseToStoredArtifacts(database: GoodImportDatabase | null | undefined): StoredArtifactRecord[] {
const records: StoredArtifactRecord[] = [];
(database?.artifacts ?? []).forEach((artifact, index) => {
const record = goodToStoredArtifact(artifact, index);
if (record) records.push(record);
});
return records;
}
+100
View File
@@ -0,0 +1,100 @@
import { describe, expect, it } from "vitest";
import {
aspectRatioLabel,
detailCropRects,
inventoryCountCropRect,
inventoryGrid,
inventoryRect,
isSixteenNine,
layoutSupportWarning,
profileDetailRect,
} from "./layoutProfile";
const HD = { width: 1920, height: 1080 };
const QHD = { width: 2560, height: 1440 };
const ULTRAWIDE = { width: 3440, height: 1440 };
describe("layoutProfile", () => {
it("detects 16:9 across common resolutions and rejects ultrawide", () => {
expect(isSixteenNine(HD)).toBe(true);
expect(isSixteenNine(QHD)).toBe(true);
expect(isSixteenNine({ width: 3840, height: 2160 })).toBe(true);
expect(isSixteenNine(ULTRAWIDE)).toBe(false);
expect(isSixteenNine({ width: 1920, height: 1200 })).toBe(false); // 16:10
});
it("labels the aspect ratio", () => {
expect(aspectRatioLabel(HD)).toBe("1.78:1");
expect(aspectRatioLabel({ width: 0, height: 0 })).toBe("unknown");
});
it("warns only for non-16:9 resolutions", () => {
expect(layoutSupportWarning(HD)).toBe("");
expect(layoutSupportWarning(QHD)).toBe("");
expect(layoutSupportWarning(ULTRAWIDE)).toContain("nicht 16:9");
expect(layoutSupportWarning({ width: 0, height: 0 })).toBe("");
});
it("keeps the detail rect inside the image and on the right half", () => {
const rect = profileDetailRect(QHD);
expect(rect.x).toBeGreaterThanOrEqual(QHD.width * 0.45);
expect(rect.x + rect.width).toBeLessThanOrEqual(QHD.width);
expect(rect.y + rect.height).toBeLessThanOrEqual(QHD.height);
});
it("matches the calibrated 1080p artifact detail panel", () => {
expect(profileDetailRect(HD)).toEqual({ x: 1308, y: 120, width: 492, height: 838 });
});
it("produces the four artifact crops in top-to-bottom order, all clamped", () => {
const detail = profileDetailRect(QHD);
const crops = detailCropRects(detail, QHD);
expect(crops.map((crop) => crop.id)).toEqual([
"artifact-title",
"artifact-main-stat",
"artifact-substats",
"artifact-footer",
]);
let previousY = -1;
for (const crop of crops) {
expect(crop.rect.x).toBeGreaterThanOrEqual(0);
expect(crop.rect.y).toBeGreaterThan(previousY);
expect(crop.rect.x + crop.rect.width).toBeLessThanOrEqual(QHD.width);
expect(crop.rect.y + crop.rect.height).toBeLessThanOrEqual(QHD.height);
previousY = crop.rect.y;
}
});
it("places the inventory count crop inside the inventory panel", () => {
const detail = profileDetailRect(QHD);
const inv = inventoryRect(QHD, detail);
const count = inventoryCountCropRect(inv, QHD);
expect(count.x).toBeGreaterThan(QHD.width * 0.75);
expect(count.x + count.width).toBeLessThanOrEqual(QHD.width);
});
it("builds the calibrated 8-column inventory grid on the left", () => {
const detail = profileDetailRect(QHD);
const grid = inventoryGrid(QHD, detail);
expect(grid.cols).toBe(8);
expect(grid.rows).toBe(5);
expect(grid.source).toBe("detected");
expect(grid.centers).toHaveLength(40);
expect(grid.centers.every((center) => center.x < detail.x)).toBe(true);
});
it("matches the live 1080p artifact grid centers", () => {
const detail = profileDetailRect(HD);
const grid = inventoryGrid(HD, detail);
expect(grid.centers[0]).toEqual({ x: 179, y: 254, row: 0, col: 0 });
expect(grid.centers[7]).toEqual({ x: 1201, y: 254, row: 0, col: 7 });
expect(grid.centers.at(-1)).toEqual({ x: 1201, y: 958, row: 4, col: 7 });
});
it("reports a missing grid when the inventory panel is too small", () => {
const tiny = { width: 320, height: 180 };
const grid = inventoryGrid(tiny, profileDetailRect(tiny));
expect(grid.source).toBe("missing");
expect(grid.centers).toHaveLength(0);
});
});
+199
View File
@@ -0,0 +1,199 @@
// Resolution-anchored layout geometry for the artifact inventory screen
// (ADR-009). Inventory Kamera's proven approach is to require borderless 16:9 and
// derive crop/grid coordinates from the client rectangle instead of detecting the
// panel by colour each frame. This module is the single, pure, unit-tested source
// of that geometry; electron/main.ts consumes it for cropping and keeps a
// colour-based detail-rect detector only as a fallback for off-profile setups.
//
// Calibrated from a 1920x1080 English artifact-inventory screenshot and scaled
// by client size. This follows Inventory Kamera's stable approach: fixed
// 16:9-relative UI regions first, visual detection only as a fallback.
export interface LayoutRect {
x: number;
y: number;
width: number;
height: number;
}
export interface CropTemplateRect {
id: string;
label: string;
rect: LayoutRect;
}
export interface InventoryGridLayout {
centers: Array<{ x: number; y: number; row: number; col: number }>;
rows: number;
cols: number;
confidence: number;
source: "detected" | "missing";
}
const SIXTEEN_NINE = 16 / 9;
export function aspectRatio(size: { width: number; height: number }): number {
if (!size.height) return 0;
return size.width / size.height;
}
export function aspectRatioLabel(size: { width: number; height: number }): string {
const ratio = aspectRatio(size);
if (ratio === 0) return "unknown";
return `${ratio.toFixed(2)}:1`;
}
// Genshin's UI is authored for 16:9; other aspect ratios letterbox or reflow and
// the anchored crops no longer line up. Allow a small tolerance for rounding.
export function isSixteenNine(size: { width: number; height: number }, tolerance = 0.02): boolean {
const ratio = aspectRatio(size);
if (ratio === 0) return false;
return Math.abs(ratio - SIXTEEN_NINE) <= SIXTEEN_NINE * tolerance;
}
// Empty when the client is a supported 16:9; otherwise a warning explaining that
// the anchored crops are unreliable off-profile (ADR-009: non-16:9 is explicitly
// unsupported for the auto scanner).
export function layoutSupportWarning(size: { width: number; height: number }): string {
if (size.width <= 0 || size.height <= 0) return "";
if (isSixteenNine(size)) return "";
return `Aufloesung ${size.width}x${size.height} ist nicht 16:9 (${aspectRatioLabel(size)}). Der Auto-Scan ist auf 16:9 im randlosen Fenstermodus ausgelegt; die Erkennung kann daneben liegen.`;
}
export function clampRect(rect: LayoutRect, imageSize: { width: number; height: number }): LayoutRect {
const x = Math.max(0, Math.min(imageSize.width - 1, rect.x));
const y = Math.max(0, Math.min(imageSize.height - 1, rect.y));
const maxWidth = Math.max(1, imageSize.width - x);
const maxHeight = Math.max(1, imageSize.height - y);
return {
x,
y,
width: Math.max(1, Math.min(maxWidth, rect.width)),
height: Math.max(1, Math.min(maxHeight, rect.height)),
};
}
// Anchored guess for the artifact detail panel on the right of the screen. Used
// as the primary rect for a clean 16:9 client and as the fallback when colour
// detection cannot find the panel.
export function profileDetailRect(imageSize: { width: number; height: number }): LayoutRect {
const { width, height } = imageSize;
if (width <= 0 || height <= 0) return { x: 0, y: 0, width: Math.max(1, width), height: Math.max(1, height) };
return clampRect(
{
x: Math.round(width * 0.681),
y: Math.round(height * 0.111),
width: Math.round(width * 0.256),
height: Math.round(height * 0.776),
},
imageSize,
);
}
// The four OCR crops inside the detail panel, as fractions of the detail rect.
export function detailCropRects(detailRect: LayoutRect, imageSize: { width: number; height: number }): CropTemplateRect[] {
const templates: CropTemplateRect[] = [
{
id: "artifact-title",
label: "Artifact title",
rect: {
x: Math.round(detailRect.x),
y: Math.round(detailRect.y),
width: Math.round(detailRect.width),
height: Math.round(detailRect.height * 0.07),
},
},
{
id: "artifact-main-stat",
label: "Main stat",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.075),
width: Math.round(detailRect.width * 0.58),
height: Math.round(detailRect.height * 0.26),
},
},
{
id: "artifact-substats",
label: "Substats",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.34),
width: Math.round(detailRect.width * 0.86),
height: Math.round(detailRect.height * 0.27),
},
},
{
id: "artifact-footer",
label: "Footer",
rect: {
x: Math.round(detailRect.x + detailRect.width * 0.055),
y: Math.round(detailRect.y + detailRect.height * 0.82),
width: Math.round(detailRect.width * 0.86),
height: Math.round(detailRect.height * 0.14),
},
},
];
return templates.map((template) => ({ ...template, rect: clampRect(template.rect, imageSize) }));
}
export function inventoryCountCropRect(inventoryRect: LayoutRect, imageSize: { width: number; height: number }): LayoutRect {
const { width, height } = imageSize;
return clampRect(
{
x: Math.round(width * 0.795),
y: Math.round(height * 0.02),
width: Math.round(width * 0.145),
height: Math.round(height * 0.055),
},
imageSize,
);
}
export function inventoryRect(imageSize: { width: number; height: number }, detailRect: LayoutRect): LayoutRect {
const { width, height } = imageSize;
return clampRect(
{
x: Math.round(width * 0.055),
y: Math.round(height * 0.155),
width: Math.max(140, Math.round(Math.min(detailRect.x - width * 0.07, width * 0.63))),
height: Math.max(140, Math.round(height * 0.74)),
},
imageSize,
);
}
export function inventoryGrid(imageSize: { width: number; height: number }, detailRect: LayoutRect): InventoryGridLayout {
const rect = inventoryRect(imageSize, detailRect);
const cols = 8;
if (imageSize.width < 800 || imageSize.height < 450 || rect.width < 160 || rect.height < 140) {
return { centers: [], rows: 0, cols: 0, confidence: 0, source: "missing" };
}
const stepX = Math.round(imageSize.width * 0.076);
const stepY = Math.round(imageSize.height * 0.163);
const visibleRows = 5;
const startX = Math.round(imageSize.width * 0.093);
const startY = Math.round(imageSize.height * 0.235);
const centers: InventoryGridLayout["centers"] = [];
for (let row = 0; row < visibleRows; row++) {
for (let col = 0; col < cols; col++) {
const x = startX + col * stepX;
const y = startY + row * stepY;
if (x < imageSize.width && y < imageSize.height) {
centers.push({ x, y, row, col });
}
}
}
const trimmed = centers.filter((center) => center.x > 0 && center.y > 0);
return {
centers: trimmed,
rows: visibleRows,
cols,
confidence: trimmed.length >= cols * 2 ? 76 : trimmed.length >= cols ? 58 : 36,
source: "detected",
};
}
+44
View File
@@ -0,0 +1,44 @@
import { describe, expect, it } from "vitest";
import { detectLockState, isLocked, lockIconCropRect, lockSignalRatio } from "./lockDetection";
import type { Bitmap } from "./ocrPreprocess";
import { profileDetailRect } from "./layoutProfile";
// Build a BGRA bitmap where `goldPixels` of the pixels are lock-gold and the rest dark.
function bitmap(goldPixels: number, total: number): Bitmap {
const data = Buffer.alloc(total * 4);
for (let pixel = 0; pixel < total; pixel++) {
const index = pixel * 4;
if (pixel < goldPixels) {
data[index] = 40; // B
data[index + 1] = 170; // G
data[index + 2] = 230; // R -> gold
}
data[index + 3] = 255;
}
return { data, width: total, height: 1 };
}
describe("lockDetection", () => {
it("places the lock crop on the lock button in the substat panel", () => {
const size = { width: 2560, height: 1440 };
const detail = profileDetailRect(size);
const rect = lockIconCropRect(detail, size);
expect(rect.x).toBeGreaterThan(detail.x + detail.width * 0.5);
expect(rect.x + rect.width).toBeLessThanOrEqual(size.width);
expect(rect.y).toBeGreaterThan(detail.y + detail.height * 0.3);
expect(rect.y).toBeLessThan(detail.y + detail.height * 0.45);
});
it("measures the gold-pixel ratio", () => {
expect(lockSignalRatio(bitmap(0, 100))).toBe(0);
expect(lockSignalRatio(bitmap(50, 100))).toBeCloseTo(0.5, 5);
expect(lockSignalRatio(bitmap(100, 100))).toBe(1);
});
it("thresholds the ratio into a locked flag", () => {
expect(isLocked(0.02)).toBe(false);
expect(isLocked(0.2)).toBe(true);
expect(detectLockState(bitmap(20, 100))).toBe(true);
expect(detectLockState(bitmap(1, 100))).toBe(false);
});
});
+52
View File
@@ -0,0 +1,52 @@
import { clampRect, type LayoutRect } from "./layoutProfile.js";
import type { Bitmap } from "./ocrPreprocess.js";
// EXPERIMENTAL, read-only lock-status detection (nice-to-have). Genshin shows a
// padlock at the top-right of the artifact detail card: a bright gold fill when
// locked, a dim outline when not. This estimates that icon region and measures
// the fraction of bright "lock-gold" pixels; above a threshold the piece is
// considered locked.
//
// The crop position and threshold need calibration against a reference 16:9
// screenshot before this is wired into the capture pipeline, so it ships pure and
// unit-tested but unused by main.ts. It never drives any in-game action - it only
// reads state for triage.
export function lockIconCropRect(detailRect: LayoutRect, imageSize: { width: number; height: number }): LayoutRect {
return clampRect(
{
x: Math.round(detailRect.x + detailRect.width * 0.735),
y: Math.round(detailRect.y + detailRect.height * 0.355),
width: Math.round(detailRect.width * 0.105),
height: Math.round(detailRect.height * 0.07),
},
imageSize,
);
}
// A gold/highlighted lock pixel: red high, green mid-high, blue low.
function isLockGold(b: number, g: number, r: number): boolean {
return r >= 180 && g >= 140 && b <= 120 && r > b + 40 && g > b + 20;
}
export function lockSignalRatio(bitmap: Bitmap): number {
const { data, width, height } = bitmap;
const pixels = width * height;
if (pixels === 0) return 0;
let gold = 0;
for (let pixel = 0; pixel < pixels; pixel++) {
const index = pixel * 4;
if (isLockGold(data[index], data[index + 1], data[index + 2])) gold++;
}
return gold / pixels;
}
export const DEFAULT_LOCK_THRESHOLD = 0.06;
export function isLocked(signalRatio: number, threshold: number = DEFAULT_LOCK_THRESHOLD): boolean {
return signalRatio >= threshold;
}
export function detectLockState(bitmap: Bitmap, threshold: number = DEFAULT_LOCK_THRESHOLD): boolean {
return isLocked(lockSignalRatio(bitmap), threshold);
}
+74
View File
@@ -0,0 +1,74 @@
import { describe, expect, it } from "vitest";
import { binarizeForOcr, computeLuminanceHistogram, otsuThreshold, type Bitmap } from "./ocrPreprocess";
// Build a BGRA bitmap from a grid of [b,g,r] pixels.
function bitmapFrom(pixels: Array<[number, number, number]>, width: number, height: number): Bitmap {
const data = Buffer.alloc(width * height * 4);
pixels.forEach(([b, g, r], index) => {
data[index * 4] = b;
data[index * 4 + 1] = g;
data[index * 4 + 2] = r;
data[index * 4 + 3] = 255;
});
return { data, width, height };
}
describe("ocrPreprocess", () => {
it("computes a luminance histogram over all pixels", () => {
const bitmap = bitmapFrom([
[0, 0, 0],
[255, 255, 255],
[0, 0, 0],
[255, 255, 255],
], 2, 2);
const histogram = computeLuminanceHistogram(bitmap);
expect(histogram[0]).toBe(2);
expect(histogram[255]).toBe(2);
expect(histogram.reduce((sum, count) => sum + count, 0)).toBe(4);
});
it("otsu splits a clean bimodal image between the two peaks", () => {
const histogram = new Array<number>(256).fill(0);
histogram[20] = 50;
histogram[220] = 50;
const threshold = otsuThreshold(histogram);
expect(threshold).toBeGreaterThanOrEqual(20);
expect(threshold).toBeLessThan(220);
});
it("otsu is safe on an empty histogram", () => {
expect(otsuThreshold(new Array<number>(256).fill(0))).toBe(127);
});
it("inverts bright foreground to black-on-white by default", () => {
// Bright text pixel + dark background pixel.
const bitmap = bitmapFrom([
[255, 255, 255], // bright -> should become black
[0, 0, 0], // dark -> should become white
], 2, 1);
const out = binarizeForOcr(bitmap, { threshold: 128 });
expect([out.data[0], out.data[1], out.data[2]]).toEqual([0, 0, 0]);
expect([out.data[4], out.data[5], out.data[6]]).toEqual([255, 255, 255]);
expect(out.data[3]).toBe(255);
});
it("keeps bright foreground white when inversion is disabled", () => {
const bitmap = bitmapFrom([
[255, 255, 255],
[0, 0, 0],
], 2, 1);
const out = binarizeForOcr(bitmap, { threshold: 128, invertBrightForeground: false });
expect(out.data[0]).toBe(255);
expect(out.data[4]).toBe(0);
});
it("preserves dimensions and always emits opaque pixels", () => {
const bitmap = bitmapFrom(Array.from({ length: 9 }, () => [100, 100, 100] as [number, number, number]), 3, 3);
const out = binarizeForOcr(bitmap);
expect(out.width).toBe(3);
expect(out.height).toBe(3);
for (let pixel = 0; pixel < 9; pixel++) {
expect(out.data[pixel * 4 + 3]).toBe(255);
}
});
});
+101
View File
@@ -0,0 +1,101 @@
// OCR preprocessing for artifact crops (ADR-009). Tesseract reads a clean, high
// contrast, dark-text-on-light image far more reliably than Genshin's native
// bright-text-on-dark UI. This binarizes a crop with Otsu thresholding and (by
// default) inverts, because artifact text is the bright foreground.
//
// Works on a raw BGRA bitmap (Electron NativeImage.getBitmap() layout on
// Windows). Kept pure and channel-order-agnostic for luminance so it is unit
// testable without Electron. Upscaling is done separately via NativeImage.resize
// before this runs - interpolated upscaling of small crops is a big Tesseract win
// and NativeImage does it better than hand-rolled JS.
export interface Bitmap {
data: Uint8Array | Buffer;
width: number;
height: number;
}
export interface BinarizeOptions {
/** Artifact text is the bright foreground, so invert to dark-on-light. */
invertBrightForeground?: boolean;
/** Override Otsu with a fixed 0-255 luminance threshold. */
threshold?: number;
}
const BYTES_PER_PIXEL = 4;
// Rec. 601 luma. Channel order does not matter for a weighted sum as long as we
// read the same three bytes; BGRA and RGBA give the same luminance here because
// we weight by position-independent coefficients applied to the actual R/G/B.
function luminanceAt(data: Uint8Array | Buffer, index: number): number {
// NativeImage on Windows is BGRA: byte0=B, byte1=G, byte2=R.
const b = data[index];
const g = data[index + 1];
const r = data[index + 2];
return 0.299 * r + 0.587 * g + 0.114 * b;
}
export function computeLuminanceHistogram(bitmap: Bitmap): number[] {
const histogram = new Array<number>(256).fill(0);
const { data, width, height } = bitmap;
const pixels = width * height;
for (let pixel = 0; pixel < pixels; pixel++) {
const value = Math.round(luminanceAt(data, pixel * BYTES_PER_PIXEL));
histogram[Math.max(0, Math.min(255, value))]++;
}
return histogram;
}
// Otsu's method: pick the threshold that maximizes between-class variance.
export function otsuThreshold(histogram: readonly number[]): number {
const total = histogram.reduce((sum, count) => sum + count, 0);
if (total === 0) return 127;
let sumAll = 0;
for (let level = 0; level < 256; level++) sumAll += level * histogram[level];
let sumBackground = 0;
let weightBackground = 0;
let maxVariance = -1;
let threshold = 127;
for (let level = 0; level < 256; level++) {
weightBackground += histogram[level];
if (weightBackground === 0) continue;
const weightForeground = total - weightBackground;
if (weightForeground === 0) break;
sumBackground += level * histogram[level];
const meanBackground = sumBackground / weightBackground;
const meanForeground = (sumAll - sumBackground) / weightForeground;
const betweenVariance = weightBackground * weightForeground * (meanBackground - meanForeground) ** 2;
if (betweenVariance > maxVariance) {
maxVariance = betweenVariance;
threshold = level;
}
}
return threshold;
}
export function binarizeForOcr(bitmap: Bitmap, options: BinarizeOptions = {}): Bitmap {
const { data, width, height } = bitmap;
const invert = options.invertBrightForeground ?? true;
const threshold = options.threshold ?? otsuThreshold(computeLuminanceHistogram(bitmap));
const output = Buffer.alloc(width * height * BYTES_PER_PIXEL);
const pixels = width * height;
for (let pixel = 0; pixel < pixels; pixel++) {
const index = pixel * BYTES_PER_PIXEL;
const isBright = luminanceAt(data, index) > threshold;
// Bright foreground text -> black; dark background -> white (inverted).
const value = invert ? (isBright ? 0 : 255) : (isBright ? 255 : 0);
output[index] = value;
output[index + 1] = value;
output[index + 2] = value;
output[index + 3] = 255;
}
return { data: output, width, height };
}
+5
View File
@@ -1,5 +1,6 @@
import type { ParsedArtifactCandidate } from "./artifactOcrParser";
import { simplifyForMatch } from "./fuzzyMatch";
import { implausibleSubstats } from "./substatRolls";
import type { CaptureResult, ReviewSampleRecord } from "../types/global";
import type { ScannerLearningRulePayload } from "../types/global";
@@ -73,6 +74,10 @@ export function shouldFlagArtifactForReview(
if (substatCount === 0) return true;
if (substatCount < 3 && substatConfidence < 70) return true;
// A substat value that matches no legal roll combination is a guaranteed OCR
// misread - never store it as fact, always review.
if (implausibleSubstats(parsed.substats ?? []).length > 0) return true;
return false;
}
+8 -1
View File
@@ -23,7 +23,7 @@ function record(overrides: Partial<StoredArtifactRecord> = {}): StoredArtifactRe
describe("storedArtifactAdapter", () => {
it("converts stored OCR artifacts into recommendation-domain artifacts", () => {
const [artifact] = storedArtifactsToDomain([record()]);
const [artifact] = storedArtifactsToDomain([record({ locked: true })]);
expect(artifact.slot).toBe("sands");
expect(artifact.setKey).toBe("viridescent_venerer");
@@ -32,6 +32,13 @@ describe("storedArtifactAdapter", () => {
expect(artifact.equipped).toBe("Sucrose");
expect(artifact.confidence).toBe(0.96);
expect(artifact.source).toBe("screen");
expect(artifact.locked).toBe(true);
});
it("does not invent lock state from confidence", () => {
const [artifact] = storedArtifactsToDomain([record({ locked: undefined, confidence: 100 })]);
expect(artifact.locked).toBe(false);
});
it("keeps flat and percent ATK substats distinct", () => {
+1 -1
View File
@@ -54,7 +54,7 @@ export function storedArtifactsToDomain(records: StoredArtifactRecord[]): Artifa
mainStat: normalizeMainStat(record.mainStat, record.mainValue),
substats: record.substats.map(parseStoredSubstat).filter(Boolean) as ArtifactSubstat[],
equipped: isUsefulEquippedName(record.equipped) ? record.equipped.trim() : undefined,
locked: !record.needsReview && record.confidence >= 90,
locked: Boolean(record.locked),
source: toSource(record.source),
confidence: Math.max(0, Math.min(1, record.confidence / 100)),
lastSeenAt: record.lastSeenAt ?? record.firstSeenAt ?? now,
+72
View File
@@ -0,0 +1,72 @@
import { describe, expect, it } from "vitest";
import {
implausibleSubstats,
inferRarity,
isPlausibleSubstat,
isPlausibleSubstatValue,
parseSubstatEntry,
} from "./substatRolls";
describe("substatRolls parsing", () => {
it("parses percent and flat entries", () => {
expect(parseSubstatEntry("CRIT DMG+13.2%")).toEqual({ stat: "CRIT DMG", value: 13.2, percent: true });
expect(parseSubstatEntry("HP+1,509")).toEqual({ stat: "HP", value: 1509, percent: false });
expect(parseSubstatEntry("garbage")).toBeNull();
});
});
describe("substatRolls plausibility", () => {
it("accepts real single-roll values (5-star)", () => {
expect(isPlausibleSubstatValue("CRIT DMG", 7.8)).toBe(true); // 7.77 high roll
expect(isPlausibleSubstatValue("CRIT DMG", 5.4)).toBe(true); // 5.44 low roll
expect(isPlausibleSubstatValue("CRIT Rate", 3.9)).toBe(true); // 3.89
expect(isPlausibleSubstatValue("ATK", 19)).toBe(true); // 19.45
expect(isPlausibleSubstatValue("HP", 269)).toBe(true); // 268.88
});
it("accepts multi-roll sums", () => {
expect(isPlausibleSubstat("CRIT DMG+13.2%")).toBe(true); // 5.44+7.77 or 6.22+6.99
expect(isPlausibleSubstat("Elemental Mastery+68")).toBe(true);
expect(isPlausibleSubstat("Energy Recharge+11.7%")).toBe(true);
});
it("rejects values that fit no roll combination at either rarity (OCR misreads)", () => {
// 8.1% CRIT DMG sits in the gap: single roll maxes at 7.8 (5-star), and the
// smallest two-roll sum is 8.2 (4-star), so it is unreachable at both.
expect(isPlausibleSubstatValue("CRIT DMG", 8.1)).toBe(false);
// 3.5% CRIT DMG is below the lowest possible roll at either rarity.
expect(isPlausibleSubstatValue("CRIT DMG", 3.5)).toBe(false);
// A dropped digit on flat ATK.
expect(isPlausibleSubstatValue("ATK", 5)).toBe(false);
});
it("does not flag unknown stats", () => {
expect(isPlausibleSubstatValue("Mystery Stat", 12.3)).toBe(true);
});
it("collects the implausible entries from a substat list", () => {
const bad = implausibleSubstats(["CRIT DMG+13.2%", "CRIT DMG+8.1%", "ATK+19"]);
expect(bad).toEqual(["CRIT DMG+8.1%"]);
});
});
describe("substatRolls rarity inference", () => {
it("is 5-star for any artifact leveled past +16", () => {
expect(inferRarity(20, ["ATK%+3.5%"])).toBe(5);
expect(inferRarity(17, [])).toBe(5);
});
it("detects 5-star from a substat that only fits the 5-star table", () => {
// 7.8% CRIT DMG is a 5-star single high roll; not reachable at 4-star.
expect(inferRarity(12, ["CRIT DMG+7.8%"])).toBe(5);
});
it("detects 4-star from a substat that only fits the 4-star table", () => {
// 4.1% CRIT DMG is a 4-star single low roll; below any 5-star roll.
expect(inferRarity(12, ["CRIT DMG+4.1%"])).toBe(4);
});
it("defaults to 5-star when ambiguous", () => {
expect(inferRarity(8, [])).toBe(5);
});
});
+139
View File
@@ -0,0 +1,139 @@
// Substat roll validation (the accuracy trick yas / Genshin Optimizer use).
// Every artifact substat value is the SUM of discrete per-roll increments: a
// substat rolls once when it appears and again at every +4 level, up to 6 rolls
// total. So a displayed value is only legitimate if it equals round(sum of N
// rolls) for some N in 1..6 from that stat's roll table. An OCR value that fits
// no combination is a misread (e.g. a dropped/extra digit) and should go to
// review instead of being stored as fact.
//
// Roll tables are keyed by the app's display stat names. 5-star values are the
// full set; 4-star values are included for the stats used to tell rarities
// apart. Flat 4-star tables are intentionally omitted (kept conservative).
const MAX_ROLLS = 6;
const ROLLS_5STAR: Record<string, number[]> = {
HP: [209.13, 239.0, 268.88, 298.75],
ATK: [13.62, 15.56, 17.51, 19.45],
DEF: [16.2, 18.52, 20.83, 23.15],
"HP%": [4.08, 4.66, 5.25, 5.83],
"ATK%": [4.08, 4.66, 5.25, 5.83],
"DEF%": [5.1, 5.83, 6.56, 7.29],
"Elemental Mastery": [16.32, 18.65, 20.98, 23.31],
"Energy Recharge": [4.53, 5.18, 5.83, 6.48],
"CRIT Rate": [2.72, 3.11, 3.5, 3.89],
"CRIT DMG": [5.44, 6.22, 6.99, 7.77],
};
const ROLLS_4STAR: Record<string, number[]> = {
"HP%": [3.06, 3.5, 3.93, 4.37],
"ATK%": [3.06, 3.5, 3.93, 4.37],
"DEF%": [3.83, 4.37, 4.92, 5.47],
"Elemental Mastery": [12.25, 13.99, 15.74, 17.48],
"Energy Recharge": [3.4, 3.89, 4.37, 4.86],
"CRIT Rate": [2.04, 2.33, 2.62, 2.91],
"CRIT DMG": [4.08, 4.66, 5.25, 5.83],
};
const PERCENT_STATS = new Set([
"HP%",
"ATK%",
"DEF%",
"Energy Recharge",
"CRIT Rate",
"CRIT DMG",
]);
function isPercent(stat: string) {
return PERCENT_STATS.has(stat);
}
function displayRound(value: number, percent: boolean) {
return percent ? Math.round(value * 10) / 10 : Math.round(value);
}
// All displayed values reachable by summing 1..MAX_ROLLS rolls from `rolls`.
function buildValidSet(rolls: number[], percent: boolean): Set<number> {
const results = new Set<number>();
let sums = new Set<number>([0]);
for (let n = 1; n <= MAX_ROLLS; n++) {
const next = new Set<number>();
for (const sum of sums) {
for (const roll of rolls) next.add(Math.round((sum + roll) * 1000) / 1000);
}
sums = next;
for (const sum of sums) results.add(displayRound(sum, percent));
}
return results;
}
const VALID_CACHE = new Map<string, Set<number>>();
function validSet(stat: string, table: Record<string, number[]>, tag: string): Set<number> | null {
const rolls = table[stat];
if (!rolls) return null;
const key = `${tag}:${stat}`;
let cached = VALID_CACHE.get(key);
if (!cached) {
cached = buildValidSet(rolls, isPercent(stat));
VALID_CACHE.set(key, cached);
}
return cached;
}
export function parseSubstatEntry(entry: string): { stat: string; value: number; percent: boolean } | null {
const plusIndex = entry.indexOf("+");
if (plusIndex <= 0) return null;
const stat = entry.slice(0, plusIndex).trim();
const raw = entry.slice(plusIndex + 1).replace(/,/g, "").replace("%", "").trim();
const value = Number.parseFloat(raw);
if (!Number.isFinite(value)) return null;
return { stat, value, percent: entry.includes("%") };
}
/** A displayed value is plausible if it matches a roll sum for 5-star or 4-star. */
export function isPlausibleSubstatValue(stat: string, value: number): boolean {
const rounded = displayRound(value, isPercent(stat));
const five = validSet(stat, ROLLS_5STAR, "5");
const four = validSet(stat, ROLLS_4STAR, "4");
// Unknown stat (no table) -> do not claim it is implausible.
if (!five && !four) return true;
return Boolean(five?.has(rounded)) || Boolean(four?.has(rounded));
}
export function isPlausibleSubstat(entry: string): boolean {
const parsed = parseSubstatEntry(entry);
if (!parsed) return true;
return isPlausibleSubstatValue(parsed.stat, parsed.value);
}
/** The substat entries whose value fits no legal roll combination. */
export function implausibleSubstats(substats: readonly string[]): string[] {
return substats.filter((entry) => !isPlausibleSubstat(entry));
}
/**
* Best-effort rarity from level + which roll table the substats fit. Conservative:
* only returns 4 when a substat clearly fits the 4-star table and not 5-star;
* otherwise defaults to 5 (the common case and the previous hardcoded value).
*/
export function inferRarity(level: number, substats: readonly string[]): number {
if (level > 16) return 5; // only 5-star artifacts level past +16
let fitsFiveOnly = 0;
let fitsFourOnly = 0;
for (const entry of substats) {
const parsed = parseSubstatEntry(entry);
if (!parsed) continue;
const rounded = displayRound(parsed.value, isPercent(parsed.stat));
const five = validSet(parsed.stat, ROLLS_5STAR, "5");
const four = validSet(parsed.stat, ROLLS_4STAR, "4");
if (!five || !four) continue;
const inFive = five.has(rounded);
const inFour = four.has(rounded);
if (inFive && !inFour) fitsFiveOnly++;
else if (inFour && !inFive) fitsFourOnly++;
}
if (fitsFiveOnly > 0) return 5;
if (fitsFourOnly > 0) return 4;
return 5;
}
+5 -2
View File
@@ -53,9 +53,12 @@ export function AppPageLayout({ controller }: AppPageLayoutProps) {
overlayIcon={<Eye size={16} />}
demoIcon={<Play size={16} />}
/>
<AppMetrics metricCards={metricCards} />
{activeView === "scan" && (
{activeView !== "diagnose" && <AppMetrics metricCards={metricCards} />}
{(activeView === "scan" || activeView === "diagnose") && (
<ScanView
mode={activeView === "diagnose" ? "diagnose" : "workspace"}
onDemoScan={handleDemoScan}
canDemoScan={!isScanning}
isScanning={isScanning}
snapshot={snapshot}
captureSources={captureSources}
+8 -1
View File
@@ -15,6 +15,8 @@ import type {
SaveScannerLearningRulesResult,
SaveSnapshotResult,
GoodDatabase,
GoodImportFileResult,
ScannerCommand,
ScannerStatusPayload,
ScannerLearningRulePayload,
} from "../types/global";
@@ -39,6 +41,7 @@ export interface AssistantBridge {
options?: CaptureOptions,
) => Promise<CaptureResult>;
exportGood: (payload: GoodDatabase) => Promise<SaveResultWithPath>;
importGoodFile: () => Promise<GoodImportFileResult>;
showOverlay: () => Promise<BooleanResult>;
loadArtifacts: () => Promise<ArtifactStoreLoadResult>;
saveArtifacts: (records: StoredArtifactRecord[]) => Promise<ArtifactStoreSaveResult>;
@@ -51,9 +54,10 @@ export interface AssistantBridge {
publishScannerStatus: (status: ScannerStatusPayload) => Promise<BooleanResult>;
focusMainWindow: () => Promise<BooleanResult>;
focusGenshin: () => Promise<FocusGenshinResult>;
focusGenshinForScanStart: () => Promise<FocusGenshinResult>;
clickScreen: (x: number, y: number) => Promise<ClickResult>;
scrollScreen: (notches: number, anchorX?: number, anchorY?: number) => Promise<ScrollResult>;
onScannerCommand: (callback: (command: "start-auto" | "stop") => void) => () => void;
onScannerCommand: (callback: (command: ScannerCommand) => void) => () => void;
}
function hasFunction(api: Record<string, unknown>, key: string): boolean {
@@ -66,6 +70,7 @@ export function getAssistantBridge(): AssistantBridge | null {
const apiRecord = api as unknown as Record<string, unknown>;
const canAutoScan = hasFunction(apiRecord, "clickScreen") && hasFunction(apiRecord, "scrollScreen");
const canReviewSamples = hasFunction(apiRecord, "loadReviewSamples") && hasFunction(apiRecord, "saveReviewSample");
const hasFocusGenshinForScanStart = hasFunction(apiRecord, "focusGenshinForScanStart");
return {
isAvailable: true,
@@ -79,6 +84,7 @@ export function getAssistantBridge(): AssistantBridge | null {
listCaptureSources: () => api.listCaptureSources(),
captureSource: (sourceId, delayMs, focusGenshin, options) => api.captureSource(sourceId, delayMs, focusGenshin, options),
exportGood: (payload) => api.exportGood(payload),
importGoodFile: () => api.importGoodFile(),
loadArtifacts: () => api.loadArtifacts(),
saveArtifacts: (records) => api.saveArtifacts(records),
loadReviewSamples: (limit = 50) => api.loadReviewSamples(limit),
@@ -90,6 +96,7 @@ export function getAssistantBridge(): AssistantBridge | null {
publishScannerStatus: (status) => api.publishScannerStatus(status),
focusMainWindow: () => api.focusMainWindow(),
focusGenshin: () => api.focusGenshin(),
focusGenshinForScanStart: () => (hasFocusGenshinForScanStart ? api.focusGenshinForScanStart() : api.focusGenshin()),
clickScreen: (x: number, y: number) => api.clickScreen(x, y),
scrollScreen: (notches: number, anchorX?: number, anchorY?: number) => api.scrollScreen(notches, anchorX, anchorY),
showOverlay: () => api.showOverlay(),
+54
View File
@@ -2182,3 +2182,57 @@ button:disabled {
}
}
/* Diagnose / Dev view — all developer info, separated from the Scan workspace. */
.diagnose-view {
display: grid;
gap: 14px;
width: 100%;
}
.diagnose-header {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 12px;
flex-wrap: wrap;
}
.diagnose-header-actions {
display: flex;
gap: 8px;
flex-wrap: wrap;
}
.diagnose-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 14px;
}
.diagnose-card {
display: grid;
gap: 10px;
align-content: start;
border: 1px solid var(--line);
border-radius: 8px;
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.05), rgba(255, 255, 255, 0.015)),
rgba(18, 12, 35, 0.7);
box-shadow: var(--glass-shadow);
backdrop-filter: blur(18px);
padding: 16px;
}
.diagnose-card-heading {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
@media (max-width: 1200px) {
.diagnose-grid {
grid-template-columns: 1fr;
}
}
+27 -1
View File
@@ -56,6 +56,12 @@ export interface CaptureResult {
source: "ocr" | "missing";
text: string;
};
locked?: boolean;
layout?: {
aspect: string;
isSixteenNine: boolean;
warning: string;
};
}
export interface WindowBounds {
@@ -120,6 +126,14 @@ export interface SaveResultWithPath {
export type SaveSnapshotResult = SaveResultWithPath;
export type ScannerCommand =
| "start-auto"
| "stop"
| {
type: "start-auto";
scanLimit?: number;
};
export interface ScannerLearningRulePayload {
textReplacements?: Record<string, string>;
}
@@ -206,6 +220,7 @@ export interface ReviewSampleRecord {
ocr?: OcrResult[];
inventoryGrid?: CaptureResult["inventoryGrid"];
inventoryCount?: CaptureResult["inventoryCount"];
locked?: boolean;
};
};
}
@@ -234,6 +249,7 @@ export interface ReviewSamplePayload {
ocr?: OcrResult[];
inventoryGrid?: CaptureResult["inventoryGrid"];
inventoryCount?: CaptureResult["inventoryCount"];
locked?: boolean;
};
[key: string]: unknown;
}
@@ -275,6 +291,14 @@ export interface GoodDatabase {
artifacts: GoodExportArtifact[];
}
export interface GoodImportFileResult {
ok: boolean;
canceled: boolean;
path: string;
database?: unknown;
error?: string;
}
declare global {
interface Window {
assistantApi?: {
@@ -288,6 +312,7 @@ declare global {
getAutomationGuard: () => Promise<AutomationGuard>;
focusMainWindow: () => Promise<BooleanResult>;
focusGenshin: () => Promise<FocusGenshinResult>;
focusGenshinForScanStart: () => Promise<FocusGenshinResult>;
getRuntimeInfo: () => Promise<RuntimeInfo>;
saveReviewSample: (sample: ReviewSamplePayload) => Promise<SaveResultWithPath>;
loadReviewSamples: (limit?: number) => Promise<ReviewSampleListResult>;
@@ -296,10 +321,11 @@ declare global {
loadArtifacts: () => Promise<ArtifactStoreLoadResult>;
saveArtifacts: (records: StoredArtifactRecord[]) => Promise<ArtifactStoreSaveResult>;
exportGood: (payload: GoodDatabase) => Promise<SaveResultWithPath>;
importGoodFile: () => Promise<GoodImportFileResult>;
publishScannerStatus: (status: ScannerStatusPayload) => Promise<BooleanResult>;
showOverlay: () => Promise<BooleanResult>;
hideOverlay: () => Promise<BooleanResult>;
onScannerCommand: (callback: (command: "start-auto" | "stop") => void) => () => void;
onScannerCommand: (callback: (command: ScannerCommand) => void) => () => void;
};
}
}
+1
View File
@@ -10,6 +10,7 @@ export interface StoredArtifactRecord {
equipped: string;
confidence: number;
needsReview: boolean;
locked?: boolean;
source: string;
firstSeenAt?: string;
lastSeenAt?: string;