feat(scanner): validate elevated live automation

2026-07-07 07:49:22 +02:00
parent 7930e369a7
commit ef65c3e6a0
37 changed files with 826 additions and 217 deletions
@@ -0,0 +1,154 @@
+# Automation Live Scan Runbook
+
+This document is the durable reference for automatic artifact scanning, mouse
+movement, click input, elevation, and live validation status.
+
+## Current Known-Good State
+
+Validated live on 2026-07-07 with Genshin open in the artifact inventory at
+1920x1080, English UI:
+
+- `npm run dev:admin` starts the app elevated after the user confirms UAC.
+- Runtime status reported `isElevated: true`, `genshinFound: true`, and
+  `targetProcess: "GenshinImpact"`.
+- The safe probe endpoint `/automation/probe-click?index=1` focused Genshin,
+  moved the cursor to the second visible inventory tile, clicked it, and changed
+  the artifact detail panel fingerprint.
+- Probe result: `clicked: true`, `inputBlocked: false`,
+  `foregroundProcess: "GenshinImpact"`, and `changed: true`.
+- A bounded live auto-scan via `/scanner/start?limit=2` completed with:
+  `clicked: 2`, `attempted: 2`, `verified: 2`, `parsed: 2`, `stored: 2`,
+  `review: 2`, `misses: 0`, `status: "done"`.
+
+This proves that the current elevated app plus helper path can deliver mouse
+movement and click input to the focused Genshin client in this environment.
+
+## Elevation And UAC
+
+Use:
+
+```powershell
+npm run dev:admin
+```
+
+The command runs `scripts/dev-admin.ps1`, which launches a new elevated
+PowerShell window running `scripts/dev-admin-start.ps1`. The elevated start is
+logged to:
+
+```text
+outputs/admin-start/admin-dev.log
+```
+
+The user must confirm the Windows UAC prompt. The app cannot and must not click
+the Secure Desktop UAC prompt for itself. After confirmation, the app can verify
+its own runtime through the dev status endpoint.
+
+Useful checks:
+
+```powershell
+Invoke-RestMethod http://127.0.0.1:17317/health
+Invoke-RestMethod http://127.0.0.1:17317/scanner/status
+```
+
+Expected runtime facts before automatic scan:
+
+- `isElevated: true`
+- `genshinFound: true`
+- `targetProcess: "GenshinImpact"`
+- hotkeys registered
+
+## Mouse And Click Validation
+
+Use the probe before broad auto-scan work:
+
+```powershell
+Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1" |
+  ConvertTo-Json -Depth 12
+```
+
+The probe performs one read-only inventory selection click. It does not delete,
+feed, enhance, lock, unlock, spend, or modify game resources.
+
+Interpretation:
+
+- `click.ok: true`, `clicked: true`, `inputBlocked: false` means Windows did not
+  block SendInput/UIPI in the current configuration.
+- `focused: true` and `foregroundProcess: "GenshinImpact"` means the click was
+  sent while Genshin was foreground.
+- `changed: true` means the detail panel changed after the click.
+- `changed: false` can be benign if the target tile was already selected or two
+  neighboring artifacts render identically; retry with another `index`, `row`,
+  or `col`.
+
+Examples:
+
+```powershell
+# Second visible tile
+Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1"
+
+# Specific grid cell
+Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?row=0&col=3"
+```
+
+## Bounded Live Auto-Scan
+
+For live validation, prefer a bounded scan first:
+
+```powershell
+Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?limit=2"
+```
+
+Then poll:
+
+```powershell
+Invoke-RestMethod "http://127.0.0.1:17317/scanner/status" |
+  ConvertTo-Json -Depth 12
+```
+
+The `/scanner/start?limit=N` endpoint sends a renderer command payload with a
+temporary scan limit. It does not change the normal UI setting. The normal
+hotkeys and buttons still use the UI's configured scan limit.
+
+## Anti-Cheat And Safety Boundary
+
+Do not describe the current implementation as bypassing anti-cheat. The app
+does not read memory, hook the process, inject code, modify game files, inspect
+packets, or interact with kernel drivers. It uses normal Windows screen capture,
+focus, cursor movement, wheel, and click input.
+
+The practical finding is narrower:
+
+- A non-elevated app can be blocked by Windows integrity/UIPI when the target
+  process is elevated or protected.
+- Running the app elevated fixed input delivery in the tested environment.
+- Genshin's anti-cheat may still affect behavior on other machines, game modes,
+  overlays, or future versions. Re-run the probe before trusting broad scans.
+
+Never add automation that deletes, feeds, enhances, locks/unlocks, spends
+resources, reads memory, hooks, injects, or modifies Genshin.
+
+## Live Layout Facts
+
+The current 16:9 layout profile is calibrated from a 1920x1080 English
+artifact-inventory capture:
+
+- detail rect approximately `x=1308`, `y=120`, `width=492`, `height=838`
+- inventory grid: `8 x 5`
+- first tile center: `x=179`, `y=254`, `row=0`, `col=0`
+- second tile center: `x=325`, `y=254`, `row=0`, `col=1`
+- inventory count crop successfully read `2059/2400` in the live session
+
+The profile is resolution-scaled for 16:9. Off-profile setups should be treated
+as higher risk and validated with Smart Capture plus the probe.
+
+## Validation Checklist
+
+Before marking an automation change done:
+
+1. Run `npm run lint`.
+2. Run `npx tsc -p tsconfig.electron.json` when Electron/preload/main changed.
+3. Run `npm test`.
+4. Run `npm run build`.
+5. If Genshin is available, run `/automation/probe-click?index=1`.
+6. For scan-loop changes, run `/scanner/start?limit=2` before any broader scan.
+7. Record new live findings in this file and in `docs/scanner-rework-status.md`.