genshin-assistant/docs/AUTOMATION_LIVE_SCAN.md

Automation Live Scan Runbook

This document is the durable reference for automatic artifact scanning, mouse movement, click input, elevation, and live validation status.

Current Known-Good State

Validated live on 2026-07-07 with Genshin open in the artifact inventory at 1920x1080, English UI:

• npm run dev:admin starts the app elevated after the user confirms UAC.
• Runtime status reported isElevated: true, genshinFound: true, and targetProcess: "GenshinImpact".
• The safe probe endpoint /automation/probe-click?index=1 focused Genshin, moved the cursor to the second visible inventory tile, clicked it, and changed the artifact detail panel fingerprint.
• Probe result: clicked: true, inputBlocked: false, foregroundProcess: "GenshinImpact", and changed: true.
• A bounded live auto-scan via /scanner/start?limit=2 completed with: clicked: 2, attempted: 2, verified: 2, parsed: 2, stored: 2, review: 2, misses: 0, status: "done".

This proves that the current elevated app plus helper path can deliver mouse movement and click input to the focused Genshin client in this environment.

Elevation And UAC

Use:

npm run dev:admin

The command runs scripts/dev-admin.ps1, which launches a new elevated PowerShell window running scripts/dev-admin-start.ps1. The elevated start is logged to:

outputs/admin-start/admin-dev.log

The user must confirm the Windows UAC prompt. The app cannot and must not click the Secure Desktop UAC prompt for itself. After confirmation, the app can verify its own runtime through the dev status endpoint.

Useful checks:

Invoke-RestMethod http://127.0.0.1:17317/health
Invoke-RestMethod http://127.0.0.1:17317/scanner/status

Expected runtime facts before automatic scan:

• isElevated: true
• genshinFound: true
• targetProcess: "GenshinImpact"
• hotkeys registered

Mouse And Click Validation

Use the probe before broad auto-scan work:

Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1" |
  ConvertTo-Json -Depth 12

The probe performs one read-only inventory selection click. It does not delete, feed, enhance, lock, unlock, spend, or modify game resources.

Interpretation:

• click.ok: true, clicked: true, inputBlocked: false means Windows did not block SendInput/UIPI in the current configuration.
• focused: true and foregroundProcess: "GenshinImpact" means the click was sent while Genshin was foreground.
• changed: true means the detail panel changed after the click.
• changed: false can be benign if the target tile was already selected or two neighboring artifacts render identically; retry with another index, row, or col.

Examples:

# Second visible tile
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1"

# Specific grid cell
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?row=0&col=3"

Bounded Live Auto-Scan

For live validation, prefer a bounded scan first:

Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?limit=2"

Then poll:

Invoke-RestMethod "http://127.0.0.1:17317/scanner/status" |
  ConvertTo-Json -Depth 12

The /scanner/start?limit=N endpoint sends a renderer command payload with a temporary scan limit. It does not change the normal UI setting. The normal hotkeys and buttons still use the UI's configured scan limit.

Anti-Cheat And Safety Boundary

Do not describe the current implementation as bypassing anti-cheat. The app does not read memory, hook the process, inject code, modify game files, inspect packets, or interact with kernel drivers. It uses normal Windows screen capture, focus, cursor movement, wheel, and click input.

The practical finding is narrower:

• A non-elevated app can be blocked by Windows integrity/UIPI when the target process is elevated or protected.
• Running the app elevated fixed input delivery in the tested environment.
• Genshin's anti-cheat may still affect behavior on other machines, game modes, overlays, or future versions. Re-run the probe before trusting broad scans.

Never add automation that deletes, feeds, enhances, locks/unlocks, spends resources, reads memory, hooks, injects, or modifies Genshin.

Live Layout Facts

The current 16:9 layout profile is calibrated from a 1920x1080 English artifact-inventory capture:

• detail rect approximately x=1308, y=120, width=492, height=838
• inventory grid: 8 x 5
• first tile center: x=179, y=254, row=0, col=0
• second tile center: x=325, y=254, row=0, col=1
• inventory count crop successfully read 2059/2400 in the live session

The profile is resolution-scaled for 16:9. Off-profile setups should be treated as higher risk and validated with Smart Capture plus the probe.

Validation Checklist

Before marking an automation change done:

1. Run npm run lint.
2. Run npx tsc -p tsconfig.electron.json when Electron/preload/main changed.
3. Run npm test.
4. Run npm run build.
5. If Genshin is available, run /automation/probe-click?index=1.
6. For scan-loop changes, run /scanner/start?limit=2 before any broader scan.
7. Record new live findings in this file and in docs/scanner-rework-status.md.