Files
genshin-assistant/docs/AUTOMATION_LIVE_SCAN.md
T
2026-07-09 08:44:50 +02:00

29 KiB

Automation Live Scan Runbook

This document is the durable reference for automatic artifact scanning, mouse movement, click input, elevation, and live validation status.

Current Known-Good State

Validated live with Genshin open in the artifact inventory at 1920x1080, English UI:

  • npm run dev:admin starts the app elevated after the user confirms UAC.
  • Runtime status reported isElevated: true, genshinFound: true, and targetProcess: "GenshinImpact".
  • The safe probe endpoint /automation/probe-click?index=1 focused Genshin, moved the cursor to the second visible inventory tile, clicked it, and changed the artifact detail panel fingerprint.
  • Probe result: clicked: true, inputBlocked: false, foregroundProcess: "GenshinImpact", and changed: true.
  • A bounded live auto-scan via /scanner/start?limit=2 completed with: clicked: 2, attempted: 2, verified: 2, parsed: 2, stored: 2, review: 2, misses: 0, status: "done".
  • On 2026-07-08, a visible-inventory 50-artifact run completed with 50/50 parsed and stored, 0 review, 0 duplicates, and 0 misses. Throughput was still slow at 61765 ms elapsed (1235 ms/artifact).
  • On 2026-07-09, the current-engine visible-inventory path completed /scanner/start?entry=visible-inventory&limit=20&engine=current with 20/20 verified and parsed, 19 stored, 1 duplicate, 0 review, and 0 misses in 8047 ms elapsed (402 ms/artifact). A same-session artifact detail capture also persisted an equipped footer as equipped: "Citlali" and an unlocked grey lock as locked: false.

This proves that the current elevated app plus helper path can deliver mouse movement and click input to the focused Genshin client in this environment.

Latest-source timing is not proven while /health.appBuild.signature differs from the APP_RUNTIME_SIGNATURE in electron/main.ts, or after source changes that have not been loaded by a fresh elevated runtime. Restart the elevated app through npm run dev:admin and confirm UAC before collecting new 50/100 artifact evidence.

Elevation And UAC

Use:

npm run dev:admin

The command runs scripts/dev-admin.ps1, which launches a new elevated PowerShell window running scripts/dev-admin-start.ps1. The elevated start is logged to:

outputs/admin-start/admin-dev.log

The user must confirm the Windows UAC prompt. The app cannot and must not click the Secure Desktop UAC prompt for itself. After confirmation, the app can verify its own runtime through the dev status endpoint.

Useful checks:

Invoke-RestMethod http://127.0.0.1:17317/health
Invoke-RestMethod http://127.0.0.1:17317/scanner/status

Expected runtime facts before automatic scan:

  • isElevated: true
  • genshinFound: true
  • targetProcess: "GenshinImpact"
  • hotkeys registered

Mouse And Click Validation

Use the probe before broad auto-scan work:

Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1" |
  ConvertTo-Json -Depth 12

The probe performs one read-only inventory selection click. It does not delete, feed, enhance, lock, unlock, spend, or modify game resources.

Interpretation:

  • click.ok: true, clicked: true, inputBlocked: false means Windows did not block SendInput/UIPI in the current configuration.
  • focused: true and foregroundProcess: "GenshinImpact" means the click was sent while Genshin was foreground.
  • changed: true means the detail panel changed after the click.
  • changed: false can be benign if the target tile was already selected or two neighboring artifacts render identically; retry with another index, row, or col.

Examples:

# Second visible tile
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?index=1"

# Specific grid cell
Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?row=0&col=3"

Bounded Live Auto-Scan

For live validation, prefer a bounded scan first:

Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?entry=visible-inventory&limit=2"

The visible-inventory path is the merge-relevant safe path. It requires the Artifact inventory to already be open with a visible artifact detail card.

The normal Auto-Scan button uses a guided start. It first takes one lightweight preflight capture without OCR, full-frame payload, review scoring, or storing. If an artifact detail card is already visible, it starts the visible-inventory scan. Otherwise it blocks with an operator-facing status and asks the user to open the Artifact inventory with a visible detail card. OCR/review/store work starts only after the artifact-detail preflight passes.

The explicit Dev-Control entry modes below remain available for targeted experiments only. They send read-only navigation, but they are not the merge-ready default because live testing showed that auto-entry can leave the app in the Paimon menu when the starting state is not what the choreography expects.

Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?entry=paimon-menu&limit=2"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?entry=auto-entry&limit=2"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?entry=visible-inventory&limit=2&engine=ik-traineddata"

Those paths send only read-only navigation. ESC is not a universal "go to world" command: from the world it opens the Paimon menu, while from the already-open Paimon menu it returns to the world. This is why the normal auto-entry path first tries B directly and uses the IK-style ESC -> B fallback only when direct entry did not reach an artifact detail card.

The scan starts only after a valid lookup package, supported 16:9 layout, detected artifact grid, Genshin-client capture, and visual artifact-detail markers are all present. If any preflight check fails, keep using the visible-inventory path while tuning the entry step.

The visual preflight also classifies the Paimon menu. The Paimon profile/card grid can look like an inventory grid if only fixed 16:9 coordinates are used, so the scanner must reject paimonMenu.present before any artifact OCR, review sample creation, store write, or grid scan starts. The guided entry may still take lightweight skip-OCR captures while navigating, but those captures are only state evidence.

The same guard also runs inside the scan loop. If the app is on the main game screen, a Paimon/menu screen, a generic primary-screen capture, or any screen without an artifact detail card, auto-scan must block instead of clicking tiles or trying OCR. After each click the loop now performs one fast artifact capture/OCR pass and uses that capture's detail fingerprint to verify that the selected artifact changed. This removes the old separate card-ready capture from the hot path. If the detail fingerprint is unchanged, the loop retries once and then follows the normal miss/block guards. The outer scan start focuses Genshin once; hot-loop fingerprint/OCR captures do not re-run the focus helper before every tile, which avoids an OS focus ping on each artifact while still relying on click readback, foreground checks, and the detail-card guard for safety. After a scroll, the loop now uses the same cheap fingerprint polling model for the inventory pane: it proceeds as soon as the next page fingerprint changed and stabilized instead of always sleeping the old fixed 760 ms settle delay. Changed but still animated inventory pages may proceed after 100 ms, again matching IK's fast-scroll wait while still blocking unchanged pages.

Then poll:

Invoke-RestMethod "http://127.0.0.1:17317/scanner/status" |
  ConvertTo-Json -Depth 12

Before live timing, verify that the endpoint is the current app instance:

Invoke-RestMethod "http://127.0.0.1:17317/health" |
  ConvertTo-Json -Depth 6

The response must include appBuild.signature and appBuild.expectedOcrWorkerPoolSize. If appBuild is missing, or /scanner/status still reports the old OCR warmup start time, the local port is still owned by a stale elevated Electron process. Close the old Administrator window/app and restart with npm run dev:admin before running scanner probes.

Use the status stats timing fields for IK comparisons: elapsedMs, activeScanMs, writeFlushMs, averageMsPerParsed, activeAverageMsPerParsed, averageCaptureMs, averageCaptureRoundTripMs, averageCaptureRoundTripOverheadMs, averageOcrMs, artifactsPerMinute, and projectedMsFor100. elapsedMs is end-to-end including queued writes; activeScanMs is the click/capture/OCR loop before the final store/review flush. A run only counts as speed evidence when parsed, stored, review, duplicates, and misses are read together; raw click count alone is not scanner throughput. If averageOcrMs dominates averageMsPerParsed, the next speed lever is an IK-style OCR worker queue. If averageCaptureRoundTripOverheadMs is high, native capture encode, Base64 transport, Electron image decode, or IPC/render scheduling is the next bottleneck. The current 3 artifacts/second target requires averageMsPerParsed at or below 333 ms on a clean 20-artifact iteration.

Latest live timing evidence on 2026-07-08:

  • Probe: /automation/probe-click?index=1 returned clicked: true, inputBlocked: false, changed: true, and captureTarget: "genshin-client".
  • Baseline after helper/hot-loop cleanup: /scanner/start?entry=visible-inventory&limit=50&engine=current completed 50/50 parsed and stored with 0 review, 0 duplicates, 0 misses, 2 pages, elapsedMs: 61765, averageMsPerParsed: 1235, averageCaptureMs: 186, averageOcrMs: 162, and averageScrollReadyMs: 844.
  • Deferred-write experiment: the same 50-artifact run completed 50/50 with 0 misses but regressed to elapsedMs: 63616 because 50 single-record writes produced writeFlushMs: 8163.
  • Current source replaces that experiment with batch persist and quiet auto-scan UI captures. This is code-validated, but the batch version still needs a fresh elevated live run; the follow-up restart was blocked because the admin runtime did not become reachable after shutdown/UAC.
  • Direct GDI hot-path validation: after skipping desktopCapturer.getSources() in auto-scan artifact captures, the 20-artifact iteration baseline improved to 20/20 parsed, 19 stored, 0 review, 1 duplicate, 0 misses, 7966 ms elapsed, 398 ms/artifact, averageCaptureMs: 193, averageOcrMs: 167, averageClickMs: 2, and writeFlushMs: 4. This is roughly 2.5 artifacts/second on the first visible page.
  • Scroll-path validation with the same direct GDI hot path: /scanner/start?entry=visible-inventory&limit=45&engine=current completed 45/45 parsed, 42 stored, 0 review, 3 duplicates, 0 misses, 2 pages, 18625 ms elapsed, 414 ms/artifact, averageCaptureMs: 187, averageOcrMs: 162, and one scroll readiness wait of 173 ms.
  • 100-artifact direct-GDI validation: /scanner/start?entry=visible-inventory&limit=100&engine=current completed on runtime signature 2026-07-08-direct-gdi-hotpath with 100/100 parsed, 97 stored, 0 review, 3 duplicates, 0 misses, 4 pages, 42064 ms elapsed, 421 ms/artifact, averageCaptureMs: 179, averageOcrMs: 154, averageClickMs: 2, writeFlushMs: 6, and 3 scroll readiness waits averaging 176 ms.
  • OCR/parser eval after this speed pass: npm run eval passed with 23/23 exact-match cases, 100% field accuracy, and 100% critical fields. This is a regression gate, not a substitute for manually checking live artifact values.
  • 3 artifacts/second preparation: auto-scan artifact captures now also omit the detail-preview payload and expose averageCaptureRoundTripMs plus averageCaptureRoundTripOverheadMs. The first live run exposed a false missing-crops-or-ocr review trigger because the hot path intentionally omits detailDataUrl; this is fixed in getAutoReviewReason.
  • 3 artifacts/second live attempts: after the review fix, a clean limit=20 run completed 20/20 parsed, 19 stored, 0 review, 1 duplicate, 0 misses, 7285 ms elapsed, or 364 ms/artifact (2.75 artifacts/second). The stable final run on signature 2026-07-08-direct-gdi-reviewfix completed 20/20, 18 stored, 0 review, 2 duplicates, 0 misses, 7973 ms elapsed, or 399 ms/artifact. 3 artifacts/second is not proven.
  • Rejected speed experiments: detail-region capture, GAA_OCR_WORKERS=5, DataURL-to-buffer decode, and substat OCR PSM.SINGLE_COLUMN were all live/benchmark tested and were slower than the direct-GDI baseline. Keep GAA_OCR_WORKERS=4 for current runs.
  • Quality-gated current-vs-IK comparison: npm run scan:goal:compare:validated produced outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json with createdAt: 2026-07-08T18:41:11.6120957+02:00. The final validator summary passed at limit=100 with winner current, activeAvg: 378 ms/artifact, projected100: 37800 ms, missRate: 0, and reviewRate: 0. The current 100-artifact run parsed 100/100, stored 97, had 0 review, 0 misses, and crossed 4 pages. The ik-traineddata 100-artifact run parsed 97/100, had 5 review and 3 misses, and was not qualified because it parsed fewer artifacts than requested.

The /scanner/start?limit=N endpoint sends a renderer command payload with a temporary scan limit. It does not change the normal UI setting. The normal hotkeys and buttons still use the UI's configured scan limit.

Lookup and benchmark utility endpoints:

Invoke-RestMethod "http://127.0.0.1:17317/scanner/lookup/status"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/lookup/regenerate"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/ocr/warmup"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/ocr/warmup?engine=ik-traineddata"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/benchmark-ocr?limit=5"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/benchmark-ocr?limit=5&engine=ik-traineddata"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/benchmark-ocr?limit=5&engine=compare"
Invoke-RestMethod "http://127.0.0.1:17317/scanner/benchmark-ocr?limit=5&profile=full"

The benchmark endpoint measures the current Tesseract.js engine and the Inventory-Kamera-traineddata Tesseract.js path against the artifact crop set and returns timing/field counts, min/p50/p90/max timing, OCR p50/p90 timing, 20/45/100-artifact projections, skipped-OCR count, and the active OCR worker pool size. It also returns per-field OCR timings under ocrFieldAverages, which is the first place to look before changing crop or parser behavior. Individual captures also report whether the artifact was detected as sanctified; level/substat crops are shifted in that state to match Inventory Kamera's crop model. By default it uses the auto-scan fast OCR profile, which omits the low-value set-effect crop and the main-stat-value crop that can be derived from slot, main-stat label, and level. The slot crop remains enabled in the fast profile because it improved live-read quality. The fast profile also uses Inventory Kamera's tighter substat crop height; full/manual captures keep the larger recovery crop for debugging difficult samples. Auto-scan also omits per-crop diagnostic Base64 images from hot-loop OCR captures while keeping the detail screenshot, OCR text, crop rect metadata, and timings. OCR crops are passed to Tesseract as PNG buffers internally, not as Base64 DataURLs, to avoid encode/decode overhead in batch scans. When skipOcrUnlessArtifactDetail blocks OCR because no artifact detail card is visible, OCR crop preprocessing is skipped too. Auto-scan readiness and scroll checks use native detail/inventory fingerprints and omit preview DataURLs in poll captures. Fast preflight/poll captures also omit crop list construction, crop images, and lock-state detection unless a caller explicitly overrides that option; add profile=full to OCR every artifact detail crop for debugging. It uses the same artifact-detail guard as auto-scan: if the current screen is not a confirmed artifact detail view, OCR is skipped and the response shows skippedOcrCaptures instead of burning time on invalid crops. The fast auto-scan profile now keeps the optional Equipped footer OCR on real artifact-read captures when the footer marker is visible, so stored artifacts can record the equipped character without requiring a separate manual capture. Name, level, main-stat label, footer, and substats remain in the OCR hot path; slot, set, and main-stat value are derived when the lookup/parser can validate them. Preflight and readiness poll captures still skip OCR/crops/lock-state work because they only prove surface and fingerprint changes. Local store/review writes are serialized through an internal queue but no longer block the next inventory click. The scan still flushes the queue before it returns its final summary, so stored and review counts remain final-state numbers. The app warms the default OCR worker pool in the background after startup; check /scanner/status -> ocrWarmup.current before timing the first artifact. Use /scanner/ocr/warmup?engine=ik-traineddata before comparing Inventory Kamera-traineddata timings so the benchmark is not dominated by worker creation. engine=ik-traineddata uses Inventory Kamera's local genshin_fast_09_04_21.traineddata through Tesseract.js when the file is found in data/tessdata, IK_TESSDATA_DIR, work/Inventory_Kamera, work/refs, or the local _ik_ref* folders. engine=compare runs current and ik-traineddata against the same visible artifact detail state. The auto-scan default must stay current until the IK traineddata path wins on the same captures. For a controlled live comparison, start the scanner with engine=ik-traineddata; this only changes the OCR worker language for that run and leaves the default UI/hotkey path on current. The OCR pool defaults to four workers because the fast artifact crop set has four useful OCR parameter groups; set GAA_OCR_WORKERS=1..8 before startup to benchmark a different worker count. Inventory Kamera's native engine pool is still the reference design, but the current app path remains Tesseract.js until native OCR is integrated and measured. Crops are scheduled across the whole worker pool and each worker caches its last Tesseract parameter profile; this is closer to Inventory Kamera's multi-engine field OCR than the earlier parameter-group-serial scheduler.

Diagnostic Evidence

The Diagnose page contains a compact evidence timeline for scanner work. It logs runtime pings, focus attempts, key presses, entry captures, artifact-tab clicks, preflight failures, grid/count metadata, detail fingerprints, and detail/inventory screenshots. The same last events are also published through:

Invoke-RestMethod "http://127.0.0.1:17317/scanner/status" |
  ConvertTo-Json -Depth 18

Use this before changing scanner behavior: run the smallest failing action, read the evidence timeline, then decide whether the failure is focus/input, entry navigation, grid detection, capture quality, OCR, or parser validation.

If Paimon entry shows entry key ESC or entry key B with eventsSent: 0, the running InputHelper.exe probably predates keyboard support or is blocked. Stop the elevated app/helper, run npm run helper:build, then restart with npm run dev:admin so the app loads the rebuilt helper.

Soak-Test Helper

After the elevated app is running and Genshin is open on the artifact inventory, the non-elevated terminal can drive the local dev-control endpoints and save a full evidence bundle:

npm run scan:soak

The helper writes timestamped JSON snapshots and a transcript to:

outputs/live-soak/<timestamp>/

Default sequence:

  1. /health
  2. /scanner/status
  3. /capture/smart?skipOcr=1
  4. /automation/probe-click?index=1
  5. /automation/probe-click?index=3
  6. /scanner/start?entry=visible-inventory&limit=2
  7. /scanner/start?entry=visible-inventory&limit=5
  8. /scanner/start?entry=visible-inventory&limit=10
  9. /scanner/start?entry=visible-inventory&limit=20
  10. /review/samples?limit=30

For the actual Inventory-Kamera speed target, use the explicit goal run after /health shows the current appBuild:

npm run scan:live:preflight
npm run scan:live:preflight:wait
npm run scan:goal
npm run scan:goal:current
npm run scan:goal:ik
npm run scan:iterate:compare:validated
npm run scan:iterate:compare:validated:wait
npm run scan:goal:compare
npm run scan:goal:compare:validated
npm run scan:goal:compare:validated:wait

scan:live:preflight checks /health, /scanner/status, the current APP_RUNTIME_SIGNATURE, elevation, and whether Genshin is visible to the helper before a long live scan is attempted. Use npm run scan:live:preflight:wait during manual startup after npm run dev:admin; it waits up to 120 seconds for the elevated dev-control server and runtime checks to become ready. The non-waiting command remains the default for validated scan chains so automation fails fast on a missing runtime.

Use npm run scan:iterate:compare:validated for fast iteration while tuning OCR, parser, capture, or readiness behavior. It runs the same preflight, compares current vs. ik-traineddata at limit=20, and validates the newest assessment with --limit=20 --summary. This is the preferred loop while debugging because it gives quality-gated feedback without waiting for the full 2, 5, 20, 45, 100 goal sequence. Use npm run scan:iterate:compare:validated:wait directly after UAC if the elevated runtime may still be starting.

The goal run first warms/benchmarks current vs. ik-traineddata, then scans limits 2, 5, 20, 45, 100 with the selected scan engine, and writes scan-run-summary.json plus scan-run-summary.csv. npm run scan:goal uses the default current scan engine; use scan:goal:ik for a native IK-traineddata scan pass. Use scan:goal:compare to run both scan engines back-to-back with the same limits and one combined CSV. The CSV is the quickest evidence for averageMsPerParsed, activeAverageMsPerParsed, averageCaptureMs, captureP50Ms, captureP90Ms, averageOcrMs, ocrP50Ms, ocrP90Ms, averageCardReadyMs, averageScrollReadyMs, artifactsPerMinute, and projectedMsFor100. The run also writes scan-performance-assessment.json, which groups results by limit, picks the best qualified engine, and labels the dominant bottleneck as OCR, capture, card-ready, or scroll-ready. A qualified winner must finish the run, parse the requested count, keep miss rate under 2%, and keep review rate at or below 15%; review and miss rates are penalized before active average speed is used as the tie-breaker. For IK-target claims, check goal100Decision; it must read qualified-comparison: winner=<engine>, and goal100.comparisonComplete must be true so a single-engine 100-artifact run is not mistaken for a current-vs-IK comparison.

Validate the saved assessment before using it as final evidence:

npm run scan:assessment:validate -- --latest
npm run scan:assessment:validate -- --input=<run-dir>\scan-performance-assessment.json

--latest searches outputs/live-soak/ for the newest scan-performance-assessment.json. Use explicit --input when comparing older or archived runs. Add --expect-winner=current or --expect-winner=ik-traineddata when validating a specific engine claim instead of accepting any qualified winner. Add --limit=20 for a short iteration run instead of the final 100-artifact proof. Add --summary when you want a short report-ready PASS/FAIL output that includes the input assessment path and assessment createdAt timestamp.

npm run scan:goal:compare:validated is the preferred final command: it runs the live preflight first, then the full comparison, and then validates the newest assessment with --summary. Use npm run scan:goal:compare:validated:wait for the same final flow when starting immediately after UAC.

The assessment ranking can be verified without Genshin or the Electron app:

npm run scan:assessment:test

This self-test rejects synthetic runs that are fast but have too many misses or too many review samples, so the final IK comparison cannot be won by speed alone.

Review-To-Eval Quality Loop

After any live scan that creates review samples, export candidates before adding anything to the permanent eval corpus:

npm run eval:review-candidates -- --limit=80

Read outputs/review-eval-candidates/review-eval-candidates.md. It is a review worklist, not ground truth. Only after the expected fields are confirmed or corrected against the real artifact should a case be moved into src/eval/corpus/confirmedReviewCorpus.ts. This prevents the parser from grading itself and keeps npm run eval meaningful. The exporter deduplicates samples, puts complete modern OCR captures first, and marks missing fast-profile fields so stale or partial captures are easier to ignore. Unconfirmed exporter output must stay in outputs/review-eval-candidates/.

For a manually checked candidate, generate a paste-ready confirmed-case snippet:

npm run eval:prepare-confirmed -- --candidate=<candidate-id> --expect-file=.\path\to\expect.json

The command requires explicit labels and writes only to the ignored outputs folder. Review the snippet before adding it to src/eval/corpus/confirmedReviewCorpus.ts.

For the current implementation summary and IK comparison rationale, see scanner-ik-progress-report.md.

Use the readiness timings to compare against Inventory Kamera's fixed waits: IK waits about 200 ms after selecting the next inventory item and about 100 ms after fast scrolls. If averageCardReadyMs or averageScrollReadyMs dominates the active average while OCR is already low, tune the fingerprint gate before touching OCR again.

The runner reads APP_RUNTIME_SIGNATURE from electron/main.ts and refuses to run against a stale Electron process when /health.appBuild.signature does not match the current source. Use -AllowStaleBuild only for deliberate debugging of an older instance. Current dev builds also expose /dev/shutdown on localhost. The start cleanup script calls it before falling back to Stop-Process, so a previous elevated app can shut itself down cleanly even when the caller cannot terminate an administrator process directly. Older builds without that endpoint still need manual close or a confirmed npm run dev:admin restart.

Review samples are saved as a compact summary by default so Vite does not try to watch large Base64 payloads under outputs/. Full review payloads can be saved with -SaveFullReviewSamples when needed.

It stops on a failed probe, blocked scan, stopped scan, or timeout unless -ContinueAfterBlocked is supplied directly:

powershell -NoProfile -ExecutionPolicy Bypass -File scripts\live-soak.ps1 -Limits 2,5 -ContinueAfterBlocked

Anti-Cheat And Safety Boundary

Do not describe the current implementation as bypassing anti-cheat. The app does not read memory, hook the process, inject code, modify game files, inspect packets, or interact with kernel drivers. It uses normal Windows screen capture, focus, cursor movement, wheel, and click input.

The practical finding is narrower:

  • A non-elevated app can be blocked by Windows integrity/UIPI when the target process is elevated or protected.
  • Running the app elevated fixed input delivery in the tested environment.
  • Genshin's anti-cheat may still affect behavior on other machines, game modes, overlays, or future versions. Re-run the probe before trusting broad scans.

Never add automation that deletes, feeds, enhances, locks/unlocks, spends resources, reads memory, hooks, injects, or modifies Genshin.

Live Layout Facts

The current 16:9 layout profile is calibrated from a 1920x1080 English artifact-inventory capture:

  • detail rect approximately x=1308, y=120, width=492, height=838
  • inventory grid: 8 x 4 safe automated targets, matching Inventory Kamera's 32-artifact full-page model. The apparent lower fifth row is in the bottom control band and is intentionally not clicked during auto-scan.
  • first tile center: x=179, y=254, row=0, col=0
  • second tile center: x=325, y=254, row=0, col=1
  • inventory count crop successfully read 2059/2400 in the live session

The profile is resolution-scaled for 16:9. Off-profile setups should be treated as higher risk and validated with Smart Capture plus the probe.

Validation Checklist

Before marking an automation change done:

  1. Run npm run lint.
  2. Run npx tsc -p tsconfig.electron.json when Electron/preload/main changed.
  3. Run npm test.
  4. Run npm run build.
  5. If Genshin is available, run /automation/probe-click?index=1.
  6. For scan-loop changes, run /scanner/start?entry=visible-inventory&limit=2 before any broader scan.
  7. Record new live findings in this file and in docs/scanner-rework-status.md.
  8. For IK-target claims, attach or cite scan-performance-assessment.json from a non-stale npm run scan:goal:compare:validated run.