Prepare scanner branch for merge
This commit is contained in:
AzuTear
+31
-8
@@ -49,10 +49,19 @@ flowchart LR
|
||||
|
||||
| Module | Responsibility |
|
||||
| --- | --- |
|
||||
| `electron/main.ts` | Window lifecycle, capture source listing, Smart Capture, OCR crop generation, overlay window IPC, input/capture sidecar orchestration, JSON artifact store, dev-only scanner control endpoints |
|
||||
| `electron/main.ts` | Electron app composition, dependency wiring, app lifecycle, hotkeys, and IPC/dev-control registration |
|
||||
| `electron/appWindowManager.ts` | Main window and overlay window lifecycle, menu-bar removal, dashboard focus behavior, and renderer window command delivery |
|
||||
| `electron/services/inputHelper.ts` | Stable JSON protocol client for the compiled C# input/capture sidecar plus PowerShell fallback startup |
|
||||
| `electron/services/inputHelperPowerShellFallback.ts` | PowerShell fallback script body for environments where the compiled helper is unavailable |
|
||||
| `electron/services/goodFileService.ts` | Local GOOD export file writing and GOOD import file dialog/read handling |
|
||||
| `electron/preload.cjs` | Safe renderer bridge exposed as `window.assistantApi` |
|
||||
| `src/lib/artifactStore.ts` | Pure signature/id/record helpers for the persistent artifact store |
|
||||
| `src/App.tsx` | Main app shell, scan view, triage view, build view, overlay preview |
|
||||
| `src/App.tsx` | Thin React entry that renders the app page |
|
||||
| `src/pages/AppPage.tsx` and `src/pages/app/*` | App page composition and high-level layout routing |
|
||||
| `src/features/scan/hooks/useScanViewController.ts` | Scan feature state composition and view-controller assembly |
|
||||
| `src/features/scan/hooks/scanViewScanActions.ts` | Manual scan and visible-grid scan orchestration |
|
||||
| `src/features/scan/hooks/scanViewEntryActions.ts` | Guided auto-entry choreography for visible inventory, direct inventory, and IK-style fallback paths |
|
||||
| `src/features/scan/hooks/useScanGoodInterop.ts` | Scan-page GOOD import/export actions against renderer repository ports |
|
||||
| `src/lib/artifactOcrParser.ts` | Converts OCR output into a parsed artifact candidate with confidence and notes |
|
||||
| `src/lib/fuzzyMatch.ts` | Generic fuzzy string matching for OCR text against known game data |
|
||||
| `src/lib/genshinLookup.ts` | Pure lookup and validation API for generated Genshin data |
|
||||
@@ -63,6 +72,9 @@ flowchart LR
|
||||
| `src/data/genshinGameData.json` | Generated local dictionary of characters, artifact sets, slots, and stats |
|
||||
| `scripts/generate-genshin-data.cjs` | Regenerates the local Genshin dictionary from `genshin-db` |
|
||||
| `src/types/*` | Shared app, capture, and domain contracts |
|
||||
| `src/styles/global.css` | Stylesheet entrypoint importing split style modules |
|
||||
| `src/styles/base.css` | Shared application, layout, scanner workspace, modal, triage, build, and overlay styles |
|
||||
| `src/styles/diagnostics.css` | Diagnose/dev-view specific styles |
|
||||
|
||||
## Dependency Rules
|
||||
|
||||
@@ -129,7 +141,8 @@ sequenceDiagram
|
||||
- SendInput's return value is checked: zero injected events (UIPI, e.g. elevated Genshin vs. non-elevated app) aborts with an explicit hint instead of silently clicking into nothing.
|
||||
- Dev-only probes under `http://127.0.0.1:17317` are used for live validation:
|
||||
`/automation/probe-click?index=N` tests one read-only tile selection, and
|
||||
`/scanner/start?limit=N` starts an auto-scan with a temporary limit payload.
|
||||
`/scanner/start?entry=visible-inventory&limit=N` starts an auto-scan with a
|
||||
temporary limit payload from an already visible artifact detail view.
|
||||
The live known-good result on 2026-07-07 is documented in
|
||||
[AUTOMATION_LIVE_SCAN.md](AUTOMATION_LIVE_SCAN.md).
|
||||
- Click verification: after each click the parsed detail-panel signature should change. An unchanged signature is a soft miss (it can also mean two OCR-identical neighbor pieces, common among +0 artifacts), so it is retried once with a small offset, logged with the stuck artifact name, and then skipped - never fatal on its own. The scan aborts only when the first ~6 clicks of page 1 produce nothing new (diagnosis hint: elevated Genshin blocks SendInput via UIPI, or grid coordinates are wrong) or a later page yields zero new artifacts.
|
||||
@@ -140,15 +153,25 @@ sequenceDiagram
|
||||
- Guided auto-entry is state gated. The normal scan button first performs a
|
||||
lightweight no-OCR preflight; OCR/store/review work starts only after the
|
||||
artifact inventory grid and artifact detail card are visually confirmed.
|
||||
- The entry path tries direct inventory (`B`) first and uses the IK-style
|
||||
ESC/B fallback only when direct entry does not reach an artifact detail card.
|
||||
- Card and page waits are fingerprint based. The scanner can proceed as soon as
|
||||
the expected visual state changes and stabilizes, while still accepting IK-like
|
||||
200 ms item and 100 ms scroll readiness points.
|
||||
- The normal scan button does not navigate into inventory when that preflight
|
||||
fails; it blocks and asks the operator to open the Artifact inventory with a
|
||||
visible detail card. Explicit Dev-Control entry modes can still test direct
|
||||
inventory or Paimon-menu choreography, but they are not the merge-ready
|
||||
default path.
|
||||
- Item verification uses the artifact OCR capture's own detail fingerprint, so
|
||||
the loop no longer performs a separate card-ready capture before OCR. Page
|
||||
waits remain fingerprint based and can proceed as soon as the inventory pane
|
||||
changes and stabilizes, while still accepting IK-like 100 ms scroll readiness
|
||||
points.
|
||||
- Scrolling sends one wheel notch per grid row with the cursor anchored over the inventory (assumption: roughly one row per notch; overlap is absorbed by dedupe, and a page without new artifacts stops the scan).
|
||||
- The scan never deletes, enhances, feeds, locks, or spends anything; it only selects tiles to read them.
|
||||
|
||||
Parsed artifacts from both modes are persisted into `artifact-store.json` keyed by a content signature that excludes the equipped character, so re-equipping updates a record instead of duplicating it. Leveling an artifact currently creates a new record (documented limitation until rescan-merge exists).
|
||||
During auto-scan, artifact store writes can be batched and flushed after the
|
||||
click/capture/OCR loop to avoid per-artifact save/reload churn in the hot path.
|
||||
Auto-scan artifact captures also bypass Electron source-list enumeration and use
|
||||
the GDI capture helper directly once the selected source/Genshin state has been
|
||||
preflighted. Manual captures and source refresh still use `desktopCapturer`.
|
||||
|
||||
## Security And Safety
|
||||
|
||||
|
||||
+203
-44
@@ -5,8 +5,8 @@ movement, click input, elevation, and live validation status.
|
||||
|
||||
## Current Known-Good State
|
||||
|
||||
Validated live on 2026-07-07 with Genshin open in the artifact inventory at
|
||||
1920x1080, English UI:
|
||||
Validated live with Genshin open in the artifact inventory at 1920x1080,
|
||||
English UI:
|
||||
|
||||
- `npm run dev:admin` starts the app elevated after the user confirms UAC.
|
||||
- Runtime status reported `isElevated: true`, `genshinFound: true`, and
|
||||
@@ -19,15 +19,24 @@ Validated live on 2026-07-07 with Genshin open in the artifact inventory at
|
||||
- A bounded live auto-scan via `/scanner/start?limit=2` completed with:
|
||||
`clicked: 2`, `attempted: 2`, `verified: 2`, `parsed: 2`, `stored: 2`,
|
||||
`review: 2`, `misses: 0`, `status: "done"`.
|
||||
- On 2026-07-08, a visible-inventory 50-artifact run completed with `50/50`
|
||||
parsed and stored, `0` review, `0` duplicates, and `0` misses. Throughput
|
||||
was still slow at `61765 ms` elapsed (`1235 ms/artifact`).
|
||||
- On 2026-07-09, the current-engine visible-inventory path completed
|
||||
`/scanner/start?entry=visible-inventory&limit=20&engine=current` with
|
||||
`20/20` verified and parsed, `19` stored, `1` duplicate, `0` review, and
|
||||
`0` misses in `8047 ms` elapsed (`402 ms/artifact`). A same-session artifact
|
||||
detail capture also persisted an equipped footer as `equipped: "Citlali"` and
|
||||
an unlocked grey lock as `locked: false`.
|
||||
|
||||
This proves that the current elevated app plus helper path can deliver mouse
|
||||
movement and click input to the focused Genshin client in this environment.
|
||||
|
||||
Latest-source timing is not proven while `/health.appBuild.signature` differs
|
||||
from the `APP_RUNTIME_SIGNATURE` in `electron/main.ts`. On 2026-07-07 the port
|
||||
was still owned by an older elevated runtime, so goal scans were intentionally
|
||||
blocked by the stale-build gate. Restart the elevated app through
|
||||
`npm run dev:admin` and confirm UAC before collecting new 100-artifact evidence.
|
||||
from the `APP_RUNTIME_SIGNATURE` in `electron/main.ts`, or after source changes
|
||||
that have not been loaded by a fresh elevated runtime. Restart the elevated app
|
||||
through `npm run dev:admin` and confirm UAC before collecting new 50/100
|
||||
artifact evidence.
|
||||
|
||||
## Elevation And UAC
|
||||
|
||||
@@ -101,23 +110,24 @@ Invoke-RestMethod "http://127.0.0.1:17317/automation/probe-click?row=0&col=3"
|
||||
For live validation, prefer a bounded scan first:
|
||||
|
||||
```powershell
|
||||
Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?limit=2"
|
||||
Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?entry=visible-inventory&limit=2"
|
||||
```
|
||||
|
||||
The visible-inventory path remains the safest first check. The normal guided
|
||||
entry tries the read-only direct world path first:
|
||||
`B -> artifact tab -> first artifact tile`. If that does not produce a visible
|
||||
artifact detail card, it falls back to the Inventory Kamera-compatible sequence:
|
||||
`ESC -> B -> artifact tab -> first artifact tile`.
|
||||
The visible-inventory path is the merge-relevant safe path. It requires the
|
||||
Artifact inventory to already be open with a visible artifact detail card.
|
||||
|
||||
The normal Auto-Scan button uses a guided start. It first takes one lightweight
|
||||
preflight capture without OCR, full-frame payload, review scoring, or storing.
|
||||
If an artifact detail card is already visible, it starts the visible-inventory
|
||||
scan. Otherwise it runs the guided entry above. OCR/review/store work starts
|
||||
only after the artifact-detail preflight passes.
|
||||
Guided entry uses short state polling for the Inventory screen, artifact grid,
|
||||
and first detail card instead of waiting the full fixed delay every time; if the
|
||||
state never appears, the same timeout budget returns the last diagnostic capture.
|
||||
scan. Otherwise it blocks with an operator-facing status and asks the user to
|
||||
open the Artifact inventory with a visible detail card. OCR/review/store work
|
||||
starts only after the artifact-detail preflight passes.
|
||||
|
||||
The explicit Dev-Control entry modes below remain available for targeted
|
||||
experiments only. They send read-only navigation, but they are not the
|
||||
merge-ready default because live testing showed that `auto-entry` can leave the
|
||||
app in the Paimon menu when the starting state is not what the choreography
|
||||
expects.
|
||||
|
||||
```powershell
|
||||
Invoke-RestMethod "http://127.0.0.1:17317/scanner/start?entry=paimon-menu&limit=2"
|
||||
@@ -147,13 +157,11 @@ The same guard also runs inside the scan loop. If the app is on the main game
|
||||
screen, a Paimon/menu screen, a generic primary-screen capture, or any screen
|
||||
without an artifact detail card, auto-scan must block instead of clicking tiles
|
||||
or trying OCR.
|
||||
After each click the loop polls the detail fingerprint with a short bounded
|
||||
budget instead of sleeping blindly. The current budget is 420 ms with 60 ms
|
||||
polls; if the card changes and stabilizes earlier, OCR starts earlier, and if it
|
||||
does not change the loop retries or stops through the normal miss guards. If the
|
||||
card changed but remains animated, the loop now proceeds after 200 ms, matching
|
||||
Inventory Kamera's select-next-item wait more closely without removing the
|
||||
detail-change guard.
|
||||
After each click the loop now performs one fast artifact capture/OCR pass and
|
||||
uses that capture's detail fingerprint to verify that the selected artifact
|
||||
changed. This removes the old separate card-ready capture from the hot path. If
|
||||
the detail fingerprint is unchanged, the loop retries once and then follows the
|
||||
normal miss/block guards.
|
||||
The outer scan start focuses Genshin once; hot-loop fingerprint/OCR captures do
|
||||
not re-run the focus helper before every tile, which avoids an OS focus ping on
|
||||
each artifact while still relying on click readback, foreground checks, and the
|
||||
@@ -186,15 +194,87 @@ window/app and restart with `npm run dev:admin` before running scanner probes.
|
||||
|
||||
Use the status `stats` timing fields for IK comparisons: `elapsedMs`,
|
||||
`activeScanMs`, `writeFlushMs`, `averageMsPerParsed`,
|
||||
`activeAverageMsPerParsed`, `averageCaptureMs`, `averageOcrMs`,
|
||||
`artifactsPerMinute`, and `projectedMsFor100`. `elapsedMs` is end-to-end
|
||||
including queued writes; `activeScanMs` is the click/capture/OCR loop before
|
||||
the final store/review flush. A run only counts as speed
|
||||
`activeAverageMsPerParsed`, `averageCaptureMs`,
|
||||
`averageCaptureRoundTripMs`, `averageCaptureRoundTripOverheadMs`,
|
||||
`averageOcrMs`, `artifactsPerMinute`, and `projectedMsFor100`.
|
||||
`elapsedMs` is end-to-end including queued writes; `activeScanMs` is the
|
||||
click/capture/OCR loop before the final store/review flush. A run only counts as speed
|
||||
evidence when `parsed`, `stored`, `review`, `duplicates`, and `misses` are read
|
||||
together; raw click count alone is not scanner throughput. If `averageOcrMs`
|
||||
dominates `averageMsPerParsed`, the next speed lever is an IK-style OCR worker
|
||||
queue. If `averageCaptureMs` dominates, crop payload/capture work is the
|
||||
bottleneck.
|
||||
queue. If `averageCaptureRoundTripOverheadMs` is high, native capture encode,
|
||||
Base64 transport, Electron image decode, or IPC/render scheduling is the next
|
||||
bottleneck. The current 3 artifacts/second target requires `averageMsPerParsed`
|
||||
at or below `333 ms` on a clean 20-artifact iteration.
|
||||
|
||||
Latest live timing evidence on 2026-07-08:
|
||||
|
||||
- Probe: `/automation/probe-click?index=1` returned `clicked: true`,
|
||||
`inputBlocked: false`, `changed: true`, and `captureTarget:
|
||||
"genshin-client"`.
|
||||
- Baseline after helper/hot-loop cleanup:
|
||||
`/scanner/start?entry=visible-inventory&limit=50&engine=current` completed
|
||||
`50/50` parsed and stored with `0` review, `0` duplicates, `0` misses,
|
||||
`2` pages, `elapsedMs: 61765`, `averageMsPerParsed: 1235`,
|
||||
`averageCaptureMs: 186`, `averageOcrMs: 162`, and
|
||||
`averageScrollReadyMs: 844`.
|
||||
- Deferred-write experiment:
|
||||
the same 50-artifact run completed `50/50` with `0` misses but regressed to
|
||||
`elapsedMs: 63616` because 50 single-record writes produced
|
||||
`writeFlushMs: 8163`.
|
||||
- Current source replaces that experiment with batch persist and quiet
|
||||
auto-scan UI captures. This is code-validated, but the batch version still
|
||||
needs a fresh elevated live run; the follow-up restart was blocked because the
|
||||
admin runtime did not become reachable after shutdown/UAC.
|
||||
- Direct GDI hot-path validation:
|
||||
after skipping `desktopCapturer.getSources()` in auto-scan artifact captures,
|
||||
the 20-artifact iteration baseline improved to `20/20` parsed, `19` stored,
|
||||
`0` review, `1` duplicate, `0` misses, `7966 ms` elapsed,
|
||||
`398 ms/artifact`, `averageCaptureMs: 193`, `averageOcrMs: 167`,
|
||||
`averageClickMs: 2`, and `writeFlushMs: 4`. This is roughly
|
||||
`2.5 artifacts/second` on the first visible page.
|
||||
- Scroll-path validation with the same direct GDI hot path:
|
||||
`/scanner/start?entry=visible-inventory&limit=45&engine=current` completed
|
||||
`45/45` parsed, `42` stored, `0` review, `3` duplicates, `0` misses,
|
||||
`2` pages, `18625 ms` elapsed, `414 ms/artifact`, `averageCaptureMs: 187`,
|
||||
`averageOcrMs: 162`, and one scroll readiness wait of `173 ms`.
|
||||
- 100-artifact direct-GDI validation:
|
||||
`/scanner/start?entry=visible-inventory&limit=100&engine=current` completed
|
||||
on runtime signature `2026-07-08-direct-gdi-hotpath` with `100/100` parsed,
|
||||
`97` stored, `0` review, `3` duplicates, `0` misses, `4` pages,
|
||||
`42064 ms` elapsed, `421 ms/artifact`, `averageCaptureMs: 179`,
|
||||
`averageOcrMs: 154`, `averageClickMs: 2`, `writeFlushMs: 6`, and `3`
|
||||
scroll readiness waits averaging `176 ms`.
|
||||
- OCR/parser eval after this speed pass: `npm run eval` passed with `23/23`
|
||||
exact-match cases, `100%` field accuracy, and `100%` critical fields. This is
|
||||
a regression gate, not a substitute for manually checking live artifact values.
|
||||
- 3 artifacts/second preparation:
|
||||
auto-scan artifact captures now also omit the detail-preview payload and
|
||||
expose `averageCaptureRoundTripMs` plus
|
||||
`averageCaptureRoundTripOverheadMs`. The first live run exposed a false
|
||||
`missing-crops-or-ocr` review trigger because the hot path intentionally omits
|
||||
`detailDataUrl`; this is fixed in `getAutoReviewReason`.
|
||||
- 3 artifacts/second live attempts:
|
||||
after the review fix, a clean `limit=20` run completed `20/20` parsed,
|
||||
`19` stored, `0` review, `1` duplicate, `0` misses, `7285 ms` elapsed,
|
||||
or `364 ms/artifact` (`2.75 artifacts/second`). The stable final run on
|
||||
signature `2026-07-08-direct-gdi-reviewfix` completed `20/20`, `18` stored,
|
||||
`0` review, `2` duplicates, `0` misses, `7973 ms` elapsed, or
|
||||
`399 ms/artifact`. 3 artifacts/second is not proven.
|
||||
- Rejected speed experiments:
|
||||
detail-region capture, `GAA_OCR_WORKERS=5`, DataURL-to-buffer decode, and
|
||||
substat OCR `PSM.SINGLE_COLUMN` were all live/benchmark tested and were slower
|
||||
than the direct-GDI baseline. Keep `GAA_OCR_WORKERS=4` for current runs.
|
||||
- Quality-gated current-vs-IK comparison:
|
||||
`npm run scan:goal:compare:validated` produced
|
||||
`outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json`
|
||||
with `createdAt: 2026-07-08T18:41:11.6120957+02:00`.
|
||||
The final validator summary passed at `limit=100` with winner `current`,
|
||||
`activeAvg: 378 ms/artifact`, `projected100: 37800 ms`, `missRate: 0`, and
|
||||
`reviewRate: 0`. The `current` 100-artifact run parsed `100/100`, stored `97`,
|
||||
had `0` review, `0` misses, and crossed `4` pages. The `ik-traineddata`
|
||||
100-artifact run parsed `97/100`, had `5` review and `3` misses, and was not
|
||||
qualified because it parsed fewer artifacts than requested.
|
||||
|
||||
The `/scanner/start?limit=N` endpoint sends a renderer command payload with a
|
||||
temporary scan limit. It does not change the normal UI setting. The normal
|
||||
@@ -222,9 +302,9 @@ pool size. It also returns per-field OCR timings under
|
||||
parser behavior. Individual captures also report whether the artifact was
|
||||
detected as `sanctified`; level/substat crops are shifted in that state to match
|
||||
Inventory Kamera's crop model. By default it uses the auto-scan `fast` OCR profile,
|
||||
which omits the low-value set-effect crop, the slot crop that can be derived
|
||||
from the matched artifact piece name, and the main-stat-value crop that can be
|
||||
derived from slot, main-stat label, and level. The fast profile also uses
|
||||
which omits the low-value set-effect crop and the main-stat-value crop that can
|
||||
be derived from slot, main-stat label, and level. The slot crop remains enabled
|
||||
in the fast profile because it improved live-read quality. The fast profile also uses
|
||||
Inventory Kamera's tighter substat crop height; full/manual captures keep the
|
||||
larger recovery crop for debugging difficult samples. Auto-scan also omits per-crop diagnostic Base64 images from hot-loop OCR
|
||||
captures while keeping the detail screenshot, OCR text, crop rect metadata, and
|
||||
@@ -239,10 +319,13 @@ detection unless a caller explicitly overrides that option; add
|
||||
artifact-detail guard as auto-scan: if the current screen is not a confirmed artifact detail view, OCR is
|
||||
skipped and the response shows `skippedOcrCaptures` instead of burning time on
|
||||
invalid crops.
|
||||
For speed, the fast auto-scan profile also skips the optional Equipped footer
|
||||
OCR. Name, level, main-stat label, and substats remain in the OCR hot path;
|
||||
The fast auto-scan profile now keeps the optional Equipped footer OCR on real
|
||||
artifact-read captures when the footer marker is visible, so stored artifacts
|
||||
can record the equipped character without requiring a separate manual capture.
|
||||
Name, level, main-stat label, footer, and substats remain in the OCR hot path;
|
||||
slot, set, and main-stat value are derived when the lookup/parser can validate
|
||||
them. Use a full/manual capture when equipped ownership or every debug crop matters.
|
||||
them. Preflight and readiness poll captures still skip OCR/crops/lock-state
|
||||
work because they only prove surface and fingerprint changes.
|
||||
Local store/review writes are serialized through an internal queue but no longer
|
||||
block the next inventory click. The scan still flushes the queue before it
|
||||
returns its final summary, so `stored` and `review` counts remain final-state
|
||||
@@ -314,23 +397,45 @@ Default sequence:
|
||||
3. `/capture/smart?skipOcr=1`
|
||||
4. `/automation/probe-click?index=1`
|
||||
5. `/automation/probe-click?index=3`
|
||||
6. `/scanner/start?limit=2`
|
||||
7. `/scanner/start?limit=5`
|
||||
8. `/scanner/start?limit=10`
|
||||
9. `/scanner/start?limit=20`
|
||||
6. `/scanner/start?entry=visible-inventory&limit=2`
|
||||
7. `/scanner/start?entry=visible-inventory&limit=5`
|
||||
8. `/scanner/start?entry=visible-inventory&limit=10`
|
||||
9. `/scanner/start?entry=visible-inventory&limit=20`
|
||||
10. `/review/samples?limit=30`
|
||||
|
||||
For the actual Inventory-Kamera speed target, use the explicit goal run after
|
||||
`/health` shows the current `appBuild`:
|
||||
|
||||
```powershell
|
||||
npm run scan:live:preflight
|
||||
npm run scan:live:preflight:wait
|
||||
npm run scan:goal
|
||||
npm run scan:goal:current
|
||||
npm run scan:goal:ik
|
||||
npm run scan:iterate:compare:validated
|
||||
npm run scan:iterate:compare:validated:wait
|
||||
npm run scan:goal:compare
|
||||
npm run scan:goal:compare:validated
|
||||
npm run scan:goal:compare:validated:wait
|
||||
```
|
||||
|
||||
That run first warms/benchmarks `current` vs. `ik-traineddata`, then scans
|
||||
`scan:live:preflight` checks `/health`, `/scanner/status`, the current
|
||||
`APP_RUNTIME_SIGNATURE`, elevation, and whether Genshin is visible to the helper
|
||||
before a long live scan is attempted.
|
||||
Use `npm run scan:live:preflight:wait` during manual startup after `npm run
|
||||
dev:admin`; it waits up to 120 seconds for the elevated dev-control server and
|
||||
runtime checks to become ready. The non-waiting command remains the default for
|
||||
validated scan chains so automation fails fast on a missing runtime.
|
||||
|
||||
Use `npm run scan:iterate:compare:validated` for fast iteration while tuning OCR,
|
||||
parser, capture, or readiness behavior. It runs the same preflight, compares
|
||||
`current` vs. `ik-traineddata` at `limit=20`, and validates the newest assessment
|
||||
with `--limit=20 --summary`. This is the preferred loop while debugging because
|
||||
it gives quality-gated feedback without waiting for the full `2, 5, 20, 45, 100`
|
||||
goal sequence. Use `npm run scan:iterate:compare:validated:wait` directly after
|
||||
UAC if the elevated runtime may still be starting.
|
||||
|
||||
The goal run first warms/benchmarks `current` vs. `ik-traineddata`, then scans
|
||||
limits `2, 5, 20, 45, 100` with the selected scan engine, and writes
|
||||
`scan-run-summary.json` plus `scan-run-summary.csv`. `npm run scan:goal`
|
||||
uses the default `current` scan engine; use `scan:goal:ik` for a native
|
||||
@@ -345,7 +450,32 @@ limit, picks the best qualified engine, and labels the dominant bottleneck as
|
||||
OCR, capture, card-ready, or scroll-ready. A qualified winner must finish the
|
||||
run, parse the requested count, keep miss rate under 2%, and keep review rate
|
||||
at or below 15%; review and miss rates are penalized before active average speed
|
||||
is used as the tie-breaker.
|
||||
is used as the tie-breaker. For IK-target claims, check `goal100Decision`; it
|
||||
must read `qualified-comparison: winner=<engine>`, and
|
||||
`goal100.comparisonComplete` must be `true` so a single-engine 100-artifact run
|
||||
is not mistaken for a current-vs-IK comparison.
|
||||
|
||||
Validate the saved assessment before using it as final evidence:
|
||||
|
||||
```powershell
|
||||
npm run scan:assessment:validate -- --latest
|
||||
npm run scan:assessment:validate -- --input=<run-dir>\scan-performance-assessment.json
|
||||
```
|
||||
|
||||
`--latest` searches `outputs/live-soak/` for the newest
|
||||
`scan-performance-assessment.json`. Use explicit `--input` when comparing older
|
||||
or archived runs. Add `--expect-winner=current` or
|
||||
`--expect-winner=ik-traineddata` when validating a specific engine claim instead
|
||||
of accepting any qualified winner. Add `--limit=20` for a short iteration run
|
||||
instead of the final 100-artifact proof. Add `--summary` when you want a short
|
||||
report-ready PASS/FAIL output that includes the input assessment path and
|
||||
assessment `createdAt` timestamp.
|
||||
|
||||
`npm run scan:goal:compare:validated` is the preferred final command: it runs
|
||||
the live preflight first, then the full comparison, and then validates the
|
||||
newest assessment with `--summary`. Use
|
||||
`npm run scan:goal:compare:validated:wait` for the same final flow when starting
|
||||
immediately after UAC.
|
||||
|
||||
The assessment ranking can be verified without Genshin or the Electron app:
|
||||
|
||||
@@ -356,6 +486,34 @@ npm run scan:assessment:test
|
||||
This self-test rejects synthetic runs that are fast but have too many misses or
|
||||
too many review samples, so the final IK comparison cannot be won by speed alone.
|
||||
|
||||
## Review-To-Eval Quality Loop
|
||||
|
||||
After any live scan that creates review samples, export candidates before adding
|
||||
anything to the permanent eval corpus:
|
||||
|
||||
```powershell
|
||||
npm run eval:review-candidates -- --limit=80
|
||||
```
|
||||
|
||||
Read `outputs/review-eval-candidates/review-eval-candidates.md`. It is a review
|
||||
worklist, not ground truth. Only after the expected fields are confirmed or
|
||||
corrected against the real artifact should a case be moved into
|
||||
`src/eval/corpus/confirmedReviewCorpus.ts`. This prevents the parser from
|
||||
grading itself and keeps `npm run eval` meaningful. The exporter deduplicates
|
||||
samples, puts complete modern OCR captures first, and marks missing fast-profile
|
||||
fields so stale or partial captures are easier to ignore. Unconfirmed exporter
|
||||
output must stay in `outputs/review-eval-candidates/`.
|
||||
|
||||
For a manually checked candidate, generate a paste-ready confirmed-case snippet:
|
||||
|
||||
```powershell
|
||||
npm run eval:prepare-confirmed -- --candidate=<candidate-id> --expect-file=.\path\to\expect.json
|
||||
```
|
||||
|
||||
The command requires explicit labels and writes only to the ignored outputs
|
||||
folder. Review the snippet before adding it to
|
||||
`src/eval/corpus/confirmedReviewCorpus.ts`.
|
||||
|
||||
For the current implementation summary and IK comparison rationale, see
|
||||
[scanner-ik-progress-report.md](scanner-ik-progress-report.md).
|
||||
|
||||
@@ -429,7 +587,8 @@ Before marking an automation change done:
|
||||
3. Run `npm test`.
|
||||
4. Run `npm run build`.
|
||||
5. If Genshin is available, run `/automation/probe-click?index=1`.
|
||||
6. For scan-loop changes, run `/scanner/start?limit=2` before any broader scan.
|
||||
6. For scan-loop changes, run
|
||||
`/scanner/start?entry=visible-inventory&limit=2` before any broader scan.
|
||||
7. Record new live findings in this file and in `docs/scanner-rework-status.md`.
|
||||
8. For IK-target claims, attach or cite `scan-performance-assessment.json` from
|
||||
a non-stale `npm run scan:goal:compare` run.
|
||||
a non-stale `npm run scan:goal:compare:validated` run.
|
||||
|
||||
+9
-1
@@ -16,9 +16,17 @@
|
||||
## IK-Speed Or OCR-Engine Claim
|
||||
|
||||
- [ ] `/health.appBuild.signature` matches the current `APP_RUNTIME_SIGNATURE`.
|
||||
- [ ] `npm run scan:live:preflight` passes, or use the validated comparison command that runs it first.
|
||||
- [ ] During manual UAC startup, `npm run scan:live:preflight:wait` may be used before the scan chain.
|
||||
- [ ] `npm run scan:assessment:test` passes.
|
||||
- [ ] Use `npm run scan:iterate:compare:validated` or `npm run scan:iterate:compare:validated:wait` for short 20-artifact tuning loops.
|
||||
- [ ] Prefer `npm run scan:goal:compare:validated` or `npm run scan:goal:compare:validated:wait` for the final live comparison because it runs preflight, comparison, and assessment validation in sequence.
|
||||
- [ ] The run includes `scan-performance-assessment.json`.
|
||||
- [ ] The 100-artifact run finishes cleanly.
|
||||
- [ ] `npm run scan:assessment:validate -- --latest` or explicit `--input=<path>\scan-performance-assessment.json` passes.
|
||||
- [ ] If claiming a specific winner, the validator is run with `--expect-winner=current` or `--expect-winner=ik-traineddata`.
|
||||
- [ ] Final report cites the validator JSON or `--summary` output, including the assessment path.
|
||||
- [ ] For iteration claims, the 20-artifact run finishes cleanly and is validated with `--limit=20`.
|
||||
- [ ] For final claims, the 100-artifact run finishes cleanly.
|
||||
- [ ] Parsed count is at least the requested count.
|
||||
- [ ] Miss rate is at or below 2%.
|
||||
- [ ] Review rate is at or below 15%.
|
||||
|
||||
@@ -11,10 +11,13 @@ This document defines project engineering standards.
|
||||
## File Organization
|
||||
|
||||
- Keep Electron OS integration in `electron/`.
|
||||
- Keep Electron `main.ts` as composition/wiring. Move durable window, file, helper, capture, OCR, or dev-control responsibilities into named modules under `electron/`.
|
||||
- Keep React components in `src/`, with extraction when `App.tsx` becomes hard to review.
|
||||
- Keep feature controller hooks small enough to review. If a hook owns persistence, import/export, entry choreography, scan-loop orchestration, and UI state at once, split those concerns into feature-local hooks or services.
|
||||
- Keep pure domain logic in `src/lib/`.
|
||||
- Keep shared contracts in `src/types/`.
|
||||
- Keep generated outputs in `dist/`, `dist-electron/`, and `outputs/`.
|
||||
- Keep `src/styles/global.css` as the stylesheet entrypoint. Put broad app styles in `base.css` and dev/diagnostic-only styling in `diagnostics.css` unless a more specific style module is introduced.
|
||||
|
||||
## UI Rules
|
||||
|
||||
@@ -30,6 +33,7 @@ This document defines project engineering standards.
|
||||
- Confidence and raw OCR details must remain inspectable.
|
||||
- Heuristics should fail safely into unknown fields or review notes.
|
||||
- Do not add irreversible game actions.
|
||||
- Keep auto-entry choreography separate from scan-loop execution. Entry code may navigate to a readable artifact detail state; loop code should process verified grid targets.
|
||||
|
||||
## TypeScript Rules
|
||||
|
||||
|
||||
+7
-1
@@ -323,7 +323,11 @@ engine or claiming IK parity. A qualified scan result must:
|
||||
`scripts/live-soak.ps1` writes the evidence bundle and
|
||||
`scan-performance-assessment.json`. `npm run scan:assessment:test` verifies that
|
||||
the ranking logic rejects fast but low-quality synthetic runs without needing
|
||||
Genshin.
|
||||
Genshin. The assessment also records `goal100Decision` and
|
||||
`goal100.comparisonComplete`; IK-target claims require a qualified 100-artifact
|
||||
winner and a complete `current` vs. `ik-traineddata` comparison.
|
||||
`npm run scan:assessment:validate -- --summary` prints the assessment path and
|
||||
`createdAt` timestamp so reports can cite the exact evidence file.
|
||||
|
||||
### Consequences
|
||||
|
||||
@@ -332,4 +336,6 @@ Genshin.
|
||||
closer to IK; it must win the same-capture benchmark and a qualified live run.
|
||||
- Stale elevated Electron instances are treated as invalid evidence, not as a
|
||||
harmless warning.
|
||||
- The validated `:wait` scan scripts are acceptable for manual post-UAC startup;
|
||||
non-waiting scripts remain useful when automation should fail fast.
|
||||
- The goal remains open until the 100-artifact qualified comparison is captured.
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
# Merge Readiness
|
||||
|
||||
Current branch: `codex/ik-scanner-progress`
|
||||
|
||||
This checklist records the evidence needed before merging this scanner branch
|
||||
into `main`. It separates merge-relevant proof from optional follow-up work.
|
||||
|
||||
## Merge-Relevant Evidence
|
||||
|
||||
| Area | Status | Evidence |
|
||||
| --- | --- | --- |
|
||||
| TypeScript/lint gate | Passed | `npm run lint` |
|
||||
| Unit/regression suite | Passed | `npm test` with 207 tests, including equipped footer, lock detection, and PNG lock-crop regression coverage |
|
||||
| Production build | Passed | `npm run build` |
|
||||
| Whitespace check | Passed | `git diff --check` |
|
||||
| OCR eval gate | Passed | `npm run eval` with `23/23` exact-match cases and `100%` critical fields |
|
||||
| Scan assessment self-test | Passed | `npm run scan:assessment:test`; the fixture intentionally expects `ik-traineddata` to win its synthetic `limit=100` case while real live-winner claims stay tied to archived live assessments |
|
||||
| Live runtime preflight | Passed | `npm run scan:live:preflight`; signature `2026-07-08-direct-gdi-reviewfix`, elevated yes, Genshin found |
|
||||
| Safe visible-inventory scan path | Passed | 2026-07-09 live run: `/scanner/start?entry=visible-inventory&limit=20&engine=current` completed `20/20` verified and parsed, `19` stored, `1` duplicate, `0` review, `0` misses, `8047 ms` elapsed |
|
||||
| Equipped character OCR/persist path | Passed for live smoke | 2026-07-09 live captures read equipped footers for `Citlali` and `Linnea`; parser regression covers `Equipped: Linnea l` -> `Linnea` |
|
||||
| Unlocked lock state | Passed for live smoke | 2026-07-09 Smart Capture reported `locked: false` on an unlocked artifact; after restart, `lockSignal.ratio: 0` with threshold `0.06` |
|
||||
| Positive locked lock state | Passed | 2026-07-09 Smart Capture on a visibly locked artifact reported `locked: true`, `lockSignal.ratio: 0.14797913950456323`, threshold `0.06` |
|
||||
| Lock-state persistence | Passed | 2026-07-09 `/scanner/start?entry=visible-inventory&limit=1&engine=current` stored `A Note in Spring's Leich` with `equipped: "Citlali"` and `locked: true` |
|
||||
| Lock-state diagnostics | Passed | Capture results include `lockSignal.ratio`, `lockSignal.threshold`, and the lock crop rect. Detection uses decoded PNG crop pixels to avoid native bitmap channel-order ambiguity |
|
||||
| Unsafe auto-entry default | Mitigated | Normal guided Auto-Scan now blocks when no artifact detail card is visible instead of falling back to `auto-entry` |
|
||||
| Review-to-eval export | Passed by tests | `npm run eval:review-candidates` script is covered by `src/eval/reviewEvalCandidatesScript.test.ts` and output is Git-ignored |
|
||||
|
||||
## Final Gate
|
||||
|
||||
```powershell
|
||||
npm run lint
|
||||
npm test
|
||||
npm run build
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Latest run after the lock-state fix: passed with `207` tests.
|
||||
`npm run eval` and `npm run scan:assessment:test` also passed after the
|
||||
lock-state fix.
|
||||
|
||||
## Explicitly Not Merge-Blocking
|
||||
|
||||
- `auto-entry`, `direct-inventory`, and `paimon-menu` are still experimental
|
||||
Dev-Control entry modes. They can be tested later with low limits, but they
|
||||
are no longer the normal merge path.
|
||||
- The optional `3 artifacts/second` target is not proven. The stable current
|
||||
path is closer to `2.5` to `2.75 artifacts/second` on clean 20-artifact live
|
||||
runs.
|
||||
- Native Tesseract/IK-traineddata is not the default. The latest documented
|
||||
qualified live 100-artifact comparison in this environment had `current` as
|
||||
winner; the assessment self-test is a synthetic validator fixture, not a live
|
||||
winner claim.
|
||||
|
||||
## Merge Recommendation
|
||||
|
||||
Ready for final human diff review and then merge into `main`. The previously
|
||||
open `locked: true` proof now passes for both Smart Capture and auto-scan
|
||||
persistence.
|
||||
+36
-12
@@ -5,6 +5,7 @@ This document is the source of truth for project intent, scope, runtime facts, a
|
||||
For implementation structure, see [ARCHITECTURE.md](ARCHITECTURE.md). For engineering standards, see [CONVENTIONS.md](CONVENTIONS.md).
|
||||
For the latest Inventory-Kamera comparison work, see
|
||||
[scanner-ik-progress-report.md](scanner-ik-progress-report.md).
|
||||
For the current branch merge checklist, see [MERGE_READINESS.md](MERGE_READINESS.md).
|
||||
|
||||
## Project Identity
|
||||
|
||||
@@ -60,7 +61,7 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
| --- | --- | --- |
|
||||
| Safety | Never perform irreversible in-game actions. | Code review and manual test |
|
||||
| Performance | Single artifact read should feel interactive and batch scan should not stall on false progress. | Capture latency monitored manually; auto-scan stops on blocked verification |
|
||||
| IK target | First 100 artifacts should scan with accuracy at least as good as Inventory Kamera and equal or better speed. | `npm run scan:goal:compare` quality-gated report |
|
||||
| IK target | First 100 artifacts should scan with accuracy at least as good as Inventory Kamera and equal or better speed. | `npm run scan:goal:compare:validated` or `npm run scan:goal:compare:validated:wait` quality-gated report |
|
||||
| Privacy | Captures and parsed data stay local by default. | No remote upload in scanner path |
|
||||
| Reliability | Uncertain OCR must be visible to the user. | Confidence and details view |
|
||||
| Learning loop | Scanner mistakes should become reusable local review samples. | `review-samples.jsonl` |
|
||||
@@ -104,19 +105,34 @@ The app is not intended to replace deep min-max tools. It prioritizes time savin
|
||||
- Elevated live automation is validated in the current dev environment:
|
||||
`/automation/probe-click?index=1` changed the selected artifact and
|
||||
`/scanner/start?limit=2` completed with 2/2 verified reads and 0 misses.
|
||||
- A 2026-07-08 visible-inventory 50-artifact run completed cleanly with
|
||||
50/50 parsed and stored, 0 review, 0 duplicates, and 0 misses. It is stable
|
||||
but still too slow for the 2-3 artifacts/second target.
|
||||
- Later 2026-07-08 direct-GDI hot-path runs completed 20/20 parsed with
|
||||
0 misses and 0 review. The best clean 20-artifact iteration reached
|
||||
7285 ms, or roughly 2.75 artifacts/second; the final stable
|
||||
`2026-07-08-direct-gdi-reviewfix` run completed in 7973 ms. The
|
||||
3 artifacts/second target remains unproven.
|
||||
- The same direct-GDI path completed a 100-artifact run with 100/100 parsed,
|
||||
0 review, 0 misses, and 42064 ms elapsed across 4 pages.
|
||||
- Review samples can now be exported with `npm run eval:review-candidates` into
|
||||
a Git-ignored human-labeling worklist. This is the next quality phase before
|
||||
adding more OCR corpus cases or trusting review queue data as labels.
|
||||
|
||||
### What is still structurally weak
|
||||
|
||||
- The scan experience is still partly orchestrated from `src/App.tsx`, which makes behavior changes harder than they should be.
|
||||
- Broader scan soak testing still needs to increase the live limit gradually and
|
||||
validate scroll/page transitions beyond the first visible row.
|
||||
- Broader scan soak testing has reached clean 20-, 45-, and 100-artifact runs
|
||||
with 0 misses on the current engine. The current-vs-IK-traineddata comparison
|
||||
is now captured; `current` won the qualified 100-artifact comparison on
|
||||
2026-07-08.
|
||||
- OCR quality is still inconsistent enough that some fields are recovered by fallback and derivation more often than they should be.
|
||||
- Learned fixes currently focus on text replacements; they do not yet update crop offsets, UI profile variants, or scanner targeting rules in a structured way.
|
||||
- The scan page is cleaner than before, but it still exposes too much operator/debug state in the main flow.
|
||||
- Recommendations and build logic exist, but the scanner is not yet reliable enough to make them the core focus.
|
||||
- The latest source has not yet completed the final 100-artifact live comparison
|
||||
because the current dev-control port is still owned by a stale elevated
|
||||
Electron process. Live timing must wait for a UAC-approved restart.
|
||||
- The latest source has completed the final current-vs-IK-traineddata live
|
||||
comparison for this environment. Repeatability and 3 artifacts/second are
|
||||
still open.
|
||||
|
||||
### Current product conclusion
|
||||
|
||||
@@ -171,6 +187,9 @@ Outcome:
|
||||
5. safe derivation from piece/slot/value references
|
||||
- Main stat/value inference is tightened with slot constraints and reference tables.
|
||||
- Bad parses automatically generate structured review reasons.
|
||||
- Equipped-character parsing is canonical-data constrained: noisy known names can
|
||||
match through aliases/fuzzy lookup, but unknown footer fragments stay
|
||||
`Not detected` instead of being stored as invented character names.
|
||||
|
||||
Status:
|
||||
- In progress
|
||||
@@ -191,6 +210,8 @@ Outcome:
|
||||
- resume or stop
|
||||
- Progress counts only when a new verified artifact or duplicate signature is confirmed.
|
||||
- Repeated pages, unchanged detail cards, blocked cursor movement, and scroll failures stop the scan with diagnosis instead of producing fake progress.
|
||||
- Fast artifact-read captures include the equipped footer when an equipped
|
||||
marker is visible; preflight and polling captures still skip expensive OCR.
|
||||
|
||||
Status:
|
||||
- In progress
|
||||
@@ -225,7 +246,9 @@ Outcome:
|
||||
- Review samples become both parser regression fixtures and learning inputs.
|
||||
|
||||
Status:
|
||||
- Planned
|
||||
- Prepared in code for text replacements, field aliases, constrained fixes,
|
||||
crop adjustment proposals, and UI-profile adjustment proposals. Crop/profile
|
||||
changes still require live review before being auto-applied.
|
||||
|
||||
### Phase 6 - Recommendations come back on top of a trusted scanner
|
||||
|
||||
@@ -241,11 +264,12 @@ Status:
|
||||
1. Finish scan-page cleanup so the main operator view is no longer noisy.
|
||||
2. Tighten the game data generator and parser contract, then backfill regression tests from real bad samples.
|
||||
3. Continue moving auto-scan behavior out of `App.tsx` and into isolated scanner modules.
|
||||
4. Soak-test the elevated C# helper automation path with gradually larger scan limits and page scroll transitions.
|
||||
5. Run `npm run scan:goal:compare` after `/health.appBuild.signature` matches
|
||||
the current source and use the quality-gated 100-artifact report as the IK
|
||||
target evidence.
|
||||
6. Extend the learning system from text-only fixes into crop/UI profile tuning.
|
||||
4. Repeat the qualified current-vs-IK-traineddata comparison in a later live
|
||||
session before making stronger speed/default-engine claims.
|
||||
5. Validate equipped-character footer reads and a positive `locked=true` sample
|
||||
from known artifacts; export candidates with `npm run eval:review-candidates`.
|
||||
6. Grow the confirmed OCR corpus from review samples before tightening parser
|
||||
thresholds further.
|
||||
7. Resume recommendation work only when scan accuracy is consistently trustworthy.
|
||||
|
||||
## Open Questions
|
||||
|
||||
+47
-4
@@ -3,14 +3,19 @@
|
||||
Field-level accuracy measurement for the artifact OCR parser. This is the gate
|
||||
every OCR, crop, layout, or parser change runs against (see ADR-007).
|
||||
It is necessary but not sufficient for the IK target: live scan speed and
|
||||
review/miss rates are measured by `npm run scan:goal:compare`.
|
||||
review/miss rates are measured by `npm run scan:iterate:compare:validated` for
|
||||
short iteration and `npm run scan:goal:compare:validated` for the final
|
||||
100-artifact proof. Use the `:wait` variants directly after UAC startup.
|
||||
|
||||
## Run it
|
||||
|
||||
```powershell
|
||||
npm run eval # full accuracy report for the seed corpus
|
||||
npm run eval:review-candidates # export unconfirmed review samples for human labeling
|
||||
npm test # runs the eval gate alongside the rest of the suite
|
||||
npm run scan:assessment:test # verifies quality-first scan ranking logic
|
||||
npm run scan:iterate:compare:validated:wait # 20-artifact live comparison
|
||||
npm run scan:goal:compare:validated:wait # final 100-artifact live comparison
|
||||
```
|
||||
|
||||
The report prints exact-match rate, overall field accuracy, a per-field
|
||||
@@ -31,6 +36,35 @@ breakdown (critical fields marked with `*`), and every failing case with an
|
||||
|
||||
The review queue is the corpus source. A saved review sample carries the OCR
|
||||
text plus the parser's *guess* - `reviewSampleToEvalCase` extracts both.
|
||||
For the local Electron queue, run:
|
||||
|
||||
```powershell
|
||||
npm run eval:review-candidates -- --limit=80
|
||||
```
|
||||
|
||||
This writes:
|
||||
|
||||
- `outputs/review-eval-candidates/review-eval-candidates.json`
|
||||
- `outputs/review-eval-candidates/review-eval-candidates.md`
|
||||
|
||||
The exporter deduplicates samples, puts complete modern OCR captures first,
|
||||
marks missing fast-profile fields so stale/partial captures do not crowd out
|
||||
useful cases, and surfaces ownership/lock evidence (`artifact-footer`,
|
||||
`equipped`, and `locked=true/false`) for the next validation pass.
|
||||
|
||||
After manually checking one candidate against the real artifact, create a
|
||||
confirmed corpus snippet with explicit expected labels:
|
||||
|
||||
```powershell
|
||||
npm run eval:prepare-confirmed -- --candidate=<candidate-id> --expect-file=.\path\to\expect.json
|
||||
```
|
||||
|
||||
The script reads the latest
|
||||
`outputs/review-eval-candidates/review-eval-candidates.json` by default and
|
||||
writes a `.confirmed.ts` snippet under `outputs/review-eval-candidates/`.
|
||||
It refuses to run without explicit labels, so parser guesses are not silently
|
||||
promoted to ground truth. Review that snippet, then paste the object into
|
||||
`src/eval/corpus/confirmedReviewCorpus.ts`.
|
||||
|
||||
The parser's guess is a label **candidate, not ground truth** (using it directly
|
||||
would be the parser grading itself). To add a real case:
|
||||
@@ -38,11 +72,20 @@ would be the parser grading itself). To add a real case:
|
||||
1. Convert review samples with `reviewSamplesToEvalCases(records)`.
|
||||
2. Open each produced case and confirm or correct the `expect` values against
|
||||
what the artifact actually is in-game. Set `confirmed: true`.
|
||||
3. Move the corrected case into a file under `src/eval/corpus/` and add it to the
|
||||
corpus array.
|
||||
3. Move the corrected case into
|
||||
`src/eval/corpus/confirmedReviewCorpus.ts`. The main eval gate imports
|
||||
`src/eval/corpus/index.ts`, which combines the seed corpus with confirmed
|
||||
review cases.
|
||||
|
||||
`src/eval/corpus/confirmedReviewCorpus.test.ts` rejects common corpus mistakes:
|
||||
duplicate case ids, missing OCR text, empty labels, missing source notes, or a
|
||||
case that was copied in without `confirmed: true`.
|
||||
|
||||
Prefer cases that cover new failure modes: unseen resolutions, new sets or
|
||||
characters, and OCR noise the current corpus does not exercise.
|
||||
characters, equipped footer noise, and OCR noise the current corpus does not
|
||||
exercise. `locked` is a capture-side visual flag rather than a text parser field;
|
||||
validate it from review-export metadata and live screenshots instead of adding
|
||||
it to the OCR eval labels.
|
||||
|
||||
## When a change moves a number
|
||||
|
||||
|
||||
@@ -16,17 +16,22 @@ Inventory-Kamera-style artifact scanner:
|
||||
missing-detail states block before OCR/store/review work.
|
||||
- OCR uses a fast artifact profile that skips low-value fields and derives
|
||||
slot, set, and main-stat value through lookup constraints when safe.
|
||||
- The OCR worker pool, field crop split, page model, scroll model, and readiness
|
||||
waits now mirror the relevant IK design choices more closely.
|
||||
- The OCR worker pool, field crop split, page model, scroll model, and direct
|
||||
detail-change verification now mirror the relevant IK design choices more
|
||||
closely.
|
||||
- Diagnostics now preserve state evidence, timings, screenshots where useful,
|
||||
entry events, focus/input events, preflight failures, and scan-loop reasons.
|
||||
- A live soak runner now measures throughput and quality, compares current vs.
|
||||
IK-traineddata engines, and refuses to run against stale Electron builds.
|
||||
|
||||
The requested final goal is not proven complete yet. The current live dev port is
|
||||
still owned by an older elevated Electron instance, so the latest code cannot be
|
||||
truthfully benchmarked against IK until the app is restarted with UAC approval
|
||||
and `npm run scan:goal:compare` completes a qualified 100-artifact run.
|
||||
The current-vs-IK-traineddata comparison proof is now captured. On 2026-07-08,
|
||||
`npm run scan:goal:compare:validated` passed with
|
||||
`outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json`.
|
||||
At `limit=100`, `current` won the qualified comparison with `100/100` parsed,
|
||||
`0` review, `0` misses, `378 ms/artifact` active average, and `37800 ms`
|
||||
projected time for 100 artifacts. `ik-traineddata` was not qualified at
|
||||
`limit=100` because it parsed `97/100`, had `5` review and `3` misses. The
|
||||
separate 3 artifacts/second target is still not proven.
|
||||
|
||||
## What Changed
|
||||
|
||||
@@ -51,13 +56,19 @@ against a canonical package before it is accepted.
|
||||
name, slot, main-stat label, main-stat value, level, substats, set effects,
|
||||
equipped/footer, lock, and rarity.
|
||||
- Fast auto-scan profile skips lower-value OCR work:
|
||||
set effects, slot crop, main-stat value crop, equipped footer, crop images,
|
||||
full-frame payloads, and inventory preview payloads.
|
||||
set effects, main-stat value crop, crop images, full-frame payloads, and
|
||||
inventory preview payloads. The equipped footer remains in real artifact-read
|
||||
captures when its marker is visible, because ownership now matters for phase 1
|
||||
validation. Preflight and poll captures still skip OCR/crops/lock-state work.
|
||||
The slot crop remains in the fast path because it materially improved
|
||||
real-read quality.
|
||||
- Slot, set, and main-stat value are derived when lookup, slot rules, and level
|
||||
constraints make that safe.
|
||||
- Field-specific Tesseract PSM/whitelist cleanup and preprocessing are used.
|
||||
- OCR crops are passed as PNG buffers internally instead of Base64 DataURLs.
|
||||
- Exact visual duplicates are skipped before OCR.
|
||||
- Exact visual duplicate skipping is disabled in the hottest path; duplicate
|
||||
handling now primarily uses parsed artifact signatures so OCR is not skipped
|
||||
solely from a crop fingerprint collision.
|
||||
|
||||
Why this matters:
|
||||
|
||||
@@ -76,8 +87,16 @@ manual-debug capture that OCRs every visible thing.
|
||||
- `npm run scan:goal:current`
|
||||
- `npm run scan:goal:ik`
|
||||
- `npm run scan:goal:compare`
|
||||
- `npm run scan:goal:compare:validated`
|
||||
- `npm run scan:goal:compare:validated:wait`
|
||||
- `npm run scan:iterate:compare:validated`
|
||||
- `npm run scan:iterate:compare:validated:wait`
|
||||
- `npm run scan:live:preflight`
|
||||
- `npm run scan:live:preflight:wait`
|
||||
- `scan-performance-assessment.json` ranks runs by quality first and speed
|
||||
second.
|
||||
second, and records whether the 100-artifact result is a complete
|
||||
current-vs-IK comparison through `goal100Decision` and
|
||||
`goal100.comparisonComplete`.
|
||||
|
||||
Important rule:
|
||||
|
||||
@@ -113,12 +132,15 @@ outside the artifact inventory/detail state.
|
||||
clicking the risky lower band.
|
||||
- Last/partial page planning bottom-aligns like IK, avoiding unnecessary
|
||||
duplicate reads after scroll.
|
||||
- Card readiness uses detail fingerprint polling:
|
||||
max 420 ms, 60 ms polls, changed cards may proceed after 200 ms.
|
||||
- Detail-change verification now uses the artifact OCR capture itself instead
|
||||
of a separate card-ready capture before OCR.
|
||||
- Scroll readiness uses inventory fingerprint polling:
|
||||
max 760 ms, 80 ms polls, changed pages may proceed after 100 ms.
|
||||
- Store/review writes are queued so the next tile can be clicked before disk
|
||||
writes finish. The queue is still flushed before final summary.
|
||||
- Store/review writes are held out of the click/capture/OCR hot path. The
|
||||
latest source batches artifact store writes before the final summary instead
|
||||
of issuing one save/reload cycle per artifact.
|
||||
- Auto-scan artifact captures skip Electron source enumeration in the hot path
|
||||
and call the GDI capture helper directly.
|
||||
- Focus is done once at scan start; hot-loop captures do not refocus every tile.
|
||||
|
||||
Why this matters:
|
||||
@@ -175,21 +197,67 @@ Latest repo validation after the recent changes:
|
||||
| `npm run build` | Passed |
|
||||
| `git diff --check` | Passed |
|
||||
|
||||
Live evidence already collected earlier on 2026-07-07:
|
||||
Live evidence already collected:
|
||||
|
||||
- Probe click changed artifact detail successfully.
|
||||
- Limit 2 live auto-scan completed with 2/2 parsed and 0 misses.
|
||||
- Limit 20 live soak completed on the first visible page.
|
||||
- Limit 45 live soak crossed into a scrolled page.
|
||||
- On 2026-07-08, `/scanner/start?entry=visible-inventory&limit=50&engine=current`
|
||||
completed `50/50` parsed and stored, `0` review, `0` duplicates, `0` misses,
|
||||
`2` pages, `61765 ms` elapsed, `1235 ms/artifact`, `averageCaptureMs: 186`,
|
||||
and `averageOcrMs: 162`.
|
||||
- A deferred single-write flush experiment also completed `50/50`, but regressed
|
||||
to `63616 ms` because `writeFlushMs` was `8163`; the source now uses batch
|
||||
persist instead, pending a fresh elevated live measurement.
|
||||
- After direct GDI hot-path optimization, a 20-artifact run completed
|
||||
`20/20` parsed, `19` stored, `0` review, `1` duplicate, `0` misses,
|
||||
`7966 ms` elapsed, `398 ms/artifact`, `averageCaptureMs: 193`, and
|
||||
`averageOcrMs: 167`.
|
||||
- A 45-artifact direct-GDI run completed `45/45` parsed, `42` stored,
|
||||
`0` review, `3` duplicates, `0` misses, `2` pages, `18625 ms` elapsed,
|
||||
`414 ms/artifact`, `averageCaptureMs: 187`, and `averageOcrMs: 162`.
|
||||
- A 100-artifact direct-GDI run on signature
|
||||
`2026-07-08-direct-gdi-hotpath` completed `100/100` parsed, `97` stored,
|
||||
`0` review, `3` duplicates, `0` misses, `4` pages, `42064 ms` elapsed,
|
||||
`421 ms/artifact`, `averageCaptureMs: 179`, and `averageOcrMs: 154`.
|
||||
- `npm run eval` passed after the speed work with `23/23` exact-match cases,
|
||||
`100%` field accuracy, and `100%` critical fields.
|
||||
- The 3 artifacts/second target is now prepared in code but not live-proven:
|
||||
artifact hot-path captures omit detail preview payloads, and stats expose
|
||||
capture roundtrip/overhead timing. A qualifying 20-artifact run must finish in
|
||||
`<= 6667 ms` with 0 misses and no silent OCR quality regression.
|
||||
- Follow-up 3/s attempts on 2026-07-08 fixed the false review trigger caused by
|
||||
omitted detail previews. The best clean `limit=20` run reached `7285 ms`
|
||||
(`364 ms/artifact`, about `2.75 artifacts/second`) with `20/20` parsed,
|
||||
`0` review, and `0` misses. The final stable run on
|
||||
`2026-07-08-direct-gdi-reviewfix` completed `20/20` with `0` review,
|
||||
`0` misses, and `7973 ms` elapsed (`399 ms/artifact`). Detail-region capture,
|
||||
5 OCR workers, DataURL buffer decode, and substat `PSM.SINGLE_COLUMN` were
|
||||
tested and rejected as slower than the direct-GDI baseline.
|
||||
- Final current-vs-IK-traineddata comparison on 2026-07-08:
|
||||
`npm run scan:goal:compare:validated` passed. Evidence file:
|
||||
`outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json`,
|
||||
`createdAt: 2026-07-08T18:41:11.6120957+02:00`. `goal100Decision` was
|
||||
`qualified-comparison: winner=current`; `goal100.comparisonComplete` was true.
|
||||
`current` completed `100/100` parsed, `97` stored, `0` review, `0` misses,
|
||||
`4` pages, `378 ms/artifact` active average. `ik-traineddata` completed
|
||||
`97/100` parsed, `92` stored, `5` review, `3` misses and was rejected for
|
||||
parsing fewer artifacts than requested.
|
||||
- The review queue now has a bounded corpus-growth workflow:
|
||||
`npm run eval:review-candidates` writes a deduplicated, Git-ignored worklist
|
||||
to `outputs/review-eval-candidates/`. This separates complete modern OCR
|
||||
samples from stale captures and prevents the parser's own guess from being
|
||||
promoted to ground truth without human confirmation. After manual checking,
|
||||
`npm run eval:prepare-confirmed` turns one candidate plus explicit expected
|
||||
labels into a paste-ready confirmed corpus snippet.
|
||||
|
||||
Current live limitation:
|
||||
|
||||
- `/health` still reports an older elevated build:
|
||||
`2026-07-07-ocr-pool4-hotloop-no-refocus`.
|
||||
- Current source expects:
|
||||
`2026-07-07-ik32-fastsubstats-active-timing`.
|
||||
- The live soak runner correctly refuses to benchmark the stale runtime.
|
||||
- A UAC restart attempt was canceled, so the latest code is not yet live.
|
||||
- The fast path and current-vs-IK-traineddata 100-artifact comparison are proven
|
||||
in the current live environment. 3 artifacts/second is not proven; remaining
|
||||
speed work needs a larger OCR or capture-pipeline change, not more click
|
||||
tuning.
|
||||
|
||||
## Inventory Kamera Comparison
|
||||
|
||||
@@ -199,13 +267,14 @@ Current live limitation:
|
||||
| Entry | ESC/B inventory navigation and tab click | Direct `B` path plus IK-style fallback, with preflight guards |
|
||||
| Page model | 32 artifact items per page | 32 safe targets (`8 x 4`) implemented |
|
||||
| Last page | Bottom-aligned partial page after scroll | Implemented in page planner |
|
||||
| Item wait | About 200 ms fixed wait | Fingerprint polling, accepts changed card after 200 ms |
|
||||
| Item wait | About 200 ms fixed wait | No separate wait capture; OCR capture verifies changed detail |
|
||||
| Scroll wait | About 100 ms fast wait after scroll | Fingerprint polling, accepts changed page after 100 ms |
|
||||
| OCR model | Native Tesseract worker queue and custom traineddata | Tesseract.js pool with current and IK-traineddata comparison path |
|
||||
| Capture hot path | Direct window/screen capture without source-list scan per item | Direct GDI capture in auto-scan artifact loop |
|
||||
| Field parsing | OCR plus game-data lookup | OCR plus generated lookup, GOOD keys, aliases, slot/stat constraints |
|
||||
| Quality gate | Mature behavior by design and user history | Explicit benchmark/soak quality gates added |
|
||||
| Diagnostics | Logs/screenshots in IK flow | Diagnostics timeline plus JSON evidence bundle |
|
||||
| 100-artifact proof | Reference target | Not yet proven on latest app build |
|
||||
| 100-artifact proof | Reference target | Qualified current-vs-IK comparison captured; `current` won with `100/100`, 0 review, 0 misses, 37.8s projected |
|
||||
|
||||
What is theoretically better than before:
|
||||
|
||||
@@ -220,19 +289,29 @@ What is theoretically better than before:
|
||||
What is not yet proven better than IK:
|
||||
|
||||
- Native Tesseract speed is not integrated as the default.
|
||||
- The latest code has not completed the 100-artifact live run.
|
||||
- Review rate and miss rate on the user's real inventory still need the new
|
||||
live report.
|
||||
- 3 artifacts/second is not proven.
|
||||
- Native Inventory Kamera outside this app was not re-run in the same session;
|
||||
the completed comparison is against the bundled `ik-traineddata` scan engine.
|
||||
|
||||
## Theoretical Runtime Flow
|
||||
|
||||
For short iteration while tuning:
|
||||
|
||||
1. Start current elevated app with `npm run dev:admin` and confirm UAC.
|
||||
2. Run `npm run scan:iterate:compare:validated:wait` from a visible artifact inventory
|
||||
when starting directly after UAC, or `npm run scan:iterate:compare:validated`
|
||||
if preflight already passes.
|
||||
3. Inspect `scan-performance-assessment.json`, review samples, and timings if the
|
||||
20-artifact comparison fails quality gates.
|
||||
|
||||
For the intended 100-artifact comparison:
|
||||
|
||||
1. Start current elevated app with `npm run dev:admin` and confirm UAC.
|
||||
2. Verify `/health.appBuild.signature` matches `electron/main.ts`.
|
||||
3. Warm current and IK-traineddata OCR workers.
|
||||
4. Run a small bounded probe from the artifact inventory.
|
||||
5. Run `npm run scan:goal:compare`.
|
||||
5. Run `npm run scan:goal:compare:validated:wait` directly after UAC, or
|
||||
`npm run scan:goal:compare:validated` if preflight already passes.
|
||||
6. For each engine and limit (`2, 5, 20, 45, 100`):
|
||||
- focus Genshin once,
|
||||
- verify lookup and layout,
|
||||
@@ -274,17 +353,15 @@ Expected bottleneck sequence:
|
||||
|
||||
## Risks and Remaining Work
|
||||
|
||||
1. Restart with UAC and run the latest build live.
|
||||
2. Run `npm run scan:goal:compare` from a visible artifact inventory.
|
||||
3. If the 100-artifact winner is not qualified, inspect:
|
||||
`scan-performance-assessment.json`, review samples, diagnostic timeline, and
|
||||
field timings.
|
||||
4. If `ik-traineddata` wins but Tesseract.js is still slow, evaluate native
|
||||
Tesseract integration.
|
||||
5. Grow the eval corpus with confirmed real review samples before tightening
|
||||
1. Keep `current` as the default OCR engine for now; it won the qualified
|
||||
current-vs-IK-traineddata live comparison.
|
||||
2. If pursuing 3 artifacts/second, focus on capture/OCR pipeline changes rather
|
||||
than click timing.
|
||||
3. Grow the eval corpus with confirmed real review samples before tightening
|
||||
parser thresholds further.
|
||||
6. Validate a positive locked-artifact sample.
|
||||
7. Keep recommendations secondary until scanner quality is proven.
|
||||
4. Validate a positive locked-artifact sample.
|
||||
5. Keep recommendations secondary until scanner quality remains stable across
|
||||
repeated live sessions.
|
||||
|
||||
## Definition of Done for the IK Target
|
||||
|
||||
|
||||
+110
-25
@@ -14,12 +14,19 @@ Current status:
|
||||
|
||||
- The scanner architecture now follows the relevant Inventory Kamera model:
|
||||
32 artifact targets per page, lookup-derived fields, fast artifact OCR profile,
|
||||
short readiness gates, page-overlap planning, and queued OCR/store work.
|
||||
direct detail-fingerprint verification from the OCR capture, page-overlap
|
||||
planning, and batched store work.
|
||||
- The live runner can compare `current` and `ik-traineddata` engines and rejects
|
||||
runs that are fast but fail miss/review quality thresholds.
|
||||
- The final 100-artifact IK target is not proven yet. The dev-control port is
|
||||
currently owned by an older elevated Electron build, and the runner correctly
|
||||
refuses stale timing evidence until the app is restarted with UAC approval.
|
||||
- The final current-vs-IK-traineddata 100-artifact comparison is now proven for
|
||||
the current live environment. On 2026-07-08,
|
||||
`npm run scan:goal:compare:validated` passed with evidence at
|
||||
`outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json`.
|
||||
`current` won with `100/100` parsed, `0` review, `0` misses, and
|
||||
`378 ms/artifact` active average. `ik-traineddata` was rejected at 100 because
|
||||
it parsed `97/100`, had `5` review and `3` misses. The next optional speed
|
||||
target remains `3 artifacts/second`, which means `333 ms/artifact` or faster
|
||||
on clean 20-artifact iterations.
|
||||
|
||||
## Done (implemented, unit-tested, build green)
|
||||
|
||||
@@ -55,13 +62,13 @@ Current status:
|
||||
2 review samples, and 0 misses.
|
||||
- **Auto-scan OCR performance pass** - auto-scan captures now use an artifact
|
||||
OCR mode that skips inventory-count OCR on each tile, keeps equipped-character
|
||||
OCR, raises the substat crop to catch artifact level, stores automatic review
|
||||
samples without full-screen/inventory screenshots, reads only the tail of large
|
||||
OCR on the real artifact-read captures, raises the substat crop to catch
|
||||
artifact level, stores automatic review samples without full-screen/inventory screenshots, reads only the tail of large
|
||||
JSONL files, avoids review noise when only level/equipped is missing, starts
|
||||
the scan with an OCR-free preflight capture, skips exact visual duplicates
|
||||
before OCR, prevents repeated startup review reprocessing, omits full-frame
|
||||
and inventory-preview Base64 payloads from tile captures, and applies
|
||||
crop-specific Tesseract page-segmentation/whitelist parameters.
|
||||
the scan with an OCR-free preflight capture, prevents repeated startup review
|
||||
reprocessing, omits full-frame and inventory-preview Base64 payloads from tile
|
||||
captures, and applies crop-specific Tesseract page-segmentation/whitelist
|
||||
parameters.
|
||||
- **Visible-page live soak helper** - `scripts/live-soak.ps1` now drives the
|
||||
dev-control health/status, smart-capture, probe-click, bounded scan, and
|
||||
review-tail endpoints and writes evidence to `outputs/live-soak/`. On
|
||||
@@ -105,10 +112,77 @@ Current status:
|
||||
summaries, groups results by limit, identifies timing bottlenecks, and rejects
|
||||
winners that miss the requested count, exceed 2% misses, or exceed 15% review.
|
||||
`npm run scan:assessment:test` verifies this ranking logic without Genshin.
|
||||
The assessment also reports `goal100Decision` and
|
||||
`goal100.comparisonComplete`, so a single-engine 100-artifact run cannot be
|
||||
misread as the final IK comparison. Use
|
||||
`npm run scan:iterate:compare:validated:wait` for the 20-artifact live
|
||||
iteration and `npm run scan:goal:compare:validated:wait` for the final proof
|
||||
when starting directly after UAC. The validator `--summary` output includes
|
||||
the assessment path and timestamp for reporting.
|
||||
- **State-polled guided entry** - the guided auto-entry waits for Inventory,
|
||||
artifact grid, and first detail card evidence instead of sleeping the full
|
||||
fixed delay every time. OCR/review/store work still starts only after artifact
|
||||
detail preflight passes.
|
||||
- **Hot-loop speed pass (2026-07-08)** - the scan loop no longer performs a
|
||||
separate card-ready capture before OCR; the artifact OCR capture itself
|
||||
verifies detail-fingerprint change. Routine click diagnostics and scan stat
|
||||
publishes are throttled. Auto-scan artifact captures no longer update the
|
||||
full preview/topbar UI on every tile. Store writes can be batched so the scan
|
||||
path avoids per-artifact save/reload churn. Auto-scan artifact captures now
|
||||
use a direct GDI hot path and skip Electron `desktopCapturer.getSources()` in
|
||||
the per-artifact loop.
|
||||
- **3/s instrumentation pass (2026-07-08)** - artifact hot-path captures omit
|
||||
the detail-preview payload, and scan stats now split inner capture time from
|
||||
end-to-end capture roundtrip time. Use `averageCaptureRoundTripMs` and
|
||||
`averageCaptureRoundTripOverheadMs` in the next `limit=20` live iteration to
|
||||
decide whether the next cut belongs in native capture transport or OCR.
|
||||
- **3/s live attempt (2026-07-08)** - the missing-detail-preview review trigger
|
||||
was fixed and tested. The best clean 20-artifact run reached `7285 ms`
|
||||
(`364 ms/artifact`, about `2.75 artifacts/second`) with 0 review and 0 misses.
|
||||
The final stable run on `2026-07-08-direct-gdi-reviewfix` completed `20/20`
|
||||
with 0 review, 0 misses, and `7973 ms` elapsed (`399 ms/artifact`). Detail
|
||||
region capture, 5 OCR workers, DataURL buffer decode, and substat
|
||||
`PSM.SINGLE_COLUMN` were tested and rejected as slower.
|
||||
- **Review-to-eval loop (2026-07-08)** - `npm run eval:review-candidates`
|
||||
exports the local review queue into `outputs/review-eval-candidates/` as a
|
||||
human-labeling worklist. The exporter deduplicates samples, surfaces complete
|
||||
fast-field captures first, marks stale captures, and now surfaces equipped
|
||||
footer OCR plus `locked=true/false` payload counts for the next ownership/lock
|
||||
validation pass. Its output is deliberately
|
||||
ignored by Git and must not be treated as ground truth until fields are
|
||||
confirmed against the real artifact. Confirmed review labels now have a
|
||||
dedicated corpus file, `src/eval/corpus/confirmedReviewCorpus.ts`, with tests
|
||||
that reject duplicate ids, empty labels, and unconfirmed entries. The helper
|
||||
`npm run eval:prepare-confirmed` generates a paste-ready confirmed-case
|
||||
snippet only when explicit expected labels are provided.
|
||||
- **Prepared ownership/learning loop (2026-07-08)** - fast auto-scan no longer
|
||||
drops the artifact footer by profile alone; it omits footer OCR only when the
|
||||
capture option explicitly requests that or when the footer marker is absent.
|
||||
Parser tests cover noisy equipped names, split `Equipped:`/name footers, and
|
||||
one-letter OCR fragments that must stay `Not detected`. Scanner learning now
|
||||
persists text replacements, field aliases, constrained fixes, crop adjustment
|
||||
proposals, and UI-profile adjustment proposals instead of truncating everything
|
||||
back to text replacements.
|
||||
- **Visible-inventory merge guard (2026-07-09)** - the normal guided Auto-Scan
|
||||
start no longer falls back into `auto-entry` when the artifact detail card is
|
||||
missing. It now blocks and asks the operator to open the Artifact inventory
|
||||
with a visible detail card. The explicit `auto-entry`, `direct-inventory`, and
|
||||
`paimon-menu` Dev-Control modes remain available for targeted experiments, but
|
||||
they are not the merge-ready default path.
|
||||
- **Ownership live smoke (2026-07-09)** - live artifact detail capture parsed
|
||||
and stored an equipped footer as `equipped: "Citlali"` and the grey lock state
|
||||
as `locked: false`. A same-session visible-inventory run with
|
||||
`/scanner/start?entry=visible-inventory&limit=20&engine=current` completed
|
||||
`20/20` verified and parsed, `19` stored, `1` duplicate, `0` review, and
|
||||
`0` misses in `8047 ms` elapsed (`402 ms/artifact`).
|
||||
- **Locked artifact live proof (2026-07-09)** - a visibly locked artifact was
|
||||
selected through a read-only inventory tile click. Smart Capture reported
|
||||
`locked: true` with `lockSignal.ratio: 0.14797913950456323` over threshold
|
||||
`0.06`, and `/scanner/start?entry=visible-inventory&limit=1&engine=current`
|
||||
persisted the same artifact with `equipped: "Citlali"` and `locked: true`.
|
||||
Lock detection now decodes the lock crop PNG before measuring active lock
|
||||
pixels because Electron's native bitmap channel order was ambiguous in live
|
||||
captures.
|
||||
|
||||
## Remaining — needs the live environment or a UI pass
|
||||
|
||||
@@ -122,23 +196,34 @@ resolution or without UI work best tested live:
|
||||
The current benchmark can use IK-traineddata through Tesseract.js; native
|
||||
Tesseract integration remains the next implementation step before any engine
|
||||
default changes.
|
||||
3. **Validate guided entry live** from world, visible inventory, and Paimon/menu
|
||||
states with limits 2, 20, and 45. Confirm the artifact-tab coordinate in the
|
||||
user's current 16:9 layout and keep `visible-inventory` as fallback if the
|
||||
menu path is blocked.
|
||||
4. **Validate locked=true** against a known locked artifact — unlocked/grey lock
|
||||
was live-checked; a gold locked icon still needs a positive sample.
|
||||
3. **Validate explicit entry modes separately** from world, direct inventory,
|
||||
and Paimon/menu states with low limits only. These are now Dev-Control
|
||||
experiments, not the normal merge path; the normal Auto-Scan button blocks
|
||||
unless the visible artifact detail card is already present.
|
||||
4. **Repeat locked=true on another page/session** if lock behavior changes.
|
||||
The first positive live proof passed on 2026-07-09, including store
|
||||
persistence. Further repeats are useful for confidence but no longer block
|
||||
the merge.
|
||||
|
||||
5. **Broader scan soak test** — after the bounded two-item live scan passed,
|
||||
the next automation validation should increase the limit gradually and watch
|
||||
for repeated pages, scroll behavior, duplicate handling, and OCR review rate.
|
||||
6. **100-artifact IK comparison** — after `/health.appBuild.signature` matches
|
||||
current source, run `npm run scan:goal:compare` and compare qualified
|
||||
100-artifact results.
|
||||
5. **3 artifacts/second iteration** - not reached yet. The next credible path is
|
||||
either native Tesseract/IK-traineddata integration that materially reduces
|
||||
substat OCR time, or a larger capture pipeline change that avoids full-frame
|
||||
PNG/Base64 transport without hurting safety checks. The target remains
|
||||
`<= 6667 ms` elapsed for 20 parsed artifacts with 0 misses and no silent OCR
|
||||
review regression.
|
||||
|
||||
Visible-page limits up to 20 and a scroll/page-transition limit of 45 have
|
||||
passed. The remaining soak work is now OCR accuracy, review-rate reduction, and
|
||||
larger runs after the review corpus has grown.
|
||||
6. **Broader scan soak test** — direct-GDI current-engine runs now passed at
|
||||
`20/20`, `45/45`, and `100/100` with 0 misses. Continue with repeat runs if
|
||||
duplicate rate needs tuning.
|
||||
7. **Repeatability pass** — repeat the qualified current-vs-IK-traineddata run
|
||||
in a later live session before making major OCR-engine defaults or speed
|
||||
claims beyond this environment.
|
||||
|
||||
Visible-page limits up to 20, scroll/page-transition limit 45, and the final
|
||||
100-artifact current-vs-IK-traineddata comparison have passed for the current
|
||||
environment. Remaining soak work is repeatability, OCR corpus growth, equipped
|
||||
footer confirmation repeats, locked artifact repeats, and optional 3 artifacts/second
|
||||
speed work.
|
||||
|
||||
## Grow the eval corpus
|
||||
|
||||
|
||||
Reference in New Issue
Block a user