Prepare scanner branch for merge
This commit is contained in:
+110
-25
@@ -14,12 +14,19 @@ Current status:
|
||||
|
||||
- The scanner architecture now follows the relevant Inventory Kamera model:
|
||||
32 artifact targets per page, lookup-derived fields, fast artifact OCR profile,
|
||||
short readiness gates, page-overlap planning, and queued OCR/store work.
|
||||
direct detail-fingerprint verification from the OCR capture, page-overlap
|
||||
planning, and batched store work.
|
||||
- The live runner can compare `current` and `ik-traineddata` engines and rejects
|
||||
runs that are fast but fail miss/review quality thresholds.
|
||||
- The final 100-artifact IK target is not proven yet. The dev-control port is
|
||||
currently owned by an older elevated Electron build, and the runner correctly
|
||||
refuses stale timing evidence until the app is restarted with UAC approval.
|
||||
- The final current-vs-IK-traineddata 100-artifact comparison is now proven for
|
||||
the current live environment. On 2026-07-08,
|
||||
`npm run scan:goal:compare:validated` passed with evidence at
|
||||
`outputs/live-soak/2026-07-08T18-38-35/scan-performance-assessment.json`.
|
||||
`current` won with `100/100` parsed, `0` review, `0` misses, and
|
||||
`378 ms/artifact` active average. `ik-traineddata` was rejected at 100 because
|
||||
it parsed `97/100`, had `5` review and `3` misses. The next optional speed
|
||||
target remains `3 artifacts/second`, which means `333 ms/artifact` or faster
|
||||
on clean 20-artifact iterations.
|
||||
|
||||
## Done (implemented, unit-tested, build green)
|
||||
|
||||
@@ -55,13 +62,13 @@ Current status:
|
||||
2 review samples, and 0 misses.
|
||||
- **Auto-scan OCR performance pass** - auto-scan captures now use an artifact
|
||||
OCR mode that skips inventory-count OCR on each tile, keeps equipped-character
|
||||
OCR, raises the substat crop to catch artifact level, stores automatic review
|
||||
samples without full-screen/inventory screenshots, reads only the tail of large
|
||||
OCR on the real artifact-read captures, raises the substat crop to catch
|
||||
artifact level, stores automatic review samples without full-screen/inventory screenshots, reads only the tail of large
|
||||
JSONL files, avoids review noise when only level/equipped is missing, starts
|
||||
the scan with an OCR-free preflight capture, skips exact visual duplicates
|
||||
before OCR, prevents repeated startup review reprocessing, omits full-frame
|
||||
and inventory-preview Base64 payloads from tile captures, and applies
|
||||
crop-specific Tesseract page-segmentation/whitelist parameters.
|
||||
the scan with an OCR-free preflight capture, prevents repeated startup review
|
||||
reprocessing, omits full-frame and inventory-preview Base64 payloads from tile
|
||||
captures, and applies crop-specific Tesseract page-segmentation/whitelist
|
||||
parameters.
|
||||
- **Visible-page live soak helper** - `scripts/live-soak.ps1` now drives the
|
||||
dev-control health/status, smart-capture, probe-click, bounded scan, and
|
||||
review-tail endpoints and writes evidence to `outputs/live-soak/`. On
|
||||
@@ -105,10 +112,77 @@ Current status:
|
||||
summaries, groups results by limit, identifies timing bottlenecks, and rejects
|
||||
winners that miss the requested count, exceed 2% misses, or exceed 15% review.
|
||||
`npm run scan:assessment:test` verifies this ranking logic without Genshin.
|
||||
The assessment also reports `goal100Decision` and
|
||||
`goal100.comparisonComplete`, so a single-engine 100-artifact run cannot be
|
||||
misread as the final IK comparison. Use
|
||||
`npm run scan:iterate:compare:validated:wait` for the 20-artifact live
|
||||
iteration and `npm run scan:goal:compare:validated:wait` for the final proof
|
||||
when starting directly after UAC. The validator `--summary` output includes
|
||||
the assessment path and timestamp for reporting.
|
||||
- **State-polled guided entry** - the guided auto-entry waits for Inventory,
|
||||
artifact grid, and first detail card evidence instead of sleeping the full
|
||||
fixed delay every time. OCR/review/store work still starts only after artifact
|
||||
detail preflight passes.
|
||||
- **Hot-loop speed pass (2026-07-08)** - the scan loop no longer performs a
|
||||
separate card-ready capture before OCR; the artifact OCR capture itself
|
||||
verifies detail-fingerprint change. Routine click diagnostics and scan stat
|
||||
publishes are throttled. Auto-scan artifact captures no longer update the
|
||||
full preview/topbar UI on every tile. Store writes can be batched so the scan
|
||||
path avoids per-artifact save/reload churn. Auto-scan artifact captures now
|
||||
use a direct GDI hot path and skip Electron `desktopCapturer.getSources()` in
|
||||
the per-artifact loop.
|
||||
- **3/s instrumentation pass (2026-07-08)** - artifact hot-path captures omit
|
||||
the detail-preview payload, and scan stats now split inner capture time from
|
||||
end-to-end capture roundtrip time. Use `averageCaptureRoundTripMs` and
|
||||
`averageCaptureRoundTripOverheadMs` in the next `limit=20` live iteration to
|
||||
decide whether the next cut belongs in native capture transport or OCR.
|
||||
- **3/s live attempt (2026-07-08)** - the missing-detail-preview review trigger
|
||||
was fixed and tested. The best clean 20-artifact run reached `7285 ms`
|
||||
(`364 ms/artifact`, about `2.75 artifacts/second`) with 0 review and 0 misses.
|
||||
The final stable run on `2026-07-08-direct-gdi-reviewfix` completed `20/20`
|
||||
with 0 review, 0 misses, and `7973 ms` elapsed (`399 ms/artifact`). Detail
|
||||
region capture, 5 OCR workers, DataURL buffer decode, and substat
|
||||
`PSM.SINGLE_COLUMN` were tested and rejected as slower.
|
||||
- **Review-to-eval loop (2026-07-08)** - `npm run eval:review-candidates`
|
||||
exports the local review queue into `outputs/review-eval-candidates/` as a
|
||||
human-labeling worklist. The exporter deduplicates samples, surfaces complete
|
||||
fast-field captures first, marks stale captures, and now surfaces equipped
|
||||
footer OCR plus `locked=true/false` payload counts for the next ownership/lock
|
||||
validation pass. Its output is deliberately
|
||||
ignored by Git and must not be treated as ground truth until fields are
|
||||
confirmed against the real artifact. Confirmed review labels now have a
|
||||
dedicated corpus file, `src/eval/corpus/confirmedReviewCorpus.ts`, with tests
|
||||
that reject duplicate ids, empty labels, and unconfirmed entries. The helper
|
||||
`npm run eval:prepare-confirmed` generates a paste-ready confirmed-case
|
||||
snippet only when explicit expected labels are provided.
|
||||
- **Prepared ownership/learning loop (2026-07-08)** - fast auto-scan no longer
|
||||
drops the artifact footer by profile alone; it omits footer OCR only when the
|
||||
capture option explicitly requests that or when the footer marker is absent.
|
||||
Parser tests cover noisy equipped names, split `Equipped:`/name footers, and
|
||||
one-letter OCR fragments that must stay `Not detected`. Scanner learning now
|
||||
persists text replacements, field aliases, constrained fixes, crop adjustment
|
||||
proposals, and UI-profile adjustment proposals instead of truncating everything
|
||||
back to text replacements.
|
||||
- **Visible-inventory merge guard (2026-07-09)** - the normal guided Auto-Scan
|
||||
start no longer falls back into `auto-entry` when the artifact detail card is
|
||||
missing. It now blocks and asks the operator to open the Artifact inventory
|
||||
with a visible detail card. The explicit `auto-entry`, `direct-inventory`, and
|
||||
`paimon-menu` Dev-Control modes remain available for targeted experiments, but
|
||||
they are not the merge-ready default path.
|
||||
- **Ownership live smoke (2026-07-09)** - live artifact detail capture parsed
|
||||
and stored an equipped footer as `equipped: "Citlali"` and the grey lock state
|
||||
as `locked: false`. A same-session visible-inventory run with
|
||||
`/scanner/start?entry=visible-inventory&limit=20&engine=current` completed
|
||||
`20/20` verified and parsed, `19` stored, `1` duplicate, `0` review, and
|
||||
`0` misses in `8047 ms` elapsed (`402 ms/artifact`).
|
||||
- **Locked artifact live proof (2026-07-09)** - a visibly locked artifact was
|
||||
selected through a read-only inventory tile click. Smart Capture reported
|
||||
`locked: true` with `lockSignal.ratio: 0.14797913950456323` over threshold
|
||||
`0.06`, and `/scanner/start?entry=visible-inventory&limit=1&engine=current`
|
||||
persisted the same artifact with `equipped: "Citlali"` and `locked: true`.
|
||||
Lock detection now decodes the lock crop PNG before measuring active lock
|
||||
pixels because Electron's native bitmap channel order was ambiguous in live
|
||||
captures.
|
||||
|
||||
## Remaining — needs the live environment or a UI pass
|
||||
|
||||
@@ -122,23 +196,34 @@ resolution or without UI work best tested live:
|
||||
The current benchmark can use IK-traineddata through Tesseract.js; native
|
||||
Tesseract integration remains the next implementation step before any engine
|
||||
default changes.
|
||||
3. **Validate guided entry live** from world, visible inventory, and Paimon/menu
|
||||
states with limits 2, 20, and 45. Confirm the artifact-tab coordinate in the
|
||||
user's current 16:9 layout and keep `visible-inventory` as fallback if the
|
||||
menu path is blocked.
|
||||
4. **Validate locked=true** against a known locked artifact — unlocked/grey lock
|
||||
was live-checked; a gold locked icon still needs a positive sample.
|
||||
3. **Validate explicit entry modes separately** from world, direct inventory,
|
||||
and Paimon/menu states with low limits only. These are now Dev-Control
|
||||
experiments, not the normal merge path; the normal Auto-Scan button blocks
|
||||
unless the visible artifact detail card is already present.
|
||||
4. **Repeat locked=true on another page/session** if lock behavior changes.
|
||||
The first positive live proof passed on 2026-07-09, including store
|
||||
persistence. Further repeats are useful for confidence but no longer block
|
||||
the merge.
|
||||
|
||||
5. **Broader scan soak test** — after the bounded two-item live scan passed,
|
||||
the next automation validation should increase the limit gradually and watch
|
||||
for repeated pages, scroll behavior, duplicate handling, and OCR review rate.
|
||||
6. **100-artifact IK comparison** — after `/health.appBuild.signature` matches
|
||||
current source, run `npm run scan:goal:compare` and compare qualified
|
||||
100-artifact results.
|
||||
5. **3 artifacts/second iteration** - not reached yet. The next credible path is
|
||||
either native Tesseract/IK-traineddata integration that materially reduces
|
||||
substat OCR time, or a larger capture pipeline change that avoids full-frame
|
||||
PNG/Base64 transport without hurting safety checks. The target remains
|
||||
`<= 6667 ms` elapsed for 20 parsed artifacts with 0 misses and no silent OCR
|
||||
review regression.
|
||||
|
||||
Visible-page limits up to 20 and a scroll/page-transition limit of 45 have
|
||||
passed. The remaining soak work is now OCR accuracy, review-rate reduction, and
|
||||
larger runs after the review corpus has grown.
|
||||
6. **Broader scan soak test** — direct-GDI current-engine runs now passed at
|
||||
`20/20`, `45/45`, and `100/100` with 0 misses. Continue with repeat runs if
|
||||
duplicate rate needs tuning.
|
||||
7. **Repeatability pass** — repeat the qualified current-vs-IK-traineddata run
|
||||
in a later live session before making major OCR-engine defaults or speed
|
||||
claims beyond this environment.
|
||||
|
||||
Visible-page limits up to 20, scroll/page-transition limit 45, and the final
|
||||
100-artifact current-vs-IK-traineddata comparison have passed for the current
|
||||
environment. Remaining soak work is repeatability, OCR corpus growth, equipped
|
||||
footer confirmation repeats, locked artifact repeats, and optional 3 artifacts/second
|
||||
speed work.
|
||||
|
||||
## Grow the eval corpus
|
||||
|
||||
|
||||
Reference in New Issue
Block a user