feat(scanner): complete localized artifact quality checkpoint
This commit is contained in:
@@ -0,0 +1,201 @@
|
||||
# Tomorrow Live Test Plan - 2026-07-10
|
||||
|
||||
> Historical execution plan. Its bounded acceptance matrix was completed and
|
||||
> later extended by the packaged 2,211-Artifact full-owned-inventory baseline.
|
||||
> The current authoritative result is
|
||||
> [NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md](NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md).
|
||||
|
||||
This is the ordered acceptance checklist for the first live session after the
|
||||
five evaluation phases. It intentionally separates packaged-app, UI, scanner,
|
||||
safety, quality, and performance evidence. Do not skip directly to a 100-item
|
||||
run.
|
||||
|
||||
Executed on 2026-07-10. The result table below and
|
||||
[NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md](NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md)
|
||||
are authoritative; the unchecked boxes above remain the original reusable
|
||||
operator checklist rather than a claim that the session is still pending.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- [ ] PC started fresh; no stale Electron, Vite, helper, or old dev-server process.
|
||||
- [ ] Genshin uses borderless `1920x1080` 16:9.
|
||||
- [ ] Artifact inventory is open and one Artifact detail card is visible.
|
||||
- [ ] No enhancement, feeding, deletion, locking, or resource-spending UI is involved.
|
||||
- [ ] Current source worktree and commit are recorded before testing.
|
||||
- [ ] `outputs/dist/win-unpacked/Genshin Artifact Assistant.exe` exists.
|
||||
- [ ] Existing `%APPDATA%/genshin-artifact-assistant/native-scans` data stays intact.
|
||||
|
||||
## 1. Static Recheck After Boot
|
||||
|
||||
```powershell
|
||||
npm run lint
|
||||
npm test
|
||||
npm run build
|
||||
npm run eval
|
||||
npm run package:verify
|
||||
npm audit
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- [ ] `262/262` or more tests pass.
|
||||
- [ ] OCR eval remains `26/26` exact and `77/77` fields.
|
||||
- [ ] Package verifier remains `27/27`.
|
||||
- [ ] Dependency audit remains at zero.
|
||||
|
||||
## 2. Packaged-App Launch
|
||||
|
||||
Launch the unpacked executable and approve its UAC prompt:
|
||||
|
||||
```powershell
|
||||
& "outputs\dist\win-unpacked\Genshin Artifact Assistant.exe"
|
||||
```
|
||||
|
||||
- [ ] App opens without a blank window, crash, or native menu bar.
|
||||
- [ ] Scanner, Inventory, Diagnose, Triage, Builds, and Overlay navigation render.
|
||||
- [ ] Runtime reports Electron `43.1.0` and the current app signature.
|
||||
- [ ] Elevation/automation guard reports the packaged app as elevated.
|
||||
- [ ] IK data reports version `6.7.0`, 61 sets, and 289 pieces.
|
||||
- [ ] Packaged C# helper resolves from `resources/input-helper`.
|
||||
- [ ] Browser-preview-only bridge warnings do not appear in the packaged app.
|
||||
|
||||
Stop immediately if helper resolution, preload, IK data, or elevation is broken.
|
||||
|
||||
## 3. Existing Scanned-Artifact UI
|
||||
|
||||
Open Inventory before starting any new scan.
|
||||
|
||||
- [ ] Existing native results load without changing Genshin.
|
||||
- [ ] Latest approved 100-result run shows 100 parsed/evaluated and zero Review.
|
||||
- [ ] Result rows show `xx% Roll`, extraction status, and source separately.
|
||||
- [ ] `Bewertet` filter returns evaluated rows.
|
||||
- [ ] `Wert offen` does not include normally evaluated clean rows.
|
||||
- [ ] `Roll-Wert` sorting orders numeric scores descending and leaves unknown last.
|
||||
- [ ] A +20 detail shows score reasons and per-substat roll/efficiency breakdown.
|
||||
- [ ] Detail explicitly says build fit is still open.
|
||||
- [ ] Locked/equipped state does not change the same Artifact's roll score.
|
||||
- [ ] Native PNG crop preview loads only from the selected run directory.
|
||||
|
||||
Projection samples from run `20260709-223441`:
|
||||
|
||||
- [ ] Select an under-leveled Artifact such as sequence `#96` (+16) or `#100` (+4).
|
||||
- [ ] Projection appears only in detail, not as a guaranteed result in the list.
|
||||
- [ ] `Worst <= Middle <= Best`.
|
||||
- [ ] Remaining roll count matches the current level.
|
||||
- [ ] Disclaimer states random target/roll tier and no guarantee.
|
||||
- [ ] +20 Artifacts show no future projection card.
|
||||
|
||||
## 4. Review And Promotion Safety
|
||||
|
||||
- [ ] A Review result has no normal numeric value score.
|
||||
- [ ] Invalid manual substat totals are rejected even if each value is individually plausible.
|
||||
- [ ] A 4-star-only roll value cannot be approved on a +20 Artifact.
|
||||
- [ ] Reject keeps the result blocked.
|
||||
- [ ] Approve requires IK identity, canonical main value, and legal total roll count.
|
||||
- [ ] Promotion still requires explicit selection and second confirmation.
|
||||
- [ ] Canceling promotion writes nothing.
|
||||
- [ ] If one promotion is deliberately tested, store count changes exactly once and the durable log is written.
|
||||
- [ ] No action changes anything inside the Genshin inventory.
|
||||
|
||||
## 5. Manual Smart Capture Smoke
|
||||
|
||||
This is the first live capture of the session.
|
||||
|
||||
- [ ] Genshin focus/preflight succeeds with the visible detail card.
|
||||
- [ ] One Artifact is captured and parsed.
|
||||
- [ ] Crop preview matches the selected Artifact.
|
||||
- [ ] Name, set, slot, level, main stat/value, substats, equipped, and locked state are checked visually.
|
||||
- [ ] Clean extraction receives a roll-efficiency score.
|
||||
- [ ] Uncertain extraction goes to Review and receives no normal score.
|
||||
- [ ] Stop/failsafe remains responsive.
|
||||
|
||||
## 6. Native Five-Artifact Smoke
|
||||
|
||||
```powershell
|
||||
npm run scan:native:smoke:5
|
||||
```
|
||||
|
||||
- [ ] Exactly 5 capture jobs, PNG crops, and durable results.
|
||||
- [ ] Zero processing errors and zero automatic store writes.
|
||||
- [ ] Result order and capture timestamps are preserved.
|
||||
- [ ] Evaluation is present for every clean parsed result.
|
||||
- [ ] Run the offline gate against the new run:
|
||||
|
||||
```powershell
|
||||
npm run scan:native:replay -- --run-dir="<new-run-dir>" --repeats=5
|
||||
```
|
||||
|
||||
- [ ] Replay is deterministic and has zero clean unevaluated rows.
|
||||
|
||||
## 7. Bounded 20/50/100 Progression
|
||||
|
||||
Only continue when the previous size passes.
|
||||
|
||||
For each run:
|
||||
|
||||
- [ ] Requested crop/job/result count is complete.
|
||||
- [ ] Processing errors are zero.
|
||||
- [ ] Store writes remain zero unless explicitly requested separately.
|
||||
- [ ] Review reason is inspectable for every Review row.
|
||||
- [ ] Miss rate is `<=2%` and Review rate is `<=15%`.
|
||||
- [ ] Stop works during capture and processing.
|
||||
- [ ] Offline replay with five repeats is deterministic.
|
||||
- [ ] Evidence directory and timings are recorded.
|
||||
|
||||
Final 100-item checks:
|
||||
|
||||
- [ ] Pages and scrolling advance without duplicates or skipped visible tiles.
|
||||
- [ ] Previously confirmed decimal-loss cases are repaired only when unique.
|
||||
- [ ] Ambiguous OCR stays Review.
|
||||
- [ ] Eligible under-leveled 5-star Artifacts receive projection; 4-star or
|
||||
ambiguous-rarity data does not; +20 Artifacts are complete.
|
||||
- [ ] Capture, processing, and end-to-end rates are reported separately.
|
||||
- [ ] Do not claim 3 artifacts/second unless measured end to end with quality gates.
|
||||
|
||||
## 8. Packaged Runtime Regression
|
||||
|
||||
Repeat the key five-item smoke from the packaged executable, not only dev mode:
|
||||
|
||||
- [ ] Packaged preload bridge exposes all native scanner methods.
|
||||
- [ ] Packaged helper can focus/click/capture under UAC.
|
||||
- [ ] Packaged IK resource path resolves.
|
||||
- [ ] Packaged crop preview bridge accepts valid PNG and blocks outside paths/non-PNG.
|
||||
- [ ] Close/reopen preserves saved Inventory and review/promotion state.
|
||||
|
||||
## Abort Criteria
|
||||
|
||||
Stop the current run immediately when any of these occurs:
|
||||
|
||||
- Wrong Genshin category or no visible Artifact detail card.
|
||||
- Blank/too-uniform capture passes preflight.
|
||||
- Click lands outside the validated Artifact grid.
|
||||
- Genshin loses focus and the worker continues clicking.
|
||||
- Result count exceeds requested limit.
|
||||
- Any automatic store write occurs during a dry run.
|
||||
- Review data receives a confident normal score.
|
||||
- Projection appears with missing fourth substat or claims certainty.
|
||||
- Helper/preload/IK packaged resource is missing.
|
||||
- App attempts delete, feed, enhance, lock/unlock, or resource spending.
|
||||
|
||||
## Result Record
|
||||
|
||||
Recorded after the session:
|
||||
|
||||
| Check | Result | Evidence path / note |
|
||||
| --- | --- | --- |
|
||||
| Static suite | passed | Final `npm run acceptance:offline`: 296/296 tests, 27/27 package checks, both audits clean |
|
||||
| Packaged launch | passed | Started elevated from `%TEMP%`; ASAR renderer, preload, helper and IK resources resolved |
|
||||
| Existing Inventory UI | passed | `outputs/packaged-live/packaged-final-inventory-fixed.png`; layout and latest complete-run discovery accepted |
|
||||
| Evaluation reasons | passed | New 20/50/100 five-repeat replay; 100/100 evaluated, 0 unknown |
|
||||
| Upgrade projection | passed offline, no new live sample | Existing under-level corpus passes; new live 100 contained only +20 Artifacts |
|
||||
| Review/promotion safety | passed non-mutating boundary | Main-process/tests pass; duplicates show `5 im Store, 0 Review`; no deliberate store mutation performed |
|
||||
| Single Smart Capture | passed | Real source `Genshin Impact`, `captureTarget=genshin-client`, 4x8 detected grid |
|
||||
| Native 5 | passed | Dev `20260710-102344`: 5/5, 0 Review/errors/writes |
|
||||
| Native 20 | passed | `20260710-102421`: 20/20, capture 4.50/s, processing 3.11/s |
|
||||
| Native 50 | passed | `20260710-102450`: 50/50, capture 4.41/s, processing 3.21/s |
|
||||
| Native 100 | passed | `20260710-102547`: 100/100, four pages, no duplicate page/hash evidence |
|
||||
| Focus/stop safety | passed | Focus-loss `20260710-101820` blocked at 0; stop `20260710-102735` stopped at 0 |
|
||||
| Packaged native smoke | passed | `20260710-103736`: 5/5 via renderer/preload/IPC/packaged helper, persist=false |
|
||||
|
||||
Full evidence and the next-phase decision are in
|
||||
[NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md](NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md).
|
||||
Reference in New Issue
Block a user