genshin-assistant/docs/NATIVE_SCANNER_VALIDATION_2026-07-09.md

Native Artifact Scanner Validation - 2026-07-09

Historical first-session evidence. The later-session and packaged acceptance that was still deferred here passed on 2026-07-10; see NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md.

This report records the first bounded 20/50/100-item validation of the native Artifact capture and post-capture processing path. All runs used the visible 1920x1080 16:9 Artifact inventory, category artifacts, and persist=0. No Artifact result was written to the local store.

Result Summary

Limit	Capture	Pages	Capture time	Capture rate	Parsed	Review	Errors	Processing time	Processing rate
20	20/20	1	4,544 ms	4.40/s	20/20	0	0	7,349 ms	2.72/s
50	50/50	2	11,925 ms	4.19/s	50/50	1	0	15,398 ms after fix	3.25/s
100	100/100	4	24,673 ms	4.05/s	100/100	3	0	38,343 ms	2.61/s

The 100-item run produced exactly 100 PNG crops, 100 JSONL capture jobs, and 100 durable scan results. All 100 results preserved a real native capture timestamp; none fell back to the Unix epoch.

The native capture worker stayed above 4 artifacts/second. The current smoke runner performs capture and post-capture OCR sequentially, so these numbers do not prove a 3 artifacts/second end-to-end pipeline. That target remains open.

Quality Findings

The 50-item run exposed two correctness gaps in the downstream result path:

• OCR read 31.1% as 311%, while the parser still assigned high confidence.
• Native jobs carried the correct capturedAt, but durable scan results used the Unix epoch because the processor did not forward the timestamp.

The parser now reconciles percent main values against the canonical slot, stat, and level reference. It repairs an unambiguous missing decimal and lowers confidence for unresolved <= +16 conflicts instead of silently replacing a possible 4-star value. The processor now forwards native timestamps. The existing 50-item run was reprocessed and confirmed 31.1%, the original capture time, one review, zero errors, and no persistence.

The final 100-item run placed three real OCR losses into Review:

• DEF%+11.7% was read as DEF%+1.7%.
• ATK%+15.7% was read as ATK%+156.7%.
• HP+1,165 was read as HP+1.

All three values failed the legal substat-roll check and were prevented from becoming clean, promotable results. They were then manually verified and approved through the real single-result review workflow. The corrected results are DEF%+11.7%, ATK%+15.7%, and HP+1,165; the run now has zero remaining Review results. Approval wrote three durable review-log entries and three eval samples. The confirmed cases were also added to the permanent OCR regression corpus.

The parser now applies a conservative repair for this exact class of error: on a +20 Artifact with four substats, dropped or extra percentage digits are only repaired when exactly one combination satisfies the legal 5-star total roll count. Thousands separators in flat substats are preserved. Ambiguous cases continue to Review instead of being silently changed.

Evidence Paths

• 20-item bundle: outputs/native-live-smoke/20260709-223003/
• 20-item native run: %APPDATA%/genshin-artifact-assistant/native-scans/20260709-223006/
• 50-item bundle: outputs/native-live-smoke/20260709-223038/
• 50-item native run: %APPDATA%/genshin-artifact-assistant/native-scans/20260709-223041/
• 100-item bundle: outputs/native-live-smoke/20260709-223438/
• 100-item native run: %APPDATA%/genshin-artifact-assistant/native-scans/20260709-223441/

Validation

• npm run lint: passed
• npm test: passed, 246 tests at the time of the live validation
• Manual review follow-up: 3/3 corrected and approved, 0 remaining Review rows
• OCR regression eval after follow-up: 26/26 exact cases, 77/77 fields
• npm run build: passed
• Native 20/50/100 dry runs: passed capture and processing completeness
• Artifact-store writes: zero

Decision

The native Artifact path has enough same-session scale evidence to move the next implementation focus from capture expansion, promotion, and single-result review to deterministic Artifact value evaluation. The visible-inventory path remains the production baseline until packaged behavior and later-session native repeatability are also proven.

Offline Follow-Up - 2026-07-10

The saved 20/50/100 native runs were replayed through the new deterministic roll-efficiency evaluator without opening Genshin or sending input:

Saved run	Results	Evaluated	Review	Projection	Deterministic repeats
20260709-223006	20	20	0	0	5/5
20260709-223041	50	49	1 historical	0	5/5
20260709-223441	100	100	0	9	5/5

This proves deterministic offline processing of the saved evidence, not a new live scanner run. The complete implementation and package evidence is recorded in ARTIFACT_EVALUATION_OFFLINE_VALIDATION_2026-07-10.md. The next live and packaged-app acceptance pass is intentionally deferred to TOMORROW_LIVE_TEST_PLAN_2026-07-10.md.