genshin-assistant/docs/NATIVE_SCANNER_VALIDATION_2026-07-09.md

# Native Artifact Scanner Validation - 2026-07-09

Historical first-session evidence. The later-session and packaged acceptance
that was still deferred here passed on 2026-07-10; see
[NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md](NATIVE_SCANNER_LIVE_ACCEPTANCE_2026-07-10.md).

This report records the first bounded 20/50/100-item validation of the native
Artifact capture and post-capture processing path. All runs used the visible
1920x1080 16:9 Artifact inventory, category `artifacts`, and `persist=0`.
No Artifact result was written to the local store.

## Result Summary

| Limit | Capture | Pages | Capture time | Capture rate | Parsed | Review | Errors | Processing time | Processing rate |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 20 | 20/20 | 1 | 4,544 ms | 4.40/s | 20/20 | 0 | 0 | 7,349 ms | 2.72/s |
| 50 | 50/50 | 2 | 11,925 ms | 4.19/s | 50/50 | 1 | 0 | 15,398 ms after fix | 3.25/s |
| 100 | 100/100 | 4 | 24,673 ms | 4.05/s | 100/100 | 3 | 0 | 38,343 ms | 2.61/s |

The 100-item run produced exactly 100 PNG crops, 100 JSONL capture jobs, and
100 durable scan results. All 100 results preserved a real native capture
timestamp; none fell back to the Unix epoch.

The native capture worker stayed above 4 artifacts/second. The current smoke
runner performs capture and post-capture OCR sequentially, so these numbers do
not prove a 3 artifacts/second end-to-end pipeline. That target remains open.

## Quality Findings

The 50-item run exposed two correctness gaps in the downstream result path:

- OCR read `31.1%` as `311%`, while the parser still assigned high confidence.
- Native jobs carried the correct `capturedAt`, but durable scan results used
  the Unix epoch because the processor did not forward the timestamp.

The parser now reconciles percent main values against the canonical slot,
stat, and level reference. It repairs an unambiguous missing decimal and lowers
confidence for unresolved <= +16 conflicts instead of silently replacing a
possible 4-star value. The processor now forwards native timestamps. The
existing 50-item run was reprocessed and confirmed `31.1%`, the original
capture time, one review, zero errors, and no persistence.

The final 100-item run placed three real OCR losses into Review:

- `DEF%+11.7%` was read as `DEF%+1.7%`.
- `ATK%+15.7%` was read as `ATK%+156.7%`.
- `HP+1,165` was read as `HP+1`.

All three values failed the legal substat-roll check and were prevented from
becoming clean, promotable results. They were then manually verified and
approved through the real single-result review workflow. The corrected results
are `DEF%+11.7%`, `ATK%+15.7%`, and `HP+1,165`; the run now has zero remaining
Review results. Approval wrote three durable review-log entries and three eval
samples. The confirmed cases were also added to the permanent OCR regression
corpus.

The parser now applies a conservative repair for this exact class of error: on
a +20 Artifact with four substats, dropped or extra percentage digits are only
repaired when exactly one combination satisfies the legal 5-star total roll
count. Thousands separators in flat substats are preserved. Ambiguous cases
continue to Review instead of being silently changed.

## Evidence Paths

- 20-item bundle: `outputs/native-live-smoke/20260709-223003/`
- 20-item native run: `%APPDATA%/genshin-artifact-assistant/native-scans/20260709-223006/`
- 50-item bundle: `outputs/native-live-smoke/20260709-223038/`
- 50-item native run: `%APPDATA%/genshin-artifact-assistant/native-scans/20260709-223041/`
- 100-item bundle: `outputs/native-live-smoke/20260709-223438/`
- 100-item native run: `%APPDATA%/genshin-artifact-assistant/native-scans/20260709-223441/`

## Validation

- `npm run lint`: passed
- `npm test`: passed, 246 tests at the time of the live validation
- Manual review follow-up: 3/3 corrected and approved, 0 remaining Review rows
- OCR regression eval after follow-up: 26/26 exact cases, 77/77 fields
- `npm run build`: passed
- Native 20/50/100 dry runs: passed capture and processing completeness
- Artifact-store writes: zero

## Decision

The native Artifact path has enough same-session scale evidence to move the
next implementation focus from capture expansion, promotion, and single-result
review to deterministic Artifact value evaluation. The visible-inventory path
remains the production baseline until packaged behavior and later-session
native repeatability are also proven.

## Offline Follow-Up - 2026-07-10

The saved 20/50/100 native runs were replayed through the new deterministic
roll-efficiency evaluator without opening Genshin or sending input:

| Saved run | Results | Evaluated | Review | Projection | Deterministic repeats |
| --- | ---: | ---: | ---: | ---: | ---: |
| `20260709-223006` | 20 | 20 | 0 | 0 | 5/5 |
| `20260709-223041` | 50 | 49 | 1 historical | 0 | 5/5 |
| `20260709-223441` | 100 | 100 | 0 | 9 | 5/5 |

This proves deterministic offline processing of the saved evidence, not a new
live scanner run. The complete implementation and package evidence is recorded
in
[ARTIFACT_EVALUATION_OFFLINE_VALIDATION_2026-07-10.md](ARTIFACT_EVALUATION_OFFLINE_VALIDATION_2026-07-10.md).
The next live and packaged-app acceptance pass is intentionally deferred to
[TOMORROW_LIVE_TEST_PLAN_2026-07-10.md](TOMORROW_LIVE_TEST_PLAN_2026-07-10.md).