AzuTear
639b0b7f59
Add native IK-style capture processing, Artifact Inventory, explicit promotion and single-result review. Confirm the three live OCR corrections in the eval corpus and preserve extraction/value separation.
85 lines
4.1 KiB
Markdown
85 lines
4.1 KiB
Markdown
# Native Artifact Scanner Validation - 2026-07-09
|
|
|
|
This report records the first bounded 20/50/100-item validation of the native
|
|
Artifact capture and post-capture processing path. All runs used the visible
|
|
1920x1080 16:9 Artifact inventory, category `artifacts`, and `persist=0`.
|
|
No Artifact result was written to the local store.
|
|
|
|
## Result Summary
|
|
|
|
| Limit | Capture | Pages | Capture time | Capture rate | Parsed | Review | Errors | Processing time | Processing rate |
|
|
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
| 20 | 20/20 | 1 | 4,544 ms | 4.40/s | 20/20 | 0 | 0 | 7,349 ms | 2.72/s |
|
|
| 50 | 50/50 | 2 | 11,925 ms | 4.19/s | 50/50 | 1 | 0 | 15,398 ms after fix | 3.25/s |
|
|
| 100 | 100/100 | 4 | 24,673 ms | 4.05/s | 100/100 | 3 | 0 | 38,343 ms | 2.61/s |
|
|
|
|
The 100-item run produced exactly 100 PNG crops, 100 JSONL capture jobs, and
|
|
100 durable scan results. All 100 results preserved a real native capture
|
|
timestamp; none fell back to the Unix epoch.
|
|
|
|
The native capture worker stayed above 4 artifacts/second. The current smoke
|
|
runner performs capture and post-capture OCR sequentially, so these numbers do
|
|
not prove a 3 artifacts/second end-to-end pipeline. That target remains open.
|
|
|
|
## Quality Findings
|
|
|
|
The 50-item run exposed two correctness gaps in the downstream result path:
|
|
|
|
- OCR read `31.1%` as `311%`, while the parser still assigned high confidence.
|
|
- Native jobs carried the correct `capturedAt`, but durable scan results used
|
|
the Unix epoch because the processor did not forward the timestamp.
|
|
|
|
The parser now reconciles percent main values against the canonical slot,
|
|
stat, and level reference. It repairs an unambiguous missing decimal and lowers
|
|
confidence for unresolved <= +16 conflicts instead of silently replacing a
|
|
possible 4-star value. The processor now forwards native timestamps. The
|
|
existing 50-item run was reprocessed and confirmed `31.1%`, the original
|
|
capture time, one review, zero errors, and no persistence.
|
|
|
|
The final 100-item run placed three real OCR losses into Review:
|
|
|
|
- `DEF%+11.7%` was read as `DEF%+1.7%`.
|
|
- `ATK%+15.7%` was read as `ATK%+156.7%`.
|
|
- `HP+1,165` was read as `HP+1`.
|
|
|
|
All three values failed the legal substat-roll check and were prevented from
|
|
becoming clean, promotable results. They were then manually verified and
|
|
approved through the real single-result review workflow. The corrected results
|
|
are `DEF%+11.7%`, `ATK%+15.7%`, and `HP+1,165`; the run now has zero remaining
|
|
Review results. Approval wrote three durable review-log entries and three eval
|
|
samples. The confirmed cases were also added to the permanent OCR regression
|
|
corpus.
|
|
|
|
The parser now applies a conservative repair for this exact class of error: on
|
|
a +20 Artifact with four substats, dropped or extra percentage digits are only
|
|
repaired when exactly one combination satisfies the legal 5-star total roll
|
|
count. Thousands separators in flat substats are preserved. Ambiguous cases
|
|
continue to Review instead of being silently changed.
|
|
|
|
## Evidence Paths
|
|
|
|
- 20-item bundle: `outputs/native-live-smoke/20260709-223003/`
|
|
- 20-item native run: `%APPDATA%/genshin-artifact-assistant/native-scans/20260709-223006/`
|
|
- 50-item bundle: `outputs/native-live-smoke/20260709-223038/`
|
|
- 50-item native run: `%APPDATA%/genshin-artifact-assistant/native-scans/20260709-223041/`
|
|
- 100-item bundle: `outputs/native-live-smoke/20260709-223438/`
|
|
- 100-item native run: `%APPDATA%/genshin-artifact-assistant/native-scans/20260709-223441/`
|
|
|
|
## Validation
|
|
|
|
- `npm run lint`: passed
|
|
- `npm test`: passed, 246 tests at the time of the live validation
|
|
- Manual review follow-up: 3/3 corrected and approved, 0 remaining Review rows
|
|
- OCR regression eval after follow-up: 26/26 exact cases, 77/77 fields
|
|
- `npm run build`: passed
|
|
- Native 20/50/100 dry runs: passed capture and processing completeness
|
|
- Artifact-store writes: zero
|
|
|
|
## Decision
|
|
|
|
The native Artifact path has enough same-session scale evidence to move the
|
|
next implementation focus from capture expansion, promotion, and single-result
|
|
review to deterministic Artifact value evaluation. The visible-inventory path
|
|
remains the production baseline until packaged behavior and later-session
|
|
native repeatability are also proven.
|