feat(scanner): add native artifact pipeline
Add native IK-style capture processing, Artifact Inventory, explicit promotion and single-result review. Confirm the three live OCR corrections in the eval corpus and preserve extraction/value separation.
This commit is contained in:
AzuTear
@@ -1,14 +1,22 @@
|
||||
# Scanner Results And Artifact Inventory Roadmap
|
||||
|
||||
This document defines the next product phase after the validated visible-inventory
|
||||
scanner baseline. The scanner is already fast enough for the current milestone;
|
||||
the next work should improve artifact content extraction, review safety, and a
|
||||
minimal inventory experience that makes scanned artifacts useful.
|
||||
scanner baseline and the new native IK-style capture direction. The next capture
|
||||
milestone optimizes speed by moving game-control and card-crop capture into the
|
||||
C# helper. Artifact extraction, review safety, and inventory UX stay separate
|
||||
downstream work so the Electron app remains a visual/status surface instead of
|
||||
the worker.
|
||||
|
||||
For the short current status, see [CURRENT_STATUS.md](CURRENT_STATUS.md).
|
||||
|
||||
## Product Stance
|
||||
|
||||
- Keep the scan workspace focused on operation, not analysis.
|
||||
- Keep the live preview as the dominant surface.
|
||||
- Keep Electron/React out of per-artifact scanner work in the fast path.
|
||||
- Use IK `inventorylists` 1:1 as the scanner dictionary source for artifacts.
|
||||
Other IK lists may stay vendored, but weapons, materials, and character
|
||||
details are not active product scope until their values are actually scanned.
|
||||
- Move debug metrics, OCR internals, and detailed evaluation behind details,
|
||||
diagnostics, or the inventory view.
|
||||
- Do not merge scan confidence and artifact value into one ambiguous score.
|
||||
@@ -64,29 +72,31 @@ reasons in the detail view before it becomes a recommendation source.
|
||||
|
||||
## Planned Pipeline Shape
|
||||
|
||||
The current scan loop can keep shipping while the UI and data contracts are
|
||||
built. A fuller producer/consumer pipeline is a later implementation step:
|
||||
The current TypeScript scan loop can keep shipping as fallback while the native
|
||||
pipeline takes over high-speed capture. The target producer/consumer pipeline is:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Capture["Single capture and game-control worker"]
|
||||
Queue["Bounded screenshot/crop queue"]
|
||||
Native["C# native click, scroll, capture worker"]
|
||||
Queue["Bounded card-crop queue"]
|
||||
OCR["OCR and parse workers"]
|
||||
Eval["Artifact evaluation"]
|
||||
Match["IK inventorylists matching"]
|
||||
Eval["Artifact evaluation (deferred)"]
|
||||
Aggregate["Aggregator and store"]
|
||||
UI["Live rail and inventory"]
|
||||
|
||||
Capture --> Queue
|
||||
Native --> Queue
|
||||
Queue --> OCR
|
||||
OCR --> Eval
|
||||
OCR --> Match
|
||||
Match --> Eval
|
||||
Eval --> Aggregate
|
||||
Aggregate --> UI
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
- Only one worker may control Genshin input, focus, click, scroll, or failsafe
|
||||
polling.
|
||||
- Only the native helper may control Genshin input, focus, click, scroll, card
|
||||
capture, or failsafe polling in the fast path.
|
||||
- OCR/parse/evaluation workers may run concurrently on already captured
|
||||
screenshot/crop jobs.
|
||||
- The queue must be bounded, initially around `4-8` jobs, so the scanner does
|
||||
@@ -94,28 +104,42 @@ Constraints:
|
||||
- The pipeline must preserve current safety rules: no memory reads, hooks,
|
||||
injection, game-file changes, deleting, feeding, enhancing, locking/unlocking,
|
||||
or spending resources.
|
||||
- Do not implement the queue refactor before the result/inventory contracts are
|
||||
stable, unless timing evidence shows the current loop has become the blocker.
|
||||
- Evaluation may be omitted until after capture speed and crop quality are
|
||||
validated.
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 0 - Documentation and contracts
|
||||
|
||||
Status: prepared by this document.
|
||||
Status: prepared by this document and ADR-013.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Product direction is documented.
|
||||
- Main docs point to this roadmap.
|
||||
- Acceptance criteria and checklists exist before code changes.
|
||||
- Native run artifacts are part of the scanner contract:
|
||||
`manifest.json`, `capture-jobs.jsonl`, `status.json`,
|
||||
`scan-results.json`, and `processing-report.json`.
|
||||
|
||||
### Phase 1 - Result data model
|
||||
|
||||
Status: foundation implemented for native post-capture processing. The native
|
||||
scanner reports run status and writes crop jobs; the post-capture processor now
|
||||
writes durable per-artifact `scan-results.json` entries next to the diagnostic
|
||||
`processing-report.json`. Native artifact results can now carry IK inventorylist
|
||||
match metadata, and IK set/piece/slot conflicts force review instead of clean
|
||||
extraction. The post-capture OCR/parse worker runs as a bounded queue while
|
||||
preserving result order. `scan-results.json` also stores parser field confidence
|
||||
metadata for native artifact details. Evaluation remains explicitly deferred.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Add a durable scan result entry model with sequence number, capture metadata,
|
||||
parsed artifact identity, extraction status, artifact value score, value
|
||||
status, duplicate/review flags, and timestamps.
|
||||
status, duplicate/review flags, and timestamps. Initial native entries use
|
||||
`valueStatus: "deferred"` for clean parses and `valueStatus: "review"` for
|
||||
uncertain extraction.
|
||||
- Keep existing stored artifact records compatible.
|
||||
- Add tests for status derivation so low-confidence OCR cannot become a normal
|
||||
`Good` or `Strong` result.
|
||||
@@ -130,6 +154,12 @@ Likely files:
|
||||
|
||||
### Phase 2 - Minimal live result rail
|
||||
|
||||
Status: foundation implemented for native post-capture results. The scan main
|
||||
section shows newest stored artifacts as fallback and can display the latest
|
||||
native `scan-results.json` entries after post-processing. Result rail rows can
|
||||
open the Inventory surface for crop/IK/detail inspection. Value scores are still
|
||||
pending.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Rework the scan main section into preview plus right-side result rail.
|
||||
@@ -148,6 +178,33 @@ Likely files:
|
||||
|
||||
### Phase 3 - Artifact inventory view
|
||||
|
||||
Status: foundation implemented and now scoped to active Artifact scanning only.
|
||||
The app has an `Inventory` navigation item with
|
||||
a compact browser for native `scan-results.json` entries, stored artifacts, and
|
||||
snapshot fallback rows. Filtering, sorting, a detail panel, and secure native
|
||||
crop preview loading are present. The view now surfaces the vendored IK
|
||||
Artifact version/counts, active Artifact-only scope, compact pipeline state for
|
||||
native capture, OCR queue, review, promotion, and evidence, plus per-result
|
||||
IK/GOOD match status for native artifacts. The inventory view also computes a
|
||||
dry-run promotion summary from `scan-results.json` plus the local artifact
|
||||
store, separating `speicherbar`, already stored, review, and blocked native
|
||||
results. One selected clean result can now be promoted after a second UI
|
||||
confirmation; the main process revalidates the run result, writes the store,
|
||||
updates `scan-results.json`, and appends `promotion-log.jsonl`. Weapons,
|
||||
materials, and character details
|
||||
remain hidden from the active feature UI while they are not scanned. The native
|
||||
helper still reports the category distinction in
|
||||
`supportedCategories` via `catalogAvailable`, `nativeCaptureSupported`, and
|
||||
`scanStatus`, so dev-control evidence cannot accidentally claim that every IK
|
||||
catalog has an implemented scanner. Native scan start, status, manifest, and
|
||||
capture jobs now carry an explicit scan category; only `artifacts` can currently
|
||||
produce native capture jobs. Value scoring is still pending; single-result
|
||||
review/edit/approve is implemented in the detail phase below.
|
||||
|
||||
Support code for simple IK weapon, character, and material name/GOOD-key
|
||||
matching exists, but those categories still need their own capture flows before
|
||||
they can be claimed as scanned inventory.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Add a menu item for scanned artifact inventory.
|
||||
@@ -167,6 +224,14 @@ Likely files:
|
||||
|
||||
### Phase 4 - Artifact detail view
|
||||
|
||||
Status: started for native scan results. Inventory detail can show the native
|
||||
card crop from the run directory through the Electron bridge, constrained to
|
||||
PNG files inside the active native run folder. It also shows stored parser
|
||||
field-confidence rows from native `scan-results.json`, IK/GOOD metadata, dry-run
|
||||
promotion state, and a `Naechster Schritt` card. Native Review results now have
|
||||
an inline field editor with approve/reject, authoritative validation, run logs,
|
||||
and review-to-eval export. Value reasons are still pending.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Clicking a live row or inventory item opens detail.
|
||||
@@ -185,6 +250,9 @@ Likely files:
|
||||
|
||||
### Phase 5 - Artifact value evaluation
|
||||
|
||||
Status: next product feature after the now-completed same-session native scale,
|
||||
selected promotion, and review/edit/approve workflow.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Add a deterministic artifact value evaluator before promoting build
|
||||
@@ -204,6 +272,8 @@ Likely files:
|
||||
|
||||
### Phase 6 - Upgrade projection
|
||||
|
||||
Status: later detail-level feature after artifact value evaluation.
|
||||
|
||||
Outcome:
|
||||
|
||||
- For artifacts below max level, show optional projection only in detail.
|
||||
@@ -230,6 +300,11 @@ Outcome:
|
||||
- Preserve stop/failsafe behavior and review decisions.
|
||||
- Compare throughput against the current baseline without weakening accuracy.
|
||||
|
||||
Status: started for post-capture processing. Native capture already writes
|
||||
crop jobs without waiting for OCR. The downstream processor now consumes those
|
||||
jobs with bounded parallelism and writes stable ordered reports. Live throughput
|
||||
comparison remains a final validation item.
|
||||
|
||||
Likely files:
|
||||
|
||||
- `src/lib/autoScanLoop.ts`
|
||||
@@ -239,6 +314,9 @@ Likely files:
|
||||
|
||||
### Phase 8 - Recommendation promotion
|
||||
|
||||
Status: intentionally delayed until native artifact ingestion, review,
|
||||
promotion, value scoring, and repeatability are trustworthy.
|
||||
|
||||
Outcome:
|
||||
|
||||
- Promote account-level recommendations only after scan result quality,
|
||||
@@ -263,6 +341,11 @@ Outcome:
|
||||
- `npm run lint`
|
||||
- `npm test`
|
||||
- `npm run build`
|
||||
- `git diff --check`
|
||||
- `npm run scan:native:smoke` before claiming native capture plus
|
||||
post-capture processing on live Genshin data.
|
||||
- Broader native Artifact runs with 20/50/100 items before claiming IK-style
|
||||
speed or stability.
|
||||
- Add unit tests for score/status derivation and upgrade projection.
|
||||
- For scanner-facing changes, run a low-limit visible-inventory live scan before
|
||||
wider validation.
|
||||
|
||||
Reference in New Issue
Block a user