Commit graph

3 commits

Author SHA1 Message Date
BruebachL
7d867b9745
[Oracle/Client] Report card database download progress (#7253)
Some checks failed
CodeQL / Analyze (cpp) (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
Build Desktop / Configure (push) Has been cancelled
Build Docker / Servatrice (arm) (push) Has been cancelled
Build Docker / Servatrice (x86) (push) Has been cancelled
Build Desktop / Debian 13 (push) Has been cancelled
Build Desktop / Debian 12 (push) Has been cancelled
Build Desktop / Fedora 44 (push) Has been cancelled
Build Desktop / Fedora 43 (push) Has been cancelled
Build Desktop / Servatrice_Debian 12 (push) Has been cancelled
Build Desktop / Ubuntu 26.04 (push) Has been cancelled
Build Desktop / Ubuntu 24.04 (push) Has been cancelled
Build Desktop / Arch (push) Has been cancelled
Build Desktop / macOS 13 Intel (push) Has been cancelled
Build Desktop / macOS 14 (push) Has been cancelled
Build Desktop / macOS 15 (push) Has been cancelled
Build Desktop / macOS 26 Debug (push) Has been cancelled
Build Desktop / Windows 10 (push) Has been cancelled
Build Docker / Publish multi-platform Servatrice image (push) Has been cancelled
* [Oracle/Client] Report card database download progress

Card database updates ran invisibly: MTGJSON parsing spun an indeterminate
bar on the UI thread and the set import blocked the window, while the
onboarding wizard spawned `oracle -b` with no progress to show at all.

- Add byte-level scan progress to `RawJson::scanSetRanges` via an optional
  callback, throttled to ~100 reports per scan.
- Emit `OracleImporter::dataReadProgress` during the scan and import sets on a
  worker thread, driving the wizard's progress bar per set.
- With `-b`, write machine-readable `PROGRESS <stage> <done> <total>` lines to
  stdout for the download/scan/import stages; stderr keeps the log output.
- Parse the oracle stdout in `MainWindow` and forward it to the onboarding
  wizard, giving the card database step a determinate bar with stage-specific
  status text.
- Guard the async workers against the wizard being closed mid-run.
- Add Google Test coverage for scan progress reporting.

* [Oracle/Client] Harden oracle progress workers and quit prompt

Address review feedback on the download-progress change: decompress and read
sets files off the UI thread, cancel the load/import workers before the
wizard can tear down the importer, and show an 'Extracting file...' status
plus a clean 100% tail so the poll never looks stuck. Quitting Cockatrice
while a card database update runs now asks for confirmation.

* Show 100% for 500ms on complete.

* Disable buttons on set import until done.

* Clean up progress bar.

* Drop wrapper around lambda

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
2026-09-09 22:27:16 +02:00
BruebachL
c011ea7ceb
[Oracle] Parse sets lazily to slash importer peak memory (#7217)
* [Oracle] Parse sets lazily to slash importer peak memory

- Add a raw JSON scanner that splits the document into per-set byte ranges
  without materializing the JSON tree
- Keep only the raw document bytes and parse one set at a time in startImport
- Take readSetsFromByteArray by value so the wizard's buffer is moved, not copied
- Clear the retained raw data in releaseSetData()/clear()
- Cover the scanner and lazy parsing with tests

Took 2 minutes

* [Oracle] Fix nesting-depth cap, tolerate unescaped control chars, lazy-parse review fixes

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
2026-09-05 20:35:28 +02:00
BruebachL
61e6a9913e
[Oracle] Add oracle importer tests and fix set parsing details (#7215)
Some checks are pending
CodeQL / Analyze (cpp) (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
Build Desktop / Configure (push) Waiting to run
Build Desktop / Debian 13 (push) Blocked by required conditions
Build Desktop / Debian 12 (push) Blocked by required conditions
Build Desktop / Fedora 44 (push) Blocked by required conditions
Build Desktop / Fedora 43 (push) Blocked by required conditions
Build Desktop / Servatrice_Debian 12 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 (push) Blocked by required conditions
Build Desktop / Ubuntu 24.04 (push) Blocked by required conditions
Build Desktop / Arch (push) Blocked by required conditions
Build Desktop / macOS 13 Intel (push) Blocked by required conditions
Build Desktop / macOS 14 (push) Blocked by required conditions
Build Desktop / macOS 15 (push) Blocked by required conditions
Build Desktop / macOS 26 Debug (push) Blocked by required conditions
Build Desktop / Windows 10 (push) Blocked by required conditions
Build Docker / Servatrice (arm) (push) Waiting to run
Build Docker / Servatrice (x86) (push) Waiting to run
Build Docker / Publish multi-platform Servatrice image (push) Blocked by required conditions
* [Oracle] Add oracle importer tests and fix set parsing details

- Add oracle_importer_test and oracle_importer_benchmark_test targets
- Preserve the first printing's legalities when an existing card is reused
- Concatenate split-card coloridentity and sort/dedupe card colors
- Use a raw string for the Basic Land format regex
- Pre-allocate the card hash and micro-optimize string handling

Took 2 minutes

* [Oracle/Tests] Pin cmc coercion in CI run; scope the reserve pass

The #7214 coercion assertion lived only in oracle_importer_benchmark_test,
which gets no add_test and so never runs under ctest. Add NumericManaValueCoercedToCmc
and LegacyConvertedManaCostCoercedToCmc to oracle_importer_test (a CI-ran
binary): manaValue/convertedManaCost are JSON numbers in AllPrintings, and
QJsonValue::toString() would drop them to an empty cmc without the
#7214 coercion fix.

Wrap the distinct-name reserve pass in a bare block so the ~35k name
QStrings are handed back before the memory-heavy import loop starts.

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
2026-09-04 22:14:16 +02:00