jeirmeister/Cockatrice
1
0
Fork 0
mirror of https://github.com/Cockatrice/Cockatrice.git synced 2026-09-21 09:05:10 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

[Oracle] Parse sets lazily to slash importer peak memory (#7217)

Browse source 
* [Oracle] Parse sets lazily to slash importer peak memory

- Add a raw JSON scanner that splits the document into per-set byte ranges
  without materializing the JSON tree
- Keep only the raw document bytes and parse one set at a time in startImport
- Take readSetsFromByteArray by value so the wizard's buffer is moved, not copied
- Clear the retained raw data in releaseSetData()/clear()
- Cover the scanner and lazy parsing with tests

Took 2 minutes

* [Oracle] Fix nesting-depth cap, tolerate unescaped control chars, lazy-parse review fixes

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
This commit is contained in:
BruebachL 2026-09-05 20:35:28 +02:00 • committed by GitHub
parent 1dc54617ba
commit c011ea7ceb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 992 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

7
tests/oracle/oracle_importer_benchmark_test.cpp
View file

@ -157,9 +157,10 @@ TEST(OracleBenchmark, ParseJsonThroughput)
for (int i = 0; i < iterations; ++i) {
OracleImporter importer;
QByteArray source = data;
QElapsedTimer timer;
timer.start();
bool ok = importer.readSetsFromByteArray(data);
bool ok = importer.readSetsFromByteArray(std::move(source));
ASSERT_TRUE(ok);
totalMs += timer.elapsed();
}
@ -448,7 +449,7 @@ TEST(OracleBenchmark, ImportRamUsage)
QElapsedTimer timer;
timer.start();
ASSERT_TRUE(importer.readSetsFromByteArray(data));
ASSERT_TRUE(importer.readSetsFromByteArray(std::move(data)));
const qint64 parseMs = timer.elapsed();
const MemorySnapshot afterParse = MemorySnapshot::current();
@ -540,7 +541,7 @@ TEST(OracleBenchmark, ImportRamUsageAllPrintings)
QElapsedTimer timer;
timer.start();
ASSERT_TRUE(importer.readSetsFromByteArray(setsData));
ASSERT_TRUE(importer.readSetsFromByteArray(std::move(setsData)));
const qint64 parseMs = timer.elapsed();
const MemorySnapshot afterParse = MemorySnapshot::current();