jeirmeister/Cockatrice
1
0
Fork 0
mirror of https://github.com/Cockatrice/Cockatrice.git synced 2026-09-22 09:35:08 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

[Oracle] Parse sets lazily to slash importer peak memory (#7217)

Browse source 
* [Oracle] Parse sets lazily to slash importer peak memory

- Add a raw JSON scanner that splits the document into per-set byte ranges
  without materializing the JSON tree
- Keep only the raw document bytes and parse one set at a time in startImport
- Take readSetsFromByteArray by value so the wizard's buffer is moved, not copied
- Clear the retained raw data in releaseSetData()/clear()
- Cover the scanner and lazy parsing with tests

Took 2 minutes

* [Oracle] Fix nesting-depth cap, tolerate unescaped control chars, lazy-parse review fixes

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
This commit is contained in:
BruebachL 2026-09-05 20:35:28 +02:00 • committed by GitHub
parent 1dc54617ba
commit c011ea7ceb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 992 additions and 45 deletions
Download patch file Download diff file Expand all files Collapse all files

4
tests/oracle/CMakeLists.txt
View file

@ -11,7 +11,7 @@ add_test(NAME parse_cipt_test COMMAND parse_cipt_test)
# Oracle importer unit tests
add_executable(
oracle_importer_test ${VERSION_STRING_CPP} ../../oracle/src/oracleimporter.cpp ../../oracle/src/parsehelpers.cpp
oracle_importer_test.cpp
../../oracle/src/raw_json_scanner.cpp oracle_importer_test.cpp
)
if(NOT GTEST_FOUND)
@ -51,7 +51,7 @@ endif()
add_executable(
oracle_importer_benchmark_test
${VERSION_STRING_CPP} ../../oracle/src/oracleimporter.cpp ../../oracle/src/parsehelpers.cpp
oracle_importer_benchmark_test.cpp ${_ORACLE_BENCH_EXTRA_SOURCES}
../../oracle/src/raw_json_scanner.cpp oracle_importer_benchmark_test.cpp ${_ORACLE_BENCH_EXTRA_SOURCES}
)
if(NOT GTEST_FOUND)