* [Tests] Register each test case with CTest instead of each test binary
Registering a test cost four edits across a dozen CMakeLists files: add_test,
add_executable, a conditional add_dependencies on gtest, and target_link_libraries.
The conditional branch was the trap -- it is only needed when GTest was fetched and
add_subdirectory'd, and omitting it yields undefined gtest symbols at link time --
so it was easy to leave out and impossible to notice until the link failed.
Fold all four steps into cockatrice_add_test() and migrate every test directory
onto it, with a cockatrice_add_manual_test() wrapper for the three benchmarks that
are built but deliberately kept out of ctest. The calls now read as a declaration
of what a test is rather than a transcript of how to register one.
While rewriting the registrations, replace add_test(NAME x COMMAND x) with
gtest_discover_tests(). Naming the binary as a single CTest entry meant a CI failure
reported only which binary failed, and ctest -R Suite.Case could not select an
individual case. Discovery raises the suite from 35 opaque entries to 469 real ones:
ctest -R ThemeMigrationTest.IsIdempotent now runs exactly that case, and
ctest -L server / -LE bench work off labels applied at registration.
Give every test a 60s TIMEOUT. CTest's implicit default is 1500s, so one hung
socket wait would stall CI for 25 minutes, and single_instance_manager_test already
does real QLocalSocket waits.
Dropping per-case granularity costs wall-clock time, since the suite becomes
process-spawn bound rather than CPU bound: serial ctest went from 3.8s to 30.8s. Run
it with -j in CI to recover it -- 7.6s at -j4, and 3.5s at -j8, both at or below
the old serial time. The binaries are parallel-safe: the only test claiming a fixed
resource scopes its local socket name to its own pid, so concurrent copies cannot
collide.
Also delete dummy_test, which asserted 1 == 1 and only proved the harness was
wired up -- something the other 468 cases already establish.
No test is added, removed, or changed by this commit; the 469 cases are the same
ones the 35 binaries contained.
* [Tests] Make QtTest available to test targets
QtTest was not among the modules test targets could link, so the Qt model and
signal contracts had no way to be verified from a test: QAbstractItemModelTester
and QSignalSpy both live there.
Add Test to _TEST_NEEDED in cmake/FindQtRuntime.cmake. That is the single place the
union of test-only Qt modules is declared, so this is the only file that needs to
change to make the module reachable from every test target.
Nothing uses it yet. This lands the capability ahead of the model-layer tests that
need it, so those can be added as tests alone rather than as test plus build-system
plumbing.
* [Tests] Correct the note on the oracle test compression defines
add_definitions() populates the directory's COMPILE_DEFINITIONS property, which is
consumed at generate time, so it reaches every target in the directory no matter
where it is called relative to add_executable(). The previous note claimed the
flags stayed confined to the benchmark because the find_package block came after
it; oracle_importer_test, created above that block, carries both defines too.
Verified harmless rather than a live bug: HAS_ZLIB and HAS_LZMA gate code only in
oracle/src/pages.cpp and oracle/src/pagetemplates.cpp, and no test target compiles
either. The ordering and the leaked defines are both pre-existing, so this only
corrects the explanation.
* [Tests] Carry one CTest label per test instead of several
gtest_discover_tests() can hand CTest exactly one label. GoogleTestAddTests.cmake
accumulates its generated script with `foreach(arg ${ARGN})`, so a label list reaches
the CTestTestfile as bare `LABELS ci server`, and CTest reads only the first value of a
property and drops the rest without a word.
The `ci` label that is appended to every registered test was therefore absent from every
test that also asked for `server` or `bench`: `ctest -L ci` reported 400 of 469 cases,
and `ctest --print-labels` never listed a second label anywhere in the build.
Reduce each test to exactly one of `ci`, `server`, or `bench`, which is all the
selections need. The three now partition the suite exactly, 400 + 64 + 5 = 469, and
`ctest -LE bench` still means everything but benchmarks. A second label is now a
configure-time FATAL_ERROR rather than a line that quietly means something else.
Anything orthogonal to a test's kind belongs in a helper flag beside the label, so it
composes with the group instead of competing for the single slot.
Also raise DISCOVERY_TIMEOUT off GoogleTestAddTests' 5s default. A plain debug binary
answers --gtest_list_tests in 0.04s, but that budget is thin once several binaries
discover concurrently behind a -j build, and being over it fails at the *link* step with
a bare "Process terminated due to timeout" that points nowhere near the cause.
* [Tests] Link gtest_main on the fetched-gtest path
The downloaded-gtest branch set GTEST_BOTH_LIBRARIES to `gtest` alone, while
FindGTest sets it to both halves. Every test target links that list rather than
picking an entry point, so on this path nothing provides main() and any test
source that does not define its own fails at link with `undefined reference to
main` pointing at crt1.o, naming neither gtest nor the line responsible.
That stayed latent while every test carried its own four-line main(). A test that
omits it now fails, and the error points at the C runtime rather than at gtest,
which is a long way from the cause. Restoring both halves makes the two paths
agree and drops the boilerplate requirement on new test sources.
The 18 sources that still define main() keep linking: gtest_main.cc defines
nothing but main, so the linker leaves its object in the archive unused rather
than reporting a duplicate.
* [Build] Run the test suite under ASan and UBSan (#7400)
* [Build] Add an ENABLE_SANITIZERS option for ASan and UBSan
Adds -DENABLE_SANITIZERS, off by default, building with AddressSanitizer and
UndefinedBehaviorSanitizer. The bulk of the work is in the three ways the obvious
implementation silently does the wrong thing.
Probe the flags on the link step, not just the compile step. check_cxx_compiler_flag()
only compiles, so every sanitizer flag comes back unsupported, because linking an ASan
object without -fsanitize=address leaves __asan_init undefined. The result is not a
build failure but a build that appears instrumented and is not, which is worse. Use
check_cxx_source_compiles() with CMAKE_REQUIRED_LINK_OPTIONS instead, and treat an
unsupported sanitizer pair as fatal since the cosmetic flags are allowed to degrade.
Opt vendored sources out per source, through a cockatrice_disable_sanitizers_for()
helper rather than a list of set_source_files_properties() calls at the root. The
helper has to exist because CMake resolves a source-file property against the directory
scope its target was declared in: setting the property from the root with absolute paths
is a no-op, silently, which is why the call sites sit next to the add_library and
add_executable that pull the vendored code in. SFMT, minizip, the LZMA SDK and Qxt SMTP
are excluded. oracle/src/qt-json is in the history but not in the tree, removed in
#7214.
Also mark those sources SKIP_PRECOMPILE_HEADERS. The PCH is built with the sanitizer
flags, so it defines __SANITIZE_ADDRESS__; a translation unit compiled with
-fno-sanitize=all does not and GCC then refuses the PCH, which is a fatal
-Werror=invalid-pch in the servatrice and oracle builds.
-fno-sanitize=vptr is not optional. Qt constructs QObject subclasses into raw storage
and writes the vptr afterwards, so the vptr check reports on essentially every QObject
instantiation, and it has no value here because nothing outside moc-generated code
inherits QObject.
Verified on the resulting build: our own translation units carry ASan and UBSan calls,
SFMT.o carries neither, and the full suite of 464 cases passes clean under
ASAN_OPTIONS=detect_leaks=0:abort_on_error=1 with UBSAN_OPTIONS=halt_on_error=0.
The test suite needs two adjustments to make this option usable at all, both of which
land here because they are consequences of COCKATRICE_SANITIZERS existing rather than
independent changes.
DISCOVERY_TIMEOUT rises from 60s to 300s under sanitizers. An instrumented binary takes
2.8s to answer --gtest_list_tests against 0.04s uninstrumented, and discovery runs
several binaries at once behind a -j build.
One target is excluded via a NO_SANITIZE flag, server_developer_role_test. Reaching
google::protobuf::TextFormat::Printer's destructor aborts on an absl assertion inside
system protobuf 33.4 whenever the consumer translation unit is built without NDEBUG,
which a Debug sanitizer build is. This reproduces with no Cockatrice code at all, just
"TextFormat::Printer p; p.PrintToString(msg, &out);" built with -fsanitize=address, so it
is upstream rather than ours. It is a flag and not a fourth label because "skip this in
the sanitizer leg" has to compose with a test's group rather than compete for the single
label slot, and because a flag excludes by not registering the cases at all, which
cannot rot the way a label silently can. The target still builds and still runs on the
legs without sanitizers.
* [CI] Run the suite once under ASan and UBSan
Adds --sanitize to .ci/compile.sh, mirroring --test, and turns it on for one entry of
the existing Linux Docker matrix: Ubuntu 26.04, which already had a test-enabled leg.
That reuses the image, the ccache layer and the existing step shape, so the CI diff is a
matrix key plus the argument plumbing. Debian 13 and Fedora 44 keep covering the
uninstrumented suite, so nothing is lost by instrumenting one leg.
Sanitizers stay off the release and package legs, which build with --release for
packaging: instrumenting those would change the shipped binaries' optimization profile
and build time for no benefit. macOS and Windows stay uninstrumented too, since ASan is
unreliable against the Qt build there and MSVC has no equivalent without /fsanitize=address.
--sanitize does not imply --test, so an instrumented build can be produced locally
without paying for ctest. When both are given, ctest runs with detect_leaks=0 because Qt
and the sanitizer runtimes both leak at exit by design and LeakSanitizer's default buries
real findings under hundreds of process-teardown reports, and with UBSan halt_on_error=0 so
one run collects many findings instead of stopping at the first. ASan keeps
abort_on_error=1, since a bad access leaves the process unusable and continuing only
produces a second, more confusing report from the wreckage.
* [Build] Fix sanitizer flag leak into release package and make UBSan fail CI
- compile.sh always passes -DENABLE_SANITIZERS explicitly so the
release-package reconfigure in the same build dir cannot inherit ON
from the sanitized debug step (which shipped libasan/libubsan DEB deps).
- Sanitizers.cmake resets COCKATRICE_SANITIZERS to OFF when the option is
off (CACHE INTERNAL otherwise sticks forever, keeping NO_SANITIZE skips
and the 300s discovery timeout).
- UBSan now fails the job: -fno-sanitize-recover=undefined at compile time
plus UBSAN_OPTIONS halt_on_error=1 (recover mode exits 0, so the old
halt_on_error=0 could never fail).
---------
Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
---------
Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
* fix(cmake): guard filter_string_test behind WITH_ORACLE or WITH_CLIENT. filter_string_test links against libcockatrice_filters, which is only built when WITH_ORACLE or WITH_CLIENT is enabled. Without this guard, test-only builds fail at configure time because the target doesn't exist. The guard condition mirrors the one in the root CMakeLists.txt that controls whether libcockatrice_filters is built.
* fix(cmake): centralize TEST_QT_MODULES in FindQtRuntime.cmake Each test CMakeLists.txt was independently defining TEST_QT_MODULES with its own subset of Qt modules. This duplicated knowledge that already lives in FindQtRuntime.cmake (which handles module discovery for all other targets: SERVATRICE, COCKATRICE, ORACLE). Consolidate into a single definition using the union of all test requirements (Concurrent Network Svg Widgets), matching the existing pattern for application-target modules. This ensures test-only builds (-DTEST=ON without application targets) discover all necessary Qt components.
* fix(cmake): guard libcockatrice_network behind application targets. libcockatrice_network is only needed by the client, server, and oracle targets. Other application-specific libraries (settings, models, filters) already have similar guards. This was an oversight that caused test-only builds to fail when network dependencies weren't available.
* [DeckList] Refactor load from plaintext to take normalizer as param
* update usages
* weaken unit test
* weaken unit test more
* revert unit test
* move CardNameNormalizer to libcockatrice_card
* update unit test
* formatting
Add lock around deleting arrows for commanding cards
Add support for Qt6 w/ Backwards Qt5
Handle Qt5/6 cross compilation better
Last cleanups
caps matter
Fix serv
Prevent crash on 6.3.0 Linux & bump to 5.8 min
Prevent out of bounds indexing
Delete shutdown timer if it exists
Fixup ticket comments, remove unneeded guards
Try to add support for missing OSes
Update .ci/release_template.md
Update PR based on comments
Update XML name after done and remove Hirsute
Address local game crash
Address comments from PR (again)
Tests don't work on mac, will see if a problem on other OSes
make soundengine more consistent across qt versions
disable tests on distros that are covered by others
Fix Oracle Crash due to bad memory access
Update Oracle to use new Qt6 way of adding translations
Add support for Qt5/Qt6 compiling of Cockatrice
Remove unneeded calls to QtMath/cmath/math.h
Update how we handle bitwise comparisons for enums with Tray Icon
Change header guards to not duplicate function
Leave comment & Fix Path for GHA Qt
Update common/server.h
Update cockatrice/src/window_main.cpp
Rollback change on cmake module path for NSIS
check docker image requirements
add size limit to ccache
put variables in quotes
properly set build type on mac
avoid names used in cmake
fix up cmake module path
cmake 3.10 does not recognize prepend
Support Tests in FindQtRuntime
set ccache size on non debug builds as well
immediately return when removing non existing client
handle incTxBytes with a signal instead
don't set common link libraries in cockatrice/CMakeLists.txt
add comments
set macos qt version to 6
Try upgrading XCode versions to latest they can be supported on
Ensure Qt gets linked
add tmate so i can see what's going on
Qt6 points two directories further down than Qt5 with regard to the top lib path, so we need to account for this
Establish Plugins directory for Qt6
Establish TLS plugins for Qt6 services
Minor change for release channel network manager
Let windows build in parallel cores
Wrong symbols
Qt6 patch up for signal
add missing qt6 package on deb builds
boolean expressions are hard
negative indexes should go to the end
Intentionally fail cache
move size checks to individual zone types
Hardcode libs needed for building on Windows, as the regex was annoying
Update wording
use the --parallel option in all builds
clean up the .ci scripts some more
tweak fedora build
add os parameter to compile.sh
I don't really like this but it seems the easiest way
I'd prefer if these types of quirks would live in the main configuration
file, the yml
fixup yml
readd appended cache key to vcpkg step
fix windows 32 quirk
the json hash is already added to the key as well
remove os parameter and clean up ci files
set name_build.sh to output relative paths
set backwards compatible version of xcode and qt on mac
set QTDIR for mac builds on qt5
has no effect for qt6
export BUILD_DIR to name_build.sh
merge mac build steps
merge homebrew steps, set package suffix
link qt5
remove brew link
set qtdir to qt5 only
compile.sh vars need to be empty not 0
fix sets manager search bar on qt 5.12/15
fix oracle subprocess errors being ignored on qt 5
clean up translation loading
move en@source translation file so it will not get included in packages
NOTE: this needs to be done at transifex as well!
Use generator platform over osname
Short circuit if not Win defined
* improve loading from plain text
Fixes the loadFromStream_Plain function that is used to load plain text
decklists.
The rewritten function uses more regexes and is a bit cleaner.
This fixes multiple bugs with loading the various sources of decklists.
Note that the new function still has a few issues that are shared with
the original version like creating duplicate cards.
* clang format comments
apparently clang-format even complains about the spacing in your
comments
* refactor loading_from_clipboard tests
Remove all heap allocation and use references.
Use std::pair and std::string so gtest will show the cardnames in error messages.
(note that using QPair or QString does not work with gtest)
Improve the last two testcases to include weird names; and name and
comments testing.
Remove empty header file.
* fix compatibility with more formats
skip "sideboard" line
include everything in mainboard when there are multiple empty lines
add removal of the mwdeck cardversion selector in round braces
add replacal of lowercase ae combination that should never occur
Set cardname to lowercase as apparently our checks are hardcoded to only
accept lowercase.
* remove bugged test
The current load from plain is simply broken, removed checking the
comments for correct contents.
* rework load_from_clipboard tests again
rework the test to have less code duplication
add more tests and more special cases
note that text is still all lowercase
* improve loading from plain text
Fixes the loadFromStream_Plain function that is used to load plain text
decklists.
The rewritten function uses more regexes and is a bit cleaner.
This fixes multiple bugs with loading the various sources of decklists.
Note that the new function still has a few issues that are shared with
the original version like creating duplicate cards.
* clang format comments
apparently clang-format even complains about the spacing in your
comments
* refactor loading_from_clipboard tests
Remove all heap allocation and use references.
Use std::pair and std::string so gtest will show the cardnames in error messages.
(note that using QPair or QString does not work with gtest)
Improve the last two testcases to include weird names; and name and
comments testing.
Remove empty header file.
* fix compatibility with more formats
skip "sideboard" line
include everything in mainboard when there are multiple empty lines
add removal of the mwdeck cardversion selector in round braces
add replacal of lowercase ae combination that should never occur
Set cardname to lowercase as apparently our checks are hardcoded to only
accept lowercase.
* remove bugged test
The current load from plain is simply broken, removed checking the
comments for correct contents.
* rework load_from_clipboard tests again
rework the test to have less code duplication
add more tests and more special cases
note that text is still all lowercase
* remove forcing of lowercase cardnames
Cardnames in DeckList::loadFromStream_Plain will no longer be forced
lowercase if they aren't found in the database.
Empty lines in the comments of plaintext decklists will not be skipped.
The loading_from_clipboard_test gets its functions declared in a
separate header "clipboard_testing.h".
Add more edgecase tests.
Refactor code.
* add old QHash version support
QT 5.5 does not support using initializer lists for QHash.
Implement a preprocessor version check for conditionally using inserts
instead of a const with initializer list.
* add old QHash version support
QT 5.5 does not support using initializer lists for QHash.
Implement a preprocessor version check for conditionally using [] access
instead of a const with initializer list.
* add qHash on QRegularExpression below QT 5.6
Apparently QRegularExpression can't be hashed in lower QT versions,
so we add our own hash function, and everyone lived happily ever after,
and none the wiser.
* add header guards to clipboard_testing.h
