[CI] Publish a gcovr line-coverage report from the test run

Sanitizers tell us what a run did wrong; they say nothing about what it never
reached. This adds the other half: an instrumented build whose report says which
of our own lines no test executes.

Add an ENABLE_COVERAGE option that compiles with --coverage -O0 -g, mirroring the
ENABLE_SANITIZERS opt-in including the per-source carve-out that keeps vendored
trees out of the denominator. -O0 is deliberate: at higher optimisation levels GCC
merges and relocates lines, so the report would describe code that no longer exists
in that form. Only GCC is supported, since Clang's intermediate format needs
llvm-cov; asking for coverage under Clang is a hard error rather than a build that
silently produces unreadable .gcno files.

Keep the filters in a committed gcovr.cfg rather than in the invocation. The tests
themselves are excluded because they are compiled with the same instrumentation
and are only ever executed by the run doing the measuring, so leaving them in would
report a healthier project than we have. The process entry points and the client's
window chrome go too: each needs a display, an interactive terminal or a live
database before it does anything a headless test could observe, so they would
otherwise sit at 0% forever and bury real movement.

Give coverage its own matrix leg rather than a second flag on the sanitizer leg.
The two answer different questions, and combining them slows both down without
telling us more. The new variant key gives the leg its own Docker image and ccache
entry so the two legs cannot race on the same tag.

The leg publishes an HTML report with per-file pages, LCOV, Cobertura XML and a
JSON summary. gcovr is a report-time dependency rather than a build dependency, so
a missing one warns and skips instead of failing an otherwise green build.
This commit is contained in:
Lukas Brübach 2026-10-05 14:04:51 +02:00
parent dd639ed52c
commit 2c249438d5
11 changed files with 242 additions and 9 deletions

View file

@ -9,14 +9,16 @@
# --server compiles servatrice
# --test runs tests
# --sanitize builds with AddressSanitizer and UndefinedBehaviorSanitizer, gcc/clang only
# --coverage instruments the test run for line coverage and prints a gcovr summary, gcc only
# --debug or --release sets the build type ie CMAKE_BUILD_TYPE
# --ccache [<size>] uses ccache and shows stats, optionally provide size
# --evict-ccache <age> runs ccache eviction based on given age after build
# --dir <dir> sets the name of the build dir, default is "build"
# --cmake-generator <generator> sets CMAKE_GENERATOR as used by cmake
# --target-macos-version <version> sets the min os version - only used for macOS builds
# uses env: BUILDTYPE MAKE_INSTALL MAKE_PACKAGE PACKAGE_TYPE PACKAGE_SUFFIX MAKE_SERVER MAKE_NO_CLIENT MAKE_TEST USE_CCACHE CCACHE_SIZE CCACHE_EVICTION_AGE BUILD_DIR CMAKE_GENERATOR TARGET_MACOS_VERSION
# (correspond to args: --debug/--release --install --package <package type> --suffix <suffix> --server --test --ccache <ccache_size> --dir <dir>)
# uses env: BUILDTYPE MAKE_INSTALL MAKE_PACKAGE PACKAGE_TYPE PACKAGE_SUFFIX MAKE_SERVER MAKE_NO_CLIENT MAKE_TEST USE_CCACHE CCACHE_SIZE CCACHE_EVICTION_AGE BUILD_DIR CMAKE_GENERATOR TARGET_MACOS_VERSION MAKE_SANITIZE MAKE_COVERAGE
# (correspond to args: --debug/--release --install --package <package type> --suffix <suffix> --server --test --sanitize --coverage --ccache <ccache_size> --dir <dir>)
# exitcode: 1 for failure, 3 for invalid arguments
# Read arguments
@ -62,6 +64,10 @@ while [[ $# != 0 ]]; do
MAKE_SANITIZE=1
shift
;;
'--coverage')
MAKE_COVERAGE=1
shift
;;
'--debug')
BUILDTYPE="Debug"
shift
@ -138,6 +144,10 @@ fi
mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"
# Captured before the cd because $0 is relative to the directory the script was invoked from,
# and after the cd it no longer resolves to anything. The coverage report needs the source root.
SRC_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
# Set minimum CMake Version
export CMAKE_POLICY_VERSION_MINIMUM=3.10
@ -155,6 +165,9 @@ fi
if [[ $MAKE_SANITIZE ]]; then
flags+=("-DENABLE_SANITIZERS=ON")
fi
if [[ $MAKE_COVERAGE ]]; then
flags+=("-DENABLE_COVERAGE=ON")
fi
if [[ $USE_CCACHE ]]; then
flags+=("-DUSE_CCACHE=1")
# PCH-aware caching is required or ccache refuses to cache any TU that
@ -334,6 +347,31 @@ if [[ $MAKE_TEST ]]; then
echo "::endgroup::"
fi
if [[ $MAKE_COVERAGE ]]; then
# gcovr is not a build dependency and is not available everywhere, so a missing one is reported
# and skipped rather than failing an otherwise green build.
echo "::group::Coverage report"
GCOVR="$(command -v gcovr || true)"
if [[ -z $GCOVR ]]; then
echo "gcovr not found; install it to produce a coverage report"
else
"$GCOVR" --version
# This script configures with `cmake ..`, so the build dir it was started from is the object
# directory and the source root is one level up. Both are passed explicitly: gcovr otherwise
# walks the current directory and trips over in-tree build directories left by other tooling,
# whose stale .gcno files cannot be resolved. --config is named too, so the filters apply no
# matter what directory gcovr happens to start from.
GCOVR_ARGS=("." --root "$SRC_ROOT" --object-directory "." --config "$SRC_ROOT/gcovr.cfg")
"$GCOVR" "${GCOVR_ARGS[@]}" --txt --decisions
"$GCOVR" "${GCOVR_ARGS[@]}" --html-details coverage-details.html --html-title "Cockatrice line coverage"
"$GCOVR" "${GCOVR_ARGS[@]}" --lcov -o coverage.lcov
"$GCOVR" "${GCOVR_ARGS[@]}" --cobertura-pretty -o coverage.xml
"$GCOVR" "${GCOVR_ARGS[@]}" --json-summary-pretty -o coverage-summary.json
echo "Coverage artifacts written to coverage-details.html, coverage.lcov, coverage.xml and coverage-summary.json"
fi
echo "::endgroup::"
fi
if [[ $MAKE_INSTALL ]]; then
echo "::group::Install"
cmake --build . --target install --config "$BUILDTYPE"