[CI] Run the sanitizer leg nightly and stop caching it (#7780)
Some checks are pending
CodeQL / Analyze (cpp) (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
Build Desktop / Configure (push) Waiting to run
Build Desktop / vcpkg baseline (push) Waiting to run
Build Desktop / Debian 13 (push) Blocked by required conditions
Build Desktop / Fedora 45 release (push) Blocked by required conditions
Build Desktop / Fedora 45 debug (push) Blocked by required conditions
Build Desktop / Servatrice_Debian 12 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 (push) Blocked by required conditions
Build Desktop / Arch (push) Blocked by required conditions
Build Desktop / Debian 12 (push) Blocked by required conditions
Build Desktop / Fedora 44 (push) Blocked by required conditions
Build Desktop / Ubuntu 24.04 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 sanitize (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 cover (push) Blocked by required conditions
Build Desktop / macOS 14 (push) Blocked by required conditions
Build Desktop / macOS 15 (push) Blocked by required conditions
Build Desktop / macOS 27 Debug (push) Blocked by required conditions
Build Desktop / Windows 10 (push) Blocked by required conditions
Build Desktop / macOS 13 Intel (push) Blocked by required conditions
Build Desktop / Prune caches (push) Waiting to run
Build Docker / Servatrice (arm) (push) Waiting to run
Build Docker / Servatrice (x86) (push) Waiting to run
Build Docker / Publish multi-platform Servatrice image (push) Blocked by required conditions

The Ubuntu 26.04 sanitize leg was the largest single cache entry in the 10 GB
quota (1.2G) and is not on the pull-request critical path: Windows sets that
wall clock. Moving it to build-linux-nightly drops the cache, since nightly
legs persist none, and frees the quota for the legs whose bounds still miss.

Measured on master 2026-11-06, two runs: Fedora 45 debug/release served 99.5%
of cacheable calls at 80% full, while Arch served 75% at 96% full, Debian 13
82% at 96% and macOS 14 42% at 99.96%. The sanitize cache was the largest of
the six and is now nightly-only.

Pull requests keep the uninstrumented suite on Debian 13 (Release) and Fedora
45 (Debug), so the only thing a PR loses is the instrumented run, which now
happens nightly and on tags. The leg keeps a 1.2G scratch ccache so the larger
instrumented objects do not thrash mid-build, and it is allow-failure so a
sanitizer finding cannot stop vcpkg-cache-import from importing the nightly
run's archives.

prune-caches reclaims ccache-Ubuntu26.04sanitize on the next master push.
This commit is contained in:
BruebachL 2026-11-07 09:31:27 +01:00 • committed by GitHub
parent 9fdbe45ffb
commit 818c0161ed
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -195,26 +195,11 @@ jobs:
version: 26.04
package: DEB
test: skip # The sanitizer leg below runs this distro's suite
# 18.1 min, and the same distro as the sanitize leg below, which runs this suite.
test: skip # Debian 13 covers the uninstrumented suite on pull requests
# 18.1 min, shorter than the Windows leg. The sanitize leg in the nightly
# matrix runs this distro's instrumented suite.
persist_ccache: 'false'
# The one leg that runs the suite under AddressSanitizer and
# UndefinedBehaviorSanitizer. Debian 13 and Fedora 45 keep covering the
# uninstrumented suite, so nothing is lost by instrumenting this one.
#
# It packages nothing. A sanitized Debug build is not something to ship, and
# carrying the package build here meant 18 minutes of Release compile queued
# behind the instrumented one in a single job. The leg above publishes this
# distro's DEB.
- distro: Ubuntu
version: 26.04
variant: sanitize # separate image and ccache, same distro
package: skip
sanitize: yes
ccache_size: 1200M # instrumented objects are larger than the 614 MB measured; estimated
name: ${{ matrix.distro }} ${{ matrix.version }}${{ matrix.variant && format(' {0}', matrix.variant) || '' }}
needs: configure
# The nightly legs live in build-linux-nightly below. A leg is in exactly one of the two
@ -244,18 +229,23 @@ jobs:
# Debian 13 Release + suite 614 MB 800M
# Fedora 45 debug Debug + suite 660 MB 850M
# Fedora 45 release Release + package 614 MB 800M
# Ubuntu 26.04 sanitize ~1.0 GB 1200M
#
# That is about 5.3 GB of bounds, and with the fixed entries above it puts the quota at
# roughly 9.8 GB of 10 - so only legs worth their share get one. A leg keeps no cache
# when it duplicates another leg's coverage (Ubuntu 26.04 behind the sanitize leg of
# the same distro; macOS 15 and macOS 27 Debug behind macOS 14; Servatrice covers the
# server alone and finishes in 4.7 min). Those four run cold: 4.7, 18.1, 16.5 and
# 15.8 min. The five that stay compile the client on the toolchains a change can break
# and are the ones a cache shortens by the most. Measured job totals on master
# 2026-11-04 also show the ceiling this leaves: Windows 10 is 29.5 min and compiles for
# 23.7 of it with no compiler cache at all, so Windows still sets the pull-request
# wall clock no matter how warm these legs are (see build-vcpkg).
# That is about 4.1 GB of bounds, and with the fixed entries above it puts the quota at
# roughly 8.6 GB of 10 - so only legs worth their share get one. A leg keeps no cache
# when it duplicates another leg's coverage (Ubuntu 26.04 behind Debian 13's Release +
# suite; macOS 15 and macOS 27 Debug behind macOS 14; Servatrice covers the server
# alone and finishes in 4.7 min). Those four run cold: 4.7, 18.1, 16.5 and 15.8 min.
# The five that stay compile the client on the toolchains a change can break and are
# the ones a cache shortens by the most. Measured job totals on master 2026-11-04 also
# show the ceiling this leaves: Windows 10 is 29.5 min and compiles for 23.7 of it
# with no compiler cache at all, so Windows still sets the pull-request wall clock no
# matter how warm these legs are (see build-vcpkg).
#
# Measured on master 2026-11-06: Fedora 45 debug/release hit 99.5% at 80% full, while
# Arch (75% hits at 96% full), Debian 13 (82% at 96%) and macOS 14 (42% at 99.96%)
# still evict entries they need. The sanitize leg's 1.2G cache was the largest of the
# six and is now nightly-only; that is the budget a later re-sizing of those three
# spends.
#
# Re-add a leg only with a bound that holds its whole configuration: a smaller one is
# worse than none.
@ -518,6 +508,28 @@ jobs:
package: DEB
test: skip # Running tests on all distros is superfluous
# The one leg that runs the suite under AddressSanitizer and
# UndefinedBehaviorSanitizer. It runs nightly rather than on pull requests: it is
# not on the pull-request critical path (Windows sets that), and its 1.2G cache
# was the largest single entry in the 10 GB quota. Debian 13 and Fedora 45 keep
# covering the uninstrumented suite on pull requests, so nothing is lost there.
#
# It packages nothing. A sanitized Debug build is not something to ship; the
# pull-request Ubuntu 26.04 leg publishes this distro's DEB.
- distro: Ubuntu
version: 26.04
variant: sanitize # separate image, same distro as the coverage leg below
package: skip
sanitize: yes
# A sanitizer finding must not redden the nightly run: vcpkg-cache-import only
# imports the archives of a run whose conclusion is success. The job itself
# still reports failure.
allow-failure: yes
# Scratch only (nightly legs persist no cache); sized so the instrumented
# objects, which are larger than the 614 MB measured, do not thrash mid-build.
ccache_size: 1200M
# The one leg that runs instrumented for line coverage and publishes the report as
# a workflow artifact. It stays a separate leg rather than a second flag on the
# sanitizer leg because the two instruments answer different questions: the
@ -542,7 +554,7 @@ jobs:
name: ${{ matrix.distro }} ${{ matrix.version }}${{ matrix.variant && format(' {0}', matrix.variant) || '' }}
needs: configure
# A nightly and a tag build cover the legs a pull request or a merge does not wait for:
# the previous release of each distro, and the one instrument that only reports.
# the previous release of each distro, and the instrumented legs.
if: github.event_name == 'schedule' || github.ref_type == 'tag'
runs-on: ubuntu-latest # https://github.com/actions/runner-images
continue-on-error: ${{ matrix.allow-failure == 'yes' }}
@ -555,7 +567,7 @@ jobs:
# A cold nightly build costs minutes on a runner nobody waits for; a cold pull request
# build costs a review. CCACHE_SIZE is still passed so ccache trims its scratch dir.
PERSIST_CCACHE: 'false'
CCACHE_SIZE: 600M
CCACHE_SIZE: ${{ matrix.ccache_size || '600M' }}
CMAKE_GENERATOR: 'Ninja'
NAME: ${{ matrix.distro }}${{ matrix.version }}
IMAGE_SUFFIX: ${{ matrix.variant }}
@ -1216,7 +1228,7 @@ jobs:
# The nightly legs and the pull-request legs that finish under the Windows leg
# keep none, so their leftover entries are reclaimed here instead of waiting out
# the age rule.
LIVE_CCACHE_KEYS: 'ccache-Arch ccache-Debian13 ccache-Fedora45debug ccache-Fedora45release ccache-Ubuntu26.04sanitize ccache-macOS14-xcode-27'
LIVE_CCACHE_KEYS: 'ccache-Arch ccache-Debian13 ccache-Fedora45debug ccache-Fedora45release ccache-macOS14-xcode-27'
run: |
gh api --paginate "repos/$GITHUB_REPOSITORY/actions/caches?per_page=100" \
--jq '.actions_caches[] | [.id, .ref, .key, .last_accessed_at] | @tsv' >"$RUNNER_TEMP/caches.tsv"