jeirmeister/Cockatrice
1
0
Fork 0
mirror of https://github.com/Cockatrice/Cockatrice.git synced 2026-11-07 18:43:26 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

[CI] Key the vcpkg binary cache per ABI and let pull requests heal it

Browse source 
The cache had no prefix, and TAServers/vcpkg-cache restores every entry it
finds under the prefix rather than only the hashes vcpkg asks for. With all
legs sharing the single vcpkg/ namespace, each of the five of them downloads
all 66 archives of all five on every run. Each restore also refreshes the
entry's last access date, so a dead ABI generation stays fresh forever and
GitHub's least recently used eviction can never reach it.

Name the scope in the key after the platform, target and toolchain it belongs
to, so a leg restores only its own archives. macOS 14, 15 and 26 on the same
Xcode build identical binaries when they share the lowest deployment target,
so point all three at 14 and let them share one scope instead of three.

A pull request cannot write the shared cache either. Its entries are scoped to
refs/pull/<n>/merge, so nothing it saves is ever readable again, and letting
it write anyway only burns the repository's quota, which is what the previous
attempt at this did. Keep the build read-only and hand the archives it had to
build itself to a new workflow running in the default branch scope, which
saves them where master can restore them. Forks stay read-only, they are the
one case whose archives must not be trusted.

Because the prefix changes every key, add a job that deletes the entries of
retired scopes, which is the one-time clean up of the current flat namespace,
and the per-run ones pull requests leave behind, which age based eviction
also never reaches.

Also pin the Linux ccache cache to its own subdirectory, since the directory
is shared with the docker build context, and drop the clang-format install
from the build images, which format nothing at runtime.
This commit is contained in:
Lukas Brübach 2026-10-03 17:55:51 +02:00
parent 5dc5390880
commit ccf92e2ea2
8 changed files with 262 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

144
.github/workflows/desktop-build.yml vendored
View file

@ -150,7 +150,7 @@ jobs:
continue-on-error: ${{ matrix.allow-failure == 'yes' }}
timeout-minutes: 70
env:
CACHE: ${{ github.workspace }}/.cache/${{ matrix.distro }}${{ matrix.version }} # directory for caching docker image and ccache
CACHE: ${{ github.workspace }}/.cache/${{ matrix.distro }}${{ matrix.version }} # directory for caching the compiler cache
CCACHE_EVICTION_AGE: 7d
CCACHE_SIZE: 600M # space of all repo is 10Gi: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy
CMAKE_GENERATOR: 'Ninja'
@ -165,7 +165,7 @@ jobs:
uses: actions/cache/restore@v6
with:
key: ccache-${{ matrix.distro }}${{ matrix.version }}
path: ${{ env.CACHE }}
path: ${{ env.CACHE }}/.ccache
- name: "Build ${{ matrix.distro }} ${{ matrix.version }} Docker image"
shell: bash
@ -219,7 +219,7 @@ jobs:
uses: actions/cache/save@v6
with:
key: ${{ steps.restore_ccache.outputs.cache-primary-key }}
path: ${{ env.CACHE }}
path: ${{ env.CACHE }}/.ccache
- name: "Upload artifact"
id: upload_artifact
@ -277,6 +277,7 @@ jobs:
soc: Intel
type: Release
use_ccache: 1
vcpkg_scope: 'vcpkg/macos13-x64-xcode26.3/'
xcode: "26.3"
- os: macOS
@ -293,6 +294,8 @@ jobs:
soc: Apple
type: Release
use_ccache: 1
vcpkg_deployment_target: 14
vcpkg_scope: 'vcpkg/macos14-arm64-xcode26.6/'
xcode: "26.6"
- os: macOS
@ -309,6 +312,8 @@ jobs:
soc: Apple
type: Release
use_ccache: 1
vcpkg_deployment_target: 14
vcpkg_scope: 'vcpkg/macos14-arm64-xcode26.6/'
xcode: "26.6"
- os: macOS
@ -322,6 +327,8 @@ jobs:
soc: Apple
type: Debug
use_ccache: 1
vcpkg_deployment_target: 14
vcpkg_scope: 'vcpkg/macos14-arm64-xcode26.6/'
xcode: "26.6"
- os: Windows
@ -335,6 +342,7 @@ jobs:
qt_version: 6.11.*
qt_modules: qtimageformats qtmultimedia qtshadertools qtwebsockets
type: Release
vcpkg_scope: 'vcpkg/windows10-x64-vs2026/'
name: ${{ matrix.os }} ${{ matrix.target }}${{ matrix.soc == 'Intel' && ' Intel' || '' }}${{ matrix.type == 'Debug' && ' Debug' || '' }}
needs: configure
@ -432,6 +440,15 @@ jobs:
id: vcpkg-cache
uses: TAServers/vcpkg-cache@v3
with:
# Every entry is keyed by vcpkg ABI hash, and one hash set exists per platform,
# target and toolchain. The action restores every entry it finds under the
# prefix, not just the hashes vcpkg asks for, so with one shared vcpkg/
# namespace every leg downloads all 66 archives of all 5 legs on every run.
# Each of those restores also refreshes the entry's last access date, which
# is why a dead ABI generation is never evicted by age. Naming the scope is
# what makes those generations identifiable, and prune-caches below able to
# delete them.
prefix: ${{ matrix.vcpkg_scope }}
token: ${{ secrets.GITHUB_TOKEN }}
# Uses environment variables, see compile.sh for more details
@ -453,9 +470,15 @@ jobs:
PACKAGE_SUFFIX: '${{ matrix.package_suffix }}'
TARGET_MACOS_VERSION: ${{ matrix.override_target }}
USE_CCACHE: ${{ matrix.use_ccache }}
# Only master may write the cache itself. A pull request run has its own
# merge ref scope, which nothing else can read, so anything it saved there
# would be unreachable for every later run. Handing the new archives over
# to vcpkg-cache-import.yml is what lets a pull request heal the shared
# cache after a runner image update changes every ABI hash.
VCPKG_BINARY_SOURCES: "clear;files,${{ steps.vcpkg-cache.outputs.path }},${{ case(github.ref == 'refs/heads/master', 'readwrite', 'read') }}"
VCPKG_DISABLE_METRICS: 1
VCPKG_FEATURE_FLAGS: dependencygraph
VCPKG_TARGET_MACOS_VERSION: ${{ matrix.vcpkg_deployment_target }}
run: .ci/compile.sh --server --test --vcpkg
# CPack reports a failed makensis run as "Problem running NSIS command"
@ -469,6 +492,57 @@ jobs:
shell: bash
run: find . -name NSISOutput.log -exec cat {} +
# A pull request cannot write the shared cache itself: its cache entries are scoped
# to refs/pull/<n>/merge and stay unreadable for every other run, so writing them
# from a pull request only burns the repository's quota. So hand over only the
# archives this leg had to build itself and let vcpkg-cache-import.yml, which runs
# in the default branch scope, save them where later runs can restore them.
# Forks are skipped, they are the one case where the archives must not be trusted.
- name: "Collect new vcpkg entries"
id: vcpkg_new
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
shell: bash
env:
CACHE_DIR: '${{ steps.vcpkg-cache.outputs.path }}'
GH_TOKEN: ${{ github.token }}
VCPKG_SCOPE: '${{ matrix.vcpkg_scope }}'
run: |
# The archive name is the ABI hash, so a missing key is exactly what vcpkg had
# to compile from source in this job.
gh api --paginate \
"repos/$GITHUB_REPOSITORY/actions/caches?ref=refs/heads/master&key=$VCPKG_SCOPE&per_page=100" \
--jq '.actions_caches[].key' | sort -u >"$RUNNER_TEMP/master-keys.txt"
find "$CACHE_DIR" -type f -name '*.zip' -exec basename {} .zip \; |
sed "s|^|$VCPKG_SCOPE|" | sort -u >"$RUNNER_TEMP/local-keys.txt"
comm -23 "$RUNNER_TEMP/local-keys.txt" "$RUNNER_TEMP/master-keys.txt" >"$RUNNER_TEMP/new-keys.txt"
mkdir -p "$RUNNER_TEMP/vcpkg-new"
: >"$RUNNER_TEMP/vcpkg-new/keys.txt"
while read -r key; do
hash="${key##*/}"
archive="$(find "$CACHE_DIR" -type f -name "$hash.zip" -print -quit)"
dir="$RUNNER_TEMP/vcpkg-new/${hash:0:2}" # vcpkg's own two character fanout
mkdir -p "$dir"
cp "$archive" "$dir/"
echo "$key ${hash:0:2}/$hash.zip" >>"$RUNNER_TEMP/vcpkg-new/keys.txt"
done <"$RUNNER_TEMP/new-keys.txt"
count="$(wc -l <"$RUNNER_TEMP/new-keys.txt")"
echo "vcpkg scope $VCPKG_SCOPE: $count entries to import"
echo "count=$count" >>"$GITHUB_OUTPUT"
echo "slug=$(sed -e 's|^vcpkg/||' -e 's|/$||' -e 's|/|-|g' <<<"$VCPKG_SCOPE")" >>"$GITHUB_OUTPUT"
- name: "Upload new vcpkg entries"
# An empty count means the step above did not run at all, on master and on forks
if: steps.vcpkg_new.outputs.count != '' && steps.vcpkg_new.outputs.count != '0'
uses: actions/upload-artifact@v7
with:
archive: false
if-no-files-found: error # a count above zero with nothing staged is a bug
name: vcpkg-new-${{ steps.vcpkg_new.outputs.slug }}
path: ${{ runner.temp }}/vcpkg-new
retention-days: 1
# Delete used cache to emulate a ccache update. See https://github.com/actions/cache/issues/342
- name: "[macOS] Delete remote compiler cache (ccache)"
if: matrix.os == 'macOS' && matrix.use_ccache == 1 && github.ref == 'refs/heads/master' && steps.restore_ccache.outputs.cache-hit
@ -581,3 +655,67 @@ jobs:
BUILD_PATH: ${{ steps.build.outputs.path }}
GH_TOKEN: ${{ github.token }}
run: gh attestation verify "$BUILD_PATH" --repo Cockatrice/Cockatrice
prune-caches:
name: Prune caches
# Only master writes caches at all, so only master may clean them up too
if: github.ref == 'refs/heads/master'
runs-on: ubuntu-slim # https://github.com/actions/runner-images/blob/main/images/ubuntu-slim/ubuntu-slim-Readme.md
continue-on-error: true # freeing space must never fail a build
steps:
# The repository holds 10Gi of cache against entries that want more, and GitHub
# evicts the least recently used entry whenever that is exceeded. The vcpkg
# entries are the reason that never happens: the restore above touches every
# entry under the prefix on every run, so a dead ABI generation keeps its access
# date fresh forever and age based eviction can never reach it. Deleting by age
# alone cannot fix that, only knowing which scope an entry belongs to can.
- name: "Delete unusable cache entries"
shell: bash
env:
GH_TOKEN: ${{ github.token }}
# Must list every vcpkg_scope of the build-vcpkg matrix above, or the entries of
# a scope that is still in use get deleted on the next run.
LIVE_VCPKG_SCOPES: 'vcpkg/macos13-x64-xcode26.3/ vcpkg/macos14-arm64-xcode26.6/ vcpkg/windows10-x64-vs2026/'
run: |
gh api --paginate "repos/$GITHUB_REPOSITORY/actions/caches?per_page=100" \
--jq '.actions_caches[] | [.id, .ref, .key, .last_accessed_at] | @tsv' >"$RUNNER_TEMP/caches.tsv"
cutoff="$(( $(date -u +%s) - 7 * 24 * 3600 ))"
live=" ${LIVE_VCPKG_SCOPES} "
deleted=0
while IFS=$'\t' read -r id ref key accessed; do
[[ -n $id ]] || continue
# Only ever touch the key shapes this workflow creates, so that an entry of
# some other workflow can never be deleted by accident
case "$key" in
vcpkg/* | ccache-* | Qt*) ;;
*) continue ;;
esac
reason=""
if [[ $ref != refs/heads/master ]]; then
reason="scoped to $ref"
elif [[ $key == vcpkg/* ]]; then
# "<scope>/<abi hash>", cut the hash off the end to get the scope
scope="vcpkg/${key#vcpkg/}"
scope="${scope%/*}/"
if [[ $live != *" $scope "* ]]; then
reason="no matrix leg uses $scope"
fi
fi
if [[ -z $reason ]]; then
stamp="$(date -u -d "$accessed" +%s 2>/dev/null || echo 0)"
if [[ $stamp -ne 0 && $stamp -lt $cutoff ]]; then
reason="unused since $accessed"
fi
fi
if [[ -n $reason ]]; then
echo "::notice::deleting $key, $reason"
gh api -X DELETE "repos/$GITHUB_REPOSITORY/actions/caches/$id"
deleted=$((deleted + 1))
fi
done <"$RUNNER_TEMP/caches.tsv"
echo "::notice::deleted $deleted of $(wc -l <"$RUNNER_TEMP/caches.tsv") cache entries"

112
.github/workflows/vcpkg-cache-import.yml vendored Normal file
View file

@ -0,0 +1,112 @@
name: Import vcpkg cache
# A pull request run can only write caches into its own refs/pull/<n>/merge scope, which
# nothing else is allowed to restore. So after a runner image update changes every port's
# ABI hash, the first pull request rebuilds protobuf, openssl and abseil from source and
# the next one does the same, until somebody merges and a master run finally repopulates
# the shared cache. This workflow takes the archives such a run produced and saves them
# in the default branch scope instead, where every later run can restore them.
#
# repository_dispatch is not used on purpose: it would also resolve to the default branch,
# but it counts as a low-trust trigger and may only write there when it declares a
# write-capable cache-mode, which puts it in the same position as workflow_run.
permissions:
actions: write # needed to save entries into the default branch's cache scope
contents: read
on:
workflow_run:
workflows: ["Build Desktop"]
types: [completed]
# Every import writes the same scopes, so let them queue instead of racing each other
concurrency:
group: vcpkg-cache-import
cancel-in-progress: false
jobs:
list:
name: List new vcpkg entries
# Only a successful same-repository pull request has archives worth importing. Forks
# build untrusted code and must never reach a cache that master runs restore and link.
if: >-
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.event == 'pull_request' &&
github.event.workflow_run.head_repository.full_name == github.repository
runs-on: ubuntu-slim # https://github.com/actions/runner-images/blob/main/images/ubuntu-slim/ubuntu-slim-Readme.md
# workflow_run is a low-trust trigger, so cache access is read-only by default even
# though this run resolves to the default branch. Assert below that this took effect,
# because a save under read-only access only warns and would import nothing silently.
cache-mode: write
outputs:
entries: ${{ steps.scan.outputs.entries }}
steps:
- name: "Download new vcpkg entries"
uses: actions/download-artifact@v8
with:
pattern: vcpkg-new-*
path: incoming
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: "Assert write access"
shell: bash
run: |
if [[ $ACTIONS_CACHE_MODE != *write* ]]; then
echo "::error::expected a write-capable cache, got '$ACTIONS_CACHE_MODE'"
exit 1
fi
- name: "List cache entries"
id: scan
shell: bash
working-directory: incoming
run: |
entries="$(find . -name keys.txt -type f -exec sh -c '
for manifest; do
artifact="$(basename "$(dirname "$manifest")")"
while read -r key path; do
printf "{\"artifact\": \"%s\", \"key\": \"%s\", \"path\": \"%s\"}\n" \
"$artifact" "$key" "$path"
done <"$manifest"
done
' _ {} +)"
[[ -n $entries ]] || exit 0
# Compact, because a multi line value cannot be written to GITHUB_OUTPUT
entries="$(jq -cs '.' <<<"$entries")"
echo "entries=$entries" >>"$GITHUB_OUTPUT"
echo "::notice::importing $(jq length <<<"$entries") vcpkg entries"
save:
name: "Save ${{ matrix.entry.key }}"
needs: list
if: needs.list.outputs.entries
runs-on: ubuntu-slim
cache-mode: write
strategy:
# One job per archive, they run in parallel and each only uploads a single file
fail-fast: false
max-parallel: 20
matrix:
entry: ${{ fromJSON(needs.list.outputs.entries) }}
steps:
- name: "Download new vcpkg entries"
uses: actions/download-artifact@v8
with:
pattern: ${{ matrix.entry.artifact }}
path: incoming
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: "Save vcpkg entry"
uses: actions/cache/save@v6
with:
# The payload is a Windows or macOS archive while the upload runs on Linux, and
# without this the entry would be stamped as a Linux cache that neither of them
# is allowed to restore.
enableCrossOsArchive: true
key: ${{ matrix.entry.key }}
path: incoming/${{ matrix.entry.path }}