Compare commits

...

2 commits

Author SHA1 Message Date
BruebachL
0c10798ef4
[CI] Build the Windows solution on more than one MSBuild node (#7379)
Some checks are pending
CodeQL / Analyze (cpp) (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
Build Desktop / Configure (push) Waiting to run
Build Desktop / Debian 13 (push) Blocked by required conditions
Build Desktop / Debian 12 (push) Blocked by required conditions
Build Desktop / Fedora 44 (push) Blocked by required conditions
Build Desktop / Fedora 43 (push) Blocked by required conditions
Build Desktop / Servatrice_Debian 12 (push) Blocked by required conditions
Build Desktop / Ubuntu 24.04 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 (push) Blocked by required conditions
Build Desktop / Arch (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 cover (push) Blocked by required conditions
Build Desktop / Windows 10 (push) Blocked by required conditions
Build Desktop / macOS 13 Intel (push) Blocked by required conditions
Build Desktop / macOS 14 (push) Blocked by required conditions
Build Desktop / macOS 15 (push) Blocked by required conditions
Build Desktop / macOS 26 Debug (push) Blocked by required conditions
Build Desktop / Prune caches (push) Waiting to run
Build Docker / Servatrice (arm) (push) Waiting to run
Build Docker / Servatrice (x86) (push) Waiting to run
Build Docker / Publish multi-platform Servatrice image (push) Blocked by required conditions
cmake --build only passes a job count to MSBuild when it is asked for
one, and .ci/compile.sh never asked, so every Visual Studio generator
build ran single-node: projects were built one after another and MTT
only ever parallelized the files of whichever project happened to be
building. Ask for one node per core, overridable with
BUILD_PARALLEL_LEVEL, and fall back to a single node when the core
count is not exposed.

Alongside it set EnforceProcessCountAcrossBuilds and CL_MPCount, the
knobs Microsoft's build throughput write-up pairs with MTT: without
them the node count and MTT multiply into more cl.exe processes than
the runner has cores for.

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
2026-10-14 16:15:01 +02:00
BruebachL
eefe0bd672
[CI] Collect an MSBuild performance summary and binary log on request (#7378)
The Windows job spends about 28 minutes inside one "Build project"
group, with nothing in it that says where the time goes: MSBuild runs
single-node because no --parallel is passed, so projects build one
after another and it is not visible which projects, tasks or phases are
responsible.

Add a --profile flag that makes the Windows build write
-flp:PerformanceSummary and -bl, repeat the summary in the job log so
the numbers need no download, and report the core count, which decides
how much parallelism is worth asking for. The binary log is uploaded as
an artifact only on the runs that asked for it.

The workflow drives it from a build_profile matrix key that is empty by
default, so no build changes behaviour until someone sets it.

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
2026-10-14 14:27:58 +02:00
2 changed files with 67 additions and 3 deletions

View file

@ -17,8 +17,9 @@
# --dir <dir> sets the name of the build dir, default is "build"
# --cmake-generator <generator> sets CMAKE_GENERATOR as used by cmake
# --target-macos-version <version> sets the min os version - only used for macOS builds
# uses env: BUILDTYPE MAKE_INSTALL MAKE_PACKAGE PACKAGE_TYPE PACKAGE_SUFFIX MAKE_SERVER MAKE_NO_CLIENT MAKE_TEST USE_CCACHE CCACHE_SIZE CCACHE_EVICTION_AGE BUILD_DIR CMAKE_GENERATOR TARGET_MACOS_VERSION MAKE_SANITIZE MAKE_COVERAGE VCPKG_TARGET_MACOS_VERSION
# (correspond to args: --debug/--release --install --package <package type> --suffix <suffix> --server --test --sanitize --coverage --ccache <ccache_size> --dir <dir>)
# --profile writes an MSBuild performance summary and binary log, Windows only
# uses env: BUILDTYPE MAKE_INSTALL MAKE_PACKAGE PACKAGE_TYPE PACKAGE_SUFFIX MAKE_SERVER MAKE_NO_CLIENT MAKE_TEST USE_CCACHE CCACHE_SIZE CCACHE_EVICTION_AGE BUILD_DIR CMAKE_GENERATOR TARGET_MACOS_VERSION MAKE_SANITIZE MAKE_COVERAGE VCPKG_TARGET_MACOS_VERSION BUILD_PARALLEL_LEVEL BUILD_PROFILE
# (correspond to args: --debug/--release --install --package <package type> --suffix <suffix> --server --test --sanitize --coverage --ccache <ccache_size> --dir <dir> --profile)
# exitcode: 1 for failure, 3 for invalid arguments
# Read arguments
@ -115,6 +116,10 @@ while [[ $# != 0 ]]; do
export CMAKE_GENERATOR=$1
shift
;;
'--profile')
BUILD_PROFILE=1
shift
;;
'--target-macos-version')
shift
if [[ $# == 0 ]]; then
@ -209,6 +214,22 @@ function ccachestatsverbose() {
fi
}
function cpuinfo() {
# The number of cores decides how much build parallelism is useful; the hosted
# runners are not all sized alike, so read it off the machine instead of assuming.
case "$RUNNER_OS" in
Windows)
echo "cores: ${NUMBER_OF_PROCESSORS:-unknown}"
;;
macOS)
echo "cores: $(sysctl -n hw.ncpu)"
;;
*)
echo "cores: $(nproc 2>/dev/null || echo unknown)"
;;
esac
}
# Compile
if [[ $RUNNER_OS == macOS ]]; then
# QTDIR is needed for macOS since we actually only use the cached thin Qt binaries instead of the install-qt-action,
@ -293,9 +314,27 @@ if [[ $RUNNER_OS == macOS ]]; then
fi
elif [[ $RUNNER_OS == Windows ]]; then
# cmake only hands MSBuild a job count when one is asked for, and without it
# MSBuild builds a single node: one project at a time, whatever MTT does inside
# it. Ask for a node per core.
if [[ ! $BUILD_PARALLEL_LEVEL ]]; then
BUILD_PARALLEL_LEVEL="${NUMBER_OF_PROCESSORS:-1}"
fi
buildflags+=(--parallel "$BUILD_PARALLEL_LEVEL")
# Enable MTT, see https://devblogs.microsoft.com/cppblog/improved-parallelism-in-msbuild/
# and https://devblogs.microsoft.com/cppblog/cpp-build-throughput-investigation-and-tune-up/#multitooltask-mtt
buildflags+=(-- -p:UseMultiToolTask=true -p:EnableClServerMode=true)
# EnforceProcessCountAcrossBuilds makes the CL_MPCount ceiling apply across all
# the projects the nodes are building, instead of each node running its own
# cl.exe processes and /m and MTT multiplying into an oversubscribed machine.
buildflags+=(-- -p:UseMultiToolTask=true -p:EnableClServerMode=true -p:EnforceProcessCountAcrossBuilds=true -p:CL_MPCount="$BUILD_PARALLEL_LEVEL")
if [[ $BUILD_PROFILE ]]; then
# A performance summary says which projects and tasks cost what, the binary log
# keeps the full timeline; both are far too noisy to always produce. The files
# land next to the solution, i.e. in the build dir the workflow uploads. These
# join the MTT options above, which already carried the "--" separator.
buildflags+=("-flp:PerformanceSummary;v=q;LogFile=msbuild-perf.log" -bl:msbuild.binlog)
fi
fi
if [[ $USE_CCACHE ]]; then
@ -311,10 +350,19 @@ cmake .. "${flags[@]}"
echo "::endgroup::"
echo "::group::Build project"
cpuinfo
echo "Running cmake --build with flags: ${buildflags[*]}"
cmake --build . "${buildflags[@]}"
echo "::endgroup::"
if [[ $BUILD_PROFILE && $RUNNER_OS == Windows && -f msbuild-perf.log ]]; then
# Repeat the summary in the job log so the numbers are readable without
# downloading the binary log, which is hundreds of megabytes.
echo "::group::MSBuild performance summary"
cat msbuild-perf.log
echo "::endgroup::"
fi
if [[ $USE_CCACHE ]]; then
if [[ $CCACHE_EVICTION_AGE ]]; then
echo "::group::evict ccache files older than $CCACHE_EVICTION_AGE"

View file

@ -385,6 +385,10 @@ jobs:
target: 10
runner: windows-2025 # https://github.com/actions/runner-images/blob/main/images/windows/Windows2025-VS2026-Readme.md
# Set build_profile: 1 to have the build write an MSBuild performance
# summary and a binary log; empty on every entry that runs by default.
build_profile: ''
cmake_generator: "Visual Studio 18 2026"
cmake_generator_platform: x64
make_package: 1
@ -400,6 +404,7 @@ jobs:
runs-on: ${{ matrix.runner }}
timeout-minutes: 100
env:
BUILD_PROFILE: '${{ matrix.build_profile }}'
CCACHE_DIR: ${{ github.workspace }}/.cache/
CCACHE_SIZE: 600M # space of all repo is 10Gi: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy
@ -701,6 +706,17 @@ jobs:
build/oracle/Release/*.pdb
build/servatrice/Release/*.pdb
# Written only when the matrix entry sets build_profile, so the artifact
# exists exactly on the runs that were asked to produce one.
- name: "[Windows] Upload MSBuild binary log"
if: always() && matrix.os == 'Windows' && env.BUILD_PROFILE == '1'
uses: actions/upload-artifact@v7
with:
if-no-files-found: error
name: ${{ steps.build.outputs.name }}-msbuild
path: build/msbuild.binlog
retention-days: 7
- name: "Upload to release"
if: needs.configure.outputs.tag != null && matrix.make_package == '1'
id: upload_release