Cockatrice/cockatrice/src/client/crash/crash_reporter.h
BruebachL 9f59f5c495
[CrashReporting] Breakpad capture, session detection and symbol upload (#7523)
* [CrashReporting] Breakpad everywhere, session detection, symbol CI (inc 3)

Breakpad on all platforms (no crashpad packaging); remove the Windows
MiniDumpWithFullMemory filter and the installer WER LocalDumps full-dump
registration. QLockFile per-process session tracking replaces the
Windows-only .dmp scan, placed after parser.process and the instance
hand-off so help/version exits and handed-off secondaries never leak a
lock. CI: Release-only DSN bake from SENTRY_DSN secret, tag-only
sentry-cli symbol upload (org cockatrice, project native).

Tests: 533/533 ctest green, format clean, code gate PASS.

* [CrashReporting] Enable http_retry, harden symbol upload, fix NSIS context

Sets sentry_options_set_http_retry so the cache_keep-held crash
envelope in Ask mode is written with a retry filename and actually
flushes on giveConsent() instead of sitting in <db>/cache until
pruning.

Adds continue-on-error to both sentry-cli install steps so a Sentry
registry outage or broken Git Bash install cannot block release
assets.

Cleans the legacy WER CrashDumps folder under SetShellVarContext
current (under all it resolves to ProgramData, not the user's
AppData) in installer and uninstaller; registry delete works in
either context.

* [CrashReporting] Extract the startup crash-report flow from main() per review

* [CrashReporting] Break up the CrashReporter class doc per review

* [CrashReporting] Drop the invalid --release from the symbol upload

`debug-files upload` never accepted `--release` (checked against the pinned
sentry-cli 3.8.0: the subcommand declares --org/--project and a set of
type/id/scan options, and execute() reads no release; the only global flags are
--url/--header/--auth-token/--log-level/--quiet/--allow-failure). Clap rejects
the unknown flag before any upload happens, so every tag build has been shipping
without symbols, with continue-on-error keeping the release pipeline green.

Release association is also the wrong mechanism: debug information files are
matched to events by debug ID via the event's debug_meta. The Sentry release of
a native event comes from the event itself (COCKATRICE_SENTRY_RELEASE), not from
the DIF upload. Remove the flag and the now-unused SENTRY_RELEASE env from both
invocations (Linux, and Windows/macOS).

The symbol upload stays continue-on-error (a Sentry outage must not block the
release assets); revisit once it has proven itself on a tag.

* [CrashReporting] Split debug information out of packaged release binaries

-g is required so the CI symbol upload has DWARF to send, but it was left in
the shipped binaries too: the Linux Release client is 563 MiB, 552 MiB of it
debug info, and every distribution channel pays for that.

Keep the build tree unchanged (CI uploads symbols from build/) and strip only
the packaged copy. A CPack pre-build script extracts each ELF's debug info to
COCKATRICE_DEBUG_SYMBOL_DIR and then runs `strip --strip-debug --strip-unneeded`.
.eh_frame is allocatable and survives, so unwinding is unaffected, and the build
ID is identical in the stripped binary and the sidecar.

Measured on a local Release build (GCC -O2 -g, WITH_CRASH_REPORTING=ON),
bytes:

  binary      build tree  packaged before  packaged after  .debug sidecar
  cockatrice 590,722,560      590,722,560      28,086,480     579,623,576
  oracle      53,432,608       53,432,608       1,649,888      52,075,344
  tar.gz of both trees: 289,964,521 -> 19,018,058 (~15x smaller).

Only ELF (Linux/BSD) is handled: Windows already links /DEBUG with separate
PDBs, and macOS extracts a dSYM from the bundle. Opt-out via
COCKATRICE_SPLIT_DEBUG_SYMBOLS; skipped with a warning when objcopy/strip are
missing. CPACK_PRE_BUILD_SCRIPTS (not CPACK_INSTALL_SCRIPTS, which runs before
staging) invokes it, and the script refuses to run unless CPACK_TEMPORARY_DIRECTORY
is a real CPack staging tree so a missing variable can never turn its glob into
a filesystem-wide walk.

* [CrashReporting] Upload Windows PDBs from where Ninja actually puts them

The Windows branch of the symbol upload looked in build/<target>/Release, which
is the Visual Studio generator's layout. The Windows leg uses Ninja, which is
single-config: linker PDBs land in build/<target>/, next to the binaries — the
same paths the "[Windows] Upload PDBs" artifact step below globs with
if-no-files-found: error.

So paths was always empty on Windows, the step printed "No symbol paths found,
skipping symbol upload" and exited 0: a silent no-op behind continue-on-error.
Every Windows build would then report "A required debug information file was
missing" for cockatrice.exe with a PDB-style debug id (GUID-age), exactly as if
no upload step existed at all.

Use the artifact step's globs. Also turn the empty-path case into a failure in
both symbol steps: a silent skip is indistinguishable from a successful upload
(which is how the wrong path went unnoticed), and continue-on-error already
keeps a broken upload from failing the release.

Verified by extracting both steps' run blocks and executing the Windows branch
against a Ninja-style tree: it now passes the three linker PDBs to sentry-cli
(and not the CMakeFiles/*.pdb intermediates), and exits non-zero with an
::error:: annotation when there is nothing to upload.

* [CrashReporting] Archive PDBs on tag builds so a failed upload is recoverable

The "[Windows] Upload PDBs" artifact step skipped tag builds
(github.ref_type != 'tag'), so for a release the symbols existed only inside
the Sentry upload — which is continue-on-error. A failed upload therefore lost
the ability to symbolicate that release permanently: no rebuild produces the
same PDB, since the debug id is a per-build GUID+age. That is how every
Windows crash reported so far became unreadable.

Archive the PDBs for tag builds as well, and order the step after
"Upload to release" so a missing PDB can never block a release from being
published. Retention stays at the repository default; nothing consumes the
artifact automatically, it is the manual recovery path
(gh run download <run> -n <name>-PDBs, then sentry-cli debug-files upload).

Verified by re-parsing the workflow and asserting the step order
(release -> PDBs -> attest) and the broadened condition.

* [CrashReporting] Upload the Windows executables with their PDBs

Sentry needs two things per image on Windows: symbolication and stack
unwinding. The PDB covers symbolication, but for a 64-bit PE the Call Frame
Information (.pdata/.xdata) lives in the executable and is not copied into the
PDB, so uploading PDBs alone leaves the image at "Stack Unwinding: Missing".
That is exactly what the event's Images panel showed for cockatrice.exe, while
Sentry's own Microsoft symbol server resolved KERNELBASE.dll and ucrtbase.dll
from the same dump.

sentry-cli also needs the pair in the same invocation: only then does it
associate the PE's code identifier with the PDB's debug identifier, otherwise
"the identifiers diverge and Sentry might not be able to resolve the correct
file for symbolication" (docs: Debug Identifiers).

Do the same for macOS, where the dSYM carries the debug information but the
Mach-O carries the unwind information, and include the executables in the
symbol artifact so a re-upload still has the pair to offer.

Verified by extracting the step and running both branches against stub
harnesses: Windows passes the three exe/pdb pairs in a single sentry-cli
invocation and still fails loudly on an empty tree; macOS passes the dSYM and
the Mach-O together.

* [CrashReporting] Upload the shipped Qt runtime DLLs too

Qt ships no PDBs with its MSVC SDK, so every Qt frame in a crash report
resolved to nothing: an image with neither PDB nor PE has no unwind
information (on 64-bit Windows the Call Frame Information lives in the PE
and is not copied into the PDB) and no symbol source to fall back on.

The list is required-runtime.txt, the manifest .ci/compile.sh already
asserts the installer against, so what gets uploaded is exactly what an
install carries. Qt frames now get unwinding plus export-table names; line
numbers still need Qt's debug information files, which the SDK omits.

The guards are not cosmetic: the step runs under `bash -eo pipefail`, where
a sed over a missing file or an unmatched ls glob aborts the whole step, and
continue-on-error makes that look identical to a successful upload.

* [CrashReporting] Upload only the binaries, archive symbols only on tags

The Linux leg handed sentry-cli whole build directories: 471 candidate files,
of which 7 were object files that can never match a stack frame. Only the
three linked binaries carry the build-ids Sentry looks up; the packaged copy
is split and stripped but keeps that same build-id.

The Windows symbol archive ran on every packaging build even though only tag
builds upload to Sentry, so a PR build paid for an artifact nothing could
ever consume. Tags keep it as the durable copy for a failed upload.

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>
2026-10-19 14:57:04 +02:00

177 lines
7.2 KiB
C++

/**
* @file crash_reporter.h
* @ingroup Core
*/
#ifndef CRASH_REPORTER_H
#define CRASH_REPORTER_H
#include <QByteArray>
#include <QLoggingCategory>
#include <QString>
#include <QStringList>
#include <libcockatrice/settings/personal_settings.h>
#include <memory>
#ifndef COCKATRICE_WITH_CRASH_REPORTING
#define COCKATRICE_WITH_CRASH_REPORTING 0
#endif
#ifndef COCKATRICE_SENTRY_DSN
#define COCKATRICE_SENTRY_DSN ""
#endif
#ifndef COCKATRICE_SENTRY_RELEASE
#define COCKATRICE_SENTRY_RELEASE ""
#endif
class QLockFile;
Q_DECLARE_LOGGING_CATEGORY(CrashReportLog);
/**
* @brief Sentry-backed crash-report facade, owned by main().
*
* An empty DSN (or Never consent) skips the SDK entirely, so the mock dialog
* fallback behaves exactly as before. All SDK calls are guarded so OFF builds
* compile to no-ops.
*
* @par Consent gating
* init() arms require-user-consent and before_send scrubbing. Stale persisted
* consent from a previous AlwaysSend run is cleared before init when the mode is
* not AlwaysSend, and cached crash envelopes are held for the dialog instead of
* being dropped. giveConsent()/revokeConsent() gate uploads.
*
* @par Envelope and log helpers
* attachToPendingCrashEnvelopes() folds the redacted log and the optional user
* note into the pending native crash event(s) before consent releases them, so
* dump, stack, log and note arrive as ONE Sentry event. sendReport() is the
* fallback for unclean exits that produced no native crash event, capturing a
* redacted message event with the log preview attached. The log redaction
* helpers are static and SDK-independent.
*
* @par Session tracking
* beginSession()/endSession() are SDK-independent: a QLockFile per process in
* the database dir detects unclean exits cross-platform, replacing the legacy
* Windows-only .dmp scan. The held lock makes the reporter non-copyable and
* non-movable.
*/
class CrashReporter
{
public:
CrashReporter();
CrashReporter(const CrashReporter &) = delete;
CrashReporter &operator=(const CrashReporter &) = delete;
CrashReporter(CrashReporter &&) = delete;
CrashReporter &operator=(CrashReporter &&) = delete;
~CrashReporter();
void init(const QString &dsn,
const QString &release,
const QString &databasePath,
PersonalSettings::CrashReportingMode consentMode);
[[nodiscard]] bool isReady() const;
void shutdown();
void giveConsent();
void revokeConsent();
bool sendReport(const QString &redactedPreview, const QString &userNote = QString());
/**
* @brief Append the redacted log (and optional user note) to every pending
* native crash envelope in the local database, so the crashed event uploads
* with dump, stack, log and note together instead of the log arriving as a
* separate event that has to be correlated by hand. Both storage locations
* are covered: consent-held envelopes in <database>/cache (Ask mode) and
* in-flight envelopes dumped into <database>/<run>.run (AlwaysSend mode).
*
* For the AlwaysSend variant run this BEFORE init() (static overload): the
* startup replay of *.run envelopes happens inside sentry_init. SDK-independent
* plain-file surgery on the Sentry envelope format, so it stays unit-testable
* in OFF builds.
*
* @return the number of envelopes that received the attachments.
*/
int attachToPendingCrashEnvelopes(const QString &redactedLogText, const QString &userNoteText);
static int attachToPendingCrashEnvelopes(const QString &databasePath,
const QString &redactedLogText,
const QString &userNoteText);
/**
* @brief Append one attachment item to a cached Sentry envelope file.
*
* An envelope is a newline-separated stream of item-header-line + payload
* pairs. The retry transport replays cached envelopes byte-for-byte
* (sentry-native 0.17.3 reads them raw and serializes raw envelopes back
* verbatim), so appending well-formed items associates them with the event
* already in the file; the first line (event id header) is never touched.
*
* @param filename ASCII-only item filename, embedded verbatim in the header.
*/
static bool
appendAttachmentToEnvelope(const QString &envelopePath, const QString &filename, const QByteArray &payload);
/**
* @brief Pending crash envelopes in <database>: the consent-held files in
* <database>/cache and the in-flight files in <database>/<run>.run.
*
* Matches the cache filename conventions sentry-native accepts (see
* sentry__parse_cache_filename): bare "<uuid>.envelope" and retry
* "<ts>-<count>-<uuid>.envelope"; run dirs accept plain "<uuid>.envelope"
* minus the SDK's own "__sentry-crash.envelope" marker.
*/
static QStringList pendingEnvelopePaths(const QString &databasePath);
/**
* @brief SDK-independent gating decision, unit-testable in OFF builds.
*/
static bool isReportingEnabled(const QString &dsn, PersonalSettings::CrashReportingMode consentMode);
/**
* @brief Whether a DSN has the shape the build and the SDK require:
* <scheme>://<key>@<host>/<project-id>.
*
* Deliberately the same unanchored pattern cmake/CrashReporting.cmake matches (CMake's MATCHES
* searches rather than anchoring), so a DSN this build accepted is never rejected here. The SDK
* refuses to even build a request for a DSN it considers invalid, so an unchecked typo disables
* uploading with no error anywhere.
*/
static bool isDsnWellFormed(const QString &dsn);
/**
* @brief Human-readable destination of a DSN, e.g. "o0.ingest.sentry.io (project 1234567)".
*
* Safe to log: the public key is never part of the result. Empty for a malformed DSN.
*/
static QString describeDsn(const QString &dsn);
/**
* @brief Describe the captured-for DSN of a pending envelope when it points at a different
* destination than the DSN this session would upload it with; empty when they agree.
*
* sentry-native posts every envelope with the DSN held in the transport state of the session
* doing the sending (envelopes restored from disk are raw and their own "dsn" header is never
* read), so a cached crash envelope captured for one project can silently be uploaded to
* another - the exact way a tester's report lands in the wrong place. Compares destinations
* (host and project id), not raw strings, so a different key for the same project stays quiet.
*/
static QString describeOriginMismatch(const QString &envelopePath, const QString &currentDsn);
/**
* @brief Claim this process's session lock, reporting a previous unclean exit.
* @return true if a leftover session lock from a dead process (or an unreadable
* garbage file) was found and cleaned; live concurrent instances are ignored.
*/
bool beginSession(const QString &dir);
void endSession();
static QString sanitizeLogLine(const QString &line);
static QString redactCockatriceUrls(const QString &line);
private:
bool ready = false;
QString sentryDatabasePath;
QString sessionLockPath;
std::unique_ptr<QLockFile> sessionLock;
};
#endif