[Tests] Capture screenshots and dump the widget tree from a playback (#7818)

pqWidgetTestEventPlayer adds the widget-level commands the UI tests were
missing:

- screenshot grabs the addressed widget and saves it, creating the parent
  directory and expanding $NAME / ${NAME} from the environment so the runner
  decides where captures land
- dumpTree writes the widget paths the tests address, for finding the path of a
  widget while writing a test
- resize sets the top-level window size, since the layout owns the children

The runner sets COCKATRICE_SCREENSHOT_DIR to build/screenshots, which is
ignored, so captures do not show up as untracked files.
Testing/screenshot_test.xml captures the main window and the settings dialog,
and Testing/README.md documents the commands, the pause/process_events split,
path drift and the capture workflow.

Testing/README.md

@@ -2,8 +2,8 @@
 
 Widget-level playback tests for the Cockatrice client, driven by
 [QtTesting](https://github.com/Kitware/QtTesting) (`external/qttesting`, a submodule).
-Each `*_test.xml` file is a list of events that QtTesting replays against the running
+Each `*_test.xml` file lists events that QtTesting replays against the running
-application; a test passes when every event is handled without error.
+application. A test passes when every event is handled without error.
 
 ## Running
 
@@ -17,39 +17,43 @@ cmake --build build/debug -j 24
 ./Testing/run_test.sh --visual Testing/smoke_test.xml   # shows the window
 ```
 
-`WITH_GUI_TEST` is on by default; a checkout without the submodule configures with a
+`WITH_GUI_TEST` is on by default. Without the submodule the build configures with a
 warning and no harness. `COCKATRICE_BIN` points the runner at another build, and
-`TEST_TIMEOUT` (seconds, default 180) bounds a single test — a failing event inside a
+`TEST_TIMEOUT` (seconds, default 180) bounds a single test, because a failing event
-modal dialog can otherwise leave the process alive. The client also hard-exits a few
+inside a modal dialog can leave the process alive. The client also hard-exits a few
 seconds after playback returns, whatever the result.
 
-The tests are **not** part of `ctest`: they need a card database and a developer's
+The tests are **not** part of `ctest`. They need a card database and a developer's
 settings, so they stay a local loop for now.
 
 ## Caveats
 
 - **Not hermetic.** The client reads the local settings and card database, so a test can
   pass on one machine and fail on another. Avoid asserting things that restored
-  workspace state or user preferences decide (for example the number of open tabs).
+  workspace state or user preferences decide, such as the number of open tabs.
-- **English labels only.** Menu entries are addressed by their (translated) text, so the
+- **English labels only.** Menu entries are addressed by their translated text, so the
   tests expect the source language. Widgets are addressed by objectName paths, which are
   stable across translations.
-- Actions that open a dialog are followed by a `pause` event: playback continues in the
+- Actions that open a dialog are followed by a `pause` event. Playback continues in the
   dialog's own event loop, and the pause gives the UI time to build itself.
+- `pause` sleeps the thread, so nothing is delivered while it runs. A reply from a server
+  never arrives during a pause. Waiting for the client to talk to a server needs
+  `process_events`, which runs the event loop (see *Screenshots* below).
 - Widget paths include Qt-internal names where a QTabWidget is involved
   (`qt_tabwidget_stackedwidget`), and the deck views in a game are indexed
   (`tabbedDeckView_0`, `tabbedDeckView_1`).
 
 ## Writing a test
 
-A test is a `<events>` list. Widgets are named by their path from the main window, e.g.
+A test is an `<events>` list. Widgets are named by their path from the main window, e.g.
 `MainWindow/tabSupervisor/stackedWidget/TabGame/phasesMenu`. Useful commands:
 
 | Command | Arguments | Effect |
 |---------|-----------|--------|
 | `activate` | action or menu name | triggers a QAction (menus are opened first) |
 | `key` | Qt key code | presses that key on the focused widget |
-| `pause` | milliseconds | waits, letting the UI settle |
+| `pause` | milliseconds | sleeps, letting the UI settle (no events are processed) |
+| `process_events` | milliseconds | runs the event loop, so timers and network replies are handled |
 | `clickCard` | `card,zone` | double click the card found in that zone (plays it from hand) |
 | `flipCard` | `card,zone` | right-click → *Turn Over* |
 | `cardAction` | `card,zone,label` | right-click → first menu entry whose label contains `label` (submenus included) |
@@ -59,8 +63,61 @@ A test is a `<events>` list. Widgets are named by their path from the main windo
 | `selectItem` | `item label` | selects the tree/list entry with that label |
 | `typeText` | `text` | types text one key event at a time |
 | `expectProperty` | `property,value` | fails playback unless the widget property matches |
+| `screenshot` | `path` | saves an image of the addressed widget; `$VAR` expands from the environment |
+| `dumpTree` | `path` | writes the addressed widget's subtree with the paths tests address |
+| `resize` | `WIDTHxHEIGHT` | resizes the addressed widget's window |
 
 The card, item and property commands are `pqCockatriceEventPlayer`
 (`cockatrice/src/game_graphics/pq_cockatrice_event_player.h`). They resolve cards by name
 and zone through the `QGraphicsScene` and send real mouse events at the card's current
 viewport position, so a test does not depend on the window size or screen resolution.
+
+The capture commands are `pqWidgetTestEventPlayer`
+(`cockatrice/src/interface/pq_widget_test_event_player.h`).
+
+## Screenshots
+
+`run_test.sh` sets `COCKATRICE_SCREENSHOT_DIR` to `build/screenshots`, and `screenshot`
+and `dumpTree` write there. Override it in the environment to send captures elsewhere.
+That directory is not part of the source tree, so a test run does not leave untracked
+files in the repository:
+
+```xml
+<event widget="MainWindow" command="resize" arguments="1920x1080" />
+<event widget="MainWindow" command="screenshot" arguments="$COCKATRICE_SCREENSHOT_DIR/main-window.png" />
+<event widget="MainWindow" command="dumpTree" arguments="$COCKATRICE_SCREENSHOT_DIR/widget-tree.txt" />
+```
+
+The addressed widget is what gets grabbed, so `MainWindow` is the whole window and a
+panel's path is that panel. The format comes from the file extension. A path with no
+known extension is a test failure.
+
+Capture at a fixed, generous size. `1920x1080` is a safe default, and narrower windows
+clip columns of wide tables. A headless run renders through the offscreen platform, so
+the palette and layout are the real ones, but `raise()`, keyboard-grab and
+`propagateSizeHints()` warnings on stderr are normal.
+
+### Waiting for a server
+
+`pause` blocks, so a reply that has to arrive over the network never does. Use
+`process_events` with a budget that covers the round trip:
+
+```xml
+<event widget="MainWindow" command="process_events" arguments="8000" />  <!-- login -->
+<event widget="MainWindow" command="process_events" arguments="4000" />  <!-- report list -->
+```
+
+### Paths drift between runs
+
+`pqObjectNaming` names an unnamed widget `<class><index>`, where the index counts
+same-class unnamed siblings and a leading digit records visibility (`1` visible, `0`
+hidden). The numbering depends on what the previous events left on screen, so a path that
+worked in one playback can miss in the next. The player retries a leading `0` as `1`,
+which resolves some hidden-page paths but not all of them.
+
+Record the tree in the same playback that clicks, and copy paths from that dump:
+
+```bash
+./Testing/run_test.sh Testing/screenshot_test.xml
+less build/screenshots/widget-tree.txt
+```

Testing/run_test.sh

@@ -12,6 +12,10 @@
 # Environment:
 #   COCKATRICE_BIN   Path to the client binary (default: build/debug/cockatrice/cockatrice,
 #                    or the .app bundle binary on macOS).
+#   COCKATRICE_SCREENSHOT_DIR
+#                    Directory the tests write screenshots and widget dumps to
+#                    (default: build/screenshots). Tests address it as
+#                    $COCKATRICE_SCREENSHOT_DIR/name.png.
 #   TEST_TIMEOUT     Seconds a single test may run before it is killed (default: 180).
 #                    Uses timeout, or gtimeout from coreutils on macOS.
 #
@@ -32,6 +36,12 @@ fi
 TEST_DIR="$REPO_ROOT/Testing"
 TEST_TIMEOUT="${TEST_TIMEOUT:-180}"
 
+# Tests that capture the UI address this as $COCKATRICE_SCREENSHOT_DIR/name.png.
+# It lives under the ignored build/ tree, so a run does not leave untracked files.
+SCREENSHOT_DIR="${COCKATRICE_SCREENSHOT_DIR:-$REPO_ROOT/build/screenshots}"
+mkdir -p "$SCREENSHOT_DIR"
+export COCKATRICE_SCREENSHOT_DIR="$SCREENSHOT_DIR"
+
 # Stock macOS has no `timeout`; coreutils provides `gtimeout`. Without either the
 # runner still works, but a hung test is not killed.
 TIMEOUT=()

Testing/screenshot_test.xml

@@ -0,0 +1,23 @@
+<events>
+    <!--
+        Screenshot and widget-tree capture.
+
+        The runner sets COCKATRICE_SCREENSHOT_DIR (build/screenshots by default),
+        so the images and the dump land in an ignored directory. Use the dump to
+        find the paths other tests can address.
+    -->
+    <event widget="MainWindow" command="resize" arguments="1280x800" />
+    <event widget="MainWindow" command="process_events" arguments="500" />
+    <event widget="MainWindow" command="screenshot" arguments="$COCKATRICE_SCREENSHOT_DIR/main-window.png" />
+
+    <event widget="MainWindow/menubar" command="activate" arguments="cockatriceMenu" />
+    <event widget="MainWindow/menubar/cockatriceMenu" command="activate" arguments="actionSettings" />
+    <event widget="MainWindow" command="pause" arguments="500" />
+    <event widget="MainWindow/settingsDialog/settingsView/pagesContainer/pagesWidget" command="expectProperty" arguments="currentIndex,0" />
+
+    <!-- Taken while the dialog is open, so the dump covers its widgets too. -->
+    <event widget="MainWindow" command="dumpTree" arguments="$COCKATRICE_SCREENSHOT_DIR/widget-tree.txt" />
+    <event widget="MainWindow/settingsDialog/settingsView" command="screenshot" arguments="$COCKATRICE_SCREENSHOT_DIR/settings.png" />
+
+    <event widget="MainWindow/settingsDialog/settingsOkButton" command="activate" arguments="" />
+</events>

cockatrice/CMakeLists.txt

@@ -599,9 +599,12 @@ target_link_libraries(
 cockatrice_target_enable_crash_reporting(cockatrice)
 
 if(WITH_GUI_TEST)
-  # These two translate between XML test files and the QtTesting event pipeline;
+  # QtTesting integration and the Cockatrice-specific playback commands. They
-  # they need the QtTesting headers, so they only exist with the harness.
+  # need the QtTesting headers, so they only exist with the harness.
-  target_sources(cockatrice PRIVATE src/game_graphics/pq_cockatrice_event_player.cpp src/interface/recording.cpp)
+  target_sources(
+    cockatrice PRIVATE src/game_graphics/pq_cockatrice_event_player.cpp src/interface/pq_widget_test_event_player.cpp
+                       src/interface/recording.cpp
+  )
   target_link_libraries(cockatrice PRIVATE QtTesting)
   target_compile_definitions(cockatrice PRIVATE WITH_GUI_TEST)
 endif()

cockatrice/src/interface/pq_widget_test_event_player.cpp

@@ -0,0 +1,223 @@
+#include "pq_widget_test_event_player.h"
+
+#include <QApplication>
+#include <QDir>
+#include <QFile>
+#include <QFileInfo>
+#include <QLoggingCategory>
+#include <QPixmap>
+#include <QTextStream>
+#include <QWidget>
+#include <pqObjectNaming.h>
+
+Q_LOGGING_CATEGORY(PQ_WIDGET_TEST_EVENT_PLAYER, "cockatrice.guiTest.widgetEventPlayer")
+
+namespace
+{
+void dumpSubtree(QObject *object, QTextStream &out, int depth)
+{
+    if (auto *widget = qobject_cast<QWidget *>(object)) {
+        const QString path = pqObjectNaming::GetName(*widget);
+        if (!path.isEmpty()) {
+            out << QString(depth * 2, ' ') << path << "  [" << widget->metaObject()->className() << "]";
+            if (!widget->objectName().isEmpty()) {
+                out << "  objectName=" << widget->objectName();
+            }
+            out << '\n';
+        }
+    }
+
+    // children() returns a snapshot, so the recursion below cannot invalidate it.
+    const QObjectList children = object->children();
+    for (QObject *child : children) {
+        dumpSubtree(child, out, depth + 1);
+    }
+}
+
+/**
+ * Expands $NAME and ${NAME} from the environment.
+ *
+ * A test file is static, so this lets the runner send screenshots to a scratch
+ * directory instead of hardcoding a path that only exists on one machine. An
+ * unset variable is an error, not an empty string: writing to the working
+ * directory would look like a pass.
+ */
+bool expandEnvironment(const QString &command, const QString &path, QString &expanded)
+{
+    expanded.clear();
+    for (int i = 0; i < path.size(); ++i) {
+        if (path.at(i) != QLatin1Char('$')) {
+            expanded += path.at(i);
+            continue;
+        }
+
+        int nameStart = i + 1;
+        int nameEnd = nameStart;
+        if (nameStart < path.size() && path.at(nameStart) == QLatin1Char('{')) {
+            nameStart++;
+            nameEnd = path.indexOf(QLatin1Char('}'), nameStart);
+            if (nameEnd < 0) {
+                qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << command << ": unterminated ${ in path" << path;
+                return false;
+            }
+            i = nameEnd;
+        } else {
+            while (nameEnd < path.size() &&
+                   (path.at(nameEnd).isLetterOrNumber() || path.at(nameEnd) == QLatin1Char('_'))) {
+                nameEnd++;
+            }
+            i = nameEnd - 1;
+        }
+
+        const QString name = path.mid(nameStart, nameEnd - nameStart);
+        if (name.isEmpty()) {
+            expanded += QLatin1Char('$');
+            continue;
+        }
+
+        const QString value = qEnvironmentVariable(name.toUtf8().constData());
+        if (value.isEmpty()) {
+            qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << command << ": environment variable" << name << "is not set";
+            return false;
+        }
+        expanded += value;
+    }
+    return true;
+}
+} // namespace
+
+pqWidgetTestEventPlayer::pqWidgetTestEventPlayer(QObject *parent) : pqWidgetEventPlayer(parent)
+{
+}
+
+bool pqWidgetTestEventPlayer::playEvent(QObject *object, const QString &command, const QString &arguments, bool &error)
+{
+    // A command we know that fails is still a handled event: QtTesting logs
+    // "Unhandled event" when playEvent returns false, so set `error` and return
+    // true, as pqCockatriceEventPlayer does.
+    const auto run = [&error](bool ok) {
+        if (!ok) {
+            error = true;
+        }
+        return true;
+    };
+
+    if (command == "screenshot") {
+        return run(playScreenshot(object, arguments));
+    }
+    if (command == "dumpTree") {
+        return run(playDumpTree(object, arguments));
+    }
+    if (command == "resize") {
+        return run(playResize(object, arguments));
+    }
+    return false;
+}
+
+bool pqWidgetTestEventPlayer::playScreenshot(QObject *object, const QString &arguments)
+{
+    auto *widget = qobject_cast<QWidget *>(object);
+    if (!widget) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "screenshot: target is not a QWidget";
+        return false;
+    }
+
+    QString path;
+    if (!resolveOutputPath(QStringLiteral("screenshot"), arguments.trimmed(), path)) {
+        return false;
+    }
+
+    // repaint() is synchronous, even on the offscreen platform, so the grab
+    // contains what the previous events drew.
+    widget->repaint();
+    const QPixmap pixmap = widget->grab();
+    if (pixmap.isNull()) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "screenshot: grab returned a null pixmap for" << widget;
+        return false;
+    }
+    if (!pixmap.save(path)) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER)
+            << "screenshot: could not save" << path << "- an image format is chosen from the file extension";
+        return false;
+    }
+
+    qCInfo(PQ_WIDGET_TEST_EVENT_PLAYER) << "screenshot saved to" << path;
+    return true;
+}
+
+bool pqWidgetTestEventPlayer::playDumpTree(QObject *object, const QString &arguments)
+{
+    if (!qobject_cast<QWidget *>(object)) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "dumpTree: target is not a QWidget";
+        return false;
+    }
+
+    QString path;
+    if (!resolveOutputPath(QStringLiteral("dumpTree"), arguments.trimmed(), path)) {
+        return false;
+    }
+
+    QFile file(path);
+    if (!file.open(QIODevice::WriteOnly | QIODevice::Text | QIODevice::Truncate)) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "dumpTree: could not open" << path << file.errorString();
+        return false;
+    }
+
+    QTextStream out(&file);
+    dumpSubtree(object, out, 0);
+
+    qCInfo(PQ_WIDGET_TEST_EVENT_PLAYER) << "widget tree dumped to" << path;
+    return true;
+}
+
+bool pqWidgetTestEventPlayer::playResize(QObject *object, const QString &arguments)
+{
+    auto *widget = qobject_cast<QWidget *>(object);
+    if (!widget) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "resize: target is not a QWidget";
+        return false;
+    }
+
+    const QStringList parts = arguments.split(QLatin1Char('x'));
+    if (parts.size() != 2) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "resize: arguments must be 'WIDTHxHEIGHT', got:" << arguments;
+        return false;
+    }
+
+    bool okWidth = false;
+    bool okHeight = false;
+    const int width = parts.at(0).trimmed().toInt(&okWidth);
+    const int height = parts.at(1).trimmed().toInt(&okHeight);
+    if (!okWidth || !okHeight || width <= 0 || height <= 0) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << "resize: invalid size" << arguments;
+        return false;
+    }
+
+    // Resize the top-level window: the layout owns the children.
+    widget->window()->resize(width, height);
+    QApplication::processEvents();
+    return true;
+}
+
+bool pqWidgetTestEventPlayer::resolveOutputPath(const QString &command, const QString &path, QString &resolved)
+{
+    if (path.isEmpty()) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << command << ": output path is empty";
+        return false;
+    }
+
+    QString expanded;
+    if (!expandEnvironment(command, path, expanded)) {
+        return false;
+    }
+
+    const QFileInfo info(expanded);
+    const QString directory = info.absolutePath();
+    if (!QDir().mkpath(directory)) {
+        qCCritical(PQ_WIDGET_TEST_EVENT_PLAYER) << command << ": could not create directory" << directory;
+        return false;
+    }
+
+    resolved = info.absoluteFilePath();
+    return true;
+}

cockatrice/src/interface/pq_widget_test_event_player.h

@@ -0,0 +1,45 @@
+#ifndef PQ_WIDGET_TEST_EVENT_PLAYER_H
+#define PQ_WIDGET_TEST_EVENT_PLAYER_H
+
+#include "pqWidgetEventPlayer.h"
+
+/**
+ * Widget-level commands for the UI tests.
+ *
+ * pqCockatriceEventPlayer handles cards and zones inside the game scene. This
+ * player covers the rest of the client: capturing a widget as an image,
+ * dumping the paths QtTesting addresses widgets by, and sizing the window.
+ *
+ * Commands:
+ *   screenshot "<path>"
+ *       grabs the addressed widget and saves it as an image. Parent directories
+ *       are created, and $NAME / ${NAME} in the path expand from the
+ *       environment, so the runner decides where captures land
+ *   dumpTree "<path>"
+ *       writes the addressed widget's subtree with the paths QtTesting
+ *       addresses it by, for writing a test
+ *   resize "WIDTHxHEIGHT"
+ *       resizes the addressed widget's top-level window
+ */
+class pqWidgetTestEventPlayer : public pqWidgetEventPlayer
+{
+    Q_OBJECT
+    typedef pqWidgetEventPlayer Superclass;
+
+public:
+    explicit pqWidgetTestEventPlayer(QObject *parent = nullptr);
+
+    using Superclass::playEvent;
+
+    bool playEvent(QObject *object, const QString &command, const QString &arguments, bool &error) override;
+
+private:
+    bool playScreenshot(QObject *object, const QString &arguments);
+    bool playDumpTree(QObject *object, const QString &arguments);
+    bool playResize(QObject *object, const QString &arguments);
+
+    /** @brief Expands $NAME / ${NAME} and creates the file's parent directory. */
+    static bool resolveOutputPath(const QString &command, const QString &path, QString &resolved);
+};
+
+#endif // PQ_WIDGET_TEST_EVENT_PLAYER_H

cockatrice/src/interface/window_main.cpp

@@ -62,6 +62,7 @@
 #include "pqEventSource.h"
 #include "pqEventTypes.h"
 #include "pqTestUtility.h"
+#include "pq_widget_test_event_player.h"
 #include "recording.h"
 #endif
 
@@ -692,6 +693,7 @@ MainWindow::MainWindow(QWidget *parent)
     testUtility->addEventObserver("xml", new XMLEventObserver(this));
     testUtility->addEventSource("xml", new XMLEventSource(this));
     testUtility->eventPlayer()->addWidgetEventPlayer(new pqCockatriceEventPlayer(this));
+    testUtility->eventPlayer()->addWidgetEventPlayer(new pqWidgetTestEventPlayer(this));
 #endif
 }