mirror of
https://github.com/Cockatrice/Cockatrice.git
synced 2026-11-09 11:33:25 -08:00
[Tests] Capture screenshots and dump the widget tree from a playback (#7818)
Some checks are pending
CodeQL / Analyze (cpp) (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
Build Desktop / Configure (push) Waiting to run
Build Desktop / vcpkg baseline (push) Waiting to run
Build Desktop / Debian 13 (push) Blocked by required conditions
Build Desktop / Fedora 45 release (push) Blocked by required conditions
Build Desktop / Fedora 45 debug (push) Blocked by required conditions
Build Desktop / Servatrice_Debian 12 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 (push) Blocked by required conditions
Build Desktop / Arch (push) Blocked by required conditions
Build Desktop / Debian 12 (push) Blocked by required conditions
Build Desktop / Fedora 44 (push) Blocked by required conditions
Build Desktop / Ubuntu 24.04 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 sanitize (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 cover (push) Blocked by required conditions
Build Desktop / macOS 14 (push) Blocked by required conditions
Build Desktop / macOS 15 (push) Blocked by required conditions
Build Desktop / macOS 27 Debug (push) Blocked by required conditions
Build Desktop / Windows 10 (push) Blocked by required conditions
Build Desktop / macOS 13 Intel (push) Blocked by required conditions
Build Desktop / Prune caches (push) Waiting to run
Build Docker / Servatrice (arm) (push) Waiting to run
Build Docker / Servatrice (x86) (push) Waiting to run
Build Docker / Publish multi-platform Servatrice image (push) Blocked by required conditions
Some checks are pending
CodeQL / Analyze (cpp) (push) Waiting to run
CodeQL / Analyze (actions) (push) Waiting to run
Build Desktop / Configure (push) Waiting to run
Build Desktop / vcpkg baseline (push) Waiting to run
Build Desktop / Debian 13 (push) Blocked by required conditions
Build Desktop / Fedora 45 release (push) Blocked by required conditions
Build Desktop / Fedora 45 debug (push) Blocked by required conditions
Build Desktop / Servatrice_Debian 12 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 (push) Blocked by required conditions
Build Desktop / Arch (push) Blocked by required conditions
Build Desktop / Debian 12 (push) Blocked by required conditions
Build Desktop / Fedora 44 (push) Blocked by required conditions
Build Desktop / Ubuntu 24.04 (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 sanitize (push) Blocked by required conditions
Build Desktop / Ubuntu 26.04 cover (push) Blocked by required conditions
Build Desktop / macOS 14 (push) Blocked by required conditions
Build Desktop / macOS 15 (push) Blocked by required conditions
Build Desktop / macOS 27 Debug (push) Blocked by required conditions
Build Desktop / Windows 10 (push) Blocked by required conditions
Build Desktop / macOS 13 Intel (push) Blocked by required conditions
Build Desktop / Prune caches (push) Waiting to run
Build Docker / Servatrice (arm) (push) Waiting to run
Build Docker / Servatrice (x86) (push) Waiting to run
Build Docker / Publish multi-platform Servatrice image (push) Blocked by required conditions
pqWidgetTestEventPlayer adds the widget-level commands the UI tests were
missing:
- screenshot grabs the addressed widget and saves it, creating the parent
directory and expanding $NAME / ${NAME} from the environment so the runner
decides where captures land
- dumpTree writes the widget paths the tests address, for finding the path of a
widget while writing a test
- resize sets the top-level window size, since the layout owns the children
The runner sets COCKATRICE_SCREENSHOT_DIR to build/screenshots, which is
ignored, so captures do not show up as untracked files.
Testing/screenshot_test.xml captures the main window and the settings dialog,
and Testing/README.md documents the commands, the pause/process_events split,
path drift and the capture workflow.
This commit is contained in:
parent
3d24597134
commit
3b379618c3
7 changed files with 377 additions and 14 deletions
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
Widget-level playback tests for the Cockatrice client, driven by
|
||||
[QtTesting](https://github.com/Kitware/QtTesting) (`external/qttesting`, a submodule).
|
||||
Each `*_test.xml` file is a list of events that QtTesting replays against the running
|
||||
application; a test passes when every event is handled without error.
|
||||
Each `*_test.xml` file lists events that QtTesting replays against the running
|
||||
application. A test passes when every event is handled without error.
|
||||
|
||||
## Running
|
||||
|
||||
|
|
@ -17,39 +17,43 @@ cmake --build build/debug -j 24
|
|||
./Testing/run_test.sh --visual Testing/smoke_test.xml # shows the window
|
||||
```
|
||||
|
||||
`WITH_GUI_TEST` is on by default; a checkout without the submodule configures with a
|
||||
`WITH_GUI_TEST` is on by default. Without the submodule the build configures with a
|
||||
warning and no harness. `COCKATRICE_BIN` points the runner at another build, and
|
||||
`TEST_TIMEOUT` (seconds, default 180) bounds a single test — a failing event inside a
|
||||
modal dialog can otherwise leave the process alive. The client also hard-exits a few
|
||||
`TEST_TIMEOUT` (seconds, default 180) bounds a single test, because a failing event
|
||||
inside a modal dialog can leave the process alive. The client also hard-exits a few
|
||||
seconds after playback returns, whatever the result.
|
||||
|
||||
The tests are **not** part of `ctest`: they need a card database and a developer's
|
||||
The tests are **not** part of `ctest`. They need a card database and a developer's
|
||||
settings, so they stay a local loop for now.
|
||||
|
||||
## Caveats
|
||||
|
||||
- **Not hermetic.** The client reads the local settings and card database, so a test can
|
||||
pass on one machine and fail on another. Avoid asserting things that restored
|
||||
workspace state or user preferences decide (for example the number of open tabs).
|
||||
- **English labels only.** Menu entries are addressed by their (translated) text, so the
|
||||
workspace state or user preferences decide, such as the number of open tabs.
|
||||
- **English labels only.** Menu entries are addressed by their translated text, so the
|
||||
tests expect the source language. Widgets are addressed by objectName paths, which are
|
||||
stable across translations.
|
||||
- Actions that open a dialog are followed by a `pause` event: playback continues in the
|
||||
- Actions that open a dialog are followed by a `pause` event. Playback continues in the
|
||||
dialog's own event loop, and the pause gives the UI time to build itself.
|
||||
- `pause` sleeps the thread, so nothing is delivered while it runs. A reply from a server
|
||||
never arrives during a pause. Waiting for the client to talk to a server needs
|
||||
`process_events`, which runs the event loop (see *Screenshots* below).
|
||||
- Widget paths include Qt-internal names where a QTabWidget is involved
|
||||
(`qt_tabwidget_stackedwidget`), and the deck views in a game are indexed
|
||||
(`tabbedDeckView_0`, `tabbedDeckView_1`).
|
||||
|
||||
## Writing a test
|
||||
|
||||
A test is a `<events>` list. Widgets are named by their path from the main window, e.g.
|
||||
A test is an `<events>` list. Widgets are named by their path from the main window, e.g.
|
||||
`MainWindow/tabSupervisor/stackedWidget/TabGame/phasesMenu`. Useful commands:
|
||||
|
||||
| Command | Arguments | Effect |
|
||||
|---------|-----------|--------|
|
||||
| `activate` | action or menu name | triggers a QAction (menus are opened first) |
|
||||
| `key` | Qt key code | presses that key on the focused widget |
|
||||
| `pause` | milliseconds | waits, letting the UI settle |
|
||||
| `pause` | milliseconds | sleeps, letting the UI settle (no events are processed) |
|
||||
| `process_events` | milliseconds | runs the event loop, so timers and network replies are handled |
|
||||
| `clickCard` | `card,zone` | double click the card found in that zone (plays it from hand) |
|
||||
| `flipCard` | `card,zone` | right-click → *Turn Over* |
|
||||
| `cardAction` | `card,zone,label` | right-click → first menu entry whose label contains `label` (submenus included) |
|
||||
|
|
@ -59,8 +63,61 @@ A test is a `<events>` list. Widgets are named by their path from the main windo
|
|||
| `selectItem` | `item label` | selects the tree/list entry with that label |
|
||||
| `typeText` | `text` | types text one key event at a time |
|
||||
| `expectProperty` | `property,value` | fails playback unless the widget property matches |
|
||||
| `screenshot` | `path` | saves an image of the addressed widget; `$VAR` expands from the environment |
|
||||
| `dumpTree` | `path` | writes the addressed widget's subtree with the paths tests address |
|
||||
| `resize` | `WIDTHxHEIGHT` | resizes the addressed widget's window |
|
||||
|
||||
The card, item and property commands are `pqCockatriceEventPlayer`
|
||||
(`cockatrice/src/game_graphics/pq_cockatrice_event_player.h`). They resolve cards by name
|
||||
and zone through the `QGraphicsScene` and send real mouse events at the card's current
|
||||
viewport position, so a test does not depend on the window size or screen resolution.
|
||||
|
||||
The capture commands are `pqWidgetTestEventPlayer`
|
||||
(`cockatrice/src/interface/pq_widget_test_event_player.h`).
|
||||
|
||||
## Screenshots
|
||||
|
||||
`run_test.sh` sets `COCKATRICE_SCREENSHOT_DIR` to `build/screenshots`, and `screenshot`
|
||||
and `dumpTree` write there. Override it in the environment to send captures elsewhere.
|
||||
That directory is not part of the source tree, so a test run does not leave untracked
|
||||
files in the repository:
|
||||
|
||||
```xml
|
||||
<event widget="MainWindow" command="resize" arguments="1920x1080" />
|
||||
<event widget="MainWindow" command="screenshot" arguments="$COCKATRICE_SCREENSHOT_DIR/main-window.png" />
|
||||
<event widget="MainWindow" command="dumpTree" arguments="$COCKATRICE_SCREENSHOT_DIR/widget-tree.txt" />
|
||||
```
|
||||
|
||||
The addressed widget is what gets grabbed, so `MainWindow` is the whole window and a
|
||||
panel's path is that panel. The format comes from the file extension. A path with no
|
||||
known extension is a test failure.
|
||||
|
||||
Capture at a fixed, generous size. `1920x1080` is a safe default, and narrower windows
|
||||
clip columns of wide tables. A headless run renders through the offscreen platform, so
|
||||
the palette and layout are the real ones, but `raise()`, keyboard-grab and
|
||||
`propagateSizeHints()` warnings on stderr are normal.
|
||||
|
||||
### Waiting for a server
|
||||
|
||||
`pause` blocks, so a reply that has to arrive over the network never does. Use
|
||||
`process_events` with a budget that covers the round trip:
|
||||
|
||||
```xml
|
||||
<event widget="MainWindow" command="process_events" arguments="8000" /> <!-- login -->
|
||||
<event widget="MainWindow" command="process_events" arguments="4000" /> <!-- report list -->
|
||||
```
|
||||
|
||||
### Paths drift between runs
|
||||
|
||||
`pqObjectNaming` names an unnamed widget `<class><index>`, where the index counts
|
||||
same-class unnamed siblings and a leading digit records visibility (`1` visible, `0`
|
||||
hidden). The numbering depends on what the previous events left on screen, so a path that
|
||||
worked in one playback can miss in the next. The player retries a leading `0` as `1`,
|
||||
which resolves some hidden-page paths but not all of them.
|
||||
|
||||
Record the tree in the same playback that clicks, and copy paths from that dump:
|
||||
|
||||
```bash
|
||||
./Testing/run_test.sh Testing/screenshot_test.xml
|
||||
less build/screenshots/widget-tree.txt
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue