Compare commits

..

6 commits

Author SHA1 Message Date
Lukas Brübach
bc8146086b tests: add manual picture loader benchmark against the real card hosts
(cherry picked from commit 588d71d3b9a8278ab98b847802908f322c1035b7)
2026-09-12 17:32:33 +02:00
Lukas Brübach
30f5142b59 [PictureLoader] Let unlocked hosts skip dispatch pacing; adjust limits per URL
Two refinements to the per-host request caps:

- Unlocked hosts (UNLIMITED_HOST_QUOTA, e.g. cards.scryfall.io) no longer
  wait on the 100ms dispatch pacing or consume the global per-second quota.
  dispatchQueuedRequest fires their queued requests back-to-back, bounded
  only by their 429 backoff window and Qt's per-host connection pool, so
  an unthrottled CDN is not artificially slowed.
- The deck editor download settings page replaces the static grid of one
  spinbox per known host with an "Adjust Rate Limit" toolbar action on the
  URL list. It picks the host out of the selected URL and clamps the entry
  against the developer cap table (including for user-added URLs).

Also fixes a review finding: resetRequestQuota could write the
UNLIMITED_HOST_QUOTA sentinel (-1) into the sustained per-host quota when a
host became unlocked mid-run, permanently poisoning its allowance. Stale
entries for unlocked hosts are now dropped, and the per-second seed is
clamped against the effective ceiling so a lowered limit applies immediately.
2026-09-12 17:32:30 +02:00
Lukas Brübach
41b49ed4bc [PictureLoader] Add user-configurable per-host request caps
Picture downloads were throttled to a uniform 10 requests/second per host
with no way to tune a specific server. A rate-limited API host (Scryfall
caps at 10 req/s) can trip 429s during bursts, and CDN hosts with no rate
limit were throttled needlessly.

Introduce developer-owned per-host caps that users can only ever lower,
never raise, exposed in the download settings page:
- DownloadSettings::DEVELOPER_HOST_CAPS sets the ceiling per host
  (api.scryfall.com 9, cards.scryfall.io unlimited, others 10).
- A new hostRequestLimits setting stores user overrides in downloads.ini;
  clampHostRequestLimit() bounds them to [1, devCap] so a user can reduce
  api.scryfall.com to 5 but never raise it above 9.
- The picture worker seeds, halves on 429, and recovers its sustained
  per-host allowance against the effective ceiling instead of the global
  maximum, and skips per-host accounting entirely for unlocked hosts
  (cards.scryfall.io) while global pacing and 429 backoff still apply.
- The deck editor settings page gains one spinbox per known host, each
  clamped to its developer cap.
2026-09-12 16:58:10 +02:00
Lukas Brübach
5a6db206ce [PictureLoader] Seed per-host allowances on demand and skip hosts in 429 backoff
The quota reset re-filled every host's remaining allowance to a full
MAX_REQUESTS_PER_SEC as soon as the queue had a request for it. A server
that was just rate limited could therefore be hammered again at full speed
immediately after (or even during) recovery.

Only seed a host's allowance the first time it is dispatched in the
current second, seeded from its reduced sustained quota, and skip hosts
still inside their 429 backoff window entirely. This makes the pacing
commit's burst-free behavior hold per host too, instead of just smoothing
the global aggregate.
2026-09-12 16:41:17 +02:00
Lukas Brübach
64b3b7e0b4 [PictureLoader] Pace requests and run the throttle timers on the worker thread
Previously the whole backed-up queue was drained in a burst as soon as a
request was enqueued, sending up to 10 requests back-to-back and then
immediately re-filling the quota one second later. That hard-bursts a
rate-limited API like Scryfall's (10 requests/second) into a 30 second
lockout.

Introduce a pacing timer that dispatches a single queue entry every
100 ms, so the per-second allowance is used smoothly instead of in spikes,
and keep the quota timer at 1 second. Also fix both timers' thread
affinity: they are QTimer value members and so are not QObject children,
meaning moveToThread() on the worker left them on the main thread while
the slot code started them from the picture thread, which was a no-op that
also warned. They are moved to the worker thread explicitly and started
lazily from there.
2026-09-12 16:41:14 +02:00
Lukas Brübach
61035f8ae0 [PictureLoader] Serve cached pictures from the disk cache instead of re-fetching them
With picture downloads enabled, requests were issued with AlwaysNetwork
cache control, which per Qt never consults the disk cache. A picture that
had already been downloaded was therefore fetched from the network again
on every session start, with the queue bypass letting those re-fetches
skip the rate limit entirely.

Treat the network cache as the intent of the 'Network Cache' storage
method suggests: if the URL is already cached, serve it with AlwaysCache
(no network, no quota); only a genuine miss goes to the network, and only
when downloads are enabled. Cache hits skip the queue for free since they
never consume the per-second request allowance.
2026-09-12 16:41:10 +02:00
16 changed files with 29 additions and 97 deletions

View file

@ -94,7 +94,7 @@ QStringMap &SoundEngine::getAvailableThemes()
QDir dir;
availableThemes.clear();
// Load themes from user profile dir
// load themes from user profile dir
dir.setPath(SettingsCache::instance().getDataPath() + "/sounds");
@ -104,7 +104,7 @@ QStringMap &SoundEngine::getAvailableThemes()
}
}
// Load themes from Cockatrice system dir
// load themes from cockatrice system dir
dir.setPath(qApp->applicationDirPath() +
#ifdef Q_OS_MAC
"/../Resources/sounds"

View file

@ -11,7 +11,6 @@
#include <QDirIterator>
#include <QFileInfo>
#include <QMainWindow>
#include <QMetaObject>
#include <QMovie>
#include <QNetworkRequest>
#include <QPainter>
@ -56,14 +55,7 @@ CardPictureLoader::CardPictureLoader() : QObject(nullptr)
CardPictureLoader::~CardPictureLoader()
{
if (worker) {
// Capture the thread first: shutdownThread() blocks until the worker has been freed by the
// finished() -> deleteLater chain, after which the worker pointer must not be dereferenced.
QThread *pictureLoaderThread = worker->workerThread();
worker->shutdownThread();
worker = nullptr;
delete pictureLoaderThread;
}
worker->deleteLater();
}
void CardPictureLoader::getCardBackPixmap(QPixmap &pixmap, QSize size)
@ -303,15 +295,7 @@ void CardPictureLoader::clearPixmapCache()
void CardPictureLoader::clearNetworkCache()
{
auto &worker = *getInstance().worker;
// The disk cache and redirect cache are owned by the worker thread; clearing them from the
// UI thread would race with the worker's cache reads/writes. Block until the worker thread
// has executed the clear so the "Cached card pictures have been reset." message is truthful.
if (worker.isRunning()) {
QMetaObject::invokeMethod(&worker, "clearNetworkCache", Qt::BlockingQueuedConnection);
} else {
worker.clearNetworkCache();
}
getInstance().worker->clearNetworkCache();
}
void CardPictureLoader::cacheCardPixmaps(const QList<ExactCard> &cards)

View file

@ -60,9 +60,6 @@ CardPictureLoaderWorker::CardPictureLoaderWorker()
localLoader = new CardPictureLoaderLocal(this);
pictureLoaderThread = new QThread;
// The worker object frees itself once its thread finishes, so no event loop is left
// running and the QThread is never destroyed while still executing.
connect(pictureLoaderThread, &QThread::finished, this, &QObject::deleteLater);
pictureLoaderThread->start(QThread::LowPriority);
moveToThread(pictureLoaderThread);
@ -86,28 +83,7 @@ CardPictureLoaderWorker::CardPictureLoaderWorker()
CardPictureLoaderWorker::~CardPictureLoaderWorker()
{
saveRedirectCache();
}
void CardPictureLoaderWorker::shutdownThread()
{
// The finished() -> deleteLater chain (wired in the constructor) frees this worker as soon as
// its event loop exits, so nothing - not even a member read - may run once wait() returns.
// QThread::quit() and QThread::wait() are thread-safe and may be called from the owning thread.
QThread *thread = pictureLoaderThread;
if (thread) {
thread->quit();
thread->wait();
}
}
QThread *CardPictureLoaderWorker::workerThread() const
{
return pictureLoaderThread;
}
bool CardPictureLoaderWorker::isRunning() const
{
return pictureLoaderThread != nullptr && pictureLoaderThread->isRunning();
pictureLoaderThread->deleteLater();
}
void CardPictureLoaderWorker::queueRequest(const QUrl &url, CardPictureLoaderWorkerWork *worker)

View file

@ -74,37 +74,10 @@ public:
*/
void onHostRateLimited(const QString &host);
/**
* @brief Stops the worker thread and releases it.
*
* Called from the owning thread (CardPictureLoader) on its way out. QThread::quit() posts an
* exit request to the worker's event loop and QThread::wait() blocks until the loop has
* returned and the thread finished. Only QThread members are touched here, so this method is
* safe to call from the owning thread. The worker object itself is freed by the finished() ->
* deleteLater chain (see the constructor); the QThread object is deleted afterwards by the
* owner (CardPictureLoader::~CardPictureLoader), not by this method.
*/
void shutdownThread();
/** @return Whether the worker's thread is currently running. */
bool isRunning() const;
/**
* @brief Returns the worker's QThread.
* @return The worker thread
*
* Only meaningful while the worker object is alive; capture it before calling shutdownThread().
*/
QThread *workerThread() const;
/** @brief Clears the network cache and redirect cache. */
void clearNetworkCache();
public slots:
/**
* @brief Clears the network cache and redirect cache.
*
* Runs on the worker thread; invoke it via a queued call when coming from another thread,
* since both caches are owned by the worker thread.
*/
void clearNetworkCache();
/**
* @brief Makes a network request for the given URL using the specified worker.
* @param url URL to load

View file

@ -17,7 +17,7 @@ enum Format
PlainText,
/**
* This is Cockatrice's native deck file format, and supports deck metadata such as banner cards and tags.
* This is cockatrice's native deck file format, and supports deck metadata such as banner cards and tags.
* Stored as .cod files.
*/
Cockatrice

View file

@ -50,7 +50,7 @@ DeckLoader::loadFromFile(const QString &fileName, DeckFileFormat::Format fmt, bo
result = deckList.loadFromFile_Native(&file);
if (!result) {
qCInfo(DeckLoaderLog) << "Failed to load " << fileName
<< "as Cockatrice format; retrying as plain format";
<< "as cockatrice format; retrying as plain format";
file.seek(0);
result = deckList.loadFromFile_Plain(&file, CardNameNormalizer());
fmt = DeckFileFormat::PlainText;

View file

@ -131,7 +131,7 @@ public:
static void printDeckList(QPrinter *printer, const DeckList &deckList);
/**
* Converts the given deck's file to the Cockatrice file format.
* Converts the given deck's file to the cockatrice file format.
* Uses the lastLoadInfo in the LoadedDeck to determine the current name of the file and where to save to.
* @param deck The deck to convert. Should have valid lastLoadInfo. Will update the lastLoadInfo.
* @return Whether the conversion succeeded.

View file

@ -224,7 +224,7 @@ QStringMap &ThemeManager::getAvailableThemes()
}
}
// Load themes from Cockatrice system dir
// load themes from cockatrice system dir
dir.setPath(systemThemesBasePath());
for (QString themeName : dir.entryList(QDir::AllDirs | QDir::NoDotAndDotDot, QDir::Name)) {

View file

@ -478,13 +478,13 @@ void DlgSettings::closeEvent(QCloseEvent *event)
case Invalid:
loadErrorMessage = tr("Your card database is invalid.\n\n"
"Cockatrice may not function correctly with an invalid database\n\n"
"You may need to rerun Oracle to update your card database.\n\n"
"You may need to rerun oracle to update your card database.\n\n"
"Would you like to change your database location setting?");
break;
case VersionTooOld:
loadErrorMessage = tr("Your card database version is too old.\n\n"
"This can cause problems loading card information or images\n\n"
"Usually this can be fixed by rerunning Oracle to to update your card database.\n\n"
"Usually this can be fixed by rerunning oracle to to update your card database.\n\n"
"Would you like to change your database location setting?");
break;
case NotLoaded:

View file

@ -92,8 +92,8 @@
#include <libcockatrice/settings/updates_settings.h>
#define GITHUB_PAGES_URL "https://cockatrice.github.io"
#define GITHUB_CONTRIBUTORS_URL "https://github.com/Cockatrice/Cockatrice/graphs/contributors"
#define GITHUB_CONTRIBUTE_URL "https://github.com/Cockatrice/Cockatrice#"
#define GITHUB_CONTRIBUTORS_URL "https://github.com/Cockatrice/Cockatrice/graphs/contributors?type=c"
#define GITHUB_CONTRIBUTE_URL "https://github.com/Cockatrice/Cockatrice#cockatrice"
#define GITHUB_TRANSIFEX_TRANSLATORS_URL "https://github.com/Cockatrice/Cockatrice/wiki/Translator-Hall-of-Fame"
#define GITHUB_TRANSLATOR_FAQ_URL "https://github.com/Cockatrice/Cockatrice/wiki/Translation-FAQ"
#define GITHUB_ISSUES_URL "https://github.com/Cockatrice/Cockatrice/issues"
@ -1050,7 +1050,7 @@ void MainWindow::createCardUpdateProcess(bool background)
if (dir.exists(binaryName)) {
updaterCmd = dir.absoluteFilePath(binaryName);
} else { // try and find the directory Oracle is stored in the build directory
} else { // try and find the directory oracle is stored in the build directory
QDir findLocalDir(dir);
findLocalDir.cdUp();
findLocalDir.cd(getCardUpdaterBinaryName());

View file

@ -231,7 +231,7 @@ int main(int argc, char *argv[])
// These values are only used by the settings loader/saver
// Wrong or outdated values are kept to not break things
QCoreApplication::setOrganizationName("Cockatrice");
QCoreApplication::setOrganizationDomain("cockatrice.github.io");
QCoreApplication::setOrganizationDomain("cockatrice.de");
QCoreApplication::setApplicationName("Cockatrice");
QCoreApplication::setApplicationVersion(VERSION_STRING);
@ -250,7 +250,7 @@ int main(int argc, char *argv[])
// Command-line parser
QCommandLineParser parser;
parser.setApplicationDescription("Cockatrice Client");
parser.setApplicationDescription("Cockatrice");
parser.addHelpOption();
parser.addVersionOption();
@ -350,8 +350,8 @@ int main(int argc, char *argv[])
qCInfo(MainLog) << "MainWindow constructor finished";
ui.setWindowIcon(themePixmap(QStringLiteral("cockatrice")));
// Set name of the app desktop file; used by wayland to load the window icon
QGuiApplication::setDesktopFileName("Cockatrice");
// set name of the app desktop file; used by wayland to load the window icon
QGuiApplication::setDesktopFileName("cockatrice");
SettingsCache::instance().network().setClientID(generateClientID());

View file

@ -50,8 +50,8 @@ int main(int argc, char *argv[])
QApplication app(argc, argv);
QCoreApplication::setOrganizationName("Cockatrice");
QCoreApplication::setOrganizationDomain("Cockatrice");
// This can't be changed, as it influences the default save path for cards.xml
QCoreApplication::setOrganizationDomain("cockatrice");
// this can't be changed, as it influences the default save path for cards.xml
QCoreApplication::setApplicationName("Cockatrice");
// If the program is opened with the -s flag, it will only do spoilers. Otherwise it will do MTGJSON/Tokens
@ -83,7 +83,7 @@ int main(int argc, char *argv[])
QIcon icon("theme:appicon.svg");
wizard.setWindowIcon(icon);
// set name of the app desktop file; used by wayland to load the window icon
QGuiApplication::setDesktopFileName("Oracle");
QGuiApplication::setDesktopFileName("oracle");
wizard.show();

View file

@ -796,7 +796,7 @@ void SaveSetsPage::retranslateUi()
{
setTitle(tr("Sets imported"));
if (wizard()->downloadedPlainXml) {
setSubTitle(tr("A Cockatrice card database file of %1 MB has been downloaded.")
setSubTitle(tr("A cockatrice database file of %1 MB has been downloaded.")
.arg(qRound(wizard()->xmlData.size() / 1000000.0)));
} else {
setSubTitle(tr("The following sets have been found:"));

View file

@ -20,7 +20,7 @@
* Note that "...enters tapped unless..." returns false.
*
* @param name The name of the card
* @param text The Oracle text of the card
* @param text The oracle text of the card
*/
bool parseCipt(const QString &name, const QString &text)
{

View file

@ -54,7 +54,7 @@ using ScanProgressCallback = std::function<void(qsizetype bytesRead, qsizetype t
* @brief Scans a full MTGJSON document without materializing the JSON tree.
*
* Splits the top-level "data" object into per-set byte ranges and reads each
* set's metadata directly from the raw bytes. The Oracle importer can then
* set's metadata directly from the raw bytes. The oracle importer can then
* parse one set at a time during import, keeping peak memory far below a single
* QJsonDocument::fromJson() over the whole file.
*

View file

@ -86,10 +86,9 @@ bool Servatrice_DatabaseInterface::openDatabase()
<< dbversion << "to version" << expectedversion;
return false;
} else if (dbversion > expectedversion) {
qCCritical(DatabaseInterfaceLog)
<< poolStr << "Error opening database: the database schema version" << dbversion
<< "is too new, you need to update Servatrice" << "(Currently running Servatrice actually uses version"
<< expectedversion << ")";
qCCritical(DatabaseInterfaceLog) << poolStr << "Error opening database: the database schema version"
<< dbversion << "is too new, you need to update servatrice"
<< "(this servatrice actually uses version" << expectedversion << ")";
return false;
}
} else {