[Installer] Graceful close and a way out of a stuck update (#7687)

* [Installer] Stop calling the payload free while an executable is locked

The guard answered "nothing is holding the payload" from two lookups that can
both miss a running instance: Restart Manager does not reliably report a
program's own image or the modules it mapped, and the image path of a process
running under another account cannot be read from the probe at all. A user log
showed exactly that: the first guard passed while cockatrice.exe was running
from the directory, the old uninstaller silently skipped the file it could not
delete, and the update aborted half an uninstall later with "cancelled by user"
after a close attempt the running window ignored.

Ask the file itself. A running image is opened without write sharing, so the
write open replacing it needs is refused with ERROR_SHARING_VIOLATION for as
long as the program runs, while a file that is merely being read still accepts
it. When neither lookup named anybody and a payload executable refuses that
open, the probe answers that the file is in use and that the program holding it
could not be identified - the guard's "could not confirm" dialog, whose Retry
works once the user has closed it - instead of passing the payload as free.

Only the payload executables are tested, not every file the purge removes: a
locked executable proves a program is running and a mapped DLL belongs to a
running executable, while a locked DLL or qt.conf can be the asking process
itself, which is the loop the guard already had. The open is retried once after
300 ms, and only for the files that refused, so a scanner holding a file for a
moment cannot stop an install for nothing.

The Windows probe check now covers that answer: a payload executable held by
the asking process, so that it cannot be named, has to come back as in use
rather than as free.

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>

* [Installer] Close running clients over a local socket

The installer could only ask a running client to close with WM_CLOSE, which an instance hidden in the notification area or sitting on its own confirmation cannot answer usefully. Give every client a per-process local socket and have the holder probe ask it there first, falling back to WM_CLOSE for clients from before the socket.

* [Installer] Offer to check again or update by hand when files stay locked

Replace the two-answer guard prompt with force close, close by hand and check again, or cancel. Retry the automatic close while the set of holders changes, and point a cancelled run at the latest release so a user who does not want anything killed has a way forward that is not cancelling the update again.

---------

Co-authored-by: Lukas Brübach <Bruebach.Lukas@bdosecurity.de>

tools/installer-holder-probe/close_holders.cpp

@@ -1,5 +1,6 @@
 #include "close_holders.h"
 
+#include "installer_close_request.h"
 #include "payload_paths.h"
 #include "restart_manager_holders.h"
 
@@ -248,19 +249,21 @@ closeCockatriceHolders(const std::vector<Holder> &holders, unsigned long waitMil
         search.processId = holder.processId;
         EnumWindows(collectProcessWindow, reinterpret_cast<LPARAM>(&search));
         CloseTarget target = chooseCloseTarget(search.windows, askHiddenWindows);
-        if (target.main == nullptr) {
-            // Nothing of the application's own to post a close to: every window left is one of
-            // Qt's bookkeeping windows, or the only thing on screen is a question whose owner
-            // window is gone. Neither is a close the application would answer - a close to the
-            // notification-area window terminates without the confirmation - so the instance is
-            // named and the guard's end answer is what can free its files.
+        if (target.main == nullptr && !askHiddenWindows) {
+            // A launcher with no window on screen has already answered its own close and is
+            // stuck in shutdown (see the header): asking it again cannot reach a user who is
+            // not there, so it is named and the installer's wait ends it.
             ++result.withoutWindow;
             lines.push_back(processLabel(holder) + L" has no window of its own to close");
             continue;
         }
 
         std::wstring line = processLabel(holder);
-        if (target.hidden) {
+        if (target.main == nullptr) {
+            // No window of the application's own at all: every window left is one of Qt's
+            // bookkeeping windows, which must never be asked to close. The installer socket
+            // below is the only thing that can reach it.
+        } else if (target.hidden) {
             line += L" is hidden in the notification area";
         } else {
             if (target.toRaise == nullptr) {
@@ -269,19 +272,40 @@ closeCockatriceHolders(const std::vector<Holder> &holders, unsigned long waitMil
             line += bringForward(target.toRaise) ? L" brought forward" : L" could not be brought forward";
         }
 
-        // Posted to the instance's own window even when a question of its own is up. The close
-        // is what that window's close button does, and the application is the one that answers
-        // it: a current client refuses the close while its confirmation is pending, leaving the
-        // question in front, and only a client from before that guard accepts it and hides the
-        // window with the question. Withholding the request therefore protected nobody - it
-        // left the installer asking the user to close a program it had never asked - while the
-        // guard's end answer is what frees the files when the close cannot.
-        //
         // Kept open across the wait so a pid reused after the process exits cannot be
         // mistaken for the instance that was asked to close.
         const HANDLE process = OpenProcess(SYNCHRONIZE, FALSE, holder.processId);
         const DWORD openError = process == nullptr ? GetLastError() : ERROR_SUCCESS;
-        if (PostMessageW(target.main, WM_CLOSE, 0, 0) == FALSE) {
+
+        // Ask over the installer socket first. It is the same request as the window's close
+        // button - the application still answers its own confirmation - but it reaches an
+        // instance whose window is hidden in the notification area or is sitting on that
+        // confirmation, the two states a posted close cannot get through. A client from
+        // before the socket answers with a failed connect, so the posted close below stays
+        // the fallback and older versions keep behaving exactly as they did.
+        if (requestInstallerClose(holder.processId)) {
+            // "and" only when the line already says something about the instance's window.
+            line += target.main == nullptr ? L" asked to close over its installer socket"
+                                           : L" and asked to close over its installer socket";
+        } else if (target.main == nullptr) {
+            // Nothing of the application's own to post a close to, and no socket to ask: the
+            // instance is named and the guard's end answer is what can free its files.
+            if (process != nullptr) {
+                CloseHandle(process);
+            }
+            ++result.withoutWindow;
+            lines.push_back(line + L" has no window of its own to close");
+            continue;
+        } else if (PostMessageW(target.main, WM_CLOSE, 0, 0) == FALSE) {
+            // Posted to the instance's own window even when a question of its own is up. The
+            // close is what that window's close button does, and the application is the one
+            // that answers it: a current client refuses the close while its confirmation is
+            // pending, leaving the question in front, and only a client from before that guard
+            // accepts it and hides the window with the question. Withholding the request
+            // therefore protected nobody - it left the installer asking the user to close a
+            // program it had never asked - while the guard's end answer is what frees the files
+            // when the close cannot.
+            //
             // Read the error before CloseHandle, which is free to reset it.
             const DWORD postError = GetLastError();
             if (process != nullptr) {
@@ -290,8 +314,9 @@ closeCockatriceHolders(const std::vector<Holder> &holders, unsigned long waitMil
             lines.push_back(line + L", and the close request could not be sent (win32 " + std::to_wstring(postError) +
                             L")");
             continue;
+        } else {
+            line += L" and asked to close";
         }
-        line += L" and asked to close";
         ++result.askedToClose;
         if (process == nullptr) {
             line += L", but its exit could not be watched (win32 " + std::to_wstring(openError) + L")";
@@ -311,6 +336,10 @@ closeCockatriceHolders(const std::vector<Holder> &holders, unsigned long waitMil
             const bool exited = allExited || WaitForSingleObject(asked[index], 0) == WAIT_OBJECT_0;
             if (exited) {
                 lines[askedLines[index]] += L", exited after " + std::to_wstring(elapsed) + L" ms";
+            } else if (askedWindows[index] == nullptr) {
+                // Asked over the socket with no window of its own: there is no window state
+                // to describe, only that it is still there.
+                lines[askedLines[index]] += L", still running after " + std::to_wstring(elapsed) + L" ms";
             } else {
                 lines[askedLines[index]] += L", still running after " + std::to_wstring(elapsed) + L" ms" +
                                             describeWindowState(askedWindows[index], askedWasHidden[index]);