fix(asr): guard model memory and clean up failed loads (#5773). Principle XI.

Guard automatic Copy Assist model promotion with free-memory and total-capacity requirements. Fail Whisper weight allocation cleanly and release model/VAD resources before recovery instead of dereferencing unbacked tensors or leaking prior allocations.

Existing local ASR GPU-probe and linkage smoke tests passed 2/2. Windows prebuilt speech-library replacement remains outside this PR.

Squashed-from: #5773
Co-authored-by: Jeff Skerker <7691216+skerker@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: jensenpat <patjensen@gmail.com>

Author: 3 people

src/asr/WhisperAsrBackend.h

@@ -149,6 +149,43 @@ inline AsrTierResolution asrReconcileDefaultTier(const QString& currentTier,
     return {currentTier, false};
 }
 
+// Whether a model tier of `tierSizeBytes` (the weights file) can be expected to
+// load on a device reporting this much memory. Gates only the AUTOMATIC raise
+// to the GPU-default tier: "a GPU exists" says nothing about room, and a 1.6 GB
+// model auto-selected for a 2 GB card is #4972. An explicit operator choice is
+// never refused here — that stays the operator's call.
+//
+// The headroom is what whisper allocates beyond the weights (KV caches and
+// compute buffers). MEASURED (#4972 bench, RTX 5060 Laptop, ggml-vulkan,
+// 2026-09-16): large-v3-turbo occupies 1818 MiB against a 1549 MiB file
+// (+268 MiB), base 293 MiB against 141 MiB (+152 MiB); whisper's own load log
+// sums to the same figure. 300 MiB covers the larger of the two.
+//
+// The free figure is not always free memory: ggml-vulkan reports free == total
+// for a device without VK_EXT_memory_budget (ggml_backend_vk_get_device_memory),
+// so the free check alone can be handed the whole heap. The total must therefore
+// clear the same need plus a reserve for the desktop and AetherSDR's own
+// rendering on that card, which makes the answer independent of the reporting
+// mode. The reserve is a chosen margin, not a measurement; for scale, MEASURED
+// total minus free at the startup probe was 367 MiB (#5730 reporter log, GTX
+// 1050, 1809 of 2176 MB free) and 791 MiB (#4972 bench, 7360 of 8151 MB free).
+//
+// Both figures 0 means the device could not be asked (AsrGpuDevice) — unknown
+// is not "too small", so it keeps the previous behaviour. Integrated GPUs
+// report shared system memory and pass on their own numbers. Header-inline and
+// whisper-free, like asrReconcileDefaultTier above.
+inline constexpr quint64 kAsrTierVramHeadroomBytes = 300ull * 1024ull * 1024ull;
+inline constexpr quint64 kAsrTierVramDesktopReserveBytes = 512ull * 1024ull * 1024ull;
+
+inline bool asrTierFitsVram(quint64 vramFreeBytes, quint64 vramTotalBytes, qint64 tierSizeBytes)
+{
+    if (vramTotalBytes == 0 || tierSizeBytes <= 0) {
+        return true;
+    }
+    const quint64 need = static_cast<quint64>(tierSizeBytes) + kAsrTierVramHeadroomBytes;
+    return vramFreeBytes >= need && vramTotalBytes >= need + kAsrTierVramDesktopReserveBytes;
+}
+
 // A selectable transcription language: `code` is the ISO code passed to the
 // backend (e.g. "en", "es"); `name` is the English display name ("English").
 struct AsrLanguage {

src/gui/CopyAssistController.cpp

@@ -650,17 +650,46 @@ void CopyAssistController::applyGpuDevices(std::vector<AsrGpuDevice> gpus)
         // walk-back, the load-time fallback arm kept large-v3-turbo running
         // on CPU: the "backlog climbing, no text" symptom this PR opens with
         // (#4767 review). An explicitly chosen tier is never changed.
-        const bool resolvedGpuUsable = resolvedDevice >= 0 && [&] {
-            for (const AsrGpuDevice& g : m_gpuDevices) {
-                if (g.index == resolvedDevice) {
-                    return g.usable;
-                }
+        const AsrGpuDevice* resolvedGpu = nullptr;
+        for (const AsrGpuDevice& g : m_gpuDevices) {
+            if (g.index == resolvedDevice) {
+                resolvedGpu = &g;
+                break;
+            }
+        }
+        const bool resolvedGpuUsable =
+            resolvedDevice >= 0 && resolvedGpu != nullptr && resolvedGpu->usable;
+        // The raise additionally needs ROOM: a usable GPU that cannot hold the
+        // GPU-default tier must not be handed it (#4972 — 1.6 GB auto-selected
+        // for a 2 GB card). Only the raise is gated. A tier already running is
+        // not walked back on this figure: once a model is loaded, the device's
+        // free memory is low because of that very model.
+        const QString gpuDefaultTier = QStringLiteral("large-v3-turbo");
+        bool wantGpuDefault = m_useGpuDefaultIfAvailable;
+        if (wantGpuDefault && resolvedGpuUsable) {
+            const AsrModelTier* gpuTier = AsrModelCatalog::tierById(gpuDefaultTier);
+            const qint64 gpuTierBytes = gpuTier != nullptr ? gpuTier->sizeBytes : 0;
+            if (!asrTierFitsVram(resolvedGpu->vramFreeBytes, resolvedGpu->vramTotalBytes,
+                                 gpuTierBytes)) {
+                wantGpuDefault = false;
+                // Warning, not info: lcGui is declared QtWarningMsg, so an info
+                // line would be absent from every default support log — and
+                // this is the line that explains why the GPU tier was withheld.
+                const quint64 needMb =
+                    (static_cast<quint64>(gpuTierBytes) + kAsrTierVramHeadroomBytes)
+                    / (1024 * 1024);
+                qCWarning(lcGui).nospace()
+                    << "ASR: keeping the default model tier - " << resolvedGpu->name << " has "
+                    << (resolvedGpu->vramFreeBytes / (1024 * 1024)) << " of "
+                    << (resolvedGpu->vramTotalBytes / (1024 * 1024)) << " MB free, "
+                    << gpuDefaultTier << " needs about " << needMb << " MB free on a device of "
+                    << (needMb + kAsrTierVramDesktopReserveBytes / (1024 * 1024))
+                    << " MB or more";
             }
-            return false;
-        }();
+        }
         const AsrTierResolution tier = asrReconcileDefaultTier(
-            m_tierId, m_useGpuDefaultIfAvailable, m_gpuDefaultTierActive,
-            resolvedGpuUsable, QStringLiteral("large-v3-turbo"),
+            m_tierId, wantGpuDefault, m_gpuDefaultTierActive,
+            resolvedGpuUsable, gpuDefaultTier,
             AsrModelCatalog::defaultTierId());
         m_gpuDefaultTierActive = tier.gpuDefaultActive;
         if (tier.tierId != m_tierId) {

tests/asr_gpu_probe_test.cpp

@@ -202,6 +202,82 @@ int main()
                     "explicit choices untouched)\n");
     }
 
+    // VRAM gate on the automatic raise to the GPU-default tier (#4972): "a GPU
+    // exists" must not be enough to select a 1.6 GB model.
+    {
+        constexpr quint64 kMiB = 1024ull * 1024ull;
+        // Copy of the "large-v3-turbo" sizeBytes in AsrModelCatalog.cpp (this
+        // target does not link the catalog) — keep the two in step.
+        constexpr qint64 kTurboBytes = 1624555275;
+
+        // MEASURED (#4972 bench, RTX 5060 Laptop 8151 MiB under VRAM ballast,
+        // 2026-09-16): the app logged "VRAM free 1126 of 8151 MB" and enabling
+        // the auto-raised tier took SIGSEGV in the whisper model load.
+        if (asrTierFitsVram(1126 * kMiB, 8151 * kMiB, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] 1126 MB free was judged enough for the "
+                                 "1.6 GB tier (#4972)\n");
+            return 1;
+        }
+        // MEASURED (#5730 reporter log, GTX 1050, 2026-09-15): "VRAM free 1809
+        // of 2176 MB". The tier occupies 1818 MiB once loaded (same bench), so
+        // a 2 GB-class card is never auto-raised; choosing it stays possible.
+        if (asrTierFitsVram(1809 * kMiB, 2176 * kMiB, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] a 2 GB-class card (1809 MB free) was "
+                                 "judged to fit the 1.6 GB tier (#4972)\n");
+            return 1;
+        }
+        // MEASURED (same bench, no ballast): "VRAM free 7360 of 8151 MB".
+        if (!asrTierFitsVram(7360 * kMiB, 8151 * kMiB, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] an 8 GB card with 7360 MB free was "
+                                 "refused the GPU-default tier (#4972)\n");
+            return 1;
+        }
+        // CONSTRUCTED: the boundary itself — weights + headroom fits, one byte
+        // less does not.
+        const quint64 need = static_cast<quint64>(kTurboBytes) + kAsrTierVramHeadroomBytes;
+        if (!asrTierFitsVram(need, 4096 * kMiB, kTurboBytes)
+            || asrTierFitsVram(need - 1, 4096 * kMiB, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] VRAM gate boundary is not weights + "
+                                 "headroom (#4972)\n");
+            return 1;
+        }
+        // CONSTRUCTED input on a MEASURED total (2176 MB, #5730 reporter log):
+        // ggml-vulkan reports free == total for a device without
+        // VK_EXT_memory_budget (ggml_backend_vk_get_device_memory), so the same
+        // card can present as 2176 of 2176 MB free. No capture of a driver in
+        // that mode exists; the row pins that the answer does not depend on it.
+        if (asrTierFitsVram(2176 * kMiB, 2176 * kMiB, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] a 2 GB-class card reporting free == total "
+                                 "was judged to fit the 1.6 GB tier (#4972)\n");
+            return 1;
+        }
+        // CONSTRUCTED: the total boundary — need + desktop reserve fits, one
+        // byte less does not, with free memory ample in both.
+        const quint64 needTotal = need + kAsrTierVramDesktopReserveBytes;
+        if (!asrTierFitsVram(needTotal, needTotal, kTurboBytes)
+            || asrTierFitsVram(needTotal - 1, needTotal - 1, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] VRAM gate total boundary is not weights + "
+                                 "headroom + desktop reserve (#4972)\n");
+            return 1;
+        }
+        // CONSTRUCTED: memory unknown (AsrGpuDevice leaves both 0 when the
+        // device could not be asked) is not "too small" — previous behaviour.
+        if (!asrTierFitsVram(0, 0, kTurboBytes)) {
+            std::fprintf(stderr, "[FAIL] unknown VRAM was treated as too small "
+                                 "(#4972)\n");
+            return 1;
+        }
+        // CONSTRUCTED: an unknown tier size cannot refuse a device.
+        if (!asrTierFitsVram(64 * kMiB, 2048 * kMiB, 0)) {
+            std::fprintf(stderr, "[FAIL] an unknown tier size refused a device "
+                                 "(#4972)\n");
+            return 1;
+        }
+        std::printf("[ok] #4972 VRAM gate on the GPU-default tier "
+                    "(weights + headroom, total clears a desktop reserve, "
+                    "unknown memory passes)\n");
+    }
+
     QElapsedTimer timer;
 
 #ifdef Q_OS_MACOS

third_party/whisper.cpp/AETHERSDR-PATCHES.md

@@ -5,9 +5,10 @@ The source snapshot is pinned to ggml-org/whisper.cpp commit
 The tree is otherwise an exact — if trimmed — upstream snapshot; see
 [`AETHER_VENDORING.md`](AETHER_VENDORING.md) for what was removed.
 
-AetherSDR carries three local changes: two in the ggml Metal backend, from the
-same fix (#4535, PR #4553), and one in the ggml CPU backend, a MinGW build fix
-(#4406) landed separately.
+AetherSDR carries four local changes: two in the ggml Metal backend, from the
+same fix (#4535, PR #4553), one in the ggml CPU backend, a MinGW build fix
+(#4406) landed separately, and one in `src/whisper.cpp`, a model-load crash fix
+(#4972).
 
 1. `ggml/src/ggml-metal/CMakeLists.txt`: adds `GGML_METAL_EMBED_LIBRARY_COMPILED`.
    Upstream's `GGML_METAL_EMBED_LIBRARY` embeds the merged kernel **source** and
@@ -72,6 +73,52 @@ completion in 75 minutes on a Radeon Pro 560X.
    `ggml-org/ggml` and `ggml-org/whisper.cpp`, with no existing issue or PR —
    so a `COMMIT` bump alone would not have picked up a fix.
 
+4. `src/whisper.cpp`: fails the model load when the weight buffer cannot be
+   allocated, at both sites that share the pattern — `whisper_model_load()` and
+   the VAD loader in `whisper_vad_init_with_params()` (the latter is not called
+   by AetherSDR; it is patched so the two copies cannot drift).
+
+   Upstream writes
+
+   ```cpp
+   ggml_backend_buffer_t buf = ggml_backend_alloc_ctx_tensors_from_buft(ctx, buft);
+   if (buf) { model.buffers.emplace_back(buf); ... }
+   ```
+
+   with no `else`, then uploads the weights with `ggml_backend_tensor_set()`.
+   When the allocation fails — a GPU short of memory is the ordinary case —
+   every tensor in that context still has no buffer, and the first upload
+   dereferences it. Measured on Linux with ggml-vulkan (RTX 5060 Laptop, ~1.1 GB
+   of device memory free, `ggml-large-v3-turbo.bin`): ggml logs
+   `alloc_tensor_range: failed to allocate Vulkan1 buffer of size 551900160`,
+   then the process takes SIGSEGV at address 0x10 in
+   `ggml_vk_buffer_write_2d()` on the ASR worker thread. A signal is not an
+   exception, so the guards in `WhisperAsrBackend` cannot intercept it (#4972).
+
+   The patch adds `whisper_ctx_has_unallocated_tensor()` and an `else if` on it
+   at both sites that logs and returns failure. A bare `else` would be wrong:
+   the allocator also returns NULL when every tensor in the context was already
+   allocated, so the helper repeats the allocator's own "needs allocation" test
+   (`data == NULL && view_src == NULL`, non-zero size) rather than treating NULL
+   as the error. The patch also changes the failed-load cleanup in
+   `whisper_init_with_params_no_state()` from `delete ctx` to
+   `whisper_free(ctx)`: the model stores raw ggml context and buffer pointers,
+   so deleting the C++ context alone leaks them, including any weight buffers
+   allocated before a later group failed. The VAD allocation-failure path
+   similarly calls `whisper_vad_free(vctx)` before returning NULL. The returned
+   NULL lets `WhisperAsrBackend::load()` latch the device and retry on CPU
+   after the failed attempt's resources have been released.
+
+   Checked upstream at the time of this patch (2026-09-16): both sites read the
+   same on `ggml-org/whisper.cpp` `master`, so a `COMMIT` bump alone would not
+   pick up a fix.
+
+   **The Windows release does not compile this file.** It links the prebuilt
+   `whisper.lib` from the `whisper-gpu-<ver>` release asset
+   (`ASR_USE_PREBUILT_WHISPER_GPU`, top-level `CMakeLists.txt`), so this patch
+   reaches Windows release binaries only when that pack is rebuilt from a tree
+   that contains it and its pinned SHA-256 is updated.
+
 ## Refreshing
 
 When refreshing whisper.cpp, first check whether upstream has adopted an
@@ -95,8 +142,18 @@ configure + build (`cmake --build` from a MinGW-w64 Ninja toolchain); a
 regression here only shows up as a MinGW compile failure, not a test failure,
 since MSVC and non-Windows builds never exercise this branch.
 
+For the `src/whisper.cpp` allocation check, look at both
+`ggml_backend_alloc_ctx_tensors_from_buft()` call sites: if upstream now fails
+the load when the returned buffer is NULL, drop the local patch. Otherwise
+reapply it at both, including the full context cleanup on the failure paths.
+There is no unit seam for it — forcing the failure needs a
+GPU backend that is short of memory — so confirm on a GPU host by occupying
+device memory until the large tier cannot fit and enabling Copy Assist: the log
+must show `GPU model load failed ... retrying on CPU` and the process must
+survive.
+
 The authoritative diff for any of these files is its git history
 (`git log -p -- third_party/whisper.cpp/ggml/src/ggml-metal/<file>` or
-`.../ggml-cpu/ggml-cpu.c`). No checked-in `.patch` copy is kept for any of
+`.../ggml-cpu/ggml-cpu.c`, or `.../src/whisper.cpp`). No checked-in `.patch` copy is kept for any of
 them: it would need hand-syncing on every edit, and its context would not
 apply cleanly across an upstream bump anyway.

third_party/whisper.cpp/AETHER_VENDORING.md

@@ -8,7 +8,7 @@ upstream `LICENSE` covers the bundled `ggml/` tree too — upstream ships one MI
 file for both.
 
 Keep this a pristine mirror: do **not** modify vendored sources in place unless
-the change genuinely cannot live outside the tree. Three files currently do;
+the change genuinely cannot live outside the tree. Four files currently do;
 every one is recorded in [`AETHERSDR-PATCHES.md`](AETHERSDR-PATCHES.md), and
 anything not listed there is a drift bug.
 
@@ -52,7 +52,7 @@ fallback). `GGML_NATIVE=OFF` is forced for portable/Pi/CI binaries.
 
 ## Local patches (deviations from pristine upstream)
 
-Three vendored files carry AetherSDR-local changes; the pristine-mirror rule
+Four vendored files carry AetherSDR-local changes; the pristine-mirror rule
 above holds for everything else. Each is described — with its rationale and
 the refresh checklist — in [`AETHERSDR-PATCHES.md`](AETHERSDR-PATCHES.md),
 following the same convention as `third_party/wdsp` and
@@ -67,6 +67,10 @@ following the same convention as `third_party/wdsp` and
   throttle guard to a feature-detect so it compiles under MinGW-w64, which
   doesn't declare `THREAD_POWER_THROTTLING_STATE`. Fixes the same MinGW-only
   compile break originally raised in #4406.
+- `src/whisper.cpp` — fails the model load when the weight buffer cannot be
+  allocated instead of uploading into unbacked tensors (SIGSEGV on a GPU short
+  of memory). Fixes #4972. Not in the prebuilt Windows `whisper-gpu` pack until
+  that pack is rebuilt.
 
 The two Metal changes are kept as thin as possible: the *policy* around them —
 required toolchain, missing-toolchain behaviour, deployment target, shader
@@ -85,5 +89,5 @@ To add a different GPU backend (CUDA, Metal, …), **re-copy that backend's
 directory** from upstream at the pinned commit and turn its `GGML_<X>` option ON
 (with the matching toolchain + CI runner). To refresh: clone upstream at
 `COMMIT`, re-run the same trim (keeping `ggml-cpu`, `ggml-blas`, `ggml-vulkan`),
-and diff — then re-apply the three local patches (see **Local patches** above);
-a clean diff plus exactly those three files is the expected end state.
+and diff — then re-apply the four local patches (see **Local patches** above);
+a clean diff plus exactly those four files is the expected end state.