AMD GPU STT — --device rocm/amd/hip silently degrades to CPU and --device vulkan crashes under the default faster-whisper provider #224

New issue

Closed

opened 2026-09-02 17:27:43 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-02 17:27:43 +00:00

Owner

Observed

On an AMD-only machine (Radeon RX 5500 XT, RADV Vulkan; no CUDA, no ROCm userspace):

nix run .# -- --wake --device rocm → Loading Whisper ... (device=cuda) → GPU transcription unavailable (This CTranslate2 package was not compiled with CUDA support); falling back to CPU. The AMD token was mapped to NVIDIA CUDA (STT_DEVICE_ALIASES in zara/__main__.py), CUDA predictably failed, and the run silently continued on CPU. The AMD GPU was never used.
nix run .# -- --wake --device vulkan → ValueError: Unsupported STT device 'vulkan'; choose one of: cpu, cuda, vulkan, rocm, hip, amd — a crash at argument parsing even though vulkan is an advertised choice.

Expected

AMD device tokens (amd, rocm, hip, vulkan) request AMD GPU acceleration. The repository already records the architecture (flake.nix): "whisper.cpp is the AMD/Radeon STT path. Vulkan avoids tying Zara's local voice acceleration to ROCm GPU support matrices", and normalize_stt_device already maps those tokens to vulkan for the whisper-cpp provider. But with the default faster-whisper provider, CTranslate2 cannot use AMD GPUs (no CUDA, no Vulkan build), so today the tokens are either lied into cuda or rejected.

The CLI must honor the AMD request through the existing whisper.cpp Vulkan backend, or fail with actionable guidance — never map AMD tokens to CUDA.

Acceptance criteria

--device amd|rocm|hip|vulkan with the default provider selects the whisper.cpp Vulkan backend (with a visible notice that the provider was routed), for both wake and dictate paths.
--device rocm (and amd/hip) never resolves to cuda.
--device vulkan never raises ValueError at parse time.
--device cuda keeps NVIDIA/CUDA semantics, including the existing GPU-unavailable CPU fallback message.
Help text for --device matches the new behavior; unit tests cover the routing matrix and regressions for both observed failures.
Full repository gate green at the exact candidate head; GitHub Actions test green for the pushed SHA.

Out of scope

Making CTranslate2/faster-whisper itself Vulkan/ROCm-capable (packaging experiment; the recorded AMD path is whisper.cpp).
The daemon voice client device selection (#244 will migrate the wake listener onto ZARA/1; this fix covers the current CLI path).

## Observed On an AMD-only machine (Radeon RX 5500 XT, RADV Vulkan; no CUDA, no ROCm userspace): - `nix run .# -- --wake --device rocm` → `Loading Whisper ... (device=cuda)` → `GPU transcription unavailable (This CTranslate2 package was not compiled with CUDA support); falling back to CPU`. The AMD token was mapped to NVIDIA CUDA (`STT_DEVICE_ALIASES` in `zara/__main__.py`), CUDA predictably failed, and the run silently continued on CPU. The AMD GPU was never used. - `nix run .# -- --wake --device vulkan` → `ValueError: Unsupported STT device 'vulkan'; choose one of: cpu, cuda, vulkan, rocm, hip, amd` — a crash at argument parsing even though `vulkan` is an advertised choice. ## Expected AMD device tokens (`amd`, `rocm`, `hip`, `vulkan`) request AMD GPU acceleration. The repository already records the architecture (flake.nix): *"whisper.cpp is the AMD/Radeon STT path. Vulkan avoids tying Zara's local voice acceleration to ROCm GPU support matrices"*, and `normalize_stt_device` already maps those tokens to `vulkan` for the whisper-cpp provider. But with the default `faster-whisper` provider, CTranslate2 cannot use AMD GPUs (no CUDA, no Vulkan build), so today the tokens are either lied into `cuda` or rejected. The CLI must honor the AMD request through the existing whisper.cpp Vulkan backend, or fail with actionable guidance — never map AMD tokens to CUDA. ## Acceptance criteria 1. `--device amd|rocm|hip|vulkan` with the default provider selects the whisper.cpp Vulkan backend (with a visible notice that the provider was routed), for both wake and dictate paths. 2. `--device rocm` (and `amd`/`hip`) never resolves to `cuda`. 3. `--device vulkan` never raises `ValueError` at parse time. 4. `--device cuda` keeps NVIDIA/CUDA semantics, including the existing GPU-unavailable CPU fallback message. 5. Help text for `--device` matches the new behavior; unit tests cover the routing matrix and regressions for both observed failures. 6. Full repository gate green at the exact candidate head; GitHub Actions `test` green for the pushed SHA. ## Out of scope - Making CTranslate2/faster-whisper itself Vulkan/ROCm-capable (packaging experiment; the recorded AMD path is whisper.cpp). - The daemon voice client device selection (#244 will migrate the wake listener onto ZARA/1; this fix covers the current CLI path).

nsaspy closed this issue

2026-09-02 17:27:43 +00:00

nsaspy referenced this issue from a commit

2026-09-04 22:28:49 +00:00

Merge pull request #224 from lost-rob0t/fix/zara-016-llm-provider-contracts

nsaspy referenced this issue

2026-09-08 02:03:44 +00:00

[ORDER] ZARA-000 — Repository repair and real-time voice roadmap #1