Serve the pinned upstream GGML server on vulkan/cuda/cpu #4

Closed

nsaspy wants to merge 0 commits from feature/ggml-backends into main

nsaspy commented

2026-09-15 03:47:57 +00:00

Owner

Replaces the PyTorch/ROCm container backends with the published qwentts.cpp images pinned to 71ad93d591a2811f35db77e27c02acba091c9e9b.

Backends: vulkan (default, /dev/dri render nodes only — /dev/kfd is gone entirely), cuda, cpu
sha256-verified GGUF provisioning (0.6B talker + 12Hz codec Q8_0) from the commit-pinned Serveurperso/Qwen3-TTS-GGUF repo
voice_manager persists same-stem .txt reference transcripts so the server entrypoint registers ICL clone-mode voices
doctor pre-flight, /health status, bounded systemd unit, GGML_VK_/RADV_ passthrough escape hatches
nix modules: backend enum vulkan|cuda|cpu, vulkan default

Validated live on RX 5500 XT (gfx1012/RDNA1): Vulkan init, 896MB KV-cache allocation (the whisper.cpp#3611 crash path), GPU synthesis (~2.5s), and realtime PCM streaming all pass with zero kernel faults.

Tests: 33 passed, flake check green.

Replaces the PyTorch/ROCm container backends with the published qwentts.cpp images pinned to 71ad93d591a2811f35db77e27c02acba091c9e9b. - Backends: vulkan (default, /dev/dri render nodes only — /dev/kfd is gone entirely), cuda, cpu - sha256-verified GGUF provisioning (0.6B talker + 12Hz codec Q8_0) from the commit-pinned Serveurperso/Qwen3-TTS-GGUF repo - voice_manager persists same-stem .txt reference transcripts so the server entrypoint registers ICL clone-mode voices - doctor pre-flight, /health status, bounded systemd unit, GGML_VK_*/RADV_* passthrough escape hatches - nix modules: backend enum vulkan|cuda|cpu, vulkan default Validated live on RX 5500 XT (gfx1012/RDNA1): Vulkan init, 896MB KV-cache allocation (the whisper.cpp#3611 crash path), GPU synthesis (~2.5s), and realtime PCM streaming all pass with zero kernel faults. Tests: 33 passed, flake check green.

nsaspy closed this pull request

2026-09-15 04:01:45 +00:00