https://bugs.kde.org/show_bug.cgi?id=525706
Bug ID: 525706
Summary: Whisper GPU: model loaded in fp32 and checkpoint left
in VRAM, making large-v3 impossible on 6 GB cards
Classification: Applications
Product: kdenlive
Version First 26.08.0
Reported In:
Platform: Arch Linux
OS: Linux
Status: REPORTED
Severity: normal
Priority: NOR
Component: Title Clips & Subtitles
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
Created attachment 196128
--> https://bugs.kde.org/attachment.cgi?id=196128&action=edit
Patch to fix it locally
DESCRIPTION
Whisper model *weights* are always placed in VRAM as fp32, even though every
Whisper checkpoint ships as fp16, so the model occupies twice the memory it
needs. Separately, the checkpoint is loaded straight onto the GPU and never
freed, so a second full copy is resident while the model is moved across. On a
6 GB card the two faults together make the `large-v3` model fail outright with
`torch.OutOfMemoryError`, and they more than double peak VRAM for the models
that do fit.
Measured with `large-v3-turbo` through the normal subtitle path, peak VRAM is
5736 MiB where 2530 MiB is sufficient.
This is independent of the "Disable half precision (FP16)" setting. That
setting controls the *compute* dtype passed to `transcribe()`; the weights are
fp32 in VRAM either way. Verified by probing the loaded model, which reports
`torch.float32` parameters with the box unticked.
All of this is in Kdenlive's own copy of the model-loading code in
`data/scripts/whisper/whispertotext.py`, not in openai-whisper.
openai-whisper's own `load_model()` uses `map_location="cpu"` for exactly this
reason.
STEPS TO REPRODUCE
1. Use a machine with an Nvidia GPU with 6 GB of VRAM (tested on an RTX 4050
Laptop, driver 610.57.04).
2. In Settings, Speech to Text, select the Whisper backend, the `large-v3`
model, and the GPU as the device.
3. Run Speech Recognition on any clip.
OBSERVED RESULT
Transcription fails and no subtitles are produced. The log shows:
```
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB.
GPU 0 has a total capacity of 5.66 GiB of which 31.56 MiB is free.
File "/usr/share/kdenlive/scripts/whisper/whispertosrt.py", line 80, in main
result = whispertotext.run_whisper(source, model, device, task, args)
File "/usr/share/kdenlive/scripts/whisper/whispertotext.py", line 69, in
run_whisper
loadedModel = model.to(device)
```
EXPECTED RESULT
`large-v3` loads and transcribes. Its weights are 2.87 GiB in fp16, so it fits
in 6 GB with room to spare.
SOFTWARE/OS VERSIONS
Operating System (available in the Info Center app, or by running `kinfo` in a
terminal window): Omarchy(Arch based), was the same problem on Fedora.
KDE Plasma Version: N/A
KDE Frameworks Version: 6.29.0
Qt Version: 6.11.2 |
ADDITIONAL INFORMATION
--
You are receiving this mail because:
You are watching all bug changes.
