It seems that the issue has something to do with the fluidsynth package, when I remove it the issue goes away. So maybe this is a fluidsynth issue as it was updated along with pipewire.
- Bug#1100943: pipewire: Sporadic audio crackling/distorti... Alejandro Rosso
- Bug#1100943: Narrow down the problem Alejandro Rosso
- Bug#1100943: Narrow down the problem Dylan Aïssi
- Bug#1100943: pipewire: Same issue here, same workar... Richard Scherping
- Bug#1100943: pipewire: Sporadic audio crackling/dis... Paul Cercueil
- Bug#1100943: pipewire: Sporadic audio crackling/dis... Drew Stephens
