For error analysis of floating point filters, I recommend starting with: Bhaskar D. Rao “Floating Point Arithmetic and Digital Filters” IEEE Trans on Signal Processing 1992
On Sat, Aug 5, 2023 at 1:54 PM Sampo Syreeni <[email protected]> wrote: > So, fixpoint, with a proper gain structure, actually is easier to > analyse and work with than float. Fixed point is easier to analyze because the noise floor is approximately constant (independent of the signal level). But floating point is actually better *behaved* since the SNR is approximately constant. Which is (usually) what you want. Even if aspects of the analysis are consequently more difficult. > > That's right and I don't want that to cause a fucking exception. But > > I wanna keep having the utility of denorms. > > So as Buffy would have it, what *is* *it*? What is the utility of > denorms? I get it that they're there for a reason, but what do they > *mean* and how do they *serve*? > I have never seen any practical use for denorms in audio, and mostly when they come up it's about how to deal with cases where you can't ensure FTZ. I'm told they simplify formal proofs of algorithm behavior, but this is more of a cryptography/security thing and doesn't seem applicable to audio. Seems like they made it into the standard because the rest of the design implies their existence, and there's no strong reason not to include them (since anyone who cares can just FTZ them away). > > If given an assignment of developing an audio processing system using > > fixed-point math, I will not shrink away from the challenge, but > > **if** the project is "Hey we got this 64-bit ARM with FPU in it and > > gobs of memory, I don't want my code to be checking for saturation and > > "minding the gain structure". Fuck no. > > My point is that you can't avoid the analysis even with 64-bit floats. > The saturation and gain structure is not just design analysis. This is actual runtime overhead - reflecting limitations of the data format - and it imposes significant costs to the product and trade-offs to the design. Trade-offs and costs which *do not apply* in floating point, and hence don't require intensive analysis/design. > You might want to do that because a well-thoughtout gain structure at 16 > bits suddenly yields you four times the performance from your 64-bit > audio workstation. Unlikely, for typical algorithms. In the first place, a performance-sensitive floating point design would use 32 bit arithmetic. Likewise, the fixed point algorithm will require 32 bit arithmetic for most non-trivial calculations (this is your "well-thought-out gain structure"). So, there is no basic advantage in terms of bit depth, and meanwhile the fixed point version has the extra overhead of saturation/gain computations throughout. Which not only adds overhead directly, but also reduces pipeline efficiency/parallelism throughout the implementation. For these reasons, floating point implementation is typically significantly more performant on big, modern CPUs under discussion here. Where fixed point has the advantage is in the context of specialized low-power processors, which do *not* have floating point/vector units, and *do* have HW support for saturating arithmetic and other common fixed-point requirements. > No, it is not, unless you can somehow represent all of the sample values > perfectly along the way from perfect digitla input to perfect digital > output. As you then can't, you need to analyse fully what happens to the > errors. Do they accumulate or do they die down, over the overall gain > structure, do they cancel exactly, do they accumulate... You need to do > overall numerical analysis over the whole signal chain, and sometimes it > needs to be accurate to the bit. > This is not what people are talking about when they refer to the saturation/gain structure overhead of fixed point implementations. They are talking about designing overhead/compromises directly into the algorithm to implement this stuff. There is nothing stopping anyone from applying numerical analysis to floating point algorithm design, and indeed we have seen a profusion of well-behaved, performant floating point implementations in the modern era. E
