Am Sonntag, den 13.07.2008, 01:14 +0200 schrieb Michael Karcher: > Am Samstag, den 12.07.2008, 23:56 +0100 schrieb Reece Dunn: > If you are brave enough to really find out what happens (might be > different depending on Windows version, instruction sets available and > things like that), also try the identity matrix scaled by 2^66 and > 2^(-66). The square of 2^66 overflows a 32 bit floating point number. Sorry, forgot denormals. The square of 2^(-66) *is* exactly represenatable as float! Use 2^(-80) to be sure that the square underflows.
Regards, Michael Karcher
