On Oct 10, 2008, at 4:04 AM, Steven Hamilton wrote:
Hi folks, I think I need some help understanding floats.I have a class that calculates a balance and places it in a table column. At the end of this method I NSLog the output like so;NSLog(@"current balance is: %@",balance); and get current balance is: 18976.69 in the console. So far so good.The method plonks the balance into a table and returns it to whence it came. The table is sourced by the normal datasource delegate methods and the column has a number formatter on it set to Decimal #,##0,00## with half even rounding. I placed the last ##'s in there to check that the floats weren't producing decimals with more than 2 places but in my table column I get 18976.6995 which doesn't make sense. The float calculation is exactly 2 decimal places, I know that for a fact and NSLog proves it. Yet somehow, the table datasource appears to produce random decimal digits where i have the last ##.Am I totally misunderstanding something about floats? Or is the extra ##'s in the formatter making things up?
There are an infinite number of floating point values between any two integer numbers, so it is not possible to represent every single possible float value using a fixed-width binary number. You're most likely seeing a number that can't be exactly represented.
If you're dealing with currency, you should probably be using integer arithmetic, and store all values as pennies or tenths of pennies, depending on the precision you need.
-- Dave Carrigan [EMAIL PROTECTED] Seattle, WA, USA
PGP.sig
Description: This is a digitally signed message part
_______________________________________________ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
