On Nov 10, 2009, at 11:49 PM, Richard wrote: > I have some floating point numbers in my database which I need to test > for equality. Numbers like "0.3" are failing an equality test. > So, is there a way to store floating point numbers precisely? Or > should I be using something like: > offset = 0.000001 > (db.table.fp > v - offset)(db.table.fp < v + offset)
If you're using floating point, then yes, that's what you should do. You can write: (v-offset) < db.table.fp < (v+offset) or: abs(db.table.fp - v) < offset ...and stick it in a function if you use it much. Notice that for consistency, you need to use the same technique for < <= => > as well. a < b becomes a < (b-offset), etc, or else you get cases where a==b and a<b at the same time. Two other caveats. One is that the value of your epsilon (or offset) depends on your domain of values. 0.000001 isn't appropriate if you're dealing with (say) the mass of a proton in grams. The other, a minor one in most cases, is that the equality relationship we're talking about here isn't transitive. That is, it's possible that a == b and b == c but a != c. There's a large literature on the hazards of computing with floating point numbers. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to web2py@googlegroups.com To unsubscribe from this group, send email to web2py+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
