Any kind of computational geometry code will have subtle issues in just this area. Broadly, the issue is classification of a point or line as IN, OUT, or ON. Of these, ‘ON’ is the difficult one because floating point prevents perfect calculation where real numbers would allow it. This is because floating point is actually integer math with a companion power function, and the integer math part has the problem in cases of division where the numerators and denominators are relatively prime. This leads to differences in edge and point positions and frustrates the ON calculation, and this the in and out calculation near the edge.
I suggest that you think of this carefully because there is generally no good solution to be found by considering only floating point precision—the same problem will exist at any non-infinite precision. However, to answer your specific question, yes you can easily truncate float32 to a lower precision. Just force the bits you want to truncate to zero. Hp is high precision Lp is low precision Lp := (unsafe interpret as float)(Mask & (unsafe interpret as integer(Hp)) Mask would be used to discard the lower bits of the word and thus precision. Mask := ^uint32(0) << k //discard lowest k bits Nice examples of what this accomplishes are here: http://www.h-schmidt.net/FloatConverter/IEEE754.html From: <golang-nuts@googlegroups.com> on behalf of <xiiop...@gmail.com> Date: Saturday, September 17, 2016 at 8:57 PM To: golang-nuts <golang-nuts@googlegroups.com> Subject: [go-nuts] truncate float32 to lesser precision (22 bit mantissa) ok I have a problem with rounding errors on floats which I think is unavoidable. Just as specific background this is happening for me on tests of vectors (lines) intersecting with bounding boxes (ie oblongs, axis aligned). Two code snippets shows the core of the maths.. front_n = (x_front - s.X) / v.X ripY = s.Y + front_n*v.Y ripZ = s.Z + front_n*v.Z if (front_n > 0) && (ymin <= ripY && ripY <= ymax) && (zmin <= ripZ && ripZ <= zmax) { ... etc and so on eg : back_n = (x_back - s.X) / v.X ripY = s.Y + back_n*v.Y ripZ = s.Z + back_n*v.Z if (back_n > 0) && (ymin <= ripY && ripY <= ymax) && (zmin <= ripZ && ripZ <= zmax) { ... etc where xmin, ymin etc represent the boundaries of the box, anything .X , .Y etc is one component of a 3d vector, and anything suffix _n is a scalar. All are floats One of the two boundaries (front or back face) is occasionally skipped .. The issue occurs obviously (?) when the vector intersects and crosses the box close to or on and edge/corner are reached. It's rare, and clearly (?) switching to float64 makes it less rare, but doesn't solve the problem. I considered multiplying out the divides but that greatly complicates the boundary conditions, and I don't think it actually solves the problem.. I think using a 'delta' in the boundaries just kicks the same problem down the road. (I've got a simple workaround that doesn't solve the core problem..) I think I need to truncate the accuracy of the float to 22 bits - so the question is really about guard and round bits etc - we got anything supporting that in go ? It looked like I would have to use pkg unsafe -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.