Floating point calculations are imprecise.

https://0.30000000000000004.com/
https://www.phys.uconn.edu/~rozman/Courses/P2200_15F/downloads/floating-point-guide-2015-10-15.pdf

s

On Monday, June 1, 2020 at 11:56:55 AM UTC-7, Saksham Saxena wrote:
>
> Hello,
>
> I was trying to implement Substring Search using a rolling hash function 
> (fairly simple implementation): https://play.golang.org/p/_fjgxPC06So
>
> This works correctly and as expected. However, if you change the const 
> `baseNumber` to 128, the program is unable to do the intermediate 
> arithmetic incorrectly (see the difference after running it). 
>
> How could this be ? I've stared at it long enough and it doesn't seem like 
> it's overflowing (or is it really ? I thought float64 had great range ?)
>
> Any ideas/comments are appreciated. 
>
> Thanks!
> Saksham
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d071b2be-9fc1-45e4-9da8-baecb9968ea7%40googlegroups.com.

Reply via email to