naveentatikonda commented on issue #13519:
URL: https://github.com/apache/lucene/issues/13519#issuecomment-2209645562
> I disagree. We want `-127, 127`. Otherwise we have to handle the exotic
`-128*-128` case when doing vector comparisons and this disallows some nice
SIMD optimizations down the road.
>
@benwtrent Sorry, didn't get it. Can you pls share more details or links
related to this.
> * Applies a modification of the rounding error compensation. The idea
behind how I applied the different compensation now is that we need to handle
the linear scaling as this can cause us to snap to different buckets. I am
still not 100% sure my transformation is correct there.
@MilindShyani Can you check and comment on this. `(dx - dxq) * (dxq -
SIGNED_CORRECTION * alpha)` is the rounding error compensation added by Ben.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
