> The original would return val == 1, period == 62499999; While this does have
> some error [val / (period * 16 + 8) is slightly bigger than 1 / 10^9, error at
> 18[?] digit after dot], it's the best we can configure for the HW.

Correction. That's actually not *the best* we could configure -
 due to stopping at the first value between equal differences.
[you've already commented on that in the past, mentioning  that we
should use >= and not >].

But that doesn't change the fact that your approximation can choose
numbers which can't be configured to the HW, and as a result incorrectly
pick some that will not minimize the approximation error.

> One simple adjustment we could do is simply break from the loop If 'diff ==
> 0'. At least for small PPB value this would be hit relatively quickly.

Given the previous correction, the suggestion would also include
reversing the order of the iteration [7 -> 1 instead of 1 -> 7].

Reply via email to