Thanks. I noticed those but didn’t look into how to address. Appreciate it. I will rerun.
> On Apr 25, 2024, at 12:23 PM, Steven Hartland <stevenmhartl...@gmail.com> > wrote: > > Thanks for these, not sure if you noticed the notes from each run e.g. need > >= 4 samples to detect a difference at alpha level 0.05. > > Basically run the benchmark with a -count=6 or more and then run the tool, > and you get a the comparison values which are typically the interesting bit > > On Thu, 25 Apr 2024 at 00:29, Robert Engels <rob...@me.com > <mailto:rob...@me.com>> wrote: > │ /Users/robertengels/go1.21.5.txt │ > /Users/robertengels/go1.22.2.txt │ > > > │ sec/op │ sec/op vs base > │ > > > AddFixed-8 0.9603n ± ∞ ¹ 0.7931n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddDecimal-8 66.41n ± ∞ ¹ 66.27n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddBigInt-8 9.452n ± ∞ ¹ 10.650n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddBigFloat-8 63.26n ± ∞ ¹ 66.33n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulFixed-8 3.519n ± ∞ ¹ 3.939n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulDecimal-8 65.98n ± ∞ ¹ 67.07n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulBigInt-8 10.69n ± ∞ ¹ 10.49n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulBigFloat-8 23.72n ± ∞ ¹ 24.12n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivFixed-8 3.675n ± ∞ ¹ 3.661n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivDecimal-8 460.8n ± ∞ ¹ 469.6n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivBigInt-8 34.82n ± ∞ ¹ 34.90n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivBigFloat-8 110.4n ± ∞ ¹ 113.6n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpFixed-8 0.2529n ± ∞ ¹ 0.2784n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpDecimal-8 6.883n ± ∞ ¹ 6.475n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpBigInt-8 4.779n ± ∞ ¹ 4.805n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpBigFloat-8 4.411n ± ∞ ¹ 5.081n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringFixed-8 50.36n ± ∞ ¹ 50.64n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringNFixed-8 53.41n ± ∞ ¹ 49.66n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringDecimal-8 197.6n ± ∞ ¹ 197.0n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringBigInt-8 98.17n ± ∞ ¹ 98.00n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringBigFloat-8 386.2n ± ∞ ¹ 395.2n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > WriteTo-8 31.82n ± ∞ ¹ 31.71n ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > geomean 22.01n 22.28n +1.26% > > > > ¹ need >= 6 samples for confidence interval at level 0.95 > > > > ² need >= 4 samples to detect a difference at alpha level 0.05 > > > > > > > > │ /Users/robertengels/go1.21.5.txt │ > /Users/robertengels/go1.22.2.txt │ > > > │ B/op │ B/op vs base > │ > > > AddFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddDecimal-8 80.00 ± ∞ ¹ 80.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddBigFloat-8 48.00 ± ∞ ¹ 48.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulDecimal-8 80.00 ± ∞ ¹ 80.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivDecimal-8 384.0 ± ∞ ¹ 384.0 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivBigInt-8 8.000 ± ∞ ¹ 8.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivBigFloat-8 24.00 ± ∞ ¹ 24.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpDecimal-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringFixed-8 24.00 ± ∞ ¹ 24.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringNFixed-8 24.00 ± ∞ ¹ 24.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringDecimal-8 56.00 ± ∞ ¹ 56.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringBigInt-8 16.00 ± ∞ ¹ 16.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringBigFloat-8 176.0 ± ∞ ¹ 176.0 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > WriteTo-8 29.00 ± ∞ ¹ 28.00 ± ∞ ¹ ~ > (p=1.000 n=1) ³ > > > geomean ⁴ -0.16% > ⁴ > > > ¹ need >= 6 samples for confidence interval at level 0.95 > > > > ² all samples are equal > > > > ³ need >= 4 samples to detect a difference at alpha level 0.05 > > > > ⁴ summaries must be >0 to compute geomean > > > > > > > > │ /Users/robertengels/go1.21.5.txt │ > /Users/robertengels/go1.22.2.txt │ > > > │ allocs/op │ allocs/op vs base > │ > > > AddFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddDecimal-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > AddBigFloat-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulDecimal-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > MulBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivDecimal-8 12.00 ± ∞ ¹ 12.00 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivBigInt-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > DivBigFloat-8 2.000 ± ∞ ¹ 2.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpFixed-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpDecimal-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpBigInt-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > CmpBigFloat-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringFixed-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringNFixed-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringDecimal-8 4.000 ± ∞ ¹ 4.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringBigInt-8 1.000 ± ∞ ¹ 1.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > StringBigFloat-8 7.000 ± ∞ ¹ 7.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > WriteTo-8 0.000 ± ∞ ¹ 0.000 ± ∞ ¹ ~ > (p=1.000 n=1) ² > > > geomean ³ +0.00% > ³ > >> On Apr 24, 2024, at 6:20 PM, Steven Hartland <stevenmhartl...@gmail.com >> <mailto:stevenmhartl...@gmail.com>> wrote: >> >> What’s it look like when your run it through >> https://pkg.go.dev/golang.org/x/perf/cmd/benchstat >> <https://pkg.go.dev/golang.org/x/perf/cmd/benchstat> which will provide a >> nice side by side comparison? >> >> On Wed, 24 Apr 2024 at 19:26, 'Robert Engels' via golang-nuts >> <golang-nuts@googlegroups.com <mailto:golang-nuts@googlegroups.com>> wrote: >> I have a fairly stable project github.com/robaho/fixed >> <http://github.com/robaho/fixed> which is almost 100% cpu bound. It doesn’t >> change so it makes a great way to compare the performance of different Go >> versions using the same hardware. I took the time to re-run the tests today. >> >> Using 1.21.17: >> >> BenchmarkAddFixed-8 2000000000 0.59 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkAddDecimal-8 5000000 243 ns/op >> 176 B/op 8 allocs/op >> >> >> >> BenchmarkAddBigInt-8 100000000 14.3 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkAddBigFloat-8 20000000 78.8 ns/op >> 48 B/op 1 allocs/op >> >> >> >> BenchmarkMulFixed-8 300000000 4.88 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkMulDecimal-8 20000000 72.0 ns/op >> 80 B/op 2 allocs/op >> >> >> >> BenchmarkMulBigInt-8 100000000 17.1 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkMulBigFloat-8 30000000 35.5 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkDivFixed-8 300000000 4.71 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkDivDecimal-8 2000000 779 ns/op >> 568 B/op 21 allocs/op >> >> >> >> BenchmarkDivBigInt-8 30000000 46.1 ns/op >> 8 B/op 1 allocs/op >> >> >> >> BenchmarkDivBigFloat-8 20000000 108 ns/op >> 24 B/op 2 allocs/op >> >> >> >> BenchmarkCmpFixed-8 2000000000 0.38 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkCmpDecimal-8 200000000 8.05 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkCmpBigInt-8 300000000 5.87 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkCmpBigFloat-8 300000000 5.46 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkStringFixed-8 20000000 57.4 ns/op >> 32 B/op 1 allocs/op >> >> >> >> BenchmarkStringNFixed-8 20000000 55.6 ns/op >> 32 B/op 1 allocs/op >> >> >> >> BenchmarkStringDecimal-8 10000000 218 ns/op >> 64 B/op 5 allocs/op >> >> >> >> BenchmarkStringBigInt-8 10000000 122 ns/op >> 24 B/op 2 allocs/op >> >> >> >> BenchmarkStringBigFloat-8 3000000 416 ns/op >> 192 B/op 8 allocs/op >> >> >> >> BenchmarkWriteTo-8 30000000 45.8 ns/op >> 18 B/op 0 allocs/op >> >> and version 1.21.5: >> >> BenchmarkAddFixed-8 1000000000 0.9735 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkAddDecimal-8 14311995 69.99 ns/op >> 80 B/op 2 allocs/op >> >> >> >> BenchmarkAddBigInt-8 100000000 13.42 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkAddBigFloat-8 17506702 63.84 ns/op >> 48 B/op 1 allocs/op >> >> >> >> BenchmarkMulFixed-8 313983104 3.732 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkMulDecimal-8 18046520 66.59 ns/op >> 80 B/op 2 allocs/op >> >> >> >> BenchmarkMulBigInt-8 100000000 10.79 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkMulBigFloat-8 49186024 24.30 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkDivFixed-8 306888069 3.721 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkDivDecimal-8 2510688 462.4 ns/op >> 384 B/op 12 allocs/op >> >> >> >> BenchmarkDivBigInt-8 33993822 37.02 ns/op >> 8 B/op 1 allocs/op >> >> >> >> BenchmarkDivBigFloat-8 9415330 111.5 ns/op >> 24 B/op 2 allocs/op >> >> >> >> BenchmarkCmpFixed-8 1000000000 0.2548 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkCmpDecimal-8 168714549 7.086 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkCmpBigInt-8 234895634 4.952 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkCmpBigFloat-8 260814464 4.503 ns/op >> 0 B/op 0 allocs/op >> >> >> >> BenchmarkStringFixed-8 23725470 50.57 ns/op >> 24 B/op 1 allocs/op >> >> >> >> BenchmarkStringNFixed-8 23666628 50.67 ns/op >> 24 B/op 1 allocs/op >> >> >> >> BenchmarkStringDecimal-8 5665790 200.1 ns/op >> 56 B/op 4 allocs/op >> >> >> >> BenchmarkStringBigInt-8 10596398 100.2 ns/op >> 16 B/op 1 allocs/op >> >> >> >> BenchmarkStringBigFloat-8 2922332 391.2 ns/op >> 176 B/op 7 allocs/op >> >> >> >> BenchmarkWriteTo-8 45734523 31.53 ns/op >> 23 B/op 0 allocs/op >> which is pretty impressive across the board. >> >> Only 2 tests show any degradation and most show significant improvement. >> >> On the two that degrade, AddFixed is a fairly trivial add of 2 longs, so >> that is surprising. Strangely, WriteTo shows a different number of B/op on >> different runs (all of the other tests are stable). >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com >> <mailto:golang-nuts+unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/A6412E4C-03BD-477B-A725-06510DDD618A%40me.com >> >> <https://groups.google.com/d/msgid/golang-nuts/A6412E4C-03BD-477B-A725-06510DDD618A%40me.com?utm_medium=email&utm_source=footer>. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9DF5C6B6-4080-4487-A038-C2C0DB46D00B%40me.com.
