agree! On Wed, Nov 29, 2017 at 11:37 AM, 'Axel Wagner' via golang-nuts < golang-nuts@googlegroups.com> wrote:
> BTW, depending on what the problem is we are actually talking about, even > the numbers from the initial post might be fine. With those numbers, > sorting the 2GB should take roughly half an hour. A significant amount of > time, yes, but if you only need to do it once (or once a day or week or > whatever) it might not be worth spending any significant amount of time > making it go faster. Seems counter-intuitive, I know, but CPU time is very > cheap and human time is very expensive :) > > On Wed, Nov 29, 2017 at 8:08 PM, Michael Jones <michael.jo...@gmail.com> > wrote: > >> here's a little more detail on system / direct-type / and parallel >> direct-type sorting: >> >> celeste:double mtj$ go test -run=NONE -bench Sort >> goos: darwin >> goarch: amd64 >> pkg: double >> BenchmarkSSort250-8 100000 11285 ns/op >> BenchmarkSSort500-8 50000 31001 ns/op >> BenchmarkSSort1000-8 20000 93048 ns/op >> BenchmarkSSort2000-8 10000 218677 ns/op >> BenchmarkSSort4000-8 3000 473944 ns/op >> BenchmarkSSort8000-8 2000 1033446 ns/op >> BenchmarkSSort16000-8 1000 2208783 ns/op >> BenchmarkSSort32000-8 300 4599316 ns/op >> BenchmarkSSort64000-8 200 9571278 ns/op >> BenchmarkSSort128000-8 100 20524194 ns/op >> BenchmarkSSort256000-8 30 42263166 ns/op >> BenchmarkSSort512000-8 20 89330628 ns/op >> BenchmarkSSort1024000-8 10 190409724 ns/op >> BenchmarkQSort250-8 500000 2978 ns/op >> BenchmarkQSort500-8 200000 10271 ns/op >> BenchmarkQSort1000-8 30000 36474 ns/op >> BenchmarkQSort2000-8 20000 93520 ns/op >> BenchmarkQSort4000-8 10000 217781 ns/op >> BenchmarkQSort8000-8 3000 470050 ns/op >> BenchmarkQSort16000-8 2000 1020604 ns/op >> BenchmarkQSort32000-8 1000 2184270 ns/op >> BenchmarkQSort64000-8 300 4750204 ns/op >> BenchmarkQSort128000-8 200 9841153 ns/op >> BenchmarkQSort256000-8 100 21644805 ns/op >> BenchmarkQSort512000-8 30 46220023 ns/op >> BenchmarkQSort1024000-8 20 89012565 ns/op >> BenchmarkPSort250-8 500000 3009 ns/op >> BenchmarkPSort500-8 200000 10284 ns/op >> BenchmarkPSort1000-8 50000 36435 ns/op >> BenchmarkPSort2000-8 30000 48991 ns/op >> BenchmarkPSort4000-8 20000 87163 ns/op >> BenchmarkPSort8000-8 10000 156933 ns/op >> BenchmarkPSort16000-8 5000 289609 ns/op >> BenchmarkPSort32000-8 2000 616701 ns/op >> BenchmarkPSort64000-8 1000 1183778 ns/op >> BenchmarkPSort128000-8 500 2602444 ns/op >> BenchmarkPSort256000-8 300 5492333 ns/op >> BenchmarkPSort512000-8 100 10974195 ns/op >> BenchmarkPSort1024000-8 50 22786689 ns/op >> PASS >> ok double 104.041s >> >> ...which has parallel sorting an additional 2x faster on average. More >> CPUs helps, but not anywhere close to scalably. The real issue here is >> cache effects, memory bandwidth, and bounds checking. >> >> On Wed, Nov 29, 2017 at 10:47 AM, Michael Jones <michael.jo...@gmail.com> >> wrote: >> >>> Using a personal sort to avoid the standard sort's abstraction... >>> >>> // 50331648 bytes, 6291456 8-byte elements, 0.687317 <06873%2017> >>> seconds (48MB of uint64) >>> // 50331648 bytes, 50331648 1-byte elements, 1.605258 seconds (48MB >>> of uint8) >>> // 2147483648 <(214)%20748-3648> bytes, 268435456 8-byte elements, >>> 35.262593 seconds (2GB of uint64) >>> // 2147483648 <(214)%20748-3648> bytes, 2147483648 <(214)%20748-3648> >>> 1-byte elements, 68.793937 seconds (2GB of uint8) >>> >>> ...is good for another 2-3x. Running in parallel mode would get you >>> another 3x-5x, but i expect 5x this rate is as good as you will get. (at >>> least with battery, laptop, pyjamas, and reclining on a sofa :-) >>> -- >>> Michael T. Jones >>> michael.jo...@gmail.com >>> >> >> >> >> -- >> Michael T. Jones >> michael.jo...@gmail.com >> > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Michael T. Jones michael.jo...@gmail.com -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
