> So, if an OS thread is blocked, no goroutines can be scheduled on this thread, therefore even pure CPU operations can't be run.
The runtime will spawn a new thread to replace the one that is blocked. On Wednesday, 17 May 2017 13:05:49 UTC+10, Manish Rai Jain wrote: > > On further thought about GOMAXPROCS, and its impact on throughput: > > A file::pread would block the OS thread. Go runs one OS thread per core. > So, if an OS thread is blocked, no goroutines can be scheduled on this > thread, therefore even pure CPU operations can't be run. This would lead to > core wastage. > > This is probably the reason why increasing GOMAXPROCS improves throughput, > and running any number of goroutines >= GOMAXPROCS has little impact on > anything. The underlying OS threads are already blocked, so goroutines > can't do much. > > If this logic is valid, then in a complex system, which is doing many > random reads, while also performing calculations (like Dgraph) would > suffer; even if we set GOMAXPROCS to a factor more than number of cores. > > Ideally, the disk reads could be happening via libaio, causing the OS > threads to not block, so all goroutines can make progress, increasing the > number of read requests that can be made concurrently. This would then also > ensure that one doesn't need to set GOMAXPROCS to a value greater than > number of cores to achieve higher throughput. > > > On Wed, May 17, 2017 at 10:38 AM, Manish Rai Jain <manis...@gmail.com > <javascript:>> wrote: > >> So, I fixed the rand and removed the atomics usage (link in my original >> post). >> >> Setting GOMAXPROCS definitely helped a lot. And now it seems to make >> sense, because (the following command in) fio spawns 16 threads; and >> GOMAXPROCS would do the same thing. However, the numbers are still quite a >> bit off. >> >> I realized fio seems to overestimate, and my Go program seems to >> underestimate, so we used sar to determine the IOPS. >> >> $ fio --name=randread --ioengine=psync --iodepth=32 --rw=randread --bs=4k >> --direct=0 --size=2G --numjobs=16 --runtime=120 --group_reporting >> Gives around 62K, tested via sar -d 1 -p, while >> >> $ go build . && GOMAXPROCS=16 ./randread --dir ~/diskfio --jobs 16 --num >> 2000000 --mode 1 >> Gives around 44K, via sar. Number of cores on my machine are 4. >> >> Note that this is way better than the earlier 20K with GOMAXPROCS = >> number of cores, but still leaves much to be desired. >> >> On Tue, May 16, 2017 at 11:36 PM, Ian Lance Taylor <ia...@golang.org >> <javascript:>> wrote: >> >>> On Tue, May 16, 2017 at 4:59 AM, Manish Rai Jain <manis...@gmail.com >>> <javascript:>> wrote: >>> > >>> > 3 is slower than 2 (of course). But, 2 is never able to achieve the >>> IOPS >>> > that Fio can achieve. I've tried other things, to no luck. What I >>> notice is >>> > that Go and Fio are close to each other as long as number of >>> Goroutines is >>> > <= number of cores. Once you exceed cores, Go stays put, while Fio IOPS >>> > keeps on improving, until it reaches SSD thresholds. >>> >>> One thing I notice about your program is that each goroutine is >>> calling rand.Intn and rand.Int63n. Those functions acquire and >>> release a lock, so that single lock is being contested by every >>> goroutine. That's an unfortunate and unnecessary slowdown. Give each >>> goroutine its own source of pseudo-random numbers by using rand.New. >>> >>> You also have a point of contention on the local variable i, which you >>> are manipulating using atomic functions. It would be cheaper to give >>> each goroutine a number of operations to do rather than to compute >>> that dynamically using a contended address. >>> >>> I'll also note that if a program that should be I/O bound shows a >>> behavior change when the number of parallel goroutines exceeds the >>> number of CPUs, then it might be interesting to try setting GOMAXPROCS >>> to be higher. I don't know what effect that would have here, but it's >>> worth checking. >>> >>> Ian >>> >> >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
