Rather than guessing what is going on, I think it's time to break out the profiling tools Manish.
On Wed, 17 May 2017, 15:23 David Klempner <klemp...@google.com> wrote: > > On May 16, 2017 22:03, "Ian Lance Taylor" <i...@golang.org> wrote: > > On Tue, May 16, 2017 at 9:26 PM, Manish Rai Jain <manishrj...@gmail.com> > wrote: > >> The runtime will spawn a new thread to replace the one that is blocked. > > > > Realized that after writing my last mail. And that actually explains > some of > > the other crashes we saw, about "too many threads", if we run tens of > > thousands of goroutines to do these reads, one goroutine per read. > > > > It is obviously lot more expensive to spawn a new OS thread. It seems > like > > this exact same problem was already solved for network via netpoller > > (https://morsmachine.dk/netpoller). Blocking OS threads for disk reads > made > > sense for HDDs, which could only do 200 IOPS; for SSDs we'd need a > solution > > based on async I/O. > > Note that in the upcoming Go 1.9 release we now use the netpoller for > the os package as well. However, it's not as effective as one would > hope, because on GNU/Linux you can't use epoll for disk files. > > > There's a not very well documented API to make AIO completions kick an > eventfd. > > It > mainly helps with pipes. > > > Ian > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
