On Fri, Feb 4, 2022 at 4:13 PM envee <neeraj.vaidy...@gmail.com> wrote: > > Hi All, > I have an application which always has about 2000 goroutines running. (I > checked this by querying prometheus metrics at regular intervals.) This > number sounds reasonably correct to me because at the start I launch 1000 > goroutines and each of these 1000 goroutines spawns 1 goroutine each to make > an HTTP/2 request to a server. > > I run this application on a 40 vCPU machine which is quite lightly loaded. > Using atop or top, I can see that the overall CPU utilization is very low > before I start my application. Infact, it is idle for 99% of the time before > I run my application. > > top - 11:01:03 up 255 days, 23:33, 2 users, load average: 0.04, 0.05, 0.05 > Tasks: 525 total, 1 running, 524 sleeping, 0 stopped, 0 zombie > %Cpu(s): 0.0 us, 0.1 sy, 0.0 ni, 99.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 > st > KiB Mem : 52815753+total, 47459616+free, 4746712 used, 48814668 buff/cache > KiB Swap: 4194300 total, 4194300 free, 0 used. 51626032+avail Mem > > Now I turned on the go scheduler trace as below and run my application. > > GODEBUG=schedtrace=10 ./myapp start -c config.yaml > > I always see all of the local runqueues of the procs with a value of 0. Also, > my idleprocs is always equal to the total number of logical procs on my > machine (i.e. 40). > > SCHED 11639ms: gomaxprocs=40 idleprocs=40 threads=46 spinningthreads=0 > idlethreads=41 runqueue=0 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] > > Does this mean that the go scheduler is not scheduling the goroutines fast > enough ? I expected to see all the local run queues to conain a value of > atleast 1 when my application is running. > > In my application I make a lot of HTTP/2 calls at the rate of say 1000 rps or > 2000 rps etc. so it is a pretty busy application I would imagine. > > I noticed only 1 instance where a lot of goroutines assigned to the local run > queue of some processors. And in this instance I can see that there are very > few idleprocs. > > SCHED 11926ms: gomaxprocs=40 idleprocs=2 threads=46 spinningthreads=1 > idlethreads=3 runqueue=80 [2 0 0 0 2 1 0 0 2 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 > 162 0 0 0 0 0 0 0 0 0 0 0 0 0 0] > > Is there some other diagnostic to look at to explain why this is happening ?
A goroutine that is waiting for network I/O will not show up in the schedtrace list. Whether this is expected or not really depends on what your goroutines are doing. If they are doing a lot of computation then they should be there. But if they are just putting together a simple network request, sending it off, and waiting for a reply, then it is normal for your goroutines to be mostly idle. Ian -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWcjVe35TSgJzt%2BQa6wX%3DNjuiaQaE5Luaqs0mNcKwc5GA%40mail.gmail.com.
