Pada tanggal Sab, 14 Mar 2020 06.15, Ian Lance Taylor <i...@golang.org> menulis:
> On Fri, Mar 13, 2020 at 1:05 AM Mhd Shulhan <m.shul...@gmail.com> wrote: > > > > > > > > > On 12 Mar 2020, at 13.13, Mhd Shulhan <m.shul...@gmail.com> wrote: > > > > > > > > >> My question is any one have idea how to debug this so I can provide > more > > >> detailed report? > > >> > > >> Thank you in advance. > > > > > > So, I try to debug with gdb 9.1. Here is the sample of stack when the > CPU got high, > > > > > > ... > > > [New LWP 1885] > > > [New LWP 1886] > > > [New LWP 1887] > > > [New LWP 1888] > > > [New LWP 1889] > > > ^C > > > Thread 1 "XYZ" received signal SIGINT, Interrupt. > > > runtime.futex () at > /Users/XXX/share/go/src/runtime/sys_linux_amd64.s:568 > > > 568 /Users/X/share/go/src/runtime/sys_linux_amd64.s: No such file > or directory. > > > (gdb) where > > > #0 runtime.futex () at > /Users/XXX/share/go/src/runtime/sys_linux_amd64.s:568 > > > #1 0x0000000000431666 in runtime.futexsleep (addr=0xd6a3e8 > <runtime.m0+328>, val=0, ns=-1) at > /Users/XXX/share/go/src/runtime/os_linux.go:44 > > > #2 0x000000000040bd9f in runtime.notesleep (n=0xd6a3e8 > <runtime.m0+328>) at /Users/XXX/share/go/src/runtime/lock_futex.go:151 > > > #3 0x000000000043b858 in runtime.stoplockedm () at > /Users/XXX/share/go/src/runtime/proc.go:1972 > > > #4 0x000000000043d4c6 in runtime.schedule () at > /Users/XXX/share/go/src/runtime/proc.go:2455 > > > #5 0x000000000043d89d in runtime.park_m (gp=0xc000166c00) at > /Users/XXX/share/go/src/runtime/proc.go:2691 > > > #6 0x00000000004651fb in runtime.mcall () at > /Users/XXX/share/go/src/runtime/asm_amd64.s:318 > > > #7 0x0000000000465114 in runtime.rt0_go () at > /Users/XXX/share/go/src/runtime/asm_amd64.s:220 > > > #8 0x0000000000000000 in ?? () > > > ... > > > > > > Rebuild with `-gcflags=all="-N -l"` and running it again result in the > same stack trace. > > > > > > > > > Looking at git blame for each files does not shown any new commit > introduced since after Go 1.14. Maybe others can look. > > > > > > The next thing I will do is bissecting and rebuild and report again. > > > > According to my bisection, the following commit cause it, > > > > > > 14:18 ~/src/go > > (af1f3b0082...)|BISECTING tokenomy 0 % git bisect good > > 98858c438016bbafd161b502a148558987aa44d5 is the first bad commit > > commit 98858c438016bbafd161b502a148558987aa44d5 > > Author: Ian Lance Taylor <i...@golang.org> > > Date: Tue Feb 25 20:23:15 2020 -0800 > > > > runtime: don't panic on racy use of timers > > > > If we see a racy use of timers, as in concurrent calls to > Timer.Reset, > > do the operations in an unpredictable order, rather than crashing. > > > > Fixes #37400 > > > > Change-Id: Idbac295df2dfd551b6d762909d5040fc532c1b34 > > Reviewed-on: https://go-review.googlesource.com/c/go/+/221077 > > Run-TryBot: Ian Lance Taylor <i...@golang.org> > > TryBot-Result: Gobot Gobot <go...@golang.org> > > Reviewed-by: Michael Knyszek <mknys...@google.com> > > > > src/runtime/time.go | 216 > ++++++++++++++++---------------------------------- > > src/time/time_test.go | 40 ++++++---- > > 2 files changed, 92 insertions(+), 164 deletions(-) > > > > > > Link to CL: https://go-review.googlesource.com/c/go/+/221077 > > > > If anyone have any idea the minimal test code to test it on my VM, I > will test it. > > That seems like a fairly unlikely culprit for significantly increased > CPU usage. Can you double check? > Thanks for the response Ian. I am actually run the bisect twice. I have also test by reverting that commit on top of latest tip and rebuild and redeploy, the result is the random CPU spike doesn't happened anymore. The weird thing is between all 5 services not all of them suddenly consume high CPU, sometimes only one service, sometimes two of them but not three, four, or five at the same time. I will give it one more bisect next Monday. > If you are sure that is the CL, please open a bug report at > https://golang.org/issue. Anything you can give us to recreate the > problem ourselves would be very helpful. Thanks. > Its quite hard for me to create reproducible program since its depends on many third party modules. I try to comprehend the bad CL but its beyond my knowledge. > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMh9%2BYA-7eoOeT94vDC%2BmJJOzcG%2BVTyTbQtKFXaMpfStyd5DQQ%40mail.gmail.com.
