> On 15 Mar 2020, at 07.02, Ian Lance Taylor <i...@golang.org> wrote:
>
> On Sat, Mar 14, 2020 at 1:56 AM Mhd Shulhan <m.shul...@gmail.com> wrote:
>>
>> Pada tanggal Sab, 14 Mar 2020 06.15, Ian Lance Taylor <i...@golang.org>
>> menulis:
>>>
>>> On Fri, Mar 13, 2020 at 1:05 AM Mhd Shulhan <m.shul...@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>> On 12 Mar 2020, at 13.13, Mhd Shulhan <m.shul...@gmail.com> wrote:
>>>>>
>>>>>
>>>>>> My question is any one have idea how to debug this so I can provide more
>>>>>> detailed report?
>>>>>>
>>>>>> Thank you in advance.
>>>>>
>>>>> So, I try to debug with gdb 9.1. Here is the sample of stack when the
>>>>> CPU got high,
>>>>>
>>>>> ...
>>>>> [New LWP 1885]
>>>>> [New LWP 1886]
>>>>> [New LWP 1887]
>>>>> [New LWP 1888]
>>>>> [New LWP 1889]
>>>>> ^C
>>>>> Thread 1 "XYZ" received signal SIGINT, Interrupt.
>>>>> runtime.futex () at /Users/XXX/share/go/src/runtime/sys_linux_amd64.s:568
>>>>> 568 /Users/X/share/go/src/runtime/sys_linux_amd64.s: No such file or
>>>>> directory.
>>>>> (gdb) where
>>>>> #0 runtime.futex () at
>>>>> /Users/XXX/share/go/src/runtime/sys_linux_amd64.s:568
>>>>> #1 0x0000000000431666 in runtime.futexsleep (addr=0xd6a3e8
>>>>> <runtime.m0+328>, val=0, ns=-1) at
>>>>> /Users/XXX/share/go/src/runtime/os_linux.go:44
>>>>> #2 0x000000000040bd9f in runtime.notesleep (n=0xd6a3e8 <runtime.m0+328>)
>>>>> at /Users/XXX/share/go/src/runtime/lock_futex.go:151
>>>>> #3 0x000000000043b858 in runtime.stoplockedm () at
>>>>> /Users/XXX/share/go/src/runtime/proc.go:1972
>>>>> #4 0x000000000043d4c6 in runtime.schedule () at
>>>>> /Users/XXX/share/go/src/runtime/proc.go:2455
>>>>> #5 0x000000000043d89d in runtime.park_m (gp=0xc000166c00) at
>>>>> /Users/XXX/share/go/src/runtime/proc.go:2691
>>>>> #6 0x00000000004651fb in runtime.mcall () at
>>>>> /Users/XXX/share/go/src/runtime/asm_amd64.s:318
>>>>> #7 0x0000000000465114 in runtime.rt0_go () at
>>>>> /Users/XXX/share/go/src/runtime/asm_amd64.s:220
>>>>> #8 0x0000000000000000 in ?? ()
>>>>> ...
>>>>>
>>>>> Rebuild with `-gcflags=all="-N -l"` and running it again result in the
>>>>> same stack trace.
>>>>>
>>>>>
>>>>> Looking at git blame for each files does not shown any new commit
>>>>> introduced since after Go 1.14. Maybe others can look.
>>>>>
>>>>> The next thing I will do is bissecting and rebuild and report again.
>>>>
>>>> According to my bisection, the following commit cause it,
>>>>
>>>>
>>>> 14:18 ~/src/go
>>>> (af1f3b0082...)|BISECTING tokenomy 0 % git bisect good
>>>> 98858c438016bbafd161b502a148558987aa44d5 is the first bad commit
>>>> commit 98858c438016bbafd161b502a148558987aa44d5
>>>> Author: Ian Lance Taylor <i...@golang.org>
>>>> Date: Tue Feb 25 20:23:15 2020 -0800
>>>>
>>>> runtime: don't panic on racy use of timers
>>>>
>>>> If we see a racy use of timers, as in concurrent calls to Timer.Reset,
>>>> do the operations in an unpredictable order, rather than crashing.
>>>>
>>>> Fixes #37400
>>>>
>>>> Change-Id: Idbac295df2dfd551b6d762909d5040fc532c1b34
>>>> Reviewed-on: https://go-review.googlesource.com/c/go/+/221077
>>>> Run-TryBot: Ian Lance Taylor <i...@golang.org>
>>>> TryBot-Result: Gobot Gobot <go...@golang.org>
>>>> Reviewed-by: Michael Knyszek <mknys...@google.com>
>>>>
>>>> src/runtime/time.go | 216
>>>> ++++++++++++++++----------------------------------
>>>> src/time/time_test.go | 40 ++++++----
>>>> 2 files changed, 92 insertions(+), 164 deletions(-)
>>>>
>>>>
>>>> Link to CL: https://go-review.googlesource.com/c/go/+/221077
>>>>
>>>> If anyone have any idea the minimal test code to test it on my VM, I will
>>>> test it.
>>>
>>> That seems like a fairly unlikely culprit for significantly increased
>>> CPU usage. Can you double check?
>>
>>
>> Thanks for the response Ian.
>>
>> I am actually run the bisect twice.
>>
>> I have also test by reverting that commit on top of latest tip and rebuild
>> and redeploy, the result is the random CPU spike doesn't happened anymore.
>>
>> The weird thing is between all 5 services not all of them suddenly consume
>> high CPU, sometimes only one service, sometimes two of them but not three,
>> four, or five at the same time.
>>
>> I will give it one more bisect next Monday.
>>
>>>
>>> If you are sure that is the CL, please open a bug report at
>>> https://golang.org/issue. Anything you can give us to recreate the
>>> problem ourselves would be very helpful. Thanks.
Someone has created the issue [1] before I finish the bisection and looking for
more information. I add some information in the issue, I hope that help.
[1] https://github.com/golang/go/issues/38023
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/4F10C9ED-1BA4-493C-9739-194C4C377ABE%40gmail.com.
