Default Comment by Bridge
** Attachment added: "Call traces."
https://bugs.launchpad.net/bugs/1706033/+attachment/4920322/+files/perf_hw_breakpoint_call_traces
** Changed in: ubuntu
Assignee: (unassigned) => Ubuntu on IBM Power Systems Bug Triage
(ubuntu-power-triage)
** Package changed: ubuntu => linux (Ubuntu)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706033
Title:
[P9, Power NV][ WSP][Ubuntu 16.04.03] : perf hw breakpoint command
results in call traces and system goes for reboot.
Status in The Ubuntu-power-systems project:
New
Status in linux package in Ubuntu:
New
Bug description:
== Comment: #0 - Shriya R. Kulkarni <> - 2017-06-14 04:38:16 ==
Problem Description :
=============
While running perftool - testsuite, the perf hw breakpoint fails and it
result in call traces , hence system goes for reboot .
Machine details :
==========
System : P9 , WSP , Bare metal.
OS : Ubuntu 16.04.03
uname -a : Linux ltc-wspoon3 4.10.0-23-generic #25~16.04.1-Ubuntu SMP Fri Jun
9 10:43:34 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux
Steps to reproduce:
============
1. Install perf.
2. git clone perftool-testsuite.
https://github.com/rfmvh/perftool-testsuite
3. Do make.
4. Test fails at step : -- [ FAIL ] -- perf_stat :: test_hw_breakpoints ::
kspace address execution mem:0xc00000000035c020:x (command exitcode + output
regexp parsing
and call trace is seen as system goes for reboot.
Call traces :
=======
ubuntu@ltc-wspoon3:~$ [1602513.518414] Unable to handle kernel paging request
for data at address 0xc00000000135d3b8
[1602513.518553] Faulting instruction address: 0xc0000000002869bc
[1602513.518694] Oops: Kernel access of bad area, sig: 11 [#1]
[1602513.518782] SMP NR_CPUS=2048
[1602513.518784] NUMA
[1602513.518842] PowerNV
[1602513.518922] Modules linked in: vmx_crypto ofpart ipmi_powernv
cmdlinepart ipmi_devintf powernv_flash ipmi_msghandler ibmpowernv opal_prd mtd
at24 nvmem_core uio_pdrv_genirq uio autofs4 ast i2c_algo_bit drm_kms_helper
syscopyarea sysfillrect sysimgblt fb_sys_fops crc32c_vpmsum ttm drm tg3 ahci
libahci
[1602513.519399] CPU: 27 PID: 4069 Comm: sysctl Not tainted 4.10.0-22-generic
#24
[1602513.519524] task: c000203968c42c00 task.stack: c000203965710000
[1602513.519624] NIP: c0000000002869bc LR: c0000000003f7348 CTR:
c000000000286990
[1602513.519747] REGS: c000203965713a40 TRAP: 0300 Not tainted
(4.10.0-22-generic)
[1602513.519876] MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE>
[1602513.519889] CR: 22002448 XER: 00000000
[1602513.520058] CFAR: c0000000003f7344 DAR: c00000000135d3b8 DSISR: 00400000
SOFTE: 1
[1602513.520058] GPR00: c0000000003f7348 c000203965713cc0 c00000000145d100
c00000000134af00
[1602513.520058] GPR04: 0000000000000000 000000004ee50300 c000203965713d20
c000203965713e00
[1602513.520058] GPR08: 0000000000000000 c00000000135d100 0000000000000000
c000000000b71020
[1602513.520058] GPR12: c000000000286990 c000000007b4f300 0000000000000000
0000000000000000
[1602513.520058] GPR16: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[1602513.520058] GPR20: 0000000000000000 0000000000000000 0000000000000000
0000000000000001
[1602513.520058] GPR24: 00003fffc542f5a0 0000000000000400 c000203965713e00
000000004ee50300
[1602513.520058] GPR28: c00000000134af00 0000000000000000 c000003fee038800
0000000000000000
[1602513.521280] NIP [c0000000002869bc] dirty_ratio_handler+0x2c/0x90
[1602513.521374] LR [c0000000003f7348] proc_sys_call_handler+0x138/0x1c0
[1602513.521481] Call Trace:
[1602513.521526] [c000203965713cc0] [c000203965713d00] 0xc000203965713d00
(unreliable)
[1602513.521655] [c000203965713d00] [c0000000003f7348]
proc_sys_call_handler+0x138/0x1c0
[1602513.521797] [c000203965713d70] [c0000000003436ec] __vfs_read+0x3c/0x70
[1602513.521907] [c000203965713d90] [c00000000034516c] vfs_read+0xbc/0x1b0
[1602513.522016] [c000203965713de0] [c000000000346dd8] SyS_read+0x68/0x110
[1602513.522112] [c000203965713e30] [c00000000000b184] system_call+0x38/0xe0
[1602513.522243] Instruction dump:
[1602513.522303] 60420000 3c4c011d 38426770 7c0802a6 60000000 7c0802a6
fbc1fff0 fbe1fff8
[1602513.522445] f8010010 f821ffc1 3d22fff0 7c9f2378 <ebc902ba> 4be66da9
60000000 3d22fff0
[1602513.522564] ---[ end trace 17c76e13e641d3c6 ]---
[1602513.522657]
It goes for reboot :
After booting to Ubuntu , I see series of call traces.
Ubuntu 16.04.2 LTS ltc-wspoon3 hvc0
ltc-wspoon3 login: [ 3476.626263] Unable to handle kernel paging request for
data at address 0xc0000000013ad438
[ 3476.626422] Faulting instruction address: 0xc00000000029a140
[ 3476.626537] Oops: Kernel access of bad area, sig: 11 [#1]
[ 3476.626615] SMP NR_CPUS=2048
[ 3476.626616] NUMA
[ 3476.626673] PowerNV
[ 3476.626746] Modules linked in: ipmi_powernv at24 ipmi_devintf nvmem_core
ipmi_msghandler ofpart cmdlinepart powernv_flash mtd opal_prd vmx_crypto
ibmpowernv uio_pdrv_genirq uio autofs4 ast i2c_algo_bit ttm crc32c_vpmsum
drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops tg3 drm ahci
libahci
[ 3476.627220] CPU: 28 PID: 4529 Comm: sysctl Not tainted 4.10.0-23-generic
#25~16.04.1-Ubuntu
[ 3476.627339] task: c000203968ceec00 task.stack: c000203968d10000
[ 3476.627428] NIP: c00000000029a140 LR: c0000000004133a8 CTR:
c00000000029a110
[ 3476.627554] REGS: c000203968d13a50 TRAP: 0300 Not tainted
(4.10.0-23-generic)
[ 3476.627675] MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE>
[ 3476.627689] CR: 22002448 XER: 00000000
[ 3476.627844] CFAR: c0000000004133a4 DAR: c0000000013ad438 DSISR: 00400000
SOFTE: 1
[ 3476.627844] GPR00: c0000000004133a8 c000203968d13cd0 c0000000014ad100
c00000000139af78
[ 3476.627844] GPR04: 0000000000000000 000000003b440300 c000203968d13d30
c000203968d13e00
[ 3476.627844] GPR08: 0000000000000000 c0000000013ad100 0000000000000000
c000000000bc10a8
[ 3476.627844] GPR12: c00000000029a110 c000000007b4fc00 0000000000000000
0000000000000000
[ 3476.627844] GPR16: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
[ 3476.627844] GPR20: 0000000000000000 0000000000000000 0000000000000000
0000000000000001
[ 3476.627844] GPR24: 00003fffd410aa70 0000000000000400 c000203968d13e00
000000003b440300
[ 3476.627844] GPR28: c00000000139af78 0000000000000000 c000003fee038800
0000000000000000
[ 3476.629067] NIP [c00000000029a140] dirty_ratio_handler+0x30/0x90
[ 3476.629177] LR [c0000000004133a8] proc_sys_call_handler+0x138/0x170
[ 3476.629283] Call Trace:
[ 3476.629330] [c000203968d13cd0] [c000203968d13d10] 0xc000203968d13d10
(unreliable)
[ 3476.629462] [c000203968d13d10] [c0000000004133a8]
proc_sys_call_handler+0x138/0x170
[ 3476.629600] [c000203968d13d80] [c00000000035a4f0] __vfs_read+0x40/0x80
[ 3476.629711] [c000203968d13da0] [c00000000035c0d8] vfs_read+0xb8/0x1a0
[ 3476.629823] [c000203968d13de0] [c00000000035ddec] SyS_read+0x6c/0x110
[ 3476.629938] [c000203968d13e30] [c00000000000b184] system_call+0x38/0xe0
[ 3476.630050] Instruction dump:
[ 3476.630110] 3c4c0121 38422ff0 7c0802a6 f8010010 60000000 7c0802a6 fbc1fff0
fbe1fff8
[ 3476.630250] f8010010 f821ffc1 3d22fff0 7c9f2378 <ebc9033a> 4be5bbc5
60000000 3d22fff0
[ 3476.630396] ---[ end trace 10b22aebb5b2bf8d ]---
[ 3477.238492]
[ 3477.238534] Sending IPI to other CPUs
[ 3477.239615] IPI complete
[ 3477.240827] kexec: waiting for cpu 5 (physical 49) to ente
Attaching call traces in logs.
== Comment: #4 - Shriya R. Kulkarni <> - 2017-07-10 13:16:10 ==
The issue is fixed with upstream kernel.
Here is the testing done on upstream kernel .
Testing :
======
1. root@ltc-boston27:~/linux-next-next-20170710/tools/perf# cat
/proc/kallsyms | grep -P vm_dirty_ratio
c0000000014591e0 D vm_dirty_ratio
./perf stat -e mem:0xc0000000014591e0:rw -x';' -- sysctl vm.dirty_ratio >
/dev/null
3;;mem:0xc0000000014591e0:rw;1126624;100.00;;;;
2. root@ltc-boston27:~/linux-next-next-20170710/tools/perf# cat
/proc/kallsyms | grep pid_max
c000000001413bfc D pid_max
c000000001413c00 D pid_max_max
c000000001413c04 D pid_max_min
root@ltc-boston27:~/linux-next-next-20170710/tools/perf# ./perf record -a -e
mem:0xc000000001413bfc -g
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.161 MB perf.data (6 samples) ]
root@ltc-boston27:~/linux-next-next-20170710/tools/perf# dmesg -c
root@ltc-boston27:~/linux-next-next-20170710/tools/perf#
== Comment: #6 - Shriya R. Kulkarni <> - 2017-07-21 01:59:26 ==
This patch fixes the issue as follows ::
Patch :
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=d89ba5353f301971dd7d2f9fdf25c4432728f38e
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1706033/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
