Satheesh Rajendran - please try the kernel intended for release (Ubuntu-4.10.0-17.19) that is currently staged in -proposed. There have been multiple stable updates applied since 4.10.0-14. I'd also like to know if this is reproducible given that I've seen several bugs complaining of the swapops BUGON, though so far all of those bugs seem to have been related to the use of Firefox.
-- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1680390 Title: Ubuntu17.04: Kernel Oops: Exception in kernel mode, sig: 5 [#1] during Avocado KVM Test runs [Regression] Status in linux package in Ubuntu: New Bug description: == Comment: #0 - Satheesh Rajendran <sathe...@in.ibm.com> - 2017-03-27 12:30:45 == ---Problem Description--- Kernel hit with oops while running avocado(kvm) tests "Oops: Exception in kernel mode, sig: 5 [#1]" Contact Information = sathe...@in.ibm.com ---uname output--- Linux ltc-test-ci1 4.10.0-14-generic #16-Ubuntu SMP Fri Mar 17 15:19:05 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux Machine Type = power 8 ppc64le ---Debugger--- A debugger is not configured ---Steps to Reproduce--- 1. Run Avocado(kvm) tests #git clone git://git.linux.ibm.com/ltc-test/avocado-fvt-wrapper.git;cd avocado-fvt-wrapper #python avocado-setup.py --bootstrap --run-suite guest_cpu --guest-os Ubuntu.16.04.2.ppc64le --only-filter virtio_scsi virtio_net qcow2 2. After sometime the below mentioned traces were seen. Stack trace output: [20751.909458] ------------[ cut here ]------------ [20751.909461] kernel BUG at /build/linux-HLNhAK/linux-4.10.0/include/linux/swapops.h:129! [20751.909542] Oops: Exception in kernel mode, sig: 5 [#1] [20751.909549] SMP NR_CPUS=2048 [20751.909549] NUMA [20751.909555] PowerNV [20751.909583] Modules linked in: vhost_net macvtap macvlan xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 kvm_hv kvm_pr kvm tcm_fc libfc usb_f_tcm tcm_usb_gadget libcomposite udc_core tcm_qla2xxx qla2xxx scsi_transport_fc ib_srpt iscsi_target_mod tcm_loop vhost_scsi vhost target_core_user target_core_file target_core_iblock target_core_pscsi target_core_mod ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6t_REJECT nf_reject_ipv6 xt_conntrack ip6t_rpfilter ip_set nfnetlink ebtable_broute bridge stp llc ebtable_nat ip6table_security ip6table_mangle ip6table_raw ip6table_nat iptable_security iptable_mangle iptable_raw iptable_nat ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack [20751.910225] binfmt_misc powernv_rng powernv_op_panel ipmi_powernv ipmi_devintf ipmi_msghandler uio_pdrv_genirq uio leds_powernv vmx_crypto ib_iser rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace configfs iscsi_tcp libiscsi_tcp sunrpc libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear ses enclosure scsi_transport_sas crc32c_vpmsum tg3 ipr [20751.910629] CPU: 24 PID: 6926 Comm: CPU 24/KVM Not tainted 4.10.0-14-generic #16-Ubuntu [20751.910700] task: c0000007b29ac000 task.stack: c0000007f07b8000 [20751.910759] NIP: c00000000030d748 LR: c00000000030d658 CTR: 0000000000000000 [20751.910828] REGS: c0000007f07bb3b0 TRAP: 0700 Not tainted (4.10.0-14-generic) [20751.910897] MSR: 9000000000029033 <SF,HV,EE,ME,IR,DR,RI,LE> [20751.910903] CR: 44882882 XER: 00000000 [20751.910984] CFAR: c00000000030d884 SOFTE: 1 GPR00: c00000000030d658 c0000007f07bb630 c00000000144c900 f000000001f9f1f0 GPR04: c0000007e7c7e0e0 f000000001f9f1f0 000000001f001c61 00000000611c001f GPR08: c0000000015bc900 0000000000000001 0000000000000001 0000000000e0c7e7 GPR12: 0000000000002200 c00000000fb8d800 0000000000000000 000000000007fe1d GPR16: 0000000000010000 00003fff121c0000 0000000000000000 0000000088000000 GPR20: 0000000020000000 0000000088000000 0000000022000000 c000000004990000 GPR24: c0000000015be3d8 c0000007f05d3700 c0000007b972c030 0000000000000000 GPR28: c0000007f07bb710 3e000000000d611c f000000003584700 f000000001f9f1f0 [20751.911580] NIP [c00000000030d748] __migration_entry_wait+0x128/0x2a0 [20751.911639] LR [c00000000030d658] __migration_entry_wait+0x38/0x2a0 [20751.911697] Call Trace: [20751.911722] [c0000007f07bb630] [c00000000030d658] __migration_entry_wait+0x38/0x2a0 (unreliable) [20751.911807] [c0000007f07bb670] [c0000000002bbc6c] do_swap_page+0x73c/0x9a0 [20751.911866] [c0000007f07bb6f0] [c0000000002bfa98] handle_mm_fault+0xac8/0x1600 [20751.911937] [c0000007f07bb7e0] [c0000000002b4104] __get_user_pages+0x194/0x4e0 [20751.912008] [c0000007f07bb890] [c0000000002b47e4] get_user_pages_unlocked+0xf4/0x280 [20751.912079] [c0000007f07bb930] [c0000000002b59ac] get_user_pages_fast+0xac/0x100 [20751.912152] [c0000007f07bb980] [d00000000f66ca74] kvmppc_book3s_hv_page_fault+0x2bc/0xbb0 [kvm_hv] [20751.912236] [c0000007f07bba70] [d00000000f6696f8] kvmppc_vcpu_run_hv+0xe60/0x1220 [kvm_hv] [20751.912312] [c0000007f07bbb80] [d00000000f6131ac] kvmppc_vcpu_run+0x34/0x48 [kvm] [20751.912387] [c0000007f07bbba0] [d00000000f61030c] kvm_arch_vcpu_ioctl_run+0x64/0x170 [kvm] [20751.912462] [c0000007f07bbbe0] [d00000000f603db8] kvm_vcpu_ioctl+0x500/0x780 [kvm] [20751.912534] [c0000007f07bbd40] [c00000000035b1f4] do_vfs_ioctl+0xd4/0x8c0 [20751.912594] [c0000007f07bbde0] [c00000000035bab4] SyS_ioctl+0xd4/0xf0 [20751.912654] [c0000007f07bbe30] [c00000000000b184] system_call+0x38/0xe0 [20751.912713] Instruction dump: [20751.912749] 3d020017 79293448 39481a68 ebca0000 7fde4a14 e93e0020 712a0001 4082014c [20751.912822] 7fc9f378 e9290000 7d2948f8 792907e0 <0b090000> 39400000 3bbe001c 39000001 [20751.912899] ---[ end trace 5eaae2f83c5daa20 ]--- System Dump Info: The system is not configured to capture a system dump. == Comment: #3 - IRANNA D. ANKAD <iranna.an...@in.ibm.com> - 2017-03-28 01:50:15 == This is a regression from 4.10.0-13 kernel and blocking our regression tetsing == Comment: #10 - VIPIN K. PARASHAR <vipar...@in.ibm.com> - 2017-03-30 03:44:13 == root@ltc-test-ci1:~# uname -a Linux ltc-test-ci1 4.10.0-15-generic #17-Ubuntu SMP Fri Mar 24 17:50:37 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux root@ltc-test-ci1:~# cat /etc/os-release NAME="Ubuntu" VERSION="17.04 (Zesty Zapus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu Zesty Zapus (development branch)" VERSION_ID="17.04" HOME_URL="https://www.ubuntu.com/"; SUPPORT_URL="https://help.ubuntu.com/"; BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"; PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"; VERSION_CODENAME=zesty UBUNTU_CODENAME=zesty root@ltc-test-ci1:~# tail /proc/cpuinfo processor : 79 cpu : POWER8E (raw), altivec supported clock : 2061.000000MHz revision : 2.1 (pvr 004b 0201) timebase : 512000000 platform : PowerNV model : 8247-21L machine : PowerNV 8247-21L firmware : OPAL root@ltc-test-ci1:~# == Comment: #11 - VIPIN K. PARASHAR <vipar...@in.ibm.com> - 2017-03-30 03:58:04 == Mar 24 07:15:57 =========== [ 1955.041619] ------------[ cut here ]------------ [ 1955.041623] kernel BUG at /build/linux-HLNhAK/linux-4.10.0/include/linux/swapops.h:129! [ 1955.041633] Oops: Exception in kernel mode, sig: 5 [#1] [ 1955.041637] SMP NR_CPUS=2048 [ 1955.041638] NUMA [ 1955.041641] PowerNV [ 1955.041645] Modules linked in: vhost_net macvtap macvlan rpcsec_gss_krb5 nfsv4 nfs fscache xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 kvm_hv kvm_pr kvm tcm_fc libfc usb_f_tcm tcm_usb_gadget libcomposite udc_core tcm_qla2xxx qla2xxx scsi_transport_fc ib_srpt iscsi_target_mod tcm_loop vhost_scsi vhost target_core_user target_core_file target_core_iblock target_core_pscsi target_core_mod ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_tcpudp ipt_REJECT nf_reject_ipv4 xt_conntrack ip_set nfnetlink ebtable_broute bridge stp llc ebtable_nat ip6table_raw ip6table_security ip6table_mangle ip6table_nat iptable_raw iptable_security iptable_mangle iptable_nat ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 [ 1955.041724] nf_defrag_ipv6 nf_nat nf_conntrack binfmt_misc powernv_rng powernv_op_panel ipmi_powernv ipmi_devintf ipmi_msghandler leds_powernv uio_pdrv_genirq uio vmx_crypto nfsd auth_rpcgss nfs_acl lockd grace sunrpc ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear ses enclosure scsi_transport_sas crc32c_vpmsum tg3 ipr [ 1955.041786] CPU: 40 PID: 7306 Comm: CPU 13/KVM Not tainted 4.10.0-14-generic #16-Ubuntu [ 1955.041792] task: c0000007937b5a00 task.stack: c0000007f2a48000 [ 1955.041796] NIP: c00000000030d748 LR: c00000000030d658 CTR: 0000000000000000 [ 1955.041801] REGS: c0000007f2a4b3b0 TRAP: 0700 Not tainted (4.10.0-14-generic) [ 1955.041805] MSR: 900000000282b033 <SF,HV,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> [ 1955.041813] CR: 44882882 XER: 00000000 [ 1955.041819] CFAR: c00000000030d884 SOFTE: 1 [ 1955.041819] GPR00: c00000000030d658 c0000007f2a4b630 c00000000144c900 f000000001fb8b30 [ 1955.041819] GPR04: c0000007ee2cd2c8 f000000001fb8b30 000000001f005977 000000007759001f [ 1955.041819] GPR08: c0000000015bc900 0000000000000001 0000000000000001 0000000000d02cee [ 1955.041819] GPR12: 0000000000002200 c00000000fb96800 0000000000000000 000000000000005a [ 1955.041819] GPR16: 0000000000010000 00003ffe30590000 0000000000000000 0000000088000000 [ 1955.041819] GPR20: 0000000020000000 0000000088000000 0000000022000000 c0000000fd3d0000 [ 1955.041819] GPR24: c0000000015be3d8 c0000007c3c66880 c0000007ed9b40d8 0000000000000000 [ 1955.041819] GPR28: c0000007f2a4b710 3e00000000077759 f000000001ddd640 f000000001fb8b30 [ 1955.041874] NIP [c00000000030d748] __migration_entry_wait+0x128/0x2a0 [ 1955.041879] LR [c00000000030d658] __migration_entry_wait+0x38/0x2a0 [ 1955.041883] Call Trace: [ 1955.041886] [c0000007f2a4b630] [c00000000030d658] __migration_entry_wait+0x38/0x2a0 (unreliable) [ 1955.041894] [c0000007f2a4b670] [c0000000002bbc6c] do_swap_page+0x73c/0x9a0 [ 1955.041900] [c0000007f2a4b6f0] [c0000000002bfa98] handle_mm_fault+0xac8/0x1600 [ 1955.041906] [c0000007f2a4b7e0] [c0000000002b4104] __get_user_pages+0x194/0x4e0 [ 1955.041912] [c0000007f2a4b890] [c0000000002b47e4] get_user_pages_unlocked+0xf4/0x280 [ 1955.041918] [c0000007f2a4b930] [c0000000002b59ac] get_user_pages_fast+0xac/0x100 [ 1955.041927] [c0000007f2a4b980] [d00000000f7aca74] kvmppc_book3s_hv_page_fault+0x2bc/0xbb0 [kvm_hv] [ 1955.041935] [c0000007f2a4ba70] [d00000000f7a96f8] kvmppc_vcpu_run_hv+0xe60/0x1220 [kvm_hv] [ 1955.041947] [c0000007f2a4bb80] [d00000000f7531ac] kvmppc_vcpu_run+0x34/0x48 [kvm] [ 1955.041958] [c0000007f2a4bba0] [d00000000f75030c] kvm_arch_vcpu_ioctl_run+0x64/0x170 [kvm] [ 1955.041967] [c0000007f2a4bbe0] [d00000000f743db8] kvm_vcpu_ioctl+0x500/0x780 [kvm] [ 1955.041974] [c0000007f2a4bd40] [c00000000035b1f4] do_vfs_ioctl+0xd4/0x8c0 [ 1955.041980] [c0000007f2a4bde0] [c00000000035bab4] SyS_ioctl+0xd4/0xf0 [ 1955.041986] [c0000007f2a4be30] [c00000000000b184] system_call+0x38/0xe0 [ 1955.041990] Instruction dump: [ 1955.041994] 3d020017 79293448 39481a68 ebca0000 7fde4a14 e93e0020 712a0001 4082014c [ 1955.042003] 7fc9f378 e9290000 7d2948f8 792907e0 <0b090000> 39400000 3bbe001c 39000001 [ 1955.042016] ---[ end trace 1c0e9a056f95491f ]--- Mar 27 11:54:27 ============ [20751.909458] ------------[ cut here ]------------ [20751.909461] kernel BUG at /build/linux-HLNhAK/linux-4.10.0/include/linux/swapops.h:129! [20751.909542] Oops: Exception in kernel mode, sig: 5 [#1] [20751.909549] SMP NR_CPUS=2048 [20751.909549] NUMA [20751.909555] PowerNV [20751.909583] Modules linked in: vhost_net macvtap macvlan xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 kvm_hv kvm_pr kvm tcm_fc libfc usb_f_tcm tcm_usb_gadget libcomposite udc_core tcm_qla2xxx qla2xxx scsi_transport_fc ib_srpt iscsi_target_mod tcm_loop vhost_scsi vhost target_core_user target_core_file target_core_iblock target_core_pscsi target_core_mod ipt_REJECT nf_reject_ipv4 xt_tcpudp ip6t_REJECT nf_reject_ipv6 xt_conntrack ip6t_rpfilter ip_set nfnetlink ebtable_broute bridge stp llc ebtable_nat ip6table_security ip6table_mangle ip6table_raw ip6table_nat iptable_security iptable_mangle iptable_raw iptable_nat ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter openvswitch nf_conntrack_ipv6 nf_nat_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack [20751.910225] binfmt_misc powernv_rng powernv_op_panel ipmi_powernv ipmi_devintf ipmi_msghandler uio_pdrv_genirq uio leds_powernv vmx_crypto ib_iser rdma_cm iw_cm ib_cm ib_core nfsd auth_rpcgss nfs_acl lockd grace configfs iscsi_tcp libiscsi_tcp sunrpc libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear ses enclosure scsi_transport_sas crc32c_vpmsum tg3 ipr [20751.910629] CPU: 24 PID: 6926 Comm: CPU 24/KVM Not tainted 4.10.0-14-generic #16-Ubuntu [20751.910700] task: c0000007b29ac000 task.stack: c0000007f07b8000 [20751.910759] NIP: c00000000030d748 LR: c00000000030d658 CTR: 0000000000000000 [20751.910828] REGS: c0000007f07bb3b0 TRAP: 0700 Not tainted (4.10.0-14-generic) [20751.910897] MSR: 9000000000029033 <SF,HV,EE,ME,IR,DR,RI,LE> [20751.910903] CR: 44882882 XER: 00000000 [20751.910984] CFAR: c00000000030d884 SOFTE: 1 [20751.910984] GPR00: c00000000030d658 c0000007f07bb630 c00000000144c900 f000000001f9f1f0 [20751.910984] GPR04: c0000007e7c7e0e0 f000000001f9f1f0 000000001f001c61 00000000611c001f [20751.910984] GPR08: c0000000015bc900 0000000000000001 0000000000000001 0000000000e0c7e7 [20751.910984] GPR12: 0000000000002200 c00000000fb8d800 0000000000000000 000000000007fe1d [20751.910984] GPR16: 0000000000010000 00003fff121c0000 0000000000000000 0000000088000000 [20751.910984] GPR20: 0000000020000000 0000000088000000 0000000022000000 c000000004990000 [20751.910984] GPR24: c0000000015be3d8 c0000007f05d3700 c0000007b972c030 0000000000000000 [20751.910984] GPR28: c0000007f07bb710 3e000000000d611c f000000003584700 f000000001f9f1f0 [20751.911580] NIP [c00000000030d748] __migration_entry_wait+0x128/0x2a0 [20751.911639] LR [c00000000030d658] __migration_entry_wait+0x38/0x2a0 [20751.911697] Call Trace: [20751.911722] [c0000007f07bb630] [c00000000030d658] __migration_entry_wait+0x38/0x2a0 (unreliable) [20751.911807] [c0000007f07bb670] [c0000000002bbc6c] do_swap_page+0x73c/0x9a0 [20751.911866] [c0000007f07bb6f0] [c0000000002bfa98] handle_mm_fault+0xac8/0x1600 [20751.911937] [c0000007f07bb7e0] [c0000000002b4104] __get_user_pages+0x194/0x4e0 [20751.912008] [c0000007f07bb890] [c0000000002b47e4] get_user_pages_unlocked+0xf4/0x280 [20751.912079] [c0000007f07bb930] [c0000000002b59ac] get_user_pages_fast+0xac/0x100 [20751.912152] [c0000007f07bb980] [d00000000f66ca74] kvmppc_book3s_hv_page_fault+0x2bc/0xbb0 [kvm_hv] [20751.912236] [c0000007f07bba70] [d00000000f6696f8] kvmppc_vcpu_run_hv+0xe60/0x1220 [kvm_hv] [20751.912312] [c0000007f07bbb80] [d00000000f6131ac] kvmppc_vcpu_run+0x34/0x48 [kvm] [20751.912387] [c0000007f07bbba0] [d00000000f61030c] kvm_arch_vcpu_ioctl_run+0x64/0x170 [kvm] [20751.912462] [c0000007f07bbbe0] [d00000000f603db8] kvm_vcpu_ioctl+0x500/0x780 [kvm] [20751.912534] [c0000007f07bbd40] [c00000000035b1f4] do_vfs_ioctl+0xd4/0x8c0 [20751.912594] [c0000007f07bbde0] [c00000000035bab4] SyS_ioctl+0xd4/0xf0 [20751.912654] [c0000007f07bbe30] [c00000000000b184] system_call+0x38/0xe0 [20751.912713] Instruction dump: [20751.912749] 3d020017 79293448 39481a68 ebca0000 7fde4a14 e93e0020 712a0001 4082014c [20751.912822] 7fc9f378 e9290000 7d2948f8 792907e0 <0b090000> 39400000 3bbe001c 39000001 [20751.912899] ---[ end trace 5eaae2f83c5daa20 ]--- As pasted above, two instances of Oops are seen in kernel logs. == Comment: #15 - VIPIN K. PARASHAR <vipar...@in.ibm.com> - 2017-03-30 11:34:06 == From Linux source ============ /* * Something used the pte of a page under migration. We need to * get to the page and wait until migration is finished. * When we return from this function the fault will be retried. */ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, spinlock_t *ptl) { .. .. page = migration_entry_to_page(entry); static inline struct page *migration_entry_to_page(swp_entry_t entry) { struct page *p = pfn_to_page(swp_offset(entry)); /* * Any use of migration entries may only occur while the * corresponding page is locked */ BUG_ON(!PageLocked(p)); <------ Oops here return p; } Kernel Oops is getting invoked, due to hitting a BUG_ON in kernel, while servicing KVM ioctl and subsequent page fault with pages being migrated. Same issue has been noticed on intel as well https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1677057 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1680390/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
