On Thu, May 01, 2014 at 10:27:45AM -0400, Vince Weaver wrote: > On Thu, 1 May 2014, Vince Weaver wrote: > > > On Wed, 30 Apr 2014, Peter Zijlstra wrote: > > > > > Vince, could you add the below to whatever tracing muck you already > > > have? > > and this might be what you're looking for. This is with a different > random seed than the one I've used for other traces, your patch changes > the syscall behavior enough that the one I was using before wasn't going > down the same path. > > This WARNING is > WARN_ON(event->hlist_entry.pprev && event->hlist_entry.pprev != LIST_POISON2); > > [ 1554.910867] ------------[ cut here ]------------ > [ 1554.919535] WARNING: CPU: 5 PID: 16431 at kernel/events/core.c:3232 > __free_event+0x86/0x90() > [ 1554.931534] Modules linked in: fuse snd_hda_codec_hdmi > x86_pkg_temp_thermal intel_powerclamp coretemp i915 kvm snd_hda_codec_realtek > snd_hda_codec_generic snd_hda_intel snd_hda_controller crct10dif_pclmul > drm_kms_helper snd_hda_codec crc32_pclmul ghash_clmulni_intel aesni_intel > snd_hwdep snd_pcm drm snd_seq aes_x86_64 lrw evdev parport_pc iTCO_wdt > gf128mul snd_timer tpm_tis snd_seq_device glue_helper i2c_i801 snd > iTCO_vendor_support ablk_helper cryptd parport psmouse pcspkr serio_raw > button processor video battery i2c_algo_bit i2c_core tpm wmi lpc_ich mfd_core > mei_me mei soundcore sg sd_mod sr_mod crc_t10dif crct10dif_common cdrom ahci > libahci ehci_pci xhci_hcd ehci_hcd libata e1000e scsi_mod ptp usbcore > crc32c_intel pps_core usb_common fan thermal thermal_sys > [ 1555.010748] CPU: 5 PID: 16431 Comm: perf_fuzzer Tainted: G W > 3.15.0-rc1+ #92 > [ 1555.020142] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS > 01/26/2014 > [ 1555.028924] 0000000000000009 ffff880117147b78 ffffffff81649790 > 0000000000000000 > [ 1555.037771] ffff880117147bb0 ffffffff810646ad ffff880116daf800 > 0000000000000000 > [ 1555.046611] ffff880036e45a10 ffff8800cde07588 ffff880116dafaa0 > ffff880117147bc0 > [ 1555.055490] Call Trace: > [ 1555.058963] [<ffffffff81649790>] dump_stack+0x45/0x56 > [ 1555.065357] [<ffffffff810646ad>] warn_slowpath_common+0x7d/0xa0 > [ 1555.072646] [<ffffffff8106478a>] warn_slowpath_null+0x1a/0x20 > [ 1555.079740] [<ffffffff81132e56>] __free_event+0x86/0x90 > [ 1555.086298] [<ffffffff811339c9>] _free_event+0xc9/0x200 > [ 1555.092859] [<ffffffff81133c78>] put_event+0x178/0x1f0 > [ 1555.099293] [<ffffffff81133b68>] ? put_event+0x68/0x1f0 > [ 1555.105853] [<ffffffff81133d12>] perf_release+0x12/0x20 > [ 1555.112422] [<ffffffff811b64ec>] __fput+0xdc/0x1e0 > [ 1555.118576] [<ffffffff811b663e>] ____fput+0xe/0x10 > [ 1555.124630] [<ffffffff81085137>] task_work_run+0xa7/0xe0 > [ 1555.131307] [<ffffffff81066d5c>] do_exit+0x2cc/0xa50 > [ 1555.137606] [<ffffffff81076949>] ? get_signal_to_deliver+0x249/0x650 > [ 1555.145356] [<ffffffff8106756c>] do_group_exit+0x4c/0xc0 > [ 1555.151970] [<ffffffff81076991>] get_signal_to_deliver+0x291/0x650 > [ 1555.159542] [<ffffffff81012438>] do_signal+0x48/0x990 > [ 1555.165869] [<ffffffff81655592>] ? do_page_fault+0x22/0x30 > [ 1555.172645] [<ffffffff81012df0>] do_notify_resume+0x70/0xa0 > [ 1555.179484] [<ffffffff81651abc>] retint_signal+0x48/0x8c > [ 1555.185991] ---[ end trace 41ec7a21bb260463 ]--- > [ 1556.112904] Slab corruption (Tainted: G W ): kmalloc-2048 > start=ffff880116daf800, len=2048 > [ 1556.126221] 040: 6b 6b 6b 6b 6b 6b 6b 6b a8 75 36 ca 00 88 ff ff > kkkkkkkk.u6..... > [ 1556.137243] Prev obj: start=ffff880116daf000, len=2048 > [ 1556.145566] 000: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b > kkkkkkkkkkkkkkkk > [ 1556.156292] 010: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b > kkkkkkkkkkkkkkkk > > I can try slapping ftrace on and getting a trace if you want. > > I've been using > trace-cmd record -e kmem -e raw_syscalls -p function -l '*perf*' -n > 'perf_event_task_tick' > > which is a compromise between log size and info, but as you've seen it > loses useful info, especially all the things in core.c that don't > include *perf*. >
/proc/sys/kernel/traceoff_on_warning Is also useful, it disables tracing the moment a warn/bug hits. But yes please! -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
