On Tue, 15 Apr 2014, Thomas Gleixner wrote: > On Tue, 15 Apr 2014, Vince Weaver wrote: > > > > Still tracking memory corruption bugs found by the perf_fuzzer, I have > > about 10 different log splats that I think might all be related to the > > same underlying problem. > > > > Anyway I managed to trigger this using the perf_fuzzer: > > > > [ 221.065278] Slab corruption (Not tainted): kmalloc-2048 > > start=ffff8800cd15e800, len=2048 > > [ 221.074062] 040: 6b 6b 6b 6b 6b 6b 6b 6b 98 72 57 cd 00 88 ff ff > > kkkkkkkk.rW..... > > [ 221.082321] Prev obj: start=ffff8800cd15e000, len=2048 > > [ 221.087933] 000: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b > > kkkkkkkkkkkkkkkk > > [ 221.096224] 010: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b > > kkkkkkkkkkkkkkkk > > > > And luckily I had ftrace running at the time. > > > > The allocation of this block is by perf_event > > > > perf_fuzzer-2520 [001] 182.980563: kmalloc: > > (perf_event_alloc+0x55) call_site=ffffffff811399b5 ptr=0xffff8800cd15e800 > > bytes_req=1272 bytes_alloc=2048 gfp_flags=GFP_KERNEL|GFP_ZERO > > perf_fuzzer-2520 [000] 183.628515: kmalloc: > > (perf_event_alloc+0x55) call_site=ffffffff811399b5 ptr=0xffff8800cd15e800 > > bytes_req=1272 bytes_alloc=2048 gfp_flags=GFP_KERNEL|GFP_ZERO > > perf_fuzzer-2520 [000] 183.628521: kfree: > > (perf_event_alloc+0x2f7) call_site=ffffffff81139c57 ptr=0xffff8800cd15e800 > > perf_fuzzer-2520 [000] 183.628844: kmalloc: > > (perf_event_alloc+0x55) call_site=ffffffff811399b5 ptr=0xffff8800cd15e800 > > bytes_req=1272 bytes_alloc=2048 gfp_flags=GFP_KERNEL|GFP_ZERO > > ...(thousands of times of kmalloc/kfree) > > > > Is it worth wading through this mess to try to track down what happened? > > Definitely worth a try. Can you upload the trace file and provide the > URL or send it offlist in private mail if you cannot provide a public URL.
I've poked around the trace a bit. Possibly it looks like a struct perf_event is being used after freed, specifically the event->migrate_entry->prev value? I could be completely wrong about that. One thing to know about these fuzzer runs, the ones that cause memory corruption involve forking (with events active). I haven't seen the corruptions when forking is disabled. It's very simple forking, only one child is ever active at a time, and the child itself doesn't do anything but busy wait until it is killed. The trace shows the problem allocations happening before a fork and the poison message after. The traces I have don't include the children though so I don't have records of what happened there. I'll send a private link to the file downloads as they're a little large and the local sysadmins would probably appreicate if I limited access to them. Vince -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
