On 06/06/2012 09:43 PM, Jim Mauro wrote:
>
> I can't help but be curious about something, which perhaps you verified but
> did not post.
>
> What the data here shows is;
> - CPU 31 is buried in the kernel (100% sys).
> - CPU 31 is handling a moderate-to-high rate of xcalls.
>
> What the data does not prove empirically is that the 100% sys time of
> CPU 31 is in xcall handling.
>
> What's the hot stack when this occurs and you run this;
>
> dtrace -n 'profile-997hz /cpu == 31/ { @[stack()] = count(); }'
> Thanks for pointing this out. I ran the probe you specified and attached are the results (I had to chase the xcalls around a bit, because they were jumping around cores as I was trying to insert the probes). As I suspected, the most numerous stack trace is the one which causes cross calls because of the segkmem_zio_free+0x27 code path. While this was going on, I was getting between 80k and 300k xcalls on the core in question. The next most common stack was the one ending in mach_cpu_idle and then, so I'm not sure why the CPU reported 100% busy (perhaps the xcalls were very expensive on CPU time compared with the 1273 idle's). Cheers, -- Saso
xc_call.txt.bz2
Description: application/bzip
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
