Re: [perf-discuss] -lumem slower than -lmalloc and -lmtmalloc?

2009-05-12 Thread Bob Friesenhahn
On Tue, 12 May 2009, johan...@sun.com wrote: Ok. Was there a specific set of options that you thought might shed some additional light on this problem? So far, we seem to think that it's alignment related, but haven't yet narrowed the problem down any further. Besides alignment, there is the

Re: [perf-discuss] -lumem slower than -lmalloc and -lmtmalloc?

2009-05-12 Thread johansen
On Wed, May 13, 2009 at 03:28:11AM +0200, Roland Mainz wrote: > Bob Friesenhahn wrote: > > On Tue, 12 May 2009, Roland Mainz wrote: > > > Can you check whether the memory allocator in libast performs better in > > > this case (e.g. compile with $ cc -I/usr/include/ast/ -last ... # (note: > > > liba

Re: [perf-discuss] -lumem slower than -lmalloc and -lmtmalloc?

2009-05-12 Thread johansen
On Wed, May 13, 2009 at 03:28:19AM +0200, Roland Mainz wrote: > johan...@sun.com wrote: > > On Tue, May 12, 2009 at 01:33:15AM +0200, Roland Mainz wrote: > > > Can you check whether the memory allocator in libast performs better in > > > this case (e.g. compile with $ cc -I/usr/include/ast/ -last .

Re: [perf-discuss] -lumem slower than -lmalloc and -lmtmalloc?

2009-05-12 Thread Roland Mainz
johan...@sun.com wrote: > On Tue, May 12, 2009 at 01:33:15AM +0200, Roland Mainz wrote: > > Can you check whether the memory allocator in libast performs better in > > this case (e.g. compile with $ cc -I/usr/include/ast/ -last ... # (note: > > libast uses a |_ast_|-prefix for all symbols and does

Re: [perf-discuss] -lumem slower than -lmalloc and -lmtmalloc?

2009-05-12 Thread Roland Mainz
Bob Friesenhahn wrote: > On Tue, 12 May 2009, Roland Mainz wrote: > > Can you check whether the memory allocator in libast performs better in > > this case (e.g. compile with $ cc -I/usr/include/ast/ -last ... # (note: > > libast uses a |_ast_|-prefix for all symbols and does (currently) not > > ac

Re: [perf-discuss] Swap usage and reporting

2009-05-12 Thread johansen
Hi Ethan, On Tue, May 12, 2009 at 03:25:52PM -0700, Ethan Erchinger wrote: > Results, everything looks reasonable to me. > http://pastebin.com/m785c155e Thanks for running this. MySQL is using 44g of memory, which is pretty close to the 48gb limit for the machine. I'm sort of curious about how

Re: [perf-discuss] Swap usage and reporting

2009-05-12 Thread Ethan Erchinger
> >> If pmap -x isn't working for you, there might be another option. >> There's a -S option to pmap that shows the swap allocations. It may not >> provide as much detail as -x, but it should give you a good idea of how >> much swap each process is using. > > We noticed the same problem with -S as

Re: [perf-discuss] Swap usage and reporting

2009-05-12 Thread Ethan Erchinger
> I recently tried to debug a similar idiopathic problem with a colleague. > We weren't able to figure out what was causing the problem, partially > because the system took about a week to get into the state where a lot > of swap was in use. Once it was there, there were only a few obvious > signs

Re: [perf-discuss] Swap usage and reporting

2009-05-12 Thread johansen
On Tue, May 12, 2009 at 09:41:41AM -0700, Ethan Erchinger wrote: > Hi all, > > I'm having trouble determining what is using a large amount of swap on a > few of our OpenSolaris systems. These systems run MySQL, the 5.0.65 > version that came with snv_101, have 48G of ram, and 24G of swap. The >

Re: [perf-discuss] Swap usage and reporting

2009-05-12 Thread Ethan Erchinger
> >> As you can see, even though the resident size of MySQL is < 40G, the >> system is still using close to 16G of swap. At first we thought that >> ZFS arc cache was causing this to happen, but we've limited it to 2G, > Perhaps this is due to reserved anonymous memory mappings which are > not r

Re: [perf-discuss] Swap usage and reporting

2009-05-12 Thread Bob Friesenhahn
On Tue, 12 May 2009, Ethan Erchinger wrote: As you can see, even though the resident size of MySQL is < 40G, the system is still using close to 16G of swap. At first we thought that ZFS arc cache was causing this to happen, but we've limited it to 2G, Perhaps this is due to reserved anonymous

[perf-discuss] Swap usage and reporting

2009-05-12 Thread Ethan Erchinger
Hi all, I'm having trouble determining what is using a large amount of swap on a few of our OpenSolaris systems. These systems run MySQL, the 5.0.65 version that came with snv_101, have 48G of ram, and 24G of swap. The MySQL instances are configured to use a 36G innodb buffer pool. With the oth

Re: [perf-discuss] real meaning of ithr from mpstat

2009-05-12 Thread Chad Mynhier
On Tue, May 12, 2009 at 9:07 AM, Qihua Wu wrote: > From the man page, > ithr is "interrupts as threads", but what's the detailed meaning of > "interrupts as threads"? And under what condition will this interrupt > happen? The ithr column shows the number of interrupts that are converted to real t

[perf-discuss] real meaning of ithr from mpstat

2009-05-12 Thread Qihua Wu
>From the man page, ithr is "interrupts as threads", but what's the detailed meaning of "interrupts as threads"? And under what condition will this interrupt happen? On our system, cpu 7 have more sys cpu than other cpu, looks like on this cpu, ithr is much more than other cpu. CPU minf mjf xcal