Re: huge httpd processes

2009-09-30 Thread Perrin Harkins
On Wed, Sep 30, 2009 at 10:47 AM, Clinton Gormley wrote: >> Do either of these bear any relation to mod_perl's shared memory which >> you can use by preloading modules at startup? > > Yes - as I understand it (somebody please correct me if I'm wrong), all > of the C libraries (eg XS modules) that

Re: huge httpd processes

2009-09-30 Thread Clinton Gormley
> i. The total VIRT size of the process before was 119 Mb > ii. Before - the process shared 94 Mb with other processes > iii. After - shared has gone down and private dirty up - does this > mean that this process now 'owns' this memory and it can't be used by > other processes? Correct > > The o

Re: huge httpd processes

2009-09-30 Thread Adam Prime
Justin Wyllie wrote: Hi clint Yes. Linux and this script looks good. We've think that part of the problem is in the modules Apache is loading so this will be useful. I also have another couple of questions: I have found the errant code where our process jumps by 13 Mbs. One part does something

huge httpd processes

2009-09-30 Thread Justin Wyllie
Hi clint This script looks helpful. I don't completely understand (even after looking at the documentation for Linux::Smaps) it though Looking at this output: ##before a bit hit on an expensive part of the web site VMSIZE: 119472 kb RSS: 96184 kb total 94904 kb shared

Re: huge httpd processes

2009-09-29 Thread André Warnier
Justin Wyllie wrote: ... $file_handle->read($s, $length); #$s is about 1/2 Mb @data = unpack($format , $s); ##at this point memory usage jumps by 8 Mbs (measured using GTop->size() ) while (@data) { push @data2, [shift @data, shift @data, shift @data] ; # this isn't exact but it looks like

Re: huge httpd processes

2009-09-29 Thread Clinton Gormley
> > I tried undef'ing @data just before the return as it is no longer used > but > this only gained me 1/2 Mb. I would have expected to get all 8Mbs > back. I > don't understand why not. > Perl (as least on the OS's that I'm familiar with) doesn't release used memory back to the OS. Have a lo

huge httpd processes

2009-09-29 Thread Justin Wyllie
Hi clint Yes. Linux and this script looks good. We've think that part of the problem is in the modules Apache is loading so this will be useful. I also have another couple of questions: I have found the errant code where our process jumps by 13 Mbs. One part does something like this: $file_ha

Re: huge httpd processes

2009-09-29 Thread Torsten Foertsch
On Tue 29 Sep 2009, Clinton Gormley wrote: > > I'm wondering if anyone can advise me on how I could go about > > trying to understand where this 90 Mbs is comming from? Some of it > > must be the mod_perl and apache binaries - but how much should they > > be, and apart from the 6mb in shared memory

Re: huge httpd processes

2009-09-29 Thread Clinton Gormley
Hi Justin > > I'm wondering if anyone can advise me on how I could go about trying > to understand where this 90 Mbs is comming from? Some of it must be > the mod_perl and apache binaries - but how much should they be, and > apart from the 6mb in shared memory for my pre-loaded modules, where > i

Re: huge httpd processes

2009-09-28 Thread Perrin Harkins
On Mon, Sep 28, 2009 at 3:19 PM, Justin Wyllie wrote: > But I'm struggling to understand why my processes are so huge. Using top > with the DATA field that shows that almost all the 90 mbs is data not > executable code. If this is the correct interpretation of that? Probably not. Perl code looks

Re: huge httpd processes

2009-09-28 Thread Igor Chudov
Justin, I cannot have a good suggestion how to find out what hogs the memory. However, I want to mention that with the use of a good reverse proxy server, you will not need to have very many apache instances running, because the proxy server would take care of the slow interaction with the client'

huge httpd processes

2009-09-28 Thread Justin Wyllie
Hi I've been attempting to fine-tune my server. The apache/mod_perl processes are around 90 Mbs - to start with. I've preloaded a lot of code in the startup file and that makes the shared memory space about 6 Mb. I can now see how my processes do not grow due to code being loaded when it is r