On 18/08/2012, at 6:46 AM, Denis BUCHER <dbuche...@hsolutions.ch> wrote:
> Dear all, > > That's an unbelievable issue but we have single apache process that takes 5 > GB of memory ! And it doesn't happens always with the same URLs, it's > unpredictable and we don't understand why it is happening at all !!! > > Any help would be greatly appreciated (as well as the users of the website > and the owners) ! > > We even developed for days some software to analyze what is happening ! > > We do a pmap PID each second when the problems comes up, but the offending > line is : > > 00002aad145c2000 2929376K rw--- [ anon ] > > so what can we do with that, is there a way to know what it is ? > > We also analyze open files with lsof -p. > > I am now convinced that it only happens when the connection comes from Google > Bot (????) > > Do you have suggestions on how to analyze ? > > Notes : > > The config is Linux+Apache+PHP+Postgresql > Of course PHP memory_limit was the first point we checked but this is not the > problem : grep memory_limit /etc/php.ini => memory_limit = 32M > By the way we tried rlimitmem but either it doesn't work, either we don't > used it correctly, as it's no use : still using more than 5 gigabytes > That unanswered thread looks similar to our problem : > http://serverfault.com/questions/161478/apache-process-consuming-all-memory-on-the-server > Thanks a lot for any help and/or suggestion ! > > Denis > Do you have a robots.txt ? Maybe you have some dynamic page that when spidered returns a very deep structure or some script that returns a temporary failure, causing a loop? Look at the http log for googlebot, and see if there is any pattern to it. If you have the access logs it will tell you the script name at least. If you find anything suspicious exclude it from spidering with robots.txt If you don't maybe it is some sort of non-google bit attack, and you can block it with apache (as a hacker or snoop bot wont likely listen to robots.txt In any case if you identify the path, maybe you can help identify / fix the bug.. Cheers Brett
