It was thus said that the Great Phong Thai once stated: > Hello, > > I’m looking for advice on handling crawler-driven overload in an Apache > prefork environment.
... > - Are there Apache-level techniques to limit crawler impact without > blocking Googlebot? I found mod_limitipconn that works well in my case. It limits the number of concurrent connections from a single IP address to a configurable number. I changed the code to return a 429 instead of 503, but that's the only change I've made. I set globally the number of concurrent connections to 20 (again, from a single IP address). -spc --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
