On Wed, Nov 25, 2009 at 12:09 PM, Clay Fouts <clay.fo...@liblime.com> wrote: > I've discovered that a web crawler was opening up large numbers of > simultaneous requests for the RSS/Atom feed. Each of these requests > runs an instance of git-diff-tree, which consumes quite a lot of > memory and CPU time. Enough of these requests pile up, and the kernel > invokes its oom-killer to take care of business. This often times has > the unfortunate side effect of killing off key processes, like apache > and SSH.
Big ouch! > In order to have another repeat of this, I've disabled gitweb's > RSS/Atom feed functionality. It's a little ham-fisted, but not being a > git expert, it's the work-around that is most obvious to me right now > to stabilize things. If anyone can suggest a solution that can keep > the system stable while not removing functionality, please let me > know. > > Clay You want a robots.txt file in your root directory to tell robots to not index the git repository. http://en.wikipedia.org/wiki/Robots_exclusion_standard http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 http://www.robotstxt.org/ Best regards, gvb [snip] _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha.org http://lists.koha.org/mailman/listinfo/koha-devel
