Le 01/11/2016 à 03:52, Joel Kulesza a écrit :
Does anyone know what the instability is stemming from (hardware, software, provider, etc.)?
I would say that the problem was receiving the visit of crawl.sogou.com (220.181.125.68), which sucked 700MiB of data (along with some Amazon aws guy who claimed 320MiB). These are badly behaved bots which do not respect our robots.txt file. In this file, we request explicitly to skip indexing the /trac subdirectory.
I can blacklist by hand these bots, but they are gone now AFAIK and the harm is done. It seems possible to download automatically some blacklist of bad crawlers and blacklist them, but I do not know whether this is a good idea in real life.
So we could oversize our server just for the pleasure of letting any stupid bot spend time requesting data. But the best would probably to have a proper webadmin! We do not have anybody doing this job right now.
JMarc
