On Thursday, April 20, 2017 at 4:04:19 PM UTC+1, Erik Bray wrote: > > On Thu, Apr 20, 2017 at 4:54 PM, Dima Pasechnik <dim...@gmail.com > <javascript:>> wrote: > > > > > > On Thursday, April 20, 2017 at 3:38:12 PM UTC+1, Erik Bray wrote: > >> > >> Just posting to say I've noticed and am investigating. > >> > >> It seems that somebody is performing a scrape of the front page which > >> includes a list of users on the site [1] each of which is a link to a > >> search query for all tickets and wiki pages associated with that user. > >> That's what's causing the slowdown. > >> > >> The IP address seems to belong to Opera (maybe it's a VPN?) > >> > >> http://tools.tracemyip.org/lookup/107.167.122.77 > > > > note that the map on the right shows that the connection originates in > > China. > > Thus yes, it's a VPN. Actually, is sagemath.org blocked in China? I > don't > > know. > > Strange, well, I created a firewall rule blocking that IP for now and > everything seems to be fine. >
Does this mean that we need some robots.txt somewhere, perhaps after some restructuring, which would protect expensive resources from this sort of overload? -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
