Hi all, has someone an effective way to block away bots and spiders? There are so many ignoring robots.txt - besides facebook and tons of Java/xyz clients there are many more illegal spiders around.
So does anyone have a performance effective way to block them out? Or do you think - if performance matters, leave them crawling... regards Henrik -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
