Hello, Not sure if it can help you, because only some bots respect it and not in the same way, but you could look at the "crawl-delay" directive in the robots.txt file: https://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl-delay_directive
Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269862,269869#msg-269869 _______________________________________________ nginx mailing list nginx@nginx.org http://mailman.nginx.org/mailman/listinfo/nginx
