What is returning the 500 response here? Is php/python/perl involved? As for the scrapers, you are absolutely wasting your time customizing the response. I would just return a 403, actually.
On Fri, Sep 15, 2023 at 11:37 AM Marc <m...@f1-outsourcing.eu> wrote: > > > See the ErrorDocument directive. > > It does not seem to work. It looks like this config is skipped and the > error is loaded directly from the httpd binary. > > ErrorDocument 500 /406.html > > > Now, why is that response not suitable? And why would you respond with a > > 200 for a blocked user agent? > > I think it is better to return to scrapers 200 and empty content, instead > of notifying them so they can reconfigure their systems. > > > > > > > Where/how can I change this message? > > > > <p>The server encountered an internal error or > > misconfiguration and was unable to complete > > your request.</p> > > <p>Please contact the server administrator at > > xxx to inform them of the time this error occurred, > > and the actions you performed just before this error.</p> > > <p>More information about this error may be available > > in the server error log.</p> > > </body></html> > > > > or as a work-a-round, how can refuse access with modsecurity and > just > > generate a 200 blank page response. > > > > SecRule REQUEST_HEADERS:User-Agent "blockthisua" > > "id:'13006',phase:2,log,deny,status:200" > > > >
