I am presently using a scheme like this to prevent scraping documents.
************************************
location /images/ {
valid_referers none blocked www.example.com example.com
forums.othersite.com ;
# you can tell the browser that it can only download content from the domains
you explicitly allow
# if ($invalid_referer) {
# return 403;
if ($invalid_referer) {
return 302 $scheme://www.example.com;
***************************************
I commented out some old code which just sends an error message. I
pulled that from the nginx website. I later added the code which sends
the user to the top level of the website.
It works but the results really aren't user friendly. What I rather do
is if I find an invalid_referer to some document, I would like to
redirect the request to the html page that has my link to the document.
I am relatively sure I will need to hand code the redirection for every
file, but plan on only doing this for pdfs. Probably 20 files.
Here is a google referral I pulled from the log file
*********************************************
302 172.0.0.0 - - [05/Mar/2024:20:18:52 +0000] "GET /images/ttr/0701crash.pdf
HTTP/2.0" 145 "https://www.google.com/"; "Mozilla/5.0 (Linux; Android 10; K)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Mobile Safari/537.36"
"-"
**********************************************
So I would need to map /images/ttr/0701crash.pdf to the referring page
on the website.
_______________________________________________
nginx mailing list
nginx@nginx.org
https://mailman.nginx.org/mailman/listinfo/nginx
