On Wed, Jul 14, 2021 at 08:51:21PM +0200, Michael Niedermayer wrote: > Signed-off-by: Michael Niedermayer <mich...@niedermayer.cc> > --- > htdocs/robots.txt | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/htdocs/robots.txt b/htdocs/robots.txt > index eb05362..4bbc395 100644 > --- a/htdocs/robots.txt > +++ b/htdocs/robots.txt > @@ -1,2 +1,13 @@ > User-agent: * > -Disallow: > +Crawl-delay: 10 > +Disallow: /gitweb/ > +Disallow: /*a=search* > +Disallow: /*/search/* > +Disallow: /*a=blobdiff* > +Disallow: /*/blobdiff/* > +Disallow: /*a=commitdiff* > +Disallow: /*/commitdiff/* > +Disallow: /*a=snapshot* > +Disallow: /*/snapshot/* > +Disallow: /*a=blame* > +Disallow: /*/blame/*
This is based on https://serverfault.com/questions/506613/ideal-robots-txt-for-a-gitweb-installation i will add this link to robots.txt [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Avoid a single point of failure, be that a person or equipment.
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
