On Wed, Jan 09, 2008 at 05:58:34PM +1000, Anthony Towns wrote: > Disallow: /*/ # exclude everything but the shortcuts > Allow: /cgi-bin/bugreport.cgi?bug= > Allow: /cgi-bin/pkgreport.cgi?pkg=*;dist=unstable$ > I've set that up on rietz for Googlebot, we'll see if it works ok. I > don't think it's possible to make "Disallow: /*/" be the default for > all User-Agents since "*" is an extension, but extending it to MSN and > Yahoo should be fine.
User-Agent: * Disallow: /cgi-bin Allow: /cgi-bin/bugreport.cgi?bug= Allow: /cgi-bin/pkgreport.cgi?pkg=*;dist=unstable$ would probably work okay, actually. Cheers, aj
signature.asc
Description: Digital signature
