Hi, On 2019-08-28 10:26:35 -0700, Andres Freund wrote: > On August 28, 2019 9:22:44 AM PDT, Merlin Moncure <mmonc...@gmail.com> wrote: > >Hackers, > >[apologies if this is the incorrect list or is already discussed > >material] > > Probably should be on the -www list. Redirecting. Please trim in future > replies. > > >I've noticed that mailing list discussions in -hackers and other > >mailing lists appear to not be indexed by google -- at all. > > I noticed that there's fewer and fewer hits too. Pretty annoying. I have an > online archive I can search, but that's not something everyone should have to > do. > > I think it's because robots.txt tells search engines to ignore the lists. > Quite hard to understand how that's a good idea. > > https://www.postgresql.org/robots.txt > > User-agent: * > Disallow: /admin/ > Disallow: /account/ > Disallow: /docs/devel/ > Disallow: /list/ > Disallow: /search/ > Disallow: /message-id/raw/ > Disallow: /message-id/flat/ > > Sitemap: https://www.postgresql.org/sitemap.xml > > Without /list, there's no links to the individual messages. So there needs to > be another external reference for a search engine to arrive at individual > messages. > > Andres > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity.
For reasons that I do not understand, the previous mail had a broken html part, making the above message invisible for people viewing the html part. Greetings, Andres Freund
