Le 10/05/2023 à 19:14, Alfred M. Szmidt a écrit :
I was searching for an article with DuckDuckGo, and guess what appeared on top of the results... https://lists.gnu.org/archive/html/www-commits/2023-05/msg00082.html and https://lists.gnu.org/archive/html/www-commits/2023-05/msg00062.html !!What is the problem? I understand it wouldn't be practical to make this list private, but it could at least be off-limits for crawlers. How is www-commits different from anything else?
Because it registers every single commit to www, including to working directories that webmasters have disallowed, for instance */po/, /server/staging/, */workshop/, /prep/gnumaint/, etc. Please see https://www.gnu.org/robots.txt.
