On Thu, 3 Jul 2025 at 16:18, Kevin Kofler via devel < devel@lists.fedoraproject.org> wrote:
> Leigh Scott wrote: > > Why isn't fedora infra using Anubis to block LLM scrappers? > > Why should they? Anubis is a scourge that wastes massive energy for all Currently AI scrapers are regularly taking down most of fedora's servers every day. People can't get updates because the proxies are overloaded, and packagers can't get builds because koji, pkgs, etc are non-responsive. The infrastructure team is daily having to block various networks that the scrapers are 'renting' per day and this is also doing the same thing to various contributors who can't get into things on any day because that subnet was being used. User-agent blocks don't work anymore because most of these scrapers are quite as likely to be using a rented botnet to do their scraping as anything else. [Botnet people are making LOTs more money to various third party AI companies than they do from email spam these days.] Network blocks aren't effective because by the time you have blocked one network, they are now using a different one. What are other solutions available? > > legitimate browsers, breaks search engines, and if configured in a > particularly aggressive way as on the GNOME GitLab, even entirely locks > out > some browsers (though that is an issue with the setup at GNOME > specifically). > > Kevin Kofler > > -- > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > -- Stephen Smoogen, Red Hat Automotive Let us be kind to one another, for most of us are fighting a hard battle. -- Ian MacClaren
-- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
