On 13/04/23 at 08:29 +0800, Paul Wise wrote:
> On Wed, 2023-04-12 at 07:14 +0200, Lucas Nussbaum wrote:
>
> > There's specific code in the UDD uscan wrapper[1] to handle github's
> > rate limiting. We could have something similar for either sf.net, or the
> > sf.net redirector. Before I work on that, it would be great if someone
> > could change the sf.net redirector to return 429 instead of 500 when
> > sf.net returns 429, so that this specific case is easier to identify.
>
> This is now done, tested and deployed on the server:
>
> https://salsa.debian.org/qa/qa/commit/395d923257e954663156fa315142415f50d1be6a
>
> I elected to just pass on all SourceForge HTTP error codes,
> with the HTTP error text prefixed to clarify the error source.
Thanks!
I added code to handle sf.net's rate limiting in the UDD importer, and
triggered a refresh of all sf.net-hosted packages.
I wonder if we should close this bug. The redirector has not been fixed
(it will still hit rate limiting, but there's not much we can do about
that); but the main path by which maintainers probably access watch data
(UDD -> dashboards) has been fixed.
- Lucas
Some UDD notes for reference:
To watch the status of UDD trying to refresh all SF sources:
udd=> select status, count(*) from upstream where watch_file ~ 'sf.(net|php)'
group by status;
status | count
------------------------------+-------
newer package available | 120
up to date | 469
error | 976
only older package available | 53
(4 rows)
udd=> select warnings is null, count(*) from upstream where watch_file ~
'sf.(net|php)' group by 1;
?column? | count
----------+-------
f | 986
t | 632
(2 rows)
To force a refresh of all sf.net sources:
update upstream set last_check = null where watch_file ~ 'sf.(php|net)' and
warnings is not null;
- Lucas
