Hi Stuart,
changing subject and suggest moving the topic to Debian QA list where
it probably belongs.
Am Thu, Jan 09, 2025 at 11:54:47AM +1100 schrieb Stuart Prescott:
> Good point on anonscm as well... that really does blow out the numbers.
Unfortunately yes.
> However... some of them still work via the aliasing mechanism that was
> introduced at the time of migration to salsa.
In the migration phase from Alioth to Salsa I maintained lists of
packages for Debian Med and Debian Science team. In my practical
experience finding some working alias is a rare exception. I also think
this alias mechanism was a temporary solution that should not survive
for >5 years.
> Duck used to check them all
> but I don't think it is running any more, unfortunately. vcswatch still
> does, more on that later.
Vcswatch is a good hint.
> > > The vast majority of these packages have seen post-alioth uploads but with
> > > the broken Vcs fields still in place.
> >
> > Do you have numbers backing up this "vast majority" statement?
>
> Yes, that's in the table below. Of those 161 packages, 145 have been
> uploaded since salsa launched and alioth stopped. (updated data with anonscm
> at the bottom - the story is still the same, although not all those anonscm
> links are broken)
Ahhh, got your point now. The Bug of the Day criteria are selecting
packages that are not uploaded for a long time and thus might experience
is different.
> > > (I accidentally found 2 python-team packages without Vcs URLs yesterday -
> > > the repos were on salsa, just not listed in d/control)
> >
> > Not so nice. Did you just injected these? If not would you mind naming
> > the packages?
>
> One got uploaded because I was sorting other changes for qtpy, the other is
> fixed in git. Having looked at 20-something packages in the last 2 days, I'm
> not sure I could actually name which ones at this stage...
OK as long as these are fixed now.
> In pursuing this, you might also find the vcswatch table in udd - it lists
> 1533 packages where the VCS fields might need fixing. Some of the errors
> there are transient, but this also picks up typos in the VCS fields
> ('debain', 'debian/packages/') and repos that simply don't exist.
Good point.
> Updated queries and data appended. (and btw postgres can do regex matches
> which simplifies the sql quite a lot)
I'm aware in principle about the regexp feature. Unfortunately I have
to deal with SQL databases without this kind feature in my day job. So
I usually try to avoid PostgreSQL only features.
> Majority of packages with invalid vcs_url uploaded post salsa:
>
SELECT
DATE_PART('year', date) AS year,
COUNT(*)
FROM
sources AS s
JOIN upload_history AS h
ON s.source = h.source AND s.version = h.version
WHERE
release = 'sid'
AND vcs_url ~ '/(git|svn|alioth|anonscm).debian.org'
GROUP BY
year
ORDER BY
year ASC;
>
> year | count
> -----+-------
> 2011 | 2
> 2012 | 5
> 2013 | 7
> 2014 | 9
> 2015 | 9
> 2016 | 20
> 2017 | 102
> 2018 | 85 ← (salsa.d.o general availability)
> 2019 | 10
> 2020 | 77
> 2021 | 411
> 2022 | 115
> 2023 | 13
> 2024 | 31
> 2025 | 3
> (15 rows)
>
>
>
> Teams with packages to fix - and the packages are probably already on salsa
> so this is just metadata, not lots of work.
>
SELECT
maintainer_name, COUNT(*)
FROM sources
WHERE
release = 'sid'
AND vcs_url ~ '/(git|svn|alioth|anonscm).debian.org'
AND maintainer ~ '(team|group|lists)'
GROUP BY
maintainer_name
ORDER BY
count DESC;
>
> maintainer_name | count
> ---------------------------------+-------
> Debian Ruby Extras Maintainers | 196 (+2 that are in Uploaders)
> Debian Java Maintainers | 178
> Debian Go Packaging Team | 105
> Debian Perl Group | 83
> pkg-go | 25
> Debian Javascript Maintainers | 20
> Debian Fonts Task Force | 15
> Debian PHP PEAR Maintainers | 14
> Debian X Strike Force | 12
> Debian Science Maintainers | 11
> Debian XML/SGML Group | 5
> Debichem Team | 4
> Debian VDR Team | 4
> Debian CLI Applications Team | 2
> Debian Games Team | 2
> Debian Java maintainers | 2
> Debian Tasktools Packaging Team | 2
> Debian VoIP Team | 2
> Debian Astronomy Maintainers | 2
> Debian Privacy Tools Maintainers | 2
> Debian Clojure Maintainers | 2
> Debian Astronomy Team | 2
> Debian Telepathy maintainers | 2
> Live Systems Maintainers | 1
> The Debian Lua Team | 1
> Pulseaudio maintenance team | 1
> Android Tools Maintainers | 1
> Debian PhotoTools Maintainers | 1
> Puppet Package Maintainers | 1
> ClamAV Team | 1
> Debian-IN Team | 1
> Debian CLI Libraries Team | 1
> Debian Islamic Maintainers | 1
> Debian GNOME Maintainers | 1
> Debian Science Team | 1
> Debian Sugar Team | 1
> Debian GNUKhata Team | 1
> Debian Emacs addons team | 1
> Debian Med Packaging Team | 1
> Debian Salt Team | 1
> NeuroDebian Team | 1
>
>
>
> Find packages in your favourite team that you want to work on...
SELECT
source, vcs_url
FROM sources
WHERE
release = 'sid'
AND vcs_url ~ '/(git|svn|alioth|anonscm).debian.org'
AND maintainer ~ 'science'
ORDER BY
source;
Thank you for publishing these data - I hope this will encourage people
to look into this.
> The vcswatch table has lots of interesting things... Note that the salsa
> error "could not read Username" in the table is not a misconfiguration - it
> means that the repo couldn't be obtained anonymously, which could be that it
> doesn't exist, or that it needs permissions - both are wrong for Debian.
>
SELECT
source, url, error
FROM
vcswatch
WHERE
error IS NOT NULL
ORDER BY
source;
I've remove the quotation markers from the SQL queries to enable easy
copy-n-pasting for the readers. I confirm a couple of Debian Science
packages will not show up any more tomorrow (but some are not simple
metadata fixes since a lot has happened on code in Git which does not
build currently - at least I pinged the team in those cases).
Kind regards
Andreas.
--
https://fam-tille.de
