(dropping blends list for the moment) Le Mon, Jun 09, 2014 at 11:32:44PM +0800, Paul Wise a écrit : > On Mon, Jun 9, 2014 at 11:00 PM, Charles Plessy wrote: > > > While as a website upstream-metadata.debian.net was never off, in practice > > the > > system has been broken for a couple of months. > > As a result we had to change (see r3061 in the qa SVN) the PTS to link > to the collab-qa SVN repo instead of yaml-all on the upstream-metadata > website. As far as I can tell this isn't yet working, will you be > fixing that too? > > http://upstream-metadata.debian.net/umegaya/chromium-bsu/yaml-all > https://anonscm.debian.org/viewvc/collab-qa/packages-metadata/c/chromium-bsu.upstream?view=co
Ouch… So in the end there is a bug that I can not understand. Let me first remind how things should work. The ‘umegaya’ web service is a Perl program that ties a hash to a Berkeley DB. This database contains results from previous queries, and a timestamp for the last access, so that it is used as a cache if the last access was recent. If not, querying the webservice trigger a refreshing of the information. Then, the result of the query is returned. To refresh the information about a package, the web service searches in the Berkeley DB via the tied hash for a VCS URL where to download a metadata file. If that URL is not available, it is inferred by calling ‘debcheckout -d’. The metadata file is then parsed and its contents are inserted in the Berkeley DB. A copy is saved on the local filesystem, which is synchronised daily with the collab-qa Subversion repository on Alioth. On blends.debian.net, which hosts http://upstream-metadata.debian.net, attempts to refresh information on chromium-bsu (and probably other packages) are silently ignored. In the following example, I query the system for the ‘test’ key of the ‘chromium-bsu’ package. I added it in the Berkeley DB via the command-line tool ‘umegaya-adm’: plessy@debian-med:~$ umegaya-adm -p chromium-bsu -k test -v asfsad plessy@debian-med:~$ umegaya-adm -p chromium-bsu -k test asfsad plessy@debian-med:~$ umegaya-adm -p chromium-bsu test asfsad YAML-URL svn://anonscm.debian.org/pkg-games/packages/trunk/chromium-bsu/ At this point, the Berkeley DB contains two entries for chromium-bsu: test and YAML-URL. YAML-REFRESH-DATE is not set to querying the web service will automatically trigger a refresh. Here is what it gives in debug mode. http://upstream-metadata.debian.net/cgi-bin/umegaya?package=chromium-bsu&key=yaml-all&debug=1 Content-Type: text/plain; charset=utf-8 Debug mode on; somebody is probably working on this site… chromium-bsu is not yet in the database, trying to pull its metadata… Refreshing data for chromium-bsu Running “svn cat svn://anonscm.debian.org/pkg-games/packages/trunk/chromium-bsu/debian/upstream/metadata”) Writing /var/lib/umegaya/packages-metadata/c/chromium-bsu.upstream ... done. Running “svn cat svn://anonscm.debian.org/pkg-games/packages/trunk/chromium-bsu/debian/control”) Writing /var/lib/umegaya/packages-metadata/c/chromium-bsu.control ... done. Running “svn cat svn://anonscm.debian.org/pkg-games/packages/trunk/chromium-bsu/debian/copyright”) Writing /var/lib/umegaya/packages-metadata/c/chromium-bsu.copyright ... done. Deleting chromium-bsu:test Key type is . Setting chromium-bsu:Contact to chromium-bsu-de...@lists.sourceforge.net Key type is . Setting chromium-bsu:Bug-Submit to https://sourceforge.net/p/chromium-bsu/bugs/new/ Key type is . Setting chromium-bsu:Repository to git://git.code.sf.net/p/chromium-bsu/code Key type is . Setting chromium-bsu:FAQ to http://chromium-bsu.sourceforge.net/faq.htm Key type is . Setting chromium-bsu:Homepage to http://chromium-bsu.sourceforge.net/ Key type is . Setting chromium-bsu:Screenshots to http://sourceforge.net/projects/chromium-bsu/ Key type is . Setting chromium-bsu:Repository-Browse to http://sourceforge.net/p/chromium-bsu/code/ Key type is . Setting chromium-bsu:Changelog to http://sourceforge.net/p/chromium-bsu/code/ci/master/log/ Key type is . Setting chromium-bsu:Name to Chromium B.S.U. Key type is . Setting chromium-bsu:Watch to http://sf.net/chromium-bsu/chromium-bsu-(\d[\d\.]+)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz))) Key type is . Setting chromium-bsu:Bug-Database to http://sourceforge.net/p/chromium-bsu/bugs/ Looking for key yaml-all of package chromium-bsu. No yaml-all key for package chromium-bsu. So the metadata file for chromium-bsu is found and parsed, but not inserted in the Berkeley DB. Here is the Perl code that produces the output above: # Drops all previous keys. foreach (keys (%stored)) { next unless /^$package:/; next if /:YAML-URL$/; say "Deleting $_" if $debug; undef $stored{$_}; } # Loads the new keys. foreach (keys (%$package_metadata)) { my $key = $package_metadata->{$_}; my $keytype = ref($key); say "Key type is $keytype." if $debug; # Compound keywords are equivalent to nested mappings: # # Foo-Bar: baz # # is the same as: # # Foo: # Bar: baz # if ( $keytype eq "HASH" ) { my $left = $_; foreach ( keys (%$key)) { say "Setting ${package}:${left}-$_ to $key->{$_}" if $debug; $stored{"${package}:${left}-$_"} = $key->{$_}; } } elsif ( $keytype eq "ARRAY" ) { say "Setting ${package}:$_ to $key" if $debug; $stored{"${package}:$_"} = $key; } else { say "Setting ${package}:$_ to $key" if $debug; $stored{"${package}:$_"} = $key; } } # Keeps a backup of the 'upstream' file. $stored{"${package}:YAML-ALL"} = $package_metadata_yaml; This works for other packages and I do not understand why the changes to the hash fail silently for this package while they work for others. Also, on other computers, I have no problem with chromium-bsu. Lastly, I have seen a very similar problem for the ‘emboss’ package earlier, that I solved by restarting the Berkeley DB from scratch. Therefore, I am not able to reduce this bug to a simple test case and I am blocked… (and I exhausted my free-time-in-front-of-computer for this week-end…) Have a nice Sunday, -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-qa-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20140614231448.ga14...@falafel.plessy.net
