Mark Phippard wrote on Wed, 23 Mar 2022 11:36 +00:00: > On Tue, Mar 22, 2022 at 11:51 PM Daniel Shahaf <d...@daniel.shahaf.name> > wrote: >> >> Mark Phippard wrote on Mon, Mar 21, 2022 at 16:46:55 -0400: >> > On Mon, Mar 21, 2022 at 4:31 PM Stefan Sperling <s...@elego.de> wrote: >> > > On Mon, Mar 21, 2022 at 12:44:44PM -0400, Mark Phippard wrote: >> > > > Problem 1: Rolling the tarballs >> > > > >> > > > The process creates the tarballs but fails near the end. It looks GPG >> > > > related? >> > > >> > > > INFO:root:Building Unix tarballs >> > > > INFO:root:Moving artifacts and calculating checksums >> > > > Traceback (most recent call last): >> > > > File "trunk/tools/dist/release.py", line 1916, in <module> >> > > > main() >> > > > File "trunk/tools/dist/release.py", line 1912, in main >> > > > args.func(args) >> > > > File "trunk/tools/dist/release.py", line 983, in roll_tarballs >> > > > download_file(KEYS, filepath, None) >> > > > File "trunk/tools/dist/release.py", line 289, in download_file >> > > > response = urlopen(url) >> > > > File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen >> > > > return opener.open(url, data, timeout) >> > > > File "/usr/lib/python2.7/urllib2.py", line 435, in open >> > > > response = meth(req, response) >> > > > File "/usr/lib/python2.7/urllib2.py", line 548, in http_response >> > > > 'http', request, response, code, msg, hdrs) >> > > > File "/usr/lib/python2.7/urllib2.py", line 473, in error >> > > > return self._call_chain(*args) >> > > > File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain >> > > > result = func(*args) >> > > > File "/usr/lib/python2.7/urllib2.py", line 556, in >> > > > http_error_default >> > > > raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) >> > > > urllib2.HTTPError: HTTP Error 404: Not Found >> > > >> > > It seems ASF have removed the KEYS file our script is trying to fetch. >> > > See http://people.apache.org/keys/ where it says "Project group files are >> > > no longer created." >> > > >> > > It looks like what the script wants to do here is obtain a copy of >> > > the Subversion project's KEYS file and store it along with release >> > > artifacts. If we want to keep doing this we will have to maintain >> > > our own KEYS file on the website, I suppose. Otherwise, we could >> > > decide to no longer provide such a file and remove relevant code >> > > from the script. Not sure what is better. >> > >> > Since I do not know what this was used for, maybe someone can help get >> > us to a decision and update the script? Daniel Shahaf, if you see >> > this, I think of you as the resident expert on this stuff. Any thoughts? >> >> The design goals here are two: >> >> 1. When someone with commit access to N ASF projects updates their PGP >> key, they shouldn't have to do O(N) work to update N KEYS files. They >> should have to do either O(1) work or, ideally, nothing at all. >> >> 2. Releases should snapshot the keys that are current at the time they >> are generated, in order to remain verifiable in archive.a.o even if the >> keys in question are later removed from LDAP (by root@ as part of >> a manual password reset, or by the committer via id.a.o). >> >> The ASF-wide "generate group keys" scripts addressed #1. They used to >> generate two files, one with only the full committers' keys and one with >> both full and partial committers' keys. We used the former (to reduce >> attack surface) until it stopped getting generated. >> >> Copying the file to subversion-<version>.KEYS addressed #2. > > Thanks for responding and please forgive some basic questions. My > "knowledge" of this begins and ends with running gpg -ba to sign > releases and copying and pasting the output to an email. > > It sounds like the KEYS file is a list of all of the GPG Keys for ASF > committers? >
Yes. It's basically the output of «gpg --armor --export $args» where $args is developers' keys, as recently updated from public keyservers, with cosmetic text added before each block. > What is it used for during the RM process? Is it so that the RM can > validate signatures that come in? Does it get included in the release > somehow? > It's uploaded to dist/ and users are pointed to it. The idea is to establish a trust path for users between "whoever controls the SSL certificate on subversion.apache.org:443" and the particular PGP keys that signed a particular artifact. I assume the RM should verify signatures too, though HACKING doesn't call this out. > >> >> So, what to do? >> >> - We could talk to some ASF-wide list (comdev, infra, site-dev@) about >> resuming generation of group keys files. >> >> (If you do this, someone will probably ask you to comply with >> a markdown document that calls itself policy. I don't believe that >> document is Foundation policy, and even if it is, our practice has >> been +1ed by an Officer of the Foundation.) >> >> - We could roll our own automation, relying on the key fingerprints in >> id.a.o (LDAP) via <https://people.apache.org/keys/committer/>: >> >> for availid in $( >> perl -anE 'say $F[0] if (/^Blanket/../END ACTIVE >> FULL.*SCRIPTS LOOK FOR IT/ and /@/)' \ >> < /path/to/COMMITTERS >> ) ; do >> echo curl -sSfO >> https://people.apache.org/keys/committer/${availid}.asc || test $? -eq 22 || >> return 1; >> done >> cat *.asc > subversion-….keys >> >> - We could roll our own solution without relying even on >> <https://people.apache.org/keys/committer/>, since its API has already >> been broken twice. > > Suppose we wanted to roll out a new release soon. What would you > recommend? Can we just remove this step and ignore it or do we need > someone to code a solution? > I'd recommend to continue generating a KEYS file, perhaps with a Python variant of the code above. I don't immediately see another way to let users fetch the "authorized" public keys from https://*.apache.org/, and that seems to be a requirement, since KEYS files are deliberately excluded from being mirrored. > As for the rest ... I have figured out, or nearly figured out, a > solution to all of the build and test issues now. So I am all set with > that. > *nod* Cheers, Daniel > Thanks > > Mark
