I'm adding Joerg as an additional cc to this part of the thread to hopefully make it easier to find this discussion without wading through everything else.
Joerg's message helped me realize that I had completely misunderstood this message the first time I read it. I had thought that this part of the thread was only about the problem of the authentication check on the upload -- and had it marked to come back to if we could resolve the other issues -- but I completely missed that you were also talking about the source package verification here. I'm very sorry to have missed that! Ansgar 🙀 <ans...@43-1.org> writes: > On Mon, 2024-06-17 at 14:59 +0100, Jessica Clarke wrote: >> Would an API for tag2upload to use satisfy that concern? It feeds in a >> source package name and key fingerprint (or the signature, or >> whatever’s deemed useful), dak replies whether it’s valid for >> uploading. Then you don’t need to trust tag2upload’s authorisation >> checks beyond that it adheres to what dak says each time. > Hmm, a signed manifest solves that problem and also adds some integrity > verification and possibility for third parties to check the signature > itself as well. > An API that gets the signed data (signed tag) with some metadata (which > package, version, target suite, maybe some other bits of d/control, > would have to think; parts of that might also be in the tag) would > still allow having a single system make the decisions. My understanding is that the tag that triggers tag2upload currently has the following information in the directly signed Git tag object, so you don't need the rest of the Git tree to verify this: * The name and email of the uploader (in the Git Tagger field) * A statement that they intend to upload to Debian * The name of the source package * The version of the source package It doesn't currently have target suite (tag2upload gets that from the debian/changelog file), but I doubt that would be a problem to add to the tag directly. > A downside is that integrity verification and third parties (possibly) > verifying the data falls flat. For me integrity verification would be > somewhat nice and third parties a bit less interesting (given they can > get the tag, compare files and possibly redo what tag2upload does if > they also care about .dsc and stuff). Integrity verification of the source package construction by dak was the part of this that I was the most worried about, because that's the place where it had looked like the tag2upload goals couldn't meet the FTP team's requirements. If that's something that can be relaxed, that's huge for being able to find an implementation that hopefully makes everyone happy. > We are not very good at doing integrated services though. (Please no RPC > via SSH forced commands...) It would really be nice if we had a project infrastructure API mechanism that everyone agreed was fine and that we could use for new projects. That said, I think we can dodge the need for that initially, with the note that it would be nice to have it in the future. The ideal protocol would look something like: 1. tag2upload receives the upload request. 2. tag2upload sends the uploader-signed tag and metadata to dak for approval. 3. After receiving approval, tag2upload builds the source package. 4. tag2upload uploads everything to dak, including the uploader-signed tag object as a separate file listed in the *.changes file that it signs with its own key. You presumably want the tag object here too, just to be sure nothing weird happened in the middle of this process and so that you don't have to keep local state for an upload in progress. But I think the following protocol has all the same properties from your perspective, with the cost that there is a small risk that tag2upload may do pointless work and the dgit-repos server may receive a push of something that then failed to upload: 1. tag2upload receives the upload request. 2. tag2upload makes a guess at the authorization decision that dak will eventually make, based on a (possibly partial or out of date) set of the rules that dak has. 3. If it thinks the upload will be accepted, tag2upload builds the source package and also preserves the signed tag object with its metadata. 4. tag2upload uploads both the source package and the uploader-signed tag object to dak, including the tag object in the *.changes file that it signs with its own key. 5. dak then verifies the signed tag object and metadata and rejects the upload if it doesn't pass its authentication and authorization checks. This would avoid the need for a new RPC mechanism for now and I think results in exactly the same behavior as far as the archive itself is concerned. > With regard to integrity verification, it's probably not fully clear > what one exactly wants there. Joerg's idea to have tag2upload run by > ftp-master is related to that: if it was part of the archive, all > integrity promises still come from within the archive. (I don't really > want to run the service though; outsourcing is probably better.) I also think outsourcing is better. > Arguably we do trust some external services somewhat already (like > buildds for binary packages). Yes, I do think roughly the right way to think about tag2upload is that it's a source package buildd, where the "source of the source package" is a signed Git tree. -- Russ Allbery (r...@debian.org) <https://www.eyrie.org/~eagle/>
