[ I find the tone of your mail needlessly aggressive, we're just discussing an idea at this point and seeing if it's worth investing more time into it ]
On Fri, 26 Mar 2010, Marc 'HE' Brockschmidt wrote: > No, this can't be defined later. It's a central part. Would any new > binary lead to a dedicated overlay? How do you determine when this is > needed and when not? We have several types of transitions and it will need different kind of checks. Let me give some examples. A direct upload to sid: - must not drop a binary package that has reverse dependencies (ex: drop libfoo4 when new source package provides libfoo5) - must not change the SONAME of a public library without changing the package name - must not increase the automatic dep via shlibs/symbols files (we first want to make sure that such an updated library is built/available on all arches before it hits unstable) - must not remove a Provides where it was the only package providing it (ex: perlapi-5.8.0, phpapi-*) if there are reverse dependencies on that provides I'm not sure we can catch all transitions, I have no suggestion for python-defaults or gcc-defaults transitions for example. But we are generally aware of those transitions and they rarely happen out of the blue. I did not want to go into details because I believe that this part is doable and it doesn't bring much to the initial goal of the discussion: determining if such a system could work to better manage the transitions. > > Instead transitions should be managed in dedicated repositories > > that are overlays over sid. We would have tools (command-line, web > > interface, etc.) to manage the transition in that repository. > > What would these tools do? Why don't they exist currently? See below. They would be used to tell the system which packages are affected and should be included in the transition. (And they do not exist becuse nobody wrote them yet, although some release people have scripts to identify set of packages concerned by transitions) > > We could tell which packages need to be rebuilt (bin-NMU, sourceful upload) > > and the system would automatically rebuild the package (and it > > would also be automatically rebuilt every time that the package is updated > > in sid). > > How do you determine the set of packages that need to be transitioned? > Are these provided by the uploader? Can they be computed? A mix of both, the uploader should be able to provide a raw list and/or tell the system "all source packages whose binaries are depending on libfoo4". The release team would verify that the set of packages includes all required packages but some edos-debcheck based tools could also help to try to ensure that automatically. > > Some web interface would display the status of the transition, > > displaying which package/arch have been built or not, and which one failed > > to build from source. > > You mean like the existing pages on buildd.debian.org? You just need to > feed them the list of affected packages to get that. Good if it can be done with a simple link to buildd.debian.org! It would still be nice to be able to attach some information like bug numbers near FTBFS so that people managing the transitions know whether all failures have been reported. > > Multiple transitions will still end up mixed in sid if you push them > > before packages have migrated to testing but they are all already > > completed and you only have to deal with RC bugs and delays to ensure > > package can migrate to testing. > > "Only" deal with RC bugs? This touches an interesting point you didn't > cover at all: How are bugs reported? How can these bugs be tracked if > the same source version is fine in sid, but breaks in an overlay - or, > worse, breaks in one overlay, but not in another? Bugs are reported like usual. However you are right that we need to extend our bin-nmus versioning scheme to avoid collisions in package versions between various overlays. And/or we need to extend our tools to be able to explicitly give the source version (in fact there's such a request already in the dpkg-dev BTS: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=440094) In sid: Package: bar Version: 1.0-1 Source: barsrc In python2.6 overlay: Package: bar Version: 1.0-1+python2.6+b1 Source: barsrc (1.0-1) In libfoo5 overlay: Package: bar Version: 1.0-1+libfoo5+b1 Source: barsrc (1.0-1) It might also mean some changes to the version-tracking code. > > Dealing with transitions that way make it clear that the maintainer has to > > take the responsibility to prepare/complete the transition if he wants to > > see his package in sid and in the next release. > > I assumed that this was already the case. I also don't see how enforcing > such a technical system would solve this problem? Currently a maintainer uploading new version of the library and having given a list of packages to bin-nmu can disappear, the transition will complete a some point because otherwise it blocks you as release managers to deal with other transitions. So you ensure relevant RC bugs and FTBTS are fixed so that you can migrate packages. But it should not be your responsibility. With the proposed system you would no longer have to finish any current transition by yourself (or even prod people to to their work). If it's not yet finished, you can simply pick another one that is more advanced and do it first. If the maintainer never finishes the transition work, his updaded package will never reach sid. Transitions are prepared in parallel and are competing to be ready first. Your job as release managers is to decide of the order based on the advancement of currently pending transitions while still trying to limit disruption. > > Preparing the transition in experimental is not always done and takes > > much more energy than such a system would take. > > Why, actually? I don't an exhaustive answer but here are some points: 1/ you can't request bin-nmus of reverse-dependencies in experimental (to verify that all packages build fine with the updated package, and that's one of the main task in preparing the transition) 2/ you have to manually reupload a new source package to unstable with all the delay it induces for getting the package built on all arches 3/ some maintainers are too confident that nothing is going to break > Trying to improve the handling of transitions is a good thing. I don't > consider your mail to be a proposal, as it doesn't cover any of the > actual technical problems. It's still an idea, it's not yet a proposal with a spec but before going further and maybe trying to draft a DEP I want to know if you believe that such a system could improve the handling of transitions. I hope I responded to some of your concerns by showing that it's possible to find solutions if we know where we'd like to go. BTW, I got pointed to http://wiki.debian.org/SummerOfCode2010/DeveloperPackageRepositories and having this implemented would mean that it should not be too difficult to implement those overlays (by reusing the infrastructure). > Another interesting question: Do you plan to implement this? Do you > expect someone else to do it? I don't plan to this implement this alone. But if we decide it's something that we want to try, I will help (at least to drive the DEP, I can't promise to work on something we have not yet thought out). > And, seeing the not ending discussions about your handling of the new > dpkg source formats: Do you think that such a change could be > implemented without pissing off so many people? Sure. That's why I suggest to draft a DEP. Cheers, -- Raphaël Hertzog Like what I do? Sponsor me: http://ouaza.com/wp/2010/01/05/5-years-of-freexian/ My Debian goals: http://ouaza.com/wp/2010/01/09/debian-related-goals-for-2010/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100326141112.gb8...@rivendell
