On 04/19/2016 03:10 PM, Doug Hellmann wrote: >> On your etherpad, you wrote: >> >> "During the lead-up to preparing the final releases, one of the tracking >> tasks we have is to ensure all projects have synced their global >> requirements updates. This is another area where we could reduce the >> burden on the release team." >> >> Well, don't bother, this doesn't reflect a reality anyway (ie: maybe >> service X can use an older version of oslo.utils), so that's not really >> helpful in any way. > > Right, that's why I proposed stopping the whole business with managing > the ranges. Your proposal seems to be somewhere in the middle between > doing what we do today (obsessively keeping in sync) and what I'm > proposing (abandoning any pretense of keeping in sync). Is that right?
Absolutely ! :) >> Though having a single version that projects are allowed to test with is >> super important, so we can check everything can work together. IMO, >> that's the part which should absolutely not change. Dropping that is >> like opening a Pandora box. Relying on containers and/or venv will >> unfortunately, not work, from my stand point. > > Again, as I pointed out elsewhere in the thread, using some sort of > environment isolation tool in upstream CI testing does not imply that > the same or a similar tool needs to be used by distros for packaging. But then, we still need co-installability and avoid lib version conflicts. Aren't we then back to square one? It feels like I'm missing something in the reasoning here... >> (think: we still have loads of Python 2.6 stuff like discover, argparse >> and such that needs to be cleaned-up). > > Aside: We should figure out a way to make a list of those things, > so we can work on the cleanup. I've pushed a bit of patches here and there, but I sometimes give-up forwarding the Debian work (because lacking time). I'm hereby making the promise to always do it for the Newton cycle (as there shouldn't be too many remaining). > I'm certainly not proposing that. My proposal is point out the > economics of our current situation, which is that upstream we're > doing a lot of work that IMHO we don't need to do *for ourselves*. > I do see its usefulness, but it's not necessary for our own CI > testing. Thanks for making this explicit. I was kind of double-guessing something like that was in your mind. I'm with you now. >>> Another alternate that might work is for downstream folks to do >>> their own dependency management. >> >> I fail to understand what you think distros could do to fix the >> situation of conflicting versions, other than bundling, which isn't >> acceptable. Could you explain what you mean by "downstream folks to do >> their own dependency management"? > > How do you ensure that a given version of a python lib you package > is compatible with independent tools written to use that lib that > you also want to include in your distro? Well, you're pointing here at a major pain point. I can give you 2 examples where it has been extremely painful for me: - SQLA upgrades from 0.7.x to 0.8.x, then to 0.9.x. a year ago. - Django upgrade to 1.9 last december. Both SQLA and Django are widely used in Debian, and when one gets upgraded, package maintainers have no choice but to fix their packages ASAP, to support the new version. I don't maintain SQLA or Django, both are maintained by package maintainers who don't really care about OpenStack, and therefore, they uploaded new versions to Sid before OpenStack was ready. The result is that most of OpenStack was broken in Sid for maybe 4 or 5 months when the SQLA upgrade happened. As for Django, this broke Horizon from last December to a few days before the release. Without heroic efforts from Horizon contributors, (mostly by Rob, the new Horizon PTL. but not only him), then Horizon would still be broken. I did a bunch of Django 1.9 patches (blind hacks looking at other fixes and upstream doc, really...) myself too > Or for that matter, anything > written in any other language that supports shared libraries. For shared libraries breaking ABI, we have "binNMU" that the release team can triggers (in fact, just a rebuild of packages). For API breakage of shared libs, we have "transitions" which the Debian release team manages. It's a major pain point. Last summer, Matthias Klose (aka Doko) upgraded gcc from version 4.9 to 5.x, which included (if I'm not mistaking) a breakage in libstdc++. This broke Debian unstable for more than a month, with hundreds of packages to fix. When all was fixed, everything migrated at once from unstable to testing (yes, that's the difference between Debian testing and unstable: testing isn't affected by library transition bugs). Oh, and a few month later, Doko decided it would be fun to upgrade Python 3 from 3.4 to 3.5! :) >>> I started the discussion to solicit ideas, but I would prefer to >>> avoid proposing what downstream should do because (a) I'm sure >>> folks want options and (b) I'm sure I'm not the person to identify >>> those options. >> >> Well, I'm doing the packaging, and I'm not sure if we have any option >> either, if 2 python dependencies are conflicting. > > Surely OpenStack is not the only time when that situation arises? Of course not. I suppose you're asking what happens in distro in such cases, right? When the latest version of a given lib is uploaded, and some package break (either they can't be rebuild, or they have new bug, or anything else release critical), then they get auto-removed from Debian testing, and therefore, from the next Debian stable release if nothing is done. The *only* solution that there is, is to fix that one component that's not accepting the new version. After Django 1.9 was uploaded to Sid last December, a month and a half later (which is the normal delay), Horizon (and all its plugins) got therefore removed from Debian testing... until I uploaded the Mitaka final release of Horizon, including a not-yet-merged Django 1.9 compat patch. Oh, btw, horizon is producing loads of Django 1.10 deprecation warnings, so history may repeat itself. Except that this time, it may be the last time before the Debian 9 (aka: Stretch) release... :/ >> I've done some of that, and used to be reactive on it. But mostly, the >> way it's been done so far is very good (thanks to a lot of folks working >> on it), and it's taking too much of my time to review them. So I just >> pick it up before packaging a (beta-)version of OpenStack. > > I think you've made my point here. Reviewing these changes *is* > time consuming. I'm kind of reviewing it: I'm subscribed to the requirements repository change, and receive everything in my mail. About once a week, I read the resulting emails, and sometimes, even go to look at the gerrit review (when the subject of the patch isn't helpful enough, which is unfortunately often the case). This doesn't help getting global-requirements.txt being reviewed, but this is super helpful for my packaging workflow, as I can see what new package there is to do. Cheers, Thomas Goirand (zigo) __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
