Sean, Before everything, I'd like to thank you for insisting in making the transition to SQLA 0.8.x.
Since it has been uploaded to Sid, this SQLA <0.7.99 has been without any doubt the biggest reoccurring pain in the but with the packaging of OpenStack. Without people like you, insisting again and again, I would have loose hope that progress could happen in OpenStack! So thanks again, Sean. On 01/03/2014 11:26 PM, Sean Dague wrote: > Given that sqla 0.9 just came out, I wanted to explore, again, what the > state of the world was with sqla 0.8 (especially given that Ubuntu and > Red Hat are both shipping 0.8 in their OpenStack bundles) - > https://review.openstack.org/#/c/64831/ > > The answer is not great. But more importantly, the answer is actually > worse than the last time we checked, which I think gives this some > urgency to move forward. For me, it's been urgent since the 6th of July... SQLA 0.8.2 was uploaded to Sid on the 6th of July. Since then, I've been bugging everyone on this list about it, explaining that I have no choice but to have Debian packages to support it (since I upload in Sid, and Sid has SQL 0.8.x). It seems I still haven't been heard. Now, we're 6 months after that, and after the release of Havana which happened more than 3 months after this, and after everything was fixed in all core packages (the last one was heat, at the end of August). So, in January 2014, I'm still fixing manually most requirements.txt, which still advertize for support of only SQL <0.7.99. I currently have fixes-SQLAlchemy-requirement.patch in Cinder, Neutron, and Nova (for Havana), just to allow it and stop the software to break because it was decided it is the case, even though they perfectly work with SQLA 0.8. Some other project have SQLAlchemy>=0.7.8,<=0.7.99 in the requirements.txt, but do not break as badly as these 3 just because of the bad declaration that doesn't reflect the reality (that's the case for Heat Keystone and Glance, for example). Something is going extremely wrong here. And seeing the actual result of the SQLA transition, I am really leaning toward thinking we have a process problem. What I believe is wrong, is that instead of having project wide decisions imposing some constraints, we have leaf packages that do. Until absolutely all of OpenStack is ready and fixed, then there's no constraint applied. This is the thing that must change. It shouldn't be this way. It should be from top to bottom. While I do understand that we do need the gate to be able to continue working with all projects at any given time, we still must find a solution so that this kind of 6 months transition never happens again. This really goes against the culture that we have inside OpenStack, and our common belief that things must be able to move fast. On 01/04/2014 04:13 AM, Sean Dague wrote:> > Because of entry points any library that specifies any dependencies > that OpenStack components specify, at incompatible levels, means that > library effectively puts a hold on the rest of OpenStack and prevents > it from being able to move forward. That's exactly the kind of *very bad* habits that needs to stop in OpenStack. On 01/04/2014 04:13 AM, Sean Dague wrote: > The only other option is that libraries we own (stackforge / oslo), > for condition to be included in global- requirements, *can never* > specify a maximum version of any dependency (and I really mean > never), and can never specify a minimum greater than current global > requirements. PLEASE !!! Let's do this !!! :) On 01/04/2014 05:29 AM, Sean Dague wrote: > It actually tells us that a human, somewhere, decided that their > software did not work with some combination of other software Often it's even worse. Sometimes, a human decide that, just it case, the software will declare itself incompatible with some non-existent future version of another software, even if there's no way to know. We're even more into sci-fi when we see stuff like: pbr>=0.5.21,<1 Monty, did you decide you would release 1.0 with lots of backward incompatibility? Has the topic been raised and I missed it??? I'm convinced this isn't the case (and let's pretend it isn't, just until the end of this message). So, how does one know that the thing he's using in PBR will be the thing that will cause trouble later on? For a version which hasn't been released yet?!? Who's the person with such a looking glass, who can predict the future? I'd like to know as well some stuff in my personal future... Please, let's stop this kind of non-sense and stop pretending we can tell if something is incompatible with a version of something else that doesn't even exist yet. It's hurting and slowing down the whole of OpenStack for no reason. > One of the key problems is taskflow, which has an sqla pin, which breaks > all the cinder entry points. This was actually entirely the problem > global requirements was meant to address, but it really can't when there > are nested requirements like this, in stackforge projects that aren't > installing via git (so we can rewrite their requirements). > > So why does taskflow have the SQLA pin? And if the answer is global > requirements, then taskflow can't be installing via pypi like it is now, > because it's by nature taking a wedge. So we need a real solution here > to un bind us, because right now it's actually impossible to upgrade > sqla in requirements because of taskflow. A colleague and myself have been struggling with the packaging of taskflow. Whatever we do with its dependencies, there's some unit tests errors. One of them is that it can't find the "futures" module: DistributionNotFound: futures>=2.1.3 That's weird because I have python-concurrent.futures 2.1.3-1~bpo70+1 installed on my system. There's also lots of: RuntimeError: No 'taskflow.engines' driver found, looking for 'parallel' or RuntimeError: No 'taskflow.engines' driver found, looking for 'default' which we can't figure out the reason (but probably related to the detection of futures missing...). Though we do have python-concurrent.futures installed. Is this really the module that taskflow expects, or does it really want "futures" and not concurrent.futures? If it really wants "futures" then we have a problem, because python-futures (which isn't in Debian yet) would conflict with concurrent.futures (they would share the same files). I'd be very happy to have upstream point of view, and probably some help. Cheers, Thomas Goirand (zigo) _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
