On 4/15/16, 6:38 PM, "Clark Boylan" <cboy...@sapwetik.org> wrote:
>On Fri, Apr 15, 2016, at 02:39 PM, Steven Dake (stdake) wrote: >> Hey fellow OpenStackers, >> >> Kolla 1.0.0 has a fatal flaw in its design related to data containers. >> The details are outlined here: >> >>http://docs.openstack.org/developer/kolla/liberty-deployment-warning.html >> >> Our plan to rectify this involves taking what is in stable/mitaka and >> making it stable/liberty and placing 1-3 patches on top of >>stable/liberty >> to make it deploy the liberty version of openstack. These 1-3 patches >> include metadata like repo locations, upstream tarball locations, etc. >> No actual code. This is a one time thing; in the future we will be >> following a strict bug-backport only policy. >> >> Our options include: >> 1. make a megapatch diff of liberty vs mitaka and merge that, with a >> patch on top to fix the repos >> Example here: >> https://review.openstack.org/#/c/306625/ >> >> 2. Tag the tip of stable/liberty with liberty-early-demise, delete the >> liberty branch, then create a new stable/liberty branch from the tip of >> stable/mitaka >> >> 3. Tag the tip of stable/liberty with liberty-early-demise, and run git >> reset -hard origin/stable/mitaka outside of gerrit. Tonyb says our >>setup >> prevents non-fast-forwarded pushes so this might not be viable. >> >> This was tonyb's work here: >> http://paste.openstack.org/raw/494295/ >> >> What we want to avoid is jamming 1k+ patches through our CI system and >> having the core reviewers have to deal with acking all those patches, or >> overloading gerrit and breaking things there. >> >> I am far from a git wizard, and don't really know the best way to >> proceed, other than we must have a liberty 1.1.0 deliverable with our >> mitaka bits + liberty repo pointers. I'd really like to preserve >> history. What we need is for stable/liberty to be an exact duplicate of >> stable/mitaka codwise, while also preserving history (which option 1 >> doesn't do). >> >> Can the Kolla community get an ack on the options from the >>Infrastructure >> team one way or another, so I can get the ball rolling on our end and >> sync up with Doug on the plan? > >Is there some reason that the normal backport process used on all of the >other projects wouldn't work here? Backport the bug fixes you need from >the newer branches then release a 1.1.0. > >Clark Clark, Great question. Thanks for the response :) We have over 1000 patches delta between stable/mitaka and stable/liberty and some of them will conflict. But I guess we could do that and jam up the gerrit/jenkins with that and the core reviewers with approving all those backports. Unfortunately because of the critical flaw, we really can't just cherrypick ~30% of the commits - we need pretty much the whole repo intact. Its like a big onion unpeel. In essence we need a docker feature called named volumes to correct the critical flaw. Before we were using data containers which results in data loss. To gain access to named volumes with Ansible 1.9.4, we had to write our own docker module called kolla_docker.py. kolla_docker.py is different then the Ansible bulit-in module to integrate with Docker. The Ansible built-in module only works with docker 1.8.2(hard pin) and named volumes were introduced in Docker 1.9.0. Ansible Inc has stopped maintenance on Ansible 1.9 series and is focusing all of their efforts on Ansible 2.0. Porting our code base to Ansible 2.0 is planned for Newton but realistically its a 2-3 month job. IIUC Ansible 2.0 still has a hard pin on 1.8.2 when used with their Docker module - so we still prefer to use our own module to decouple Ansible's release cycle from the fast moving Docker upstream (6 y stream releases a year). Therefore, kolla-docker is a requirement to remove the hard pin and gain access to named volumes. We have approximately 300-500 (super rough guess) patches to port our code base to use this kolla_docker.py module. We really don't have any consistent way to identify the relevant patches. I have concerns with this type of backport approach, as we would invalidate all of the testing we have done to date with Mitaka and Liberty - it would basically be forking our own code base for one version of Kolla. It is actually much more complex then I've outlined above - there are other things in play here, involving how our diagnostics system works (and won't work with docker 1.9 and the current Liberty code) meaning we have to backport our diagnostics system intact as well. There are other parts of Kolla that fit into the same category - lots of interdependencies mostly centered around kolla_docker.py. Both Doug and Tony mentioned a backport of this type is not ideal, and a better solution should be found. I agree completely - at this point the viable options are listed above. I recognize the above approaches aren't how repos should be managed in OpenStack (or anywhere else for that matter). We have corrected our deficiencies with backport management this cycle. We are ready to go with bug-fix only backports from now into the far future because we fixed the pain with how Ansible handles Docker integration via kolla-docker.py. The mission of Kolla is to provide Operators a smooth experience deploying and operating OpenStack. We would be in violation of our mission in my opinion if we did only a partial backport of our repo. The only other alternative is to abandon the Liberty repository entirely, which has been discussed. We believe this would prematurely and permanently damage the Kolla project's viability in the eyes of Operators, and result in overall damage to OpenStack in general. We do plan to tag a 1.1.0 release as soon as one of the above approaches is executed - which we think is about 1-2 days of dev work if the repos were equivalent. I could see backporting individually using the standard process taking months and resulting in completely broken software. Our best option at this point is to take mitaka as is and point the tarball and repo files we depend on at the proper liberty locations. FWIW the liberty patch has already been produced (against master) - and while not complete, results in a working from-source deploy. A second patch needs to be developed to make from-binary deploys work. Reference: https://review.openstack.org/#/c/306630/ Hope the context helps. Regards -steve > _______________________________________________ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
