The alignment proposal sounds great, and would definitely help reduce redundancy.
However, it might be useful to define clear goals of the resulting deployment using these cookbooks. As an example - Looking at the anso recipes for swift - they appear to deploy a SAIO swift cluster. The Crowbar cookbook assumes a multi node deployment. Similarly for nova - the official cookbooks appear to focus only on flat networking (unless I'm missing something) while the Crowbar version supports multiple network configs ( e.g. Vlan). OTH, the official recipes support both MySQL and Postgres, while crowbar only supports MySQL. ( the above not intended to recommend brands of sliced bread ;) The above raises a few questions ( and I'm sure there might be more): - SAIO or multi node ? - possibly repeat of the above - are the cookbooks to be used beyond unit testing, or just serve as an example? - what coverage for the breadth of options ? Or stated differently - are the cookbooks prescriptive and opinionated about deployments, or flexible? - does the above apply just to openstack components, or 3rd party dependencies ? A. On Feb 6, 2012, at 11:53 PM, Vishvananda Ishaya <vishvana...@gmail.com> wrote: > > On Feb 6, 2012, at 6:37 PM, Jesse Andrews wrote: > >> I know that the RCB deploy team works with the Crowbar team on chef >> recipes for that project. >> >> Regarding the github.com/ansolabs & github.com/rcb recipes - I'll have >> to delegate to Vishy who worked on those. > > They were the basis of dan and matt's cookbooks, but they are now ancient > history. i've been using them as a repository for a few helper devstack > recipes, but waldon pulled those out into a separate repo so it is fine if we > torch them. > > >> >> Jesse >> >> On Mon, Feb 6, 2012 at 6:07 PM, Jay Pipes <jaypi...@gmail.com> wrote: >>> Hi Stackers, >>> >>> tl;dr >>> ----- >>> >>> There are myriad Chef cookbooks "out there" in the ecosystem and locked up >>> behind various company firewalls. It would be awesome if we could agree to: >>> >>> * Align to a single origin repository for OpenStack cookbooks >>> * Consolidate OpenStack Chef-based deployment experience into a single >>> knowledge base >>> * Have branches on the origin OpenStack cookbooks repository that align with >>> core OpenStack projects >>> * Automate the validation and testing of these cookbooks on multiple >>> supported versions of the OpenStack code base >>> >>> Details >>> ------- >>> >>> Current State of Forks >>> ====================== >>> >>> Matt Ray and I tried to outline the current state of the various OpenStack >>> Chef cookbooks this past Thursday, and we came up with the following state >>> of affairs: >>> >>> ** The "official" OpenStack Chef cookbooks ** >>> >>> https://github.com/openstack/openstack-chef >>> >>> These chef cookbooks are the ones maintained mostly by Dan Prince and Brian >>> Lamar and these are the cookbooks used by the SmokeStack project. The >>> cookbooks contained in the above repo can install all the core OpenStack >>> projects with the exception of Swift and Horizon. >>> >>> This repo is controlled by the Gerrit instance at review.openstack.org just >>> like other core OpenStack projects. >>> >>> However, these cookbooks DO NOT currently have a stable/diablo branch -- >>> they are updated when the development trunks of any OpenStack project merges >>> a commit that requires deployment or configuration-related changes to their >>> associated cookbook. >>> >>> Important note: it's easy for Dan and Brian to know when updates to these >>> cookbooks are necessary -- SmokeStack will bomb out if a >>> deployment-affecting configuration change hits a core project trunk :) >>> >>> These cookbooks are the ONLY cookbooks that contain stuff for deploying with >>> XenServer, AFAICT. >>> >>> ** NTT PF Lab Diablo Chef cookbooks ** >>> >>> https://github.com/ntt-pf-lab/openstack-chef/ >>> >>> So, NTT PF Lab forked the upstream Chef cookbooks back in Nov 11, 2011, >>> because they needed a set of Chef cookbooks for OpenStack that functioned >>> for the Diablo code base. >>> >>> While Nov 11, 2011, is not the *exact* date of the Diablo release, these >>> cookbooks do in fact work for a Diablo install -- Nati Ueno is using them >>> for the FreeCloud deployment so we know they work... >>> >>> ** OpsCode OpenStack Chef Cookbooks ** >>> >>> Matt Ray from OpsCode created a set of cookbooks for OpenStack for the >>> Cactus release of OpenStack: >>> >>> https://github.com/mattray/openstack-cookbooks >>> http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef >>> >>> These cookbooks were forked from the Anso Labs' original OpenStack cookbooks >>> from the Bexar release and were the basis for the Chef work that Dell did >>> for Crowbar. Crowbar was originally based on Cactus, and according to Matt, >>> the repositories of OpenStack cookbooks that OpsCode houses internally and >>> uses most often are Cactus-based cookbooks. (Matt, please correct me if I am >>> wrong here...) >>> >>> ** Rackspace CloudBuilders OpenStack Chef Cookbooks ** >>> >>> The RCB team also has a repository of OpenStack Chef cookbooks: >>> >>> https://github.com/cloudbuilders/openstack-cookbooks >>> >>> Now, GitHub *says* that these cookbooks were forked from the official >>> upstream cookbooks, but I do not think that is correct. Looking at this >>> repo, I believe that this repo was *actually* forked from the Anso Labs >>> OpenStack Chef Cookbooks, as the list of cookbooks is virtually identical. >>> >>> ** Anso Labs OpenStack Chef Cookbooks ** >>> >>> These older cookbooks are in this repo: >>> >>> https://github.com/ansolabs/openstack-cookbooks/tree/master/cookbooks >>> >>> Interestingly, this repo DOES contain a cookbook for Swift. >>> >>> Current State of Documentation >>> ============================== >>> >>> Documentation for best practices on using Chef for your OpenStack >>> deployments is, well, a bit scattered. Matt Ray has some good information on >>> the README on his cookbook repo and the OpsCode wiki: >>> >>> https://github.com/mattray/openstack-cookbooks/blob/cactus/README.md >>> http://wiki.opscode.com/display/chef/Deploying+OpenStack+with+Chef >>> >>> But it is unfortunately not going to help people looking to deploy Diablo >>> and later versions of OpenStack. >>> >>> Most of the other repos contain virtually no documentation on using the >>> cookbooks or how they are written. >>> >>> I have a suspicion that one of the reasons that there has been such a >>> proliferation of cookbooks has been the lack of documentation pointing >>> people to an appropriate repo, how to use the cookbooks properly, and what >>> the best practices for deployment are. That, and the fact that folks are >>> just trying to stand up complex clouds and Get Things Done, and >>> documentation is annoying to write ;) >>> >>> Proposal for Alignment >>> ====================== >>> >>> I think the following steps would be good to get done by the time Essex >>> rolls out the door in April: >>> >>> 1) Create a stable/diablo branch of the openstack/openstack-chef cookbook >>> repo and maintain it in the same way that we maintain stable branches for >>> core OpenStack projects. I propose we use the branch point that NTT PF Lab >>> used to create their fork of the upstream repo. >>> >>> 2) Work with Matt Ray and other Chef experts to combine any and all best >>> practices that may be contained in the non-official cookbook repos into the >>> upstream official repository. From a cursory overview, there are some >>> differences in how databags are handled, how certs are handled, how certain >>> cookbooks are constructed, and of course differences in the actual cookbooks >>> in the repos themselves. >>> >>> 3) Consolidate documentation on how to use the cookbooks, the best practices >>> used in constructing the cookbooks, and possibly some videos/tutorials >>> walking folks through this critical piece of the OpenStack puzzle. >>> >>> 4) Create Jenkins builders for stable branch deployment testing. We >>> currently test the official development cookbooks by way of SmokeStack gates >>> on all core OpenStack projects. Would be great to get the same testing >>> automated for non-development branches of the cookbooks. >>> >>> Thoughts and criticism most welcome, and apologies in advance if I got any >>> of the above history wrong. Feel free to correct me! >>> >>> Best, >>> -jay >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : openstack@lists.launchpad.net >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack >> Post to : openstack@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp > > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
