On 07/10/2012 04:23 PM, Matt Ray wrote: > Bluntness appreciated, this process is already in motion. > http://opscode.com/openstack was launched 2 weeks ago and I promptly > left for conferences and vacation. I am consolidating GitHub repos > here:
Awesome. > https://github.com/opscode/openstack-chef-repo > https://github.com/opscode-cookbooks/nova > https://github.com/opscode-cookbooks/glance > https://github.com/opscode-cookbooks/horizon > https://github.com/opscode-cookbooks/keystone > https://github.com/opscode-cookbooks/swift > > Work is being done in my own repos until it's ready for an initial > release, which will include a Getting Started with Chef and OpenStack > document. > https://github.com/mattray/ > > I'm working with quite a few folks already, Rackspace, Dell, DreamHost > and others and Intel is sponsoring this work. > > Jay and I chatted a bit in IRC, we're quite aligned in how we plan on > working this and the goal will be to get > github.com/openstack/chef-repo gated with Gerrit and CI and pulling > from Opscode's repos soon. Only really replying because I saw the word gated. :) I'd love to be part of any conversations that are being had on this subject, sooner rather than later. > Please feel free to join the discussion on > our new mailing list focused on this effort here: > http://groups.google.com/group/opscode-chef-openstack > > And an IRC channel: > #openstack-chef on irc.freenode.net > > Thanks, > Matt Ray > Senior Technical Evangelist | Opscode Inc. > m...@opscode.com | (512) 731-2218 > Twitter, IRC, GitHub: mattray > > > On Tue, Jul 10, 2012 at 11:25 AM, Jay Pipes <jaypi...@gmail.com> wrote: >> Apologies in advance for my blunt and somewhat dour response, Matt. I'm >> not singling you out at all, and I know you've tried your best to get >> the various Chef stakeholders to work together. Also apologies for >> top-posting, but there's not a whole lot of use inline posting this. >> >> tl;dr >> ----- >> >> We need to stop the needless fracturing of the operational knowledge of >> the Chef community and try working as a team towards some common goals >> instead of creating fork after fork of repos of Chef cookbooks. >> >> There is a ton of wasted effort in this area. >> >> Proposal: >> >> * Get our act together and treat Chef repos (and puppet/juju/whatever) >> as we do other OpenStack core and supporting projects -- use Gerrit, use >> a CI gating system, do real code reviews on it, and in general treat >> them as a supporting OpenStack project >> * Mark ALL Chef repos that are not currently maintained with an OBSELETE >> marker and/or DELETE the repo on Github >> * Consolidate all *cookbooks* into a repository in >> github.com/openstack/chef-repo >> * Use git submodules to manage cookbooks that are upstreamed in >> github.com/opscode/ that have little to no changes in them >> * Actually fix the documentation of the dang cookbooks -- right now, >> half of them include the documentation from the memcache cookbook, as >> they were lazily copy-pasted around, or the standard example doc that is >> created when using something like knife. >> * Put as much variation in deployment philosophy into *roles* and >> attribute overrides/defaults >> >> More/Rant/Details >> ----------------- >> >> Maybe it's just the open source developer in me, but I don't understand >> why there is such an aversion to coordination in the deployment/ops >> community around the scripts and deployment >> cookbooks/modules/charms/whatever. >> >> Is it that everyone has a different idea of what is best? Is it because >> deployers/ops folks think that coordinating with other contributors is >> too time-consuming? Is it because the chef repos are not controlled in >> the same way as, say, devstack or the core projects, with an automated >> patch queue manager and code review system that actually encourages >> debate over patches? A combination of all of the above? >> >> Over the last 2 years, I've worked at 3 companies in the OpenStack >> ecosystem. All three companies had their own repos of Chef cookbooks >> (still do to this day). 50-60% of the content of these cookbooks is >> identical. 10-20% of the content of these cookbooks is different -- but >> only slightly or cosmetically. And a good portion of the remaining >> 20-40% are differences that are incorrectly (IMHO) placed in the >> cookbooks and recipes instead of where they should be: in roles and >> environments, with cookbooks created that deal with variations in >> deployments with attributes and the occasional if/else block. >> >> In trying to determine the appropriate Chef repo to use for the TryStack >> project, we found the following repo: >> >> https://github.com/osops/chef-repo >> >> to have the most up-to-date. I've since been told this repo is no longer >> maintained. This is very frustrating, not because of this particular >> repo, but because this is just one in a long line of neglected and >> forgotten forks of chef cookbook repositories. The fact that the default >> Chef behaviour and Opscode documentation encourages the copy/pasting of >> cookbooks all over the place and GitHub itself encourages the random and >> promiscuous forking of repos doesn't help. >> >> Let's get real about the deployment/ops code and >> cookbooks/modules/charms. Let's treat them the same way we do code in >> the core projects and supporting projects. >> >> Thanks for your time, >> -jay >> >> On 07/10/2012 11:42 AM, Matt Ray wrote: >>> Just a heads up, I'm working on building unified community-driven >>> cookbooks over in https://github.com/opscode/openstack-chef-repo (and >>> repos for the individual cookbooks). These are forked from Rackspace's >>> cookbooks and I'm working with them and others to make reusable, >>> well-documented and supported Chef cookbooks for OpenStack. I'll make >>> a larger announcement around them once I have a working quickstart >>> document for them. >>> >>> tl;dr; https://github.com/opscode/openstack-chef-repo >>> >>> Thanks, >>> Matt Ray >>> Senior Technical Evangelist | Opscode Inc. >>> m...@opscode.com | (512) 731-2218 >>> Twitter, IRC, GitHub: mattray >>> >>> >>> On Tue, Jul 10, 2012 at 10:22 AM, Jay Pipes <jaypi...@gmail.com> wrote: >>>> Gah... probably would be good if you guys either shut down the repo or >>>> made a big notice on the README then :( >>>> >>>> -jay >>>> >>>> On 07/09/2012 05:25 PM, Joe Breu wrote: >>>>> Hi Jay, >>>>> >>>>> The chef cookbooks at https://github.com/osops are no longer maintained. >>>>> Our current cookbooks are at https://github.com/rcbops/chef-cookbooks >>>>> >>>>> >>>>> >>>>> --- >>>>> Joseph Breu >>>>> Deployment Engineer >>>>> Rackspace Cloud Builders >>>>> 210-312-3508 >>>>> >>>>> On Jul 9, 2012, at 8:01 AM, Jay Pipes wrote: >>>>> >>>>>> Vish and Ron, just getting back to this... see inline continued >>>>>> questions for you both. >>>>>> >>>>>> On 07/02/2012 04:24 PM, Vishvananda Ishaya wrote: >>>>>>> >>>>>>> On Jul 2, 2012, at 7:28 AM, Jay Pipes wrote: >>>>>>> >>>>>>>> Hi Ron, cc'ing the openstack ML for extra eyes and opinions... >>>>>>>> >>>>>>>> So, Nati and I are looking to use either the osops chef-repo or >>>>>>>> something similar as the basis of the new TryStack zone chef >>>>>>>> deployment. >>>>>>>> I've been going through the recipes and roles and I have a question on >>>>>>>> the nova-compute *role*: >>>>>>>> >>>>>>>> https://github.com/osops/chef-repo/blob/master/roles/nova-compute.rb >>>>>>>> >>>>>>>> I'm wondering why the nova-api recipe is in the runlist for >>>>>>>> nova-compute? >>>>>>> >>>>>>> Because metadata needs to run on the compute hosts in the default >>>>>>> layout. This should >>>>>>> be switched to use nova-api-metadata instead of nova-api, but the >>>>>>> split out hasn't been done yet. >>>>>> >>>>>> OK, I will work on splitting this out a bit more effectively. >>>>>> >>>>>> One additional question, though. In opening up the >>>>>> /cookbooks/nova/recipes/nova/compute.rb file, you will notice this at >>>>>> the top: >>>>>> >>>>>> include_recipe "nova::api" >>>>>> >>>>>> Therefore, unless I am mistaken, the nova-compute *role*'s runlist >>>>>> actually doesn't need to contain both nova-api AND nova-compute since >>>>>> apparently the nova-compute *recipe* already includes all of the >>>>>> nova-api recipe. >>>>>> >>>>>> Would you agree with that? >>>>>> >>>>>>>> In addition, I was wondering if y'all had considered making more use of >>>>>>>> roles instead of recipes to allow most of the attribute assignment and >>>>>>>> variation to be in the combination of roles assigned to a host, instead >>>>>>>> of recipes combined in a role? >>>>>>>> >>>>>>>> For example, right now, there is a "nova-controller" role that looks >>>>>>>> like this: >>>>>>>> >>>>>>>> name "nova-controller" >>>>>>>> description "Nova controller node (vncproxy + rabbit)" >>>>>>>> run_list( >>>>>>>> "role[base]", >>>>>>>> "recipe[nova::controller]" >>>>>>>> ) >>>>>>>> >>>>>>>> Because most of the special sauce is in the nova::controller recipe, I >>>>>>>> have to go into that recipe to see what the role is composed of: >>>>>>>> >>>>>>>> include_recipe "mysql::server" >>>>>>>> include_recipe "openssh::default" >>>>>>>> >>>>>>>> include_recipe "rabbitmq::default" >>>>>>>> include_recipe "keystone::server" >>>>>>>> include_recipe "glance::registry" >>>>>>>> include_recipe "glance::api" >>>>>>>> include_recipe "nova::nova-setup" >>>>>>>> include_recipe "nova::scheduler" >>>>>>>> include_recipe "nova::api" >>>>>>>> >>>>>>>> if platform?(%w{fedora}) >>>>>>>> # Fedora skipping vncproxy for right now >>>>>>>> else >>>>>>>> include_recipe "nova::vncproxy" >>>>>>>> end >>>>>>>> >>>>>>>> But what this recipe really is is an opinionated description of a >>>>>>>> "controller role". If the role was, instead, structured like so: >>>>>>>> >>>>>>>> name "nova-controller" >>>>>>>> description "Nova Controller - All major API services and control >>>>>>>> servers like MySQL and Rabbit" >>>>>>>> run_list( >>>>>>>> "role[base]", >>>>>>>> "recipe[mysql::server]", >>>>>>>> "recipe[openssh::default]", >>>>>>>> "recipe[rabbitmq::default]", >>>>>>>> "recipe[keystone::server]", >>>>>>>> "recipe[glance::api]", >>>>>>>> "recipe[glance::registry]", >>>>>>>> "recipe[nova::scheduler]", >>>>>>>> "recipe[nova::api]", >>>>>>>> ) >>>>>>>> >>>>>>>> Then the deployer can more easily switch up the way they deploy >>>>>>>> OpenStack servers by merely changing the role -- say, removing the >>>>>>>> Rabbit service and putting it somewhere else -- WITHOUT having to >>>>>>>> modify >>>>>>>> a recipe in a Git submodule in the upstream cookbooks. >>>>>>>> >>>>>>>> Furthermore, if we broke out more roles -- such as "control-services" >>>>>>>> which might be MySQL and Rabbit only -- than we could make the "super >>>>>>>> roles" ,like the nova-controller role above, more of a "put this and >>>>>>>> that role together, like so: >>>>>>>> >>>>>>>> name "nova-controller" >>>>>>>> description "Nova Controller - All major API services and control >>>>>>>> servers like MySQL and Rabbit" >>>>>>>> run_list( >>>>>>>> "role[base]", >>>>>>>> "role[control_services]", >>>>>>>> "recipe[keystone::server]", >>>>>>>> "recipe[glance::api]", >>>>>>>> "recipe[glance::registry]", >>>>>>>> "recipe[nova::scheduler]", >>>>>>>> "recipe[nova::api]", >>>>>>>> ) >>>>>>> >>>>>>> This all makes sense to me. Ron? >>>>>> >>>>>> Ron, any disagreement here? >>>>>> >>>>>>>> Finally, I've noticed that there are aren't any HA options in the osops >>>>>>>> recipes -- specifically around MySQL. Are there plans to do so? We use >>>>>>>> heartbeat/Pacemaker/DRBD in the original TryStack cookbooks [1] and >>>>>>>> environments to get simple HA solutions up and would love to see those >>>>>>>> in the upstream. >>>>>> >>>>>> Either of you, any thoughts on this front? >>>>>> >>>>>> Thanks! >>>>>> -jay >>>>>> >>>>>>>> Thanks and all the best guys, >>>>>>>> -jay >>>>>>>> >>>>>>>> [1] >>>>>>>> https://github.com/trystack/openstack-chef/tree/stable/diablo/cookbooks/heartbeat >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> <mailto:openstack@lists.launchpad.net> >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Mailing list: https://launchpad.net/~openstack >>>> Post to : openstack@lists.launchpad.net >>>> Unsubscribe : https://launchpad.net/~openstack >>>> More help : https://help.launchpad.net/ListHelp >> > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
