I’m coordinating with Victor to do a demo on http://join.me/dellcrowbar at 11:30<http://join.me/dellcrowbar%20at%2011:30> Central time. 30 minute time limit.
Please join us! From: crowbar-bounces On Behalf Of Victor Lowther Sent: Monday, September 09, 2013 8:56 AM To: Dearborn, Chris Cc: crowbar Subject: Re: [Crowbar] Crowbar 2.0 Status Report On Mon, Sep 9, 2013 at 8:21 AM, <christopher_dearb...@dell.com<mailto:christopher_dearb...@dell.com>> wrote: Thanks for the great information Victor! I have a few questions that hopefully you’ll be able to answer. See below… Thanks, Chris Dell Inc From: crowbar-bounces On Behalf Of Victor Lowther Sent: Saturday, September 07, 2013 7:56 PM To: crowbar Subject: [Crowbar] Crowbar 2.0 Status Report As of the latest set of pushes to development release, Crowbar 2.0 is now able to deploy an admin node that can then boot other nodes into Sledgehammer. Unlike the last time CB2.0 was able to do this, the Crowbar framework is performing all the work needed to bootstrap itself into usefulness without manual intervention. With the current codebase, you should be able to: * Build a Crowbar ISO using the usual ./dev build --os ubuntu-12.04 method. * Boot a VM (or real machine, if you have one to spare) to the generated ISO, and let it install * Once the admin node is installed, log in as crowbar/crowbar, sudo -i to root, and run /opt/dell/bin/install-crowbar your.admin.fqdn * Watch the script output with screen -r * Once screen terminates, log in to the Crowbar webui at http://192.168.124.10:3000 with crowbar/crowbar * Poke around all the new UI bits and see all the new shininess * Boot other nodes into sledgehammer and then watch them drop to a login prompt instead of doing something useful While that does not sound like much on the surface, that Rob and I have been able to go from just having a working web UI and test suite to having most of the key bits in place to act as a Crowbar admin node over the last 3 weeks in our spare time validates several of the key design features of CB 2.0, which I will go over in more detail below. Design difference between CB 1.x and CB 2.0: * Shifting from coarse-grained linear semi-implicit dependencies between barclamps to having a proper dependency graphs for roles and node-role bindings. In Crowbar 1.x, the core Crowbar framework has several partially-overlapping ad-hoc mechanisms for determining when it should do what, and they are all driven by having barclamp authors assigning integer priorities to barclamps, chef roles, configuration data, and hacks and monkey patching to work around where they do not play nicely together. Since the rules were informal and never documented, the only way to determine when things would happen is via experimenting with priority numbers until things worked the way you expected them to, or adding yet another hack if you could not get things to work properly. In Crowbar 2.0, barclamp writers only need to explicitly declare the dependencies on other roles their roles have, and make sure their declarations do not make the role dependency graph circular, which is easier to reason about. The Crowbar framework then uses the dependency graph between roles to build the graph between node-role bindings as the cluster is built out to ensure that things always happen in the right order. * Shifting the primary emphasis in interacting with Crowbar from interactions at a barclamp level to interactions at the role and node-role binding level. In Crowbar 1.x, virtually all of the interaction between the user and Crowbar happens in terms of barclamp level proposals. In Crowbar 2.0, that responsibility is divided into 3 different spheres of responsibility: * Roles, which represent a specific bundle of capabilities that can be attached to a node. For instance, dns-server is the role in Crowbar 2.0 that allows a server to act as a DNS server. * Deployments, which act as a collection of nodes along with a default set of role configuration information for any roles that may be bound to a node in the deployment. Deployments will be hierarchical, and there is a system deployment that the Crowbar framework manages at the root of that hierarchy. All newly-discovered nodes will be added to the system deployment to get their initial node-role bindings. So Deployments are containers in which nodes reside, and a Deployment can contain other Deployments? Are node roles considered to reside in the Deployment that contains their associated nodes and roles? Yes and no. All the noderoles together make up the noderole graph, which by itself does not care about deployments. However, when a noderole is created (and gets its initial configuration) and bound to the noderole graph, it is always created in the context of a deployment, which determines what configuration it will get and determines what noderoles it will be bound to to satisfy the dependencies of the role it is binding to the node. * Node-role bindings (called noderoles for short -- please suggest a better name for this!), which represent a specific instance of a role bound to a node. Noderoles have their own configuration in addition to the role configuration at a deployment level and the default role configuration. What is the configuration that a node role owns? Would this configuration be user visible, or something that would be set by the barclamp associated with the role? The node role owns any configuration data that is specific to that binding of a role to a node. As an example, every node in Crowbar will have a noderole binding the network-admin role to it, and that noderole binding will be used to carry the addresses and conduit information from the Crowbar framework to the network recipe when it executes on the node. For now, all of that data is editable by both the user and by the Crowbar framework. The other thing is that barclamps in CB2.0 are mostly just a way to package related roles their recipes/scripts/whatever, and whatever glue is needed to tie them in to the core Crowbar framework. They do not have configuration data of their own. * Not relying on Chef as a foundational component. Crowbar 1.x relies on Chef 10.x for everything, including storing all of the information needed to run the Crowbar framework. This meant that any failure on the part of Chef had catastrophic consequences on Crowbar. Crowbar 2.0 is a standalone application that does not rely on Chef to store its configuration information. Where is Crowbar storing its information in lieu of Chef? sqlite for now, because it is the easiest thing to bootstrap with. * Not relying on Chef as the only way to effect change on a node. In Crowbar 2.0, we have the concept of a jig as a component that can be used to effect change on another node. Support for using Chef in Crowbar 2.0 is provided by the Chef jig, and Crowbar 2.0 also has a Script jig that is used for bootstrapping. Every role declares what jig must be used to do what it needs on a node. Say there are 3 different orchestration engines that have jigs. As a barclamp developer, I would imagine I would want to write a barclamp, and provide the equivalent of Chef recipes for each of the jigs I wanted to support, the thought being that Crowbar would use whichever set of recipes it needed to based on what jig was present. Will this be possible? Each role declares what jig must be used to implement it. As a barclamp developer, you choose what jig will be used to implement the roles that will be part of that barclamp. There is no mechanism for having more than one jig being able to implement a role -- adding that restriction gets rid of a whole slew of problems that it we really should not want to deal with for a .0 release. Next Steps for Crowbar 2.0 (in rough dependency order): * Make framework side of the Network barclamp operate properly, instead of working in rigged demo mode. Right now, the Crowbar framework side of things is just smart enough to allocate an IP to the admin node to exercise the network recipe and bootstrap the rest of the admin node roles. It needs to be able to track IP address allocations in a generic fashion, and it needs to be able to support other networks besides the admin network. * Flesh out node discovery. We can boot nodes into Sledgehammer, but we need to wire in more functionality to allow node discovery and configuration to take place. * Come up with a reasonable OS installation story, and implement it. * De-cruft the Crowbar-specific Chef cookbooks. While the current Chef cookbooks work with minimal changes, a side-effect of how the Crowbar framework manages configuration data via the node-role graph is that it is very easy for most of our core cookbooks to operate in an entirely attribute-driven fashion. Refactoring them and stripping out the CB 1.x specific cruft would be a good way to learn the framework. Questions? Comments? Flames? Pull requests? I am at your service.
_______________________________________________ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
