[Crowbar] Quantum fixes for Mesa-1.6 - Pebbles has similar issues
Vincent, The pull request Arkady mentioned on the community call this morning is: https://github.com/crowbar/barclamp-quantum/pull/108 -John T. ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
Re: [Crowbar] [pebbles] Allow passing an incomplete json when creating a proposal
I just looked at this again in the light of my current work on https://github.com/crowbar/barclamp-provisioner/pull/177 ... Vincent Untz (vu...@suse.com) wrote: > Hi, > > I just submitted this pull request, which I guess is worth discussing > here: > https://github.com/crowbar/barclamp-crowbar/pull/589 > > Here's a copy and paste from the commit message: > > === > Until now, when creating a proposal with some existing json, it was > required that the json contained a full proposal definition. Two big > downsides of this are: > > a) it's easy to break stuff by accident when changing a proposal > b) if the schema changes to add new mandatory attributes, people can > not simply re-use old json they had around; they have to update them > > This commit allows passing an incomplete json with only a subset of > attributes. It will take the default json, and overwrite any attribute > there with the ones from the incomplete json. > > Note that this is note using Chef::Mixin::DeepMerge::deep_merge! as a > deep merge is also merging arrays in attribute values, which is > something not desired here. The hash_only_merge! API is actually > available in Chef 11 in Chef::Mixin::DeepMerge, so this is just a > backport. Why is merging of arrays not desired? There are plenty of arrays scattered around the JSON schema: $ grep -c sequence barclamps/*/chef/data_bags/crowbar/bc-template-*.schema | sort -t: -k2 -nr barclamps/network/chef/data_bags/crowbar/bc-template-network.schema:9 barclamps/dns/chef/data_bags/crowbar/bc-template-dns.schema:8 barclamps/glance/chef/data_bags/crowbar/bc-template-glance.schema:7 barclamps/deployer/chef/data_bags/crowbar/bc-template-deployer.schema:7 barclamps/swift/chef/data_bags/crowbar/bc-template-swift.schema:6 barclamps/quantum/chef/data_bags/crowbar/bc-template-quantum.schema:6 barclamps/ntp/chef/data_bags/crowbar/bc-template-ntp.schema:6 barclamps/nova_dashboard/chef/data_bags/crowbar/bc-template-nova_dashboard.schema:6 barclamps/nova/chef/data_bags/crowbar/bc-template-nova.schema:6 barclamps/logging/chef/data_bags/crowbar/bc-template-logging.schema:6 barclamps/keystone/chef/data_bags/crowbar/bc-template-keystone.schema:6 barclamps/crowbar/chef/data_bags/crowbar/bc-template-crowbar.schema:6 barclamps/cinder/chef/data_bags/crowbar/bc-template-cinder.schema:6 barclamps/ceph/chef/data_bags/crowbar/bc-template-ceph.schema:6 barclamps/test/chef/data_bags/crowbar/bc-template-test.schema:5 barclamps/tempest/chef/data_bags/crowbar/bc-template-tempest.schema:5 barclamps/rabbitmq/chef/data_bags/crowbar/bc-template-rabbitmq.schema:5 barclamps/provisioner/chef/data_bags/crowbar/bc-template-provisioner.schema:5 barclamps/nagios/chef/data_bags/crowbar/bc-template-nagios.schema:5 barclamps/loadbalancer/chef/data_bags/crowbar/bc-template-loadbalancer.schema:5 barclamps/ipmi/chef/data_bags/crowbar/bc-template-ipmi.schema:5 barclamps/git/chef/data_bags/crowbar/bc-template-git.schema:5 barclamps/ganglia/chef/data_bags/crowbar/bc-template-ganglia.schema:5 barclamps/database/chef/data_bags/crowbar/bc-template-database.schema:5 ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
Re: [Crowbar] [pebbles] Allow passing an incomplete json when creating a proposal
Le jeudi 01 août 2013, à 16:44 +0100, Adam Spiers a écrit : > I just looked at this again in the light of my current work on > > https://github.com/crowbar/barclamp-provisioner/pull/177 > > ... > > Vincent Untz (vu...@suse.com) wrote: > > Hi, > > > > I just submitted this pull request, which I guess is worth discussing > > here: > > https://github.com/crowbar/barclamp-crowbar/pull/589 > > > > Here's a copy and paste from the commit message: > > > > === > > Until now, when creating a proposal with some existing json, it was > > required that the json contained a full proposal definition. Two big > > downsides of this are: > > > > a) it's easy to break stuff by accident when changing a proposal > > b) if the schema changes to add new mandatory attributes, people can > > not simply re-use old json they had around; they have to update them > > > > This commit allows passing an incomplete json with only a subset of > > attributes. It will take the default json, and overwrite any attribute > > there with the ones from the incomplete json. > > > > Note that this is note using Chef::Mixin::DeepMerge::deep_merge! as a > > deep merge is also merging arrays in attribute values, which is > > something not desired here. The hash_only_merge! API is actually > > available in Chef 11 in Chef::Mixin::DeepMerge, so this is just a > > backport. > > Why is merging of arrays not desired? There are plenty of arrays > scattered around the JSON schema: Because if the template has: "foo": [ 1 , 2 ] and your json file to merge with the template has: "foo": [ 7, 8 ] you don't expect the resulting json to be: "foo": [ 1, 2, 7, 8 ] You only expect to have [ 7, 8 ]. In more concrete terms, look at https://github.com/crowbar/barclamp-crowbar/blob/release/pebbles/master/chef/data_bags/crowbar/bc-template-crowbar.json#L9 and imagine we have "provisioner": [ "/var/lib/config/crowbar/provisioner.json" ] in the config file. You clearly don't expect to have a "default" in the resulting json. Now, if there are cases where merging arrays is desired, then we can add an option to make that possible. But I actually believe it's very unlikely. I was also considering removing the use of deep merge when saving nodes in favor of the hash_only_merge. But it's a bit risky, so didn't want to touch that :-) Cheers, Vincent -- Les gens heureux ne sont pas pressés. ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
Re: [Crowbar] Quantum fixes for Mesa-1.6 - Pebbles has similar issues
Hi, Le jeudi 01 août 2013, à 09:05 -0500, john_terps...@dell.com a écrit : > Vincent, > > The pull request Arkady mentioned on the community call this morning is: > https://github.com/crowbar/barclamp-quantum/pull/108 This pull request does many different things. Would it be possible to know the exact error we're trying to fix? (With logs :-)) Cheers, Vincent -- Les gens heureux ne sont pas pressés. ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
[Crowbar] Using Dell Equallogic with Openstack Cinder
Hello. Hoping to find some help with an issue in my attempt to use an Equallogic array for a storage back end for Cinder with OpenStack Grizzly. Since the Crowbar Grizzly release is not complete yet, we are trying to use the Rackspace Private Cloud installer tool and so far it has worked very well. Using it with a Ceph storage back end worked great but we want to see how things work with Equallogic. We have a fresh Equallogic running 6.0.2 release firmware and have installed the eqlx.py into the appropriate driver location for the Cinder setup. The config is all in place and we can start the Cinder volume service just fine. Creating and deleting volumes works as well. The problem comes when we try to actually use the Equallogic to actually have the controller node or compute node login to an iSCSI volume to put data on it (such as making a volume from an image). The log result is as follows: 2013-08-01 13:46:20ERROR [cinder.volume.manager] Error: ['Traceback (most recent call last): ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 250, in create_volume image_location) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 189, in _create_volume image_id) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 602, in _copy_image_to_volume image_id) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 362, in copy_image_to_volume iscsi_properties, volume_path = self._attach_volume( ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 398, in _attach_volume try: ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 304, in _run_iscsiadm *iscsi_command, run_as_root=True, ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 221, in _execute return self._run_ssh(command, timeout=FLAGS.eqlx_cli_timeout) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 73, in __inner res = gt.wait() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait return self._exit_event.wait() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait return hubs.get_hub().switch() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch return self.greenlet.switch() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main result = function(*args, **kwargs) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 253, in _run_ssh raise exception.Error(msg, out) ', "Error: (u'Error executing EQL command: stty columns 255', ['iscsiadm -m node -T iqn.2001-05.com.equallogic:0-8a0906-b0976c809-216005851fab-volume-25fea6db-dd20-4f8a-9099-f1d96c122b3d -p 10.64.0.5:3260', ' ^', 'Error: Bad command', 'OpenStack-InternalGroup> ']) "] I'm no programmer but from what I can tell, it seems the eqlx driver is somehow causing Cinder-volume to try to run the iscsiadm command at the end of that log entry via SSH on the Equallogic shell. Obviously that won't work but I can't figure out why it is happening. Any insights? Thanks ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
Re: [Crowbar] Using Dell Equallogic with Openstack Cinder
Jon, Why do you say " Obviously that won't work but I can't figure out why it is happening " ? The driver uses SSH to issue commands to the EQL. That's the API access path for the EQL driver (and a common approach for other Cinder drivers) Rob -Original Message- From: crowbar-bounces On Behalf Of Jon Bayless Sent: Thursday, August 01, 2013 4:47 PM To: crowbar Subject: [Crowbar] Using Dell Equallogic with Openstack Cinder Hello. Hoping to find some help with an issue in my attempt to use an Equallogic array for a storage back end for Cinder with OpenStack Grizzly. Since the Crowbar Grizzly release is not complete yet, we are trying to use the Rackspace Private Cloud installer tool and so far it has worked very well. Using it with a Ceph storage back end worked great but we want to see how things work with Equallogic. We have a fresh Equallogic running 6.0.2 release firmware and have installed the eqlx.py into the appropriate driver location for the Cinder setup. The config is all in place and we can start the Cinder volume service just fine. Creating and deleting volumes works as well. The problem comes when we try to actually use the Equallogic to actually have the controller node or compute node login to an iSCSI volume to put data on it (such as making a volume from an image). The log result is as follows: 2013-08-01 13:46:20ERROR [cinder.volume.manager] Error: ['Traceback (most recent call last): ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 250, in create_volume image_location) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 189, in _create_volume image_id) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 602, in _copy_image_to_volume image_id) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 362, in copy_image_to_volume iscsi_properties, volume_path = self._attach_volume( ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 398, in _attach_volume try: ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 304, in _run_iscsiadm *iscsi_command, run_as_root=True, ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 221, in _execute return self._run_ssh(command, timeout=FLAGS.eqlx_cli_timeout) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 73, in __inner res = gt.wait() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in wait return self._exit_event.wait() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait return hubs.get_hub().switch() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch return self.greenlet.switch() ', ' File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in main result = function(*args, **kwargs) ', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/eqlx.py", line 253, in _run_ssh raise exception.Error(msg, out) ', "Error: (u'Error executing EQL command: stty columns 255', ['iscsiadm -m node -T iqn.2001-05.com.equallogic:0-8a0906-b0976c809-216005851fab-volume-25fea6db-dd20-4f8a-9099-f1d96c122b3d -p 10.64.0.5:3260', ' ^', 'Error: Bad command', 'OpenStack-InternalGroup> ']) "] I'm no programmer but from what I can tell, it seems the eqlx driver is somehow causing Cinder-volume to try to run the iscsiadm command at the end of that log entry via SSH on the Equallogic shell. Obviously that won't work but I can't figure out why it is happening. Any insights? Thanks ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/ ___ Crowbar mailing list Crowbar@dell.com https://lists.us.dell.com/mailman/listinfo/crowbar For more information: http://crowbar.github.com/
Re: [Crowbar] [pebbles] Allow passing an incomplete json when creating a proposal
RE: CB2 - it's a different model in which we've already decomposed the proposal
components. There's no reason to consider a forward port.
I agree it's handy but I will reiterate that _changing Crowbar 1.x core code
risks unexpected side-effects_.
My recommendation is to minimize code change here unless it's specifically
related to a defect or blocking item. IMHO, this is neither.
-Original Message-
From: crowbar-bounces On Behalf Of Adam Spiers
Sent: Monday, July 22, 2013 5:05 AM
To: crowbar
Subject: Re: [Crowbar] [pebbles] Allow passing an incomplete json when creating
a proposal
Vincent Untz (vu...@suse.com) wrote:
> I just submitted this pull request, which I guess is worth discussing
> here:
> https://github.com/crowbar/barclamp-crowbar/pull/589
>
> Here's a copy and paste from the commit message:
>
> ===
> Until now, when creating a proposal with some existing json, it was
> required that the json contained a full proposal definition. Two big
> downsides of this are:
>
> a) it's easy to break stuff by accident when changing a proposal
> b) if the schema changes to add new mandatory attributes, people can
> not simply re-use old json they had around; they have to update them
>
> This commit allows passing an incomplete json with only a subset of
> attributes. It will take the default json, and overwrite any attribute
> there with the ones from the incomplete json.
>
> Note that this is note using Chef::Mixin::DeepMerge::deep_merge! as a
> deep merge is also merging arrays in attribute values, which is
> something not desired here. The hash_only_merge! API is actually
> available in Chef 11 in Chef::Mixin::DeepMerge, so this is just a
> backport.
>
> For all that matters, this is backwards-compatible.
> ===
>
> The main benefit I see is that it makes it really easy to use a json
> like this to create a database proposal using postgresql:
>
> {
> "id": "mydbproposal",
> "attributes": {
> "database": {
> "sql_engine": "postgresql"
> }
> }
> }
>
> All the missing attributes will be fetched from the database databag.
>
> Any objection?
This sounds really nice to me - it cleanly separates out what the admin wants
to be different from the defaults, rather than mixing those values with the
defaults. This should make upgrades smoother in the future as you say,
although I expect a forward-port to CB2.0 is non-trivial. Thanks!
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/
___
Crowbar mailing list
Crowbar@dell.com
https://lists.us.dell.com/mailman/listinfo/crowbar
For more information: http://crowbar.github.com/
