For completeness sake here's what I ended up doing to resolve my ongoing issues with second Cinder node:
we're dealing with RHEL6.x here with Folsom from EPEL. For simplification of things both iptables and SELinux were dropped into permissive mode on both nodes (which I will be bringing back up right away) Multi-node Cinder ================= We will use 2 nodes: * master * openstack-keystone * openstack-cinder-api * openstack-cinder-scheduler * openstack-cinder-volume * qpidd * tgtd * ... * slave * openstack-cinder-volume * tgtd * Networking * 10.10.10.0/24 - Admin network * 10.10.10.1 - master * 10.10.10.2 - slave Note ---- Master node also hosts keystone and other core OpenStack services. Configuration ============= Make sure you're configuring your OpenStack infrastructure without the use of localhost and 127.0.0.1 references. Also make sure your tgt is configured appropriately (openstack-setup scripts will do it for you). For RHEL <6.4 :: if [ "$DISTRO_VER" -eq '6' ] ; then if [ "$DISTRO_REL" -le '3' ] ; then ## as per comment in /etc/tgt/conf.d/cinder.conf ## while not flexible or smart - it should do the trick grep -F 'include /etc/cinder/volumes/*' /etc/tgt/targets.conf || \ sed -i '0 i ##OPENSTACK-SETUP: \ include /etc/cinder/volumes/* \ ' /etc/tgt/targets.conf fi Master ------ /etc/cinder/cinder.conf ~~~~~~~~~~~~~~~~~~~~~~~ First, the contents:: [DEFAULT] logdir = /var/log/cinder state_path = /var/lib/cinder lock_path = /var/lib/cinder/tmp volumes_dir = /etc/cinder/volumes pybasedir=/usr/lib/python2.6/site-packages # Directory where cinder binaries are installed (string value) bindir=/usr/bin my_ip = 10.10.10.1 iscsi_helper = tgtadm iscsi_ip_prefix= 10.10.10.1 iscsi_ip_address= 10.10.10.1 sql_connection = mysql://cinder:cinder@localhost/cinder rpc_backend = cinder.openstack.common.rpc.impl_qpid rootwrap_config = /etc/cinder/rootwrap.conf auth_strategy = keystone [keystone_authtoken] admin_tenant_name = service admin_user = cinder admin_password = cinder auth_host = 10.10.10.1 auth_port = 35357 auth_protocol = http signing_dirname = /tmp/keystone-signing-cinder /etc/cinder/api-paste.ini ~~~~~~~~~~~~~~~~~~~~~~~~~ We'll skip the boring part and look at filter:authtoken:: .... [filter:authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory service_protocol = http service_host = 10.10.10.1 service_port = 5000 Slave ----- /etc/cinder/cinder.conf ~~~~~~~~~~~~~~~~~~~~~~~ Content is slightly different from Master:: [DEFAULT] ## See more options at: ## https://github.com/openstack/cinder/blob/master/etc/cinder/cinder.conf.sample logdir = /var/log/cinder state_path = /var/lib/cinder lock_path = /var/lib/cinder/tmp volumes_dir = /etc/cinder/volumes pybasedir=/usr/lib/python2.6/site-packages # Directory where cinder binaries are installed (string value) bindir=/usr/bin my_ip = 10.10.10.2 iscsi_helper = tgtadm iscsi_ip_prefix= 10.10.10.2 iscsi_ip_address= 10.10.10.2 sql_connection = mysql://cinder:cinder@10.10.10.1/cinder rpc_backend = cinder.openstack.common.rpc.impl_qpid # Qpid broker hostname (string value) qpid_hostname=10.10.10.1 rootwrap_config = /etc/cinder/rootwrap.conf auth_strategy = keystone [keystone_authtoken] admin_tenant_name = service admin_user = cinder admin_password = cinder auth_host = 10.10.10.1 auth_port = 35357 auth_protocol = http signing_dirname = /tmp/keystone-signing-cinder Note that we have defined qpid_hostname and our sql_connection now uses proper IP /etc/cinder/api-paste.ini ~~~~~~~~~~~~~~~~~~~~~~~~~ this one is pretty much a carbon-copy of Master:: .... [filter:authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory service_protocol = http service_host = 10.10.10.1 service_port = 5000 Looks like two things have helped: 1. Using non-localhost URLs for endpoints and other configuration 2. adding "my_ip" and "qpid_hostname" to the config in addition to existing "iscsi_ip*" stuff. _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
