And to answer the second part of my question: This all seems to be related to SElinux. From /var/log/neutron/dhcp-agent.log:
2015-06-12 14:31:58.757 7130 ERROR neutron.agent.linux.utils [-] Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-23cb6ffe-b0b2-4509-8a23-ce5cbd16b339', 'env', 'NEUTRON_NETWORK_ID=23cb6ffe-b0b2-4509-8a23-ce5cbd16b339', 'dnsmasq', '--no-hosts', '--no-resolv', '--strict-order', '--bind-interfaces', '--interface=tap3ae23814-24', '--except-interface=lo', '--pid-file=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/pid', '--dhcp-hostsfile=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/host', '--addn-hosts=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/addn_hosts', '--dhcp-optsfile=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/opts', '--leasefile-ro', '--dhcp-range=set:tag0,10.0.0.0,static,86400s', '--dhcp-lease-max=256', '--conf-file=/etc/neutron/dnsmasq-neutron.conf', '--domain=openstacklocal'] Exit code: 3 Stdout: '' Stderr: '\ndnsmasq: cannot open log /var/log/neutron/dnsmasq.log: Permission denied\n' 2015-06-12 14:31:58.757 7130 ERROR neutron.agent.dhcp_agent [-] Unable to enable dhcp for 23cb6ffe-b0b2-4509-8a23-ce5cbd16b339. 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent Traceback (most recent call last): 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.7/site-packages/neutron/agent/dhcp_agent.py", line 128, in call_driver 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent getattr(driver, action)(**action_kwargs) 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/dhcp.py", line 206, in enable 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent self.spawn_process() 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/dhcp.py", line 433, in spawn_process 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent ip_wrapper.netns.execute(cmd, addl_env=env) 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/ip_lib.py", line 550, in execute 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent check_exit_code=check_exit_code, extra_ok_codes=extra_ok_codes) 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 84, in execute 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent raise RuntimeError(m) 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent RuntimeError: 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-23cb6ffe-b0b2-4509-8a23-ce5cbd16b339', 'env', 'NEUTRON_NETWORK_ID=23cb6ffe-b0b2-4509-8a23-ce5cbd16b339', 'dnsmasq', '--no-hosts', '--no-resolv', '--strict-order', '--bind-interfaces', '--interface=tap3ae23814-24', '--except-interface=lo', '--pid-file=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/pid', '--dhcp-hostsfile=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/host', '--addn-hosts=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/addn_hosts', '--dhcp-optsfile=/var/lib/neutron/dhcp/23cb6ffe-b0b2-4509-8a23-ce5cbd16b339/opts', '--leasefile-ro', '--dhcp-range=set:tag0,10.0.0.0,static,86400s', '--dhcp-lease-max=256', '--conf-file=/etc/neutron/dnsmasq-neutron.conf', '--domain=openstacklocal'] 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent Exit code: 3 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent Stdout: '' 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent Stderr: '\ndnsmasq: cannot open log /var/log/neutron/dnsmasq.log: Permission denied\n' 2015-06-12 14:31:58.757 7130 TRACE neutron.agent.dhcp_agent 2015-06-12 14:31:58.758 7130 INFO neutron.agent.dhcp_agent [-] Synchronizing state complete And from /var/log/audit/audit.log: type=AVC msg=audit(1434112358.159:8115): avc: denied { search } for pid=7879 comm="dnsmasq" name="neutron" dev="dm-2" ino=3670017 scontext=system_u:system_r:dnsmasq_t:s0 tcontext=system_u:object_r:neutron_log_t:s0 tclass=dir type=SYSCALL msg=audit(1434112358.159:8115): arch=c000003e syscall=2 success=no exit=-13 a0=7fcd88abd2a0 a1=441 a2=1a0 a3=0 items=0 ppid=1 pid=7879 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="dnsmasq" exe="/usr/sbin/dnsmasq" subj=system_u:system_r:dnsmasq_t:s0 key=(null) So I created /var/log/neutron/dnsmasq.log , chown'ed to neutron:neutron and also chcon'ed so that SElinux ACLs are the same as from the other files in that folder: -rw-r--r--. neutron neutron system_u:object_r:neutron_log_t:s0 /var/log/neutron/dnsmasq.log I tried to add a SElinux policy for dnsmasq to allow access to /var/log/neutron/ but that produced errors: # grep dnsmasq /var/log/audit/audit.log | audit2allow -M dnsmasq_neutron_selinux # semodule -i /root/dnsmasq_neutron_selinux.pp Full path required for exclude: net:[4026532406]. Full path required for exclude: net:[4026532406]. Full path required for exclude: net:[4026532475]. Full path required for exclude: net:[4026532475]. # echo $? 0 Any suggestions? Uwe Am 12.06.2015 um 14:07 schrieb Uwe Sauter: > Gary, > > Thanks. > > That brings up the question why there are dnsmasq processes running on my > controller node (which has neutron-server running) and > not on the networking node (neutron-dhcp-agent neutron-l3-agent > neutron-metadata-agent neutron-openvswitch-agent). > > Any suggestions? > > > Regards, > > Uwe > > Am 12.06.2015 um 13:58 schrieb Gary Kotton: >> Hi, >> The DHCP agent runs the dnsmasq process. That is done on the network node. >> Thanks >> Gary >> >> On 6/12/15, 2:35 PM, "Uwe Sauter" <uwe.sauter...@gmail.com> wrote: >> >>> Hi, >>> >>> == TL;DR == >>> Which neutron service manages the DNSMASQ processes? Does this run on the >>> controller node or the networking node? >>> >>> >>> == Long story == >>> I have a five node Juno installation (1 controller, 1 storage, 1 network >>> and 2 compute nodes). >>> >>> I followed the Juno Red Hat installation guide [1] up to the point where >>> the dashboard was installed, making modifications where >>> necessary to account for the additional nodes. I'm using Neutron / ML2 as >>> networking component with GRE tenant networks. >>> >>> I am able to sucessfully start a Cirros VM but that instance won't get an >>> IP address. To resolve this I followed a link [2] that >>> told to add logging to dnsmasq. Here the relevant parts on the *network* >>> node: >>> >>> /etc/neutron/dhcp_agent.ini >>> [DEFAULT] >>> interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver >>> dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq >>> use_namespaces = True >>> dhcp_delete_namespaces = True >>> verbose = True >>> dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf >>> >>> >>> /etc/neutron/dnsmasq-neutron.conf >>> dhcp-option-force=26,1454 >>> log-facility = /var/log/neutron/dnsmasq.log >>> log-dhcp >>> >>> >>> Then I realized that there were no dnsmasq processes on the networking >>> node but only on the controller node. Is this correct? I >>> was under the impression that neutron-dhcp-agent (running on the >>> networking node) is the service that maintains DHCP on the tenant >>> networks. >>> >>> So the question is: >>> Which service manages dnsmasq and on which node should that run on? >>> >>> >>> Thanks, >>> >>> Uwe >>> >>> [1] http://docs.openstack.org/juno/install-guide/install/yum/content/ >>> [2] >>> https://ask.openstack.org/en/question/63110/unable-to-get-dhcp-lease-in-ju >>> no/ >>> >>> _______________________________________________ >>> Mailing list: >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>> Post to : openstack@lists.openstack.org >>> Unsubscribe : >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> > _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack