Hi Grant, You should have an IP address configured on the br-vxlan interface of each host.
- Can you confirm connectivity between the hosts using the IP addresses assigned to br-vxlan? Ping from within the neutron_agents container, as well, since this is ultimately where the Neutron namespace that handles DHCP lives. - If the ping fails, can you confirm bond1.21 is UP? How about br-vxlan? Is the VLAN trunked properly? Once connectivity is confirmed, try performing packet captures on vxlan-21 and bond1.21 while you boot the instance and see what’s there. Report back and we’ll go from there. James From: Grant Morley <gr...@absolutedevops.io> Date: Thursday, September 8, 2016 at 9:55 AM To: Vahric Muhtaryan <vah...@doruk.net.tr>, OpenStack Operators <openstack-operators@lists.openstack.org> Cc: "ian.ba...@serverchoice.com" <ian.ba...@serverchoice.com> Subject: Re: [Openstack-operators] VXLAN / Tenant Network Issue Hi there, thanks for replying, configs below: The following are from the neutron agents container. # Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/plugins/ml2/ml2_conf.ini.j2 # ML2 general [ml2] type_drivers = flat,vlan,vxlan,local tenant_network_types = vxlan,vlan,flat mechanism_drivers = linuxbridge,l2population extension_drivers = port_security path_mtu = 0 segment_mtu = 0 # ML2 flat networks [ml2_type_flat] flat_networks = flat # ML2 VLAN networks [ml2_type_vlan] network_vlan_ranges = vlan:101:200,vlan:301:400 # ML2 VXLAN networks [ml2_type_vxlan] vxlan_group = 239.1.1.1 vni_ranges = 1:1000 # Security groups [securitygroup] enable_security_group = True enable_ipset = True -------------------------------------------------------------------- # Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/dhcp_agent.ini.j2 # General [DEFAULT] verbose = True debug = False num_sync_threads = 6 # Drivers interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq # Default domain for DHCP leases dhcp_domain = openstacklocal # Dnsmasq options dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf dnsmasq_dns_servers = dnsmasq_lease_max = 16777216 # Metadata enable_isolated_metadata = True --------------------------------------------------------------------- # Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/l3_agent.ini.j2 # General [DEFAULT] verbose = True debug = False # While this option is deprecated in Liberty, if we remove it then it takes # a default value of 'br-ex', which we do not want. We therefore leave it # in place for now and can remove it in Mitaka. external_network_bridge = gateway_external_network_id = # Drivers interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver # Agent mode (legacy only) agent_mode = legacy # Conventional failover allow_automatic_l3agent_failover = True # HA failover ha_confs_path = /var/lib/neutron/ha_confs ha_vrrp_advert_int = 2 ha_vrrp_auth_password = bee916a2589b14dd7f ha_vrrp_auth_type = PASS handle_internal_only_routers = False send_arp_for_ha = 3 # Metadata enable_metadata_proxy = True Regards, On 08/09/16 13:51, Vahric Muhtaryan wrote: Hello Grant , Possible to share ml2_conf.ini , dhcp_agent.ini and l3_agent.ini files ? Regards VM From: Grant Morley <gr...@absolutedevops.io<mailto:gr...@absolutedevops.io>> Date: Thursday 8 September 2016 at 15:12 To: OpenStack Operators <openstack-operators@lists.openstack.org<mailto:openstack-operators@lists.openstack.org>> Cc: <ian.ba...@serverchoice.com<mailto:ian.ba...@serverchoice.com>> Subject: [Openstack-operators] VXLAN / Tenant Network Issue Hi All, We are working off the OSA deployment for a new cloud system we are building and everything seems to be working apart from the tenant VXLAN network. We have tried various troubleshooting but the initial DHCP request, is not making it out of the linux bridge on the compute node. We have checked all physical networking and switch setup and they appear to be fine. Below is an output of related networking components that we have configured. (Sorry for the long post but wanted to get as much info on here) Can anyone see what might be causing the issue or where we have gone wrong? Neutron subnet and router: neutron) net-list +--------------------------------------+----------------------------------------------------+----------------------------------------------------+ | id | name | subnets | +--------------------------------------+----------------------------------------------------+----------------------------------------------------+ | b1da0a4f-2d06-46af-92aa-962c7a7c36f9 | ext-net | 405f439c-51bb-40b6-820a-9048c2ee69fe | | | | 185.136.232.0/22 | | a256ccb2-273a-4738-97ab-bd8bfbc2a2cc | HA network tenant 7b5aad6af3ee450ea60e06aaaba2da50 | 6d98faac-2e3b-43c8-bcd6-f9a6f5dcc45e | | | | 169.254.192.0/18 | | f88ceab1-a392-4281-8c60-f57d171a8029 | vxlan-172 | 367e88eb-b09f-4ce5-bfff-5d9e0b0e14b0 | 172.16.0.0/24 +--------------------------------------+----------------------------------------------------+----------------------------------------------------+ (neutron) net-show f88ceab1-a392-4281-8c60-f57d171a8029 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | f88ceab1-a392-4281-8c60-f57d171a8029 | | mtu | 0 | | name | vxlan-172 | | port_security_enabled | True | | provider:network_type | vxlan | | provider:physical_network | | | provider:segmentation_id | 21 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | 367e88eb-b09f-4ce5-bfff-5d9e0b0e14b0 | | tenant_id | 7b5aad6af3ee450ea60e06aaaba2da50 | +---------------------------+--------------------------------------+ (neutron) router-show f31ed1fb-1b90-46e3-b869-d9374e3d08b1 +-----------------------+------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | True | | distributed | False | | external_gateway_info | {"network_id": "b1da0a4f-2d06-46af-92aa-962c7a7c36f9", "enable_snat": true, "external_fixed_ips": [{"subnet_id": | | | "405f439c-51bb-40b6-820a-9048c2ee69fe", "ip_address": "185.136.232.55"}]} | | ha | True | | id | f31ed1fb-1b90-46e3-b869-d9374e3d08b1 | | name | ext-router | | routes | | | status | ACTIVE | | tenant_id | 7b5aad6af3ee450ea60e06aaaba2da50 | +-----------------------+------------------------------------------------------------------------------------------------------------------------+ (neutron) router-port-list f31ed1fb-1b90-46e3-b869-d9374e3d08b1 +--------------------------------------+----------------------------------------+-------------------+------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+----------------------------------------+-------------------+------------------------------------------+ | 443d8a0e-833e-4dd2-9320-c2a361e97bf0 | HA port tenant | fa:16:3e:db:48:be | {"subnet_id": "6d98faac-2e3b- | | | 7b5aad6af3ee450ea60e06aaaba2da50 | | 43c8-bcd6-f9a6f5dcc45e", "ip_address": | | | | | "169.254.192.2"} | | 58312691-77d1-408a-adf2-8c74bb87d35d | HA port tenant | fa:16:3e:26:86:3c | {"subnet_id": "6d98faac-2e3b- | | | 7b5aad6af3ee450ea60e06aaaba2da50 | | 43c8-bcd6-f9a6f5dcc45e", "ip_address": | | | | | "169.254.192.1"} | | 8182e8ca-0e3d-444a-ac4f-f424027aa373 | | fa:16:3e:20:1c:08 | {"subnet_id": "405f439c-51bb-40b6-820a- | | | | | 9048c2ee69fe", "ip_address": | | | | | "185.136.232.55"} | | beaa905d-fc68-46ba-9fd3-9f620584a1f7 | | fa:16:3e:5a:8e:c0 | {"subnet_id": "367e88eb-b09f-4ce5-bfff- | | | | | 5d9e0b0e14b0", "ip_address": | | | | | "172.16.0.254"} | +--------------------------------------+----------------------------------------+-------------------+------------------------------------------+ The bridge and interface for the instance: root@compute-2:~# brctl show bridge name bridge id STP enabled interfaces br-mgmt 8000.1418775ed1bc no bond0.11 br-storage 8000.1418775ed1bc no bond0.31 br-vlan 8000.1418775ed1be no bond1 br-vxlan 8000.1418775ed1be no bond1.21 brqf88ceab1-a3 8000.0a81d25d36ce no tapf9871920-e0 vxlan-21 Network agent node namespaces: root@network-1_neutron_agents_container-f3caf6a1:~# ip netns qrouter-f31ed1fb-1b90-46e3-b869-d9374e3d08b1 qdhcp-f88ceab1-a392-4281-8c60-f57d171a8029 qdhcp-b1da0a4f-2d06-46af-92aa-962c7a7c36f9 The two qdhcp namespaces are able to ping to each other. When booting the instance the DHCP request can be seen: root@compute-2:~# dhcpdump -i tapf9871920-e0 TIME: 2016-09-08 11:49:03.646 IP: 0.0.0.0 (fa:16:3e:32:7e:79) > 255.255.255.255 (ff:ff:ff:ff:ff:ff) OP: 1 (BOOTPREQUEST) HTYPE: 1 (Ethernet) HLEN: 6 HOPS: 0 XID: 7840761a SECS: 60 FLAGS: 0 CIADDR: 0.0.0.0 YIADDR: 0.0.0.0 SIADDR: 0.0.0.0 GIADDR: 0.0.0.0 CHADDR: fa:16:3e:32:7e:79:00:00:00:00:00:00:00:00:00:00 SNAME: . FNAME: . OPTION: 53 ( 1) DHCP message type 1 (DHCPDISCOVER) OPTION: 61 ( 7) Client-identifier 01:fa:16:3e:32:7e:79 OPTION: 57 ( 2) Maximum DHCP message size 576 OPTION: 55 ( 9) Parameter Request List 1 (Subnet mask) 3 (Routers) 6 (DNS server) 12 (Host name) 15 (Domainname) 26 (Interface MTU) 28 (Broadcast address) 42 (NTP servers) 121 (Classless Static Route) OPTION: 60 ( 12) Vendor class identifier udhcp 1.20.1 OPTION: 12 ( 6) Host name cirros --------------------------------------------------------------------------- The DHCP packet is seen on the tap interface for the instance and the bridge brqf88ceab1-a3, but not on any other interface on the compute host. No DHCP packet is observed on the network agent container running the DHCP namespace. output of the instance booting: Starting network... udhcpc (v1.20.1) started Sending discover... Sending discover... Sending discover... Usage: /sbin/cirros-dhcpc <up|down> No lease, failing WARN: /etc/rc3.d/S40-network failed cirros-ds 'net' up at 181.24 Regards, -- [cid:part1.44430905.A1A46B65@absolutedevops.io] Grant Morley Cloud Lead Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io<http://www.absolutedevops.io/> gr...@absolutedevops.io<mailto:grant@absolutedevops.i> 0845 874 0580 _______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators -- [cid:image001.png@01D209E1.FCB1D490] Grant Morley Cloud Lead Absolute DevOps Ltd Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP www.absolutedevops.io<http://www.absolutedevops.io/> gr...@absolutedevops.io<mailto:grant@absolutedevops.i> 0845 874 0580
_______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
