Public bug reported: We install DVR to src mac flow only for fixed IPs but not for AAPs. As result its not possible to communicate to IP address specified in AAP via distributed router.
https://github.com/openstack/neutron/blob/f2c3e3e68c0fa77cff5cdd22d847e3f6ea7cdd84/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py#L569 Issue may be reproduced by creating the following topology. ump-01 (instance) --- netA (10.10.11.0/24) --- router ---- net B (192.168.0.0/24) ---- fw-01 (instance VIP: 192.168.0.10/32) Port for instance B1 +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | ip_address='192.168.0.10/32', mac_address='00:00:5e:00:01:32' | | | ip_address='0.0.0.0/0', mac_address='fa:16:3e:0c:7d:6c' | | binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi | | binding_profile | | | binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | normal | | created_at | 2025-01-08T11:13:05Z | | data_plane_status | None | | description | | | device_id | dd41d063-717f-4795-99b2-00783bce81f6 | | device_owner | compute:nova | | device_profile | None | | dns_assignment | fqdn='oc-virtual-lab-server-fw-01.openstack.internal.', hostname='oc-virtual-lab-server-fw-01', ip_address='192.168.0.63' | | dns_domain | | | dns_name | oc-virtual-lab-server-fw-01 | | extra_dhcp_opts | | | fixed_ips | ip_address='192.168.0.188', subnet_id='7425f886-b2db-4474-af23-dc8e164be243' | | hardware_offload_type | None | | hints | | | id | e71f0314-35a2-4a5c-8ac4-2216ded5addd | | ip_allocation | None | | mac_address | fa:16:3e:17:78:99 | | name | oc-virtual-lab-port-fw-01 | | network_id | fefa44ca-5b56-46f6-a31a-3e0a67b99ffe | | numa_affinity_policy | None | | port_security_enabled | True | | project_id | 59592b24e56641afa2d7dc705da006e7 | | propagate_uplink_status | None | | resource_request | None | | revision_number | 6 | | qos_network_policy_id | None | | qos_policy_id | None | | security_group_ids | d926157a-91f0-491b-980a-3ab33d7e4a50 | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2025-01-08T11:21:56Z | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ port show for A1 +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi | | binding_profile | | | binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | normal | | created_at | 2025-01-08T11:13:05Z | | data_plane_status | None | | description | | | device_id | c9844224-7154-4b91-a15f-92bb515964d7 | | device_owner | compute:nova | | device_profile | None | | dns_assignment | fqdn='oc-virtual-lab-server-jump-01.openstack.internal.', hostname='oc-virtual-lab-server-jump-01', ip_address='10.10.11.203' | | dns_domain | | | dns_name | oc-virtual-lab-server-jump-01 | | extra_dhcp_opts | | | fixed_ips | ip_address='10.10.11.203', subnet_id='6cc27493-bafc-4c93-9290-1cefd6c74bd0' | | hardware_offload_type | None | | hints | | | id | 006cca82-c38c-4311-a6db-0fc5acc6e977 | | ip_allocation | None | | mac_address | fa:16:3e:d1:4d:dc | | name | oc-virtual-lab-jump-port-jump-01 | | network_id | 8392d7ee-0c5f-46f6-805f-527afe68b5f8 | | numa_affinity_policy | None | | port_security_enabled | False | | project_id | 59592b24e56641afa2d7dc705da006e7 | | propagate_uplink_status | None | | resource_request | None | | revision_number | 4 | | qos_network_policy_id | None | | qos_policy_id | None | | security_group_ids | | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2025-01-08T11:13:36Z | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ On instance fw-01 start VIP 192.168.0.10/32 with keepalived and following configuration ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/keepalived/keepalived.conf vrrp_instance VI_1 { interface ens3 virtual_router_id 50 nopreempt priority 100 advert_int 1 virtual_ipaddress { 192.168.0.10/32 } notify_master "/etc/keepalived/notify_master.sh" use_vmac vmac_xmit_base } and sysctl settings ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/sysctl.d/vrrp.conf net.ipv4.conf.all.arp_ignore=1 net.ipv4.conf.all.arp_announce=1 net.ipv4.conf.all.arp_filter=0 net.ipv4.conf.ens3.arp_filter = 1 When jump-01 and fw-01 are on same host connectivity works, but when they are on different hypervisors its failed due to missing dvr flow ovs-ofctl -O OpenFlow14 dump-flows br-int |grep table=1 cookie=0xf2a4abee1cb41511, duration=328.546s, table=1, n_packets=3179, n_bytes=231080, priority=20,dl_vlan=5,dl_dst=fa:16:3e:17:78:99 actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58 cookie=0xf2a4abee1cb41511, duration=336.209s, table=1, n_packets=116933, n_bytes=11456946, priority=1 actions=drop If add this flow connectivity is restored ovs-ofctl -O OpenFlow14 add-flow br-int 'table=1,priority=20,dl_vlan=5,dl_dst=00:00:5e:00:01:32,actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58' ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2093248 Title: DVR flow is not installed for allowed address pairs mac in DVR_TO_SRC_MAC table=1 Status in neutron: New Bug description: We install DVR to src mac flow only for fixed IPs but not for AAPs. As result its not possible to communicate to IP address specified in AAP via distributed router. https://github.com/openstack/neutron/blob/f2c3e3e68c0fa77cff5cdd22d847e3f6ea7cdd84/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_dvr_neutron_agent.py#L569 Issue may be reproduced by creating the following topology. ump-01 (instance) --- netA (10.10.11.0/24) --- router ---- net B (192.168.0.0/24) ---- fw-01 (instance VIP: 192.168.0.10/32) Port for instance B1 +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | ip_address='192.168.0.10/32', mac_address='00:00:5e:00:01:32' | | | ip_address='0.0.0.0/0', mac_address='fa:16:3e:0c:7d:6c' | | binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi | | binding_profile | | | binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | normal | | created_at | 2025-01-08T11:13:05Z | | data_plane_status | None | | description | | | device_id | dd41d063-717f-4795-99b2-00783bce81f6 | | device_owner | compute:nova | | device_profile | None | | dns_assignment | fqdn='oc-virtual-lab-server-fw-01.openstack.internal.', hostname='oc-virtual-lab-server-fw-01', ip_address='192.168.0.63' | | dns_domain | | | dns_name | oc-virtual-lab-server-fw-01 | | extra_dhcp_opts | | | fixed_ips | ip_address='192.168.0.188', subnet_id='7425f886-b2db-4474-af23-dc8e164be243' | | hardware_offload_type | None | | hints | | | id | e71f0314-35a2-4a5c-8ac4-2216ded5addd | | ip_allocation | None | | mac_address | fa:16:3e:17:78:99 | | name | oc-virtual-lab-port-fw-01 | | network_id | fefa44ca-5b56-46f6-a31a-3e0a67b99ffe | | numa_affinity_policy | None | | port_security_enabled | True | | project_id | 59592b24e56641afa2d7dc705da006e7 | | propagate_uplink_status | None | | resource_request | None | | revision_number | 6 | | qos_network_policy_id | None | | qos_policy_id | None | | security_group_ids | d926157a-91f0-491b-980a-3ab33d7e4a50 | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2025-01-08T11:21:56Z | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ port show for A1 +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | vs-ps-y7eajqtcjwcm-2-v3xgrdrpb2ao-server-6rwxod6xp2yi | | binding_profile | | | binding_vif_details | bound_drivers.0='openvswitch', bridge_name='br-int', connectivity='l2', datapath_type='system', ovs_hybrid_plug='False', port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | normal | | created_at | 2025-01-08T11:13:05Z | | data_plane_status | None | | description | | | device_id | c9844224-7154-4b91-a15f-92bb515964d7 | | device_owner | compute:nova | | device_profile | None | | dns_assignment | fqdn='oc-virtual-lab-server-jump-01.openstack.internal.', hostname='oc-virtual-lab-server-jump-01', ip_address='10.10.11.203' | | dns_domain | | | dns_name | oc-virtual-lab-server-jump-01 | | extra_dhcp_opts | | | fixed_ips | ip_address='10.10.11.203', subnet_id='6cc27493-bafc-4c93-9290-1cefd6c74bd0' | | hardware_offload_type | None | | hints | | | id | 006cca82-c38c-4311-a6db-0fc5acc6e977 | | ip_allocation | None | | mac_address | fa:16:3e:d1:4d:dc | | name | oc-virtual-lab-jump-port-jump-01 | | network_id | 8392d7ee-0c5f-46f6-805f-527afe68b5f8 | | numa_affinity_policy | None | | port_security_enabled | False | | project_id | 59592b24e56641afa2d7dc705da006e7 | | propagate_uplink_status | None | | resource_request | None | | revision_number | 4 | | qos_network_policy_id | None | | qos_policy_id | None | | security_group_ids | | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2025-01-08T11:13:36Z | +-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ On instance fw-01 start VIP 192.168.0.10/32 with keepalived and following configuration ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/keepalived/keepalived.conf vrrp_instance VI_1 { interface ens3 virtual_router_id 50 nopreempt priority 100 advert_int 1 virtual_ipaddress { 192.168.0.10/32 } notify_master "/etc/keepalived/notify_master.sh" use_vmac vmac_xmit_base } and sysctl settings ubuntu@oc-virtual-lab-server-fw-01:~$ cat /etc/sysctl.d/vrrp.conf net.ipv4.conf.all.arp_ignore=1 net.ipv4.conf.all.arp_announce=1 net.ipv4.conf.all.arp_filter=0 net.ipv4.conf.ens3.arp_filter = 1 When jump-01 and fw-01 are on same host connectivity works, but when they are on different hypervisors its failed due to missing dvr flow ovs-ofctl -O OpenFlow14 dump-flows br-int |grep table=1 cookie=0xf2a4abee1cb41511, duration=328.546s, table=1, n_packets=3179, n_bytes=231080, priority=20,dl_vlan=5,dl_dst=fa:16:3e:17:78:99 actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58 cookie=0xf2a4abee1cb41511, duration=336.209s, table=1, n_packets=116933, n_bytes=11456946, priority=1 actions=drop If add this flow connectivity is restored ovs-ofctl -O OpenFlow14 add-flow br-int 'table=1,priority=20,dl_vlan=5,dl_dst=00:00:5e:00:01:32,actions=set_field:fa:16:3e:59:84:ef->eth_src,goto_table:58' To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2093248/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
