Hi Ilya Maximets,
>From the tcpdump, with or without the rewrite, the link-local address was used.
===
$ ovs-tcpdump -nn -i exit_p0
11:10:26.323938 IP6 fe80::dc03:37ff:fee2:1fef.51513 >
2403:400:31da:ffff::18:3.4789: VXLAN, flags [I] (0x08), vni 1
IP 100.87.18.60 > 192.168.1.33: ICMP echo request, id 70, seq 1, length 64
11:10:27.326875 IP6 fe80::dc03:37ff:fee2:1fef.51513 >
2403:400:31da:ffff::18:3.4789: VXLAN, flags [I] (0x08), vni 1
IP 100.87.18.60 > 192.168.1.33: ICMP echo request, id 70, seq 2, length 64
===
Here is the output of the trace without the rewrite.
===
$ ovs-appctl ofproto/trace --names br-int 'in_port=dpdk-vm101,
eth_src=52:54:00:3d:cd:0c,eth_dst=00:00:00:00:00:01,eth_type=0x0800,
nw_src=100.87.18.60,nw_dst=192.168.1.33,nw_proto=1,nw_ttl=64,nw_frag=no,
icmp_type=8,icmp_code=0'
Flow:
icmp,in_port="dpdk-vm101",vlan_tci=0x0000,dl_src=52:54:00:3d:cd:0c,dl_dst=00:00:00:00:00:01,nw_src=100.87.18.60,nw_dst=192.168.1.33,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=8,icmp_code=0
bridge("br-int")
------------------
0. in_port="dpdk-vm101", priority 32768
output:vxlan0
-> output to native tunnel
-> tunneling to fe80::920a:84ff:fe9e:9570 via br-phy
-> tunneling from de:03:37:e2:1f:ef fe80::dc03:37ff:fee2:1fef to
90:0a:84:9e:95:70 fe80::920a:84ff:fe9e:9570
bridge("br-phy")
-------------------
0. priority 10
NORMAL
-> forwarding to learned port
Final flow: unchanged
Megaflow: recirc_id=0,eth,ip,in_port="dpdk-vm101",nw_ecn=0,nw_frag=no
Datapath actions:
tnl_push(tnl_port(vxlan_sys_4789),header(size=70,type=4,eth(dst=90:0a:84:9e:95:70,src=de:03:37:e2:1f:ef,dl_type=0x86dd),ipv6(src=fe80::dc03:37ff:fee2:1fef,dst=2403:400:31da:ffff::18:3,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x1)),out_port(br-phy)),push_vlan(vid=304,pcp=0),exit_p0
===
The "tunneling to fe80::920a:84ff:fe9e:9570 via br-phy" looks a bit curious.
I'm not sure why this was picked instead of the `remote_ip` specified in the
tunnel configuration. But then the final datapath actions shows the correct
`dst` address. Why is the `local_ip` specified in the VXLAN tunnel options not
considered?
Here is the out of the trace with the rewrite. It seems the flow entry was
matched but the rewrite didn't happen.
===
$ ovs-appctl ofproto/trace --names br-int 'in_port=dpdk-vm101,
eth_src=52:54:00:3d:cd:0c,eth_dst=00:00:00:00:00:01,eth_type=0x0800,
nw_src=100.87.18.60,nw_dst=192.168.1.33,nw_proto=1,nw_ttl=64,nw_frag=no,
icmp_type=8,icmp_code=0'
Flow:
icmp,in_port="dpdk-vm101",vlan_tci=0x0000,dl_src=52:54:00:3d:cd:0c,dl_dst=00:00:00:00:00:01,nw_src=100.87.18.60,nw_dst=192.168.1.33,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=8,icmp_code=0
bridge("br-int")
------------------
0. in_port="dpdk-vm101", priority 32768
output:vxlan0
-> output to native tunnel
-> tunneling to fe80::920a:84ff:fe9e:9570 via br-phy
-> tunneling from de:03:37:e2:1f:ef fe80::dc03:37ff:fee2:1fef to
90:0a:84:9e:95:70 fe80::920a:84ff:fe9e:9570
bridge("br-phy")
-------------------
0. ipv6,ipv6_dst=2403:400:31da:ffff::18:3, priority 499
load:0x180006->NXM_NX_IPV6_SRC[0..63]
load:0x2403040031daffff->NXM_NX_IPV6_SRC[64..127]
NORMAL
-> forwarding to learned port
Final flow: unchanged
Megaflow: recirc_id=0,eth,ip,in_port="dpdk-vm101",nw_ecn=0,nw_frag=no
Datapath actions:
tnl_push(tnl_port(vxlan_sys_4789),header(size=70,type=4,eth(dst=90:0a:84:9e:95:70,src=de:03:37:e2:1f:ef,dl_type=0x86dd),ipv6(src=fe80::dc03:37ff:fee2:1fef,dst=2403:400:31da:ffff::18:3,label=0,proto=17,tclass=0x0,hlimit=64),udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x1)),out_port(br-phy)),push_vlan(vid=304,pcp=0),exit_p0
===
# VXLAN interface on bridge configuration
===
Port vxlan0
Interface vxlan0
type: vxlan
options: {dst_port="4789", key="1",
local_ip="2403:400:31da:ffff::18:6", remote_ip="2403:400:31da:ffff::18:3"}
===
Thank you,
Derrick
From: Ilya Maximets <[email protected]>
Date: Friday, February 2, 2024 at 22:27
To: Lim, Derrick | Derrick | CMD <[email protected]>,
[email protected] <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: [ovs-discuss] Encapsulate VXLAN and then process other flows
[EXTERNAL] This message comes from an external organization.
On 2/2/24 08:58, Lim, Derrick via discuss wrote:
> Hi Ilya Maximets,
>
>> The rules look mostly fine. I think the main problem you have is priority.
>> Default priority for OF rules (if not specified) is 32768, so your new rules
>> with priority 50 are too low in a priority list and not getting hit.
>
> I tried this again with the default flow at priority 50 and mine at 499 but I
> still couldn't get the flow to hit.
>
> However, I observed that if the source address is set to anything other than
> `2403:400:31da:ffff::18:6`, which is an address that exists on the phy
> `br-phy`
> interface, the lookup is a hit and the action taken.
>
> Is there anything that prevents the address from being set to that of
> something
> that is already configured on an interface?
>
> For example,
>
> $ ip addr
> 35: br-phy: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel
> state UNKNOWN group default qlen 1000
> link/ether de:03:37:e2:1f:ef brd ff:ff:ff:ff:ff:ff
> inet6 2403:400:31da:ffff::18:6/128 scope global
> valid_lft forever preferred_lft forever
> inet6 fe80::dc03:37ff:fee2:1fef/64 scope link
> valid_lft forever preferred_lft forever
>
> Set the source address to `2403:400:31da:ffff::18:5`. In the flow entry,
> `set(ipv6(src=2403:400:31da:ffff::18:5))` is applied in actions.
>
> ===
>
> $ /usr/bin/ovs-ofctl add-flow br-phy \
>
> "priority=499,ipv6,ipv6_dst=2403:400:31da:ffff::18:3,actions=set_field:2403:400:31da:ffff::18:5->ipv6_src,normal"
>
> $ /usr/bin/ovs-appctl dpctl/dump-flows -m netdev@ovs-netdev | grep
> 192.168.1.33
>
> ufid:acc4b3bc-4958-412d-90c2-9bc4b3fbfac7,
>
> skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),
> recirc_id(0),dp_hash(0/0),in_port(dpdk-vm101),packet_type(ns=0,id=0),
>
> eth(src=52:54:00:3d:cd:0c/00:00:00:00:00:00,dst=00:00:00:00:00:01/00:00:00:00:00:00),eth_type(0x0800),
>
> ipv4(src=100.87.18.60/0.0.0.0,dst=192.168.1.33/0.0.0.0,proto=1/0,tos=0/0x3,ttl=64/0,frag=no),
> icmp(type=8/0,code=0/0), packets:407, bytes:39886, used:0.661s, dp:ovs,
> actions:
> tnl_push(tnl_port(vxlan_sys_4789),header(size=70,type=4,
> eth(dst=90:0a:84:9e:95:70,src=de:03:37:e2:1f:ef,dl_type=0x86dd),
>
> ipv6(src=fe80::dc03:37ff:fee2:1fef,dst=2403:400:31da:ffff::18:3,label=0,proto=17,tclass=0x0,hlimit=64),
>
> udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x1)),out_port(br-phy)),
> set(ipv6(src=2403:400:31da:ffff::18:5)),
> push_vlan(vid=304,pcp=0),
> exit_p0,
> dp-extra-info:miniflow_bits(4,1)
>
> ===
>
>
> Set the source address to `2403:400:31da:ffff::18:6`. This is a configured
> address on `br-phy`. The `set(ipv6(src=<addr>))` part is no longer applied.
>
> ===
>
> $ /usr/bin/ovs-ofctl add-flow br-phy \
>
> "priority=499,ipv6,ipv6_dst=2403:400:31da:ffff::18:3,actions=set_field:2403:400:31da:ffff::18:6->ipv6_src,normal"
>
> $ /usr/bin/ovs-appctl dpctl/dump-flows -m netdev@ovs-netdev | grep
> 192.168.1.33
>
> ufid:acc4b3bc-4958-412d-90c2-9bc4b3fbfac7,
>
> skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),
> recirc_id(0),dp_hash(0/0),in_port(dpdk-vm101),packet_type(ns=0,id=0),
>
> eth(src=52:54:00:3d:cd:0c/00:00:00:00:00:00,dst=00:00:00:00:00:01/00:00:00:00:00:00),eth_type(0x0800),
>
> ipv4(src=100.87.18.60/0.0.0.0,dst=192.168.1.33/0.0.0.0,proto=1/0,tos=0/0x3,ttl=64/0,frag=no),
> icmp(type=8/0,code=0/0), packets:423, bytes:41454, used:0.803s, dp:ovs,
> actions:
> tnl_push(tnl_port(vxlan_sys_4789),header(size=70,type=4,
> eth(dst=90:0a:84:9e:95:70,src=de:03:37:e2:1f:ef,dl_type=0x86dd),
>
> ipv6(src=fe80::dc03:37ff:fee2:1fef,dst=2403:400:31da:ffff::18:3,label=0,proto=17,tclass=0x0,hlimit=64),
>
> udp(src=0,dst=4789,csum=0xffff),vxlan(flags=0x8000000,vni=0x1)),out_port(br-phy)),
> push_vlan(vid=304,pcp=0),
> exit_p0,
> dp-extra-info:miniflow_bits(4,1)
>
> $ /usr/bin/ovs-ofctl dump-flows br-phy
>
> cookie=0x0, duration=170.787s, table=0, n_packets=251, n_bytes=40328,
> priority=499,ipv6,ipv6_dst=2403:400:31da:ffff::18:3
>
> actions=load:0x180006->NXM_NX_IPV6_SRC[0..63],load:0x2403040031daffff->NXM_NX_IPV6_SRC[64..127],NORMAL
>
> cookie=0x0, duration=1136.132s, table=0, n_packets=10175, n_bytes=1116852,
> priority=10 actions=NORMAL
>
> ===
Hmm. This is interesting. We can see that some packets do actually hit the
OpenFlow
rule (n_packets=251). The decision to not include the set(ipv6(src=<addr>))
action
is likely made because it is the same as one already in the packet. But we can
see
from the datapath flow dump that it is supposed to be different:
tnl_push( ... ipv6(src=fe80::dc03:37ff:fee2:1fef, ... )
This is a link-local IP of that interface.
I suspect that a mishap happened somewhere and 2403:400:31da:ffff::18:6 was
used for
the actual tunnel header, or it was used to updated the local flow structure
during
the flow translation instead of a link-local address and hence OVS thinks that
the
address is already in the packet and doesn't set a new one.
Can you capture this packet on the exit_p0 interface? e.g. with ovs-tcpdump.
So we can see what is the actual IP of the outgoing packet.
Also, it might be useful if you can run ofproto/trace command to check how OVS
makes
the decision during the flow translation. It should look something like this:
$ ovs-appctl ofproto/trace --names br-int \
'in_port=dpdk-vm101,
eth_src=52:54:00:3d:cd:0c,eth_dst=00:00:00:00:00:01,eth_type=0x0800,
nw_src=100.87.18.60,nw_dst=192.168.1.33,nw_proto=1,nw_ttl=64,nw_frag=no,
icmp_type=8,icmp_code=0'
This command doesn't have any side effects by default, i.e. it doesn't generate
any
actual traffic. It only shows the logic of how the datapath actions are
generated.
Best regards, Ilya Maximets.
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
