Hello,
I am trying to trace ICMP ping reply packet on DPDK + bond gateway node and I
am struggling.
It looks like ovs-appctl ofproto/trace is not applying conntrack base data and
in trace flow
packet is not unSNAT-ed.
Of course, ICMP ping is working properly - packet is going back to the VM - the
problem seems to be with
ofproto/trace only.
Infrastructure: OpenStack VM 10.44.1.174 with tenant network only (no FIP) that
pings 8.8.8.8
and router with external gateway enabled with 194.146.52.200 gw IP that is on
network002 node.
Network Node/gw chassis: OVN v24.03.2, OVS v3.3.2, DPDK 23.11.0.
[root@network002 /]# flow=$(ovs-tcpdump -nXX -i dpdkbond -c 1 vlan 170 and host
194.146.52.200 and icmp and 'icmp[0]=0' | ovs-tcpundump)
[root@network002 /]# echo $flow
fa163e80daf6b0cf0e043dfd810000aa0800450000540000000075013e3f08080808c29234c8000054450e8288cc38d82d6700000000e3590c0000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637
####
[root@network002 /]# ovs-appctl ofproto/trace br-ex in_port=2 $flow
Flow:
icmp,in_port=2,dl_vlan=170,dl_vlan_pcp=0,vlan_tci1=0x0000,dl_src=b0:cf:0e:04:3d:fd,dl_dst=fa:16:3e:80:da:f6,nw_src=8.8.8.8,nw_dst=194.146.52.200,nw_tos=0,nw_ecn=0,nw_ttl=117,nw_frag=no,icmp_type=0,icmp_code=0
<CUT>
12. ip,reg14=0x1,metadata=0xa5,nw_dst=194.146.52.200, priority 100, cookie
0xb8e323f4
ct(table=13,zone=NXM_NX_REG12[0..15],nat)
nat
-> A clone of the packet is forked to recirculate. The forked pipeline
will be resumed at table 13.
-> Sets the packet to an untracked state, and clears all the conntrack
fields.
Final flow: unchanged
Megaflow:
recirc_id=0,eth,icmp,in_port=2,dl_vlan=170,dl_vlan_pcp=0,dl_src=b0:cf:0e:04:3d:fd,dl_dst=fa:16:3e:80:da:f6,nw_src=8.0.0.0/7,nw_dst=194.146.52.200,nw_ttl=117,nw_frag=no,icmp_type=0x0/0xf8
Datapath actions: pop_vlan,ct(zone=171,nat),recirc(0x130b9b)
===============================================================================
recirc(0x130b9b) - resume conntrack with default ct_state=trk|new (use
--ct-next to customize)
Replacing src/dst IP/ports to simulate NAT:
Initial flow:
Modified flow:
===============================================================================
Flow:
recirc_id=0x130b9b,ct_state=new|trk,ct_zone=171,eth,icmp,reg0=0xfa16,reg1=0x3e80daf6,reg9=0x4,reg11=0x97,reg12=0xab,reg14=0x1,metadata=0xa5,in_port=3,vlan_tci=0x0000,dl_src=b0:cf:0e:04:3d:fd,dl_dst=fa:16:3e:80:da:f6,nw_src=8.8.8.8,nw_dst=194.146.52.200,nw_tos=0,nw_ecn=0,nw_ttl=117,nw_frag=no,icmp_type=0,icmp_code=0
<CUT>
20. metadata=0xa5, priority 0, cookie 0xcbf3e7a5
set_field:0/0xffffffff->xxreg1
resubmit(,21)
21. ip,metadata=0xa5,nw_dst=194.146.52.128/25, priority 77, cookie 0xcdd1182
dec_ttl()
set_field:0/0xffff00000000->xreg4
move:NXM_OF_IP_DST[]->NXM_NX_XXREG0[96..127]
-> NXM_NX_XXREG0[96..127] is now 0xc29234c8
set_field:0xc29234c80000000000000000/0xffffffff0000000000000000->xxreg0
set_field:fa:16:3e:80:da:f6->eth_src
set_field:0x1->reg15
set_field:0x1/0x1->reg10
resubmit(,22)
22. reg8=0/0xffff,metadata=0xa5, priority 150, cookie 0x2d387d2e
resubmit(,23)
23. metadata=0xa5, priority 0, cookie 0x87eb379e
set_field:0/0xffff00000000->xreg4
resubmit(,24)
24. reg8=0/0xffff,metadata=0xa5, priority 150, cookie 0x129f6153
resubmit(,25)
25. ip,reg14=0x1,reg15=0x1,metadata=0xa5,nw_dst=194.146.52.200, priority 150,
cookie 0xd283cb8a
drop
Final flow:
recirc_id=0x130b9b,ct_state=new|trk,ct_zone=171,eth,icmp,reg0=0xc29234c8,reg1=0xc29234c8,reg9=0x4,reg10=0x1,reg11=0x97,reg12=0xab,reg14=0x1,reg15=0x1,metadata=0xa5,in_port=3,vlan_tci=0x0000,dl_src=fa:16:3e:80:da:f6,dl_dst=fa:16:3e:80:da:f6,nw_src=8.8.8.8,nw_dst=194.146.52.200,nw_tos=0,nw_ecn=0,nw_ttl=116,nw_frag=no,icmp_type=0,icmp_code=0
Megaflow:
recirc_id=0x130b9b,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,icmp,in_port=3,dl_src=b0:cf:0e:04:3d:fd,dl_dst=fa:16:00:00:00:00/ff:ff:00:00:00:00,nw_dst=194.146.52.200,nw_ttl=117,nw_frag=no
Datapath actions: drop
###
Somehow destination address has not been changed and after recirc(0x130b9b)
ofproto/trace show that packet is dropped.
I have played with --ct-next param but without success.
Datapath flows:
[root@network002 /]# ovs-appctl dpctl/dump-flows -m| grep 8.8.8.8 | grep
194.146.52.200
ufid:90e41d7a-0204-46da-88a3-adfb9fb92e61,
recirc_id(0x130b9a),dp_hash(0/0),skb_priority(0/0),tunnel(tun_id=0xa5,src=10.84.69.33,dst=10.84.69.24,ttl=64/0,tp_src=18711/0,tp_dst=6081/0,geneve({class=0x102/0,type=0x80/0,len=4/0,0x30002/0}),flags(-df+csum+key)),in_port(genev_sys_6081),skb_mark(0/0),ct_state(0x62/0x3f),ct_zone(0xab/0),ct_mark(0/0x1),ct_label(0/0),ct_tuple4(src=10.44.1.174/0.0.0.0,dst=8.8.8.8/0.0.0.0,proto=1/0,tp_src=8/0,tp_dst=0/0),packet_type(ns=0,id=0),eth(src=fa:16:3e:80:da:f6,dst=00:00:5e:00:01:01),eth_type(0x0800),ipv4(src=194.146.52.200/254.0.0.0,dst=8.8.8.8/128.0.0.0,proto=1/0,tos=0/0,ttl=63/0,frag=no),icmp(type=8/0,code=0/0),
packets:5658, bytes:554484, used:0.104s, dp:ovs,
actions:ct_clear,push_vlan(vid=170,pcp=0),p1, dp-extra-info:miniflow_bits(10,2)
ufid:b34d70c8-e45a-4b5b-b9df-4f6f81196c38,
recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(p0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),packet_type(ns=0,id=0),eth(src=b0:cf:0e:04:3d:fd,dst=fa:16:3e:80:da:f6),eth_type(0x8100),vlan(vid=170,pcp=0),encap(eth_type(0x0800),ipv4(src=8.8.8.8/254.0.0.0,dst=194.146.52.200,proto=1,tos=0/0,ttl=117,frag=no),icmp(type=0/0xf8,code=0/0)),
packets:5658, bytes:577116, used:0.104s, dp:ovs,
actions:pop_vlan,ct(zone=171,nat),recirc(0x130b9b),
dp-extra-info:miniflow_bits(5,3)
Conntrack:
[root@network002 /]# ovs-appctl dpctl/dump-conntrack zone=171
icmp,orig=(src=10.44.1.174,dst=8.8.8.8,id=3714,type=8,code=0),reply=(src=8.8.8.8,dst=194.146.52.200,id=3714,type=0,code=0),zone=171
What I would suspect is for recirc(0x130b9b) destination IP would be changed to
local one and traffic be pushed to VM.
Like (changed manually):
[root@network002 /]# ovs-appctl ofproto/trace br-ex
"recirc_id=0x130b9b,ct_state=new|trk,ct_zone=171,icmp,reg0=0xfa16,reg1=0x3e80daf6,reg9=0x4,reg11=0x97,reg12=0xab,reg14=0x1,metadata=0xa5,in_port=3,vlan_tci=0x0000,dl_src=b0:cf:0e:04:3d:fd,dl_dst=fa:16:3e:80:da:f6,nw_src=8.8.8.8,nw_dst=10.44.1.174,nw_tos=0,nw_ecn=0,nw_ttl=117,nw_frag=no,icmp_type=0,icmp_code=0"
<CUT>
39. reg13=0/0xffff0000,reg15=0x4,metadata=0xa4, priority 100, cookie
0x77f10a22
set_field:0xa4/0xffffff->tun_id
set_field:0x4->tun_metadata0
move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30]
-> NXM_NX_TUN_METADATA0[16..30] is now 0x2
output:1
-> output to native tunnel
-> tunneling to 10.84.69.33 via os_tunnel
-> tunneling from 56:81:8e:92:af:64 10.84.69.24 to
9e:51:6b:8a:4b:76 10.84.69.33
bridge("br-ex")
---------------
0. priority 0
NORMAL
-> forwarding to learned port
resubmit(,40)
40. priority 0
drop
pop:NXM_OF_IN_PORT[]
-> NXM_OF_IN_PORT[] is now 3
Final flow: unchanged
Megaflow:
recirc_id=0x132219,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,icmp,in_port=3,dl_src=fa:16:3e:e2:37:bc,dl_dst=fa:16:3e:32:db:00,nw_ecn=0,nw_frag=no
Datapath actions:
ct_clear,tnl_push(tnl_port(1),header(size=58,type=5,eth(dst=9e:51:6b:8a:4b:76,src=56:81:8e:92:af:64,dl_type=0x0800),ipv4(src=10.84.69.24,dst=10.84.69.33,proto=17,tos=0,ttl=64,frag=0x4000),udp(s
rc=0,dst=6081,csum=0xffff),geneve(crit,vni=0xa4,options({class=0x102,type=0x80,len=4,0x20004}))),out_port(6)),push_vlan(vid=140,pcp=0),4
###
What I am missing here?
--
Regards,
Krzystzof Tomaszewski
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
