Hi, I am a pretty bad issue where traffic is being NAT to the external_ip when it shouldn't be.
First here are the versions of code I am running
Version: (Open vSwitch) 3.2.2
Version (OVN) 23.09.1
I am using the Mellanox connect X6 and offloading everything to hardware.
I am sending traffic between two VMs , they are connected to the same
logical switch. The customer can ping between them on the private
addresses - 172.27.34.210 -> 172.27.47.201 works fine
compute_node_a (client)
private_ip: 172.27.34.210
public_ip: X.X.X.X
ovn-vtep-if: 10.26.9.32/24
external_mac: ae:29:e9:32:52:7f
representor_port: enp148s0f1np1
ofport: 623
router_port_mac: 1a:b1:b6:11:6b:62
compute_node_b (server)
private_ip 172.27.47.201
public_ip Y.Y.Y.Y
ovn-vtep-if 10.26.17.28/24
external_mac 1e:41:55:e6:8f:46
However when the packet moves from the VM through the kernel path I
can see the TCP SYN being sent on the representor see attached pcap
file.
Note the TCP source (52776) and destination ports (8080). You can see
that the client sends a TCP SYN from 172.27.34.210 to 172.27.47.201 ,
as it moves through the kernal (slow path) to the physical interface
it NAT's to the dnat_and_snat external IP address. See the packet
egress with the public IP address.
I have no idea why this would happen on TCP only , ICMP is fine
Output from an ofproto trace below
ovs-appctl ofproto/trace br-int
in_port=623,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,tcp,nw_src=172.27.34.210,nw_dst=172.27.47.201,nw_ttl=32,tcp_src=52776,tcp_dst=8080,tcp_flags=2
Flow:
tcp,in_port=623,vlan_tci=0x0000,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,nw_src=172.27.34.210,nw_dst=172.27.47.201,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=8080,tcp_flags=syn
bridge("br-int")
----------------
0. in_port=623, priority 100, cookie 0x17928c
set_field:0x78->reg13
set_field:0xb5->reg11
set_field:0x2ab->reg12
set_field:0x1d7->metadata
set_field:0x9->reg14
resubmit(,8)
8. metadata=0x1d7, priority 50, cookie 0x78cd18a6
set_field:0/0x1000->reg10
resubmit(,73)
73.
ip,reg14=0x9,metadata=0x1d7,dl_src=ae:29:e9:32:52:7f,nw_src=172.27.34.210,
priority 90, cookie 0x17928c
set_field:0/0x1000->reg10
move:NXM_NX_REG10[12]->NXM_NX_XXREG0[111]
-> NXM_NX_XXREG0[111] is now 0
resubmit(,9)
9. metadata=0x1d7, priority 0, cookie 0x8830a146
resubmit(,10)
10. metadata=0x1d7, priority 0, cookie 0x20463edc
resubmit(,11)
11. metadata=0x1d7, priority 0, cookie 0x54ff7155
resubmit(,12)
12. ip,metadata=0x1d7, priority 100, cookie 0xac030fc6
set_field:0x1000000000000000000000000/0x1000000000000000000000000->xxreg0
resubmit(,13)
13. metadata=0x1d7, priority 0, cookie 0x30de6504
resubmit(,14)
14. ip,reg0=0x1/0x1,metadata=0x1d7, priority 100, cookie 0x3534a29
ct(table=15,zone=NXM_NX_REG13[0..15])
drop
-> A clone of the packet is forked to recirculate. The forked
pipeline will be resumed at table 15.
-> Sets the packet to an untracked state, and clears all the
conntrack fields.
Final flow:
tcp,reg0=0x1,reg11=0xb5,reg12=0x2ab,reg13=0x78,reg14=0x9,metadata=0x1d7,in_port=623,vlan_tci=0x0000,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,nw_src=172.27.34.210,nw_dst=172.27.47.201,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=8080,tcp_flags=syn
Megaflow:
recirc_id=0,eth,tcp,in_port=623,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,nw_src=172.27.34.210,nw_frag=no
Datapath actions: ct(zone=120),recirc(0x26b18)
===============================================================================
recirc(0x26b18) - resume conntrack with default ct_state=trk|new (use
--ct-next to customize)
===============================================================================
Flow:
recirc_id=0x26b18,ct_state=new|trk,ct_zone=120,eth,tcp,reg0=0x1,reg11=0xb5,reg12=0x2ab,reg13=0x78,reg14=0x9,metadata=0x1d7,in_port=623,vlan_tci=0x0000,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,nw_src=172.27.34.210,nw_dst=172.27.47.201,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=8080,tcp_flags=syn
bridge("br-int")
----------------
thaw
Resuming from table 15
15. ct_state=+new-est+trk,metadata=0x1d7, priority 7, cookie 0x6e0867dd
set_field:0x80000000000000000000000000/0x80000000000000000000000000->xxreg0
set_field:0x200000000000000000000000000/0x200000000000000000000000000->xxreg0
resubmit(,16)
16. conj_id=3485242595,tcp,reg0=0x80/0x80,reg14=0x9,metadata=0x1d7,
priority 3000, cookie 0x52a834a1
set_field:0x1000000000000/0x1000000000000->xreg4
set_field:0x2000000000000000000000000/0x2000000000000000000000000->xxreg0
resubmit(,17)
17. reg8=0x10000/0x10000,metadata=0x1d7, priority 1000, cookie 0xa729a095
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,18)
18. metadata=0x1d7, priority 0, cookie 0x96baa2f
resubmit(,19)
19. metadata=0x1d7, priority 0, cookie 0x3216af2f
resubmit(,20)
20. metadata=0x1d7, priority 0, cookie 0x9e83bb36
resubmit(,21)
21. metadata=0x1d7, priority 0, cookie 0x3fc32b6a
resubmit(,22)
22. metadata=0x1d7, priority 0, cookie 0x841766eb
resubmit(,23)
23. metadata=0x1d7, priority 0, cookie 0x6ea80e05
resubmit(,24)
24. metadata=0x1d7, priority 0, cookie 0x4d31ab9
resubmit(,25)
25. metadata=0x1d7, priority 0, cookie 0xfe78236
resubmit(,26)
26. metadata=0x1d7, priority 0, cookie 0x2b009c14
resubmit(,27)
27. metadata=0x1d7, priority 0, cookie 0x7b6bf676
set_field:0/0x1000000000000->xreg4
set_field:0/0x2000000000000->xreg4
set_field:0/0x4000000000000->xreg4
resubmit(,28)
28. ip,reg0=0x2/0x2002,metadata=0x1d7, priority 100, cookie 0x36a19530
ct(commit,zone=NXM_NX_REG13[0..15],nat(src),exec(set_field:0/0x1->ct_mark))
nat(src)
set_field:0/0x1->ct_mark
-> Sets the packet to an untracked state, and clears all the
conntrack fields.
resubmit(,29)
29. metadata=0x1d7, priority 0, cookie 0xa36067fa
resubmit(,30)
30. metadata=0x1d7, priority 0, cookie 0xb56c19e
resubmit(,31)
31. metadata=0x1d7, priority 0, cookie 0x2d9fecef
resubmit(,32)
32. metadata=0x1d7, priority 0, cookie 0xad0aad2f
resubmit(,33)
33. metadata=0x1d7, priority 0, cookie 0x4e92d423
resubmit(,34)
34. metadata=0x1d7, priority 0, cookie 0x9fe31110
resubmit(,35)
35. metadata=0x1d7,dl_dst=1e:41:55:e6:8f:46, priority 50, cookie 0x6ea924d2
set_field:0x15->reg15
resubmit(,37)
37. priority 0
resubmit(,39)
39. reg15=0x15,metadata=0x1d7, priority 100, cookie 0xcdd83576
set_field:0x1d7/0xffffff->tun_id
set_field:0x15->tun_metadata0
move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30]
-> NXM_NX_TUN_METADATA0[16..30] is now 0x9
output:777
-> output to kernel tunnel
resubmit(,40)
40. priority 0
drop
Final flow:
recirc_id=0x26b18,eth,tcp,reg0=0x283,reg11=0xb5,reg12=0x2ab,reg13=0x78,reg14=0x9,reg15=0x15,tun_id=0x1d7,metadata=0x1d7,in_port=623,vlan_tci=0x0000,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,nw_src=172.27.34.210,nw_dst=172.27.47.201,nw_tos=0,nw_ecn=0,nw_ttl=32,nw_frag=no,tp_src=52776,tp_dst=8080,tcp_flags=syn
Megaflow:
recirc_id=0x26b18,ct_state=+new-est-rel-rpl-inv+trk,ct_mark=0/0x1,eth,tcp,in_port=623,dl_src=ae:29:e9:32:52:7f,dl_dst=1e:41:55:e6:8f:46,nw_src=172.27.32.0/19,nw_dst=172.27.40.0/21,nw_ecn=0,nw_frag=no,tp_src=0x8000/0x8000,tp_dst=0x1000/0xf000
Datapath actions:
ct(commit,zone=120,mark=0/0x1,nat(src)),set(tunnel(tun_id=0x1d7,dst=10.26.17.28,ttl=64,tp_dst=6081,geneve({class=0x102,type=0x80,len=4,0x90015}),flags(df|csum|key))),2
representor-enp148s0f0_1.pcap
Description: Binary data
physcial-enp148s0f0np0-export.pcap
Description: Binary data
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
