On Tue, Mar 12, 2024 at 2:53 AM Eelco Chaudron <echau...@redhat.com> wrote:
>
>
>
> On 12 Mar 2024, at 4:16, Zhangweiwei via discuss wrote:
>
> > Hi,
> > I send an ICMP echo request packet from port tapVm71( IP: 1.1.70.1) to
> > tapVm72( IP: 1.1.70.2) with CT, and there are two ICMP reply packets which
> > are captured on tapVm72. We can see the ICMP reply packet sent by tapVm72
> > is mirrored twice within recirc_id(0) and recirc_id(0xe). This issue is
> > caused by patch (ofproto-dpif-upcall: Mirror packets that are modified,
> > https://github.com/openvswitch/ovs/commit/feed7f6775056b3dd55249596a7e587bc9c5fd4a).
> > The patch resets the mirrors every time when a packet is modified. In my
> > pipline, the NXM_NX_REG4[31] is modified,thus the ICMP reply packet is
> > mirrored again in recirc_id(0xe).
> > The condition for resetting the mirror in reset_mirror_ctx() seems not
> > strict enough, which can lead to duplicated mirror during recirclation.
>
> Mike can you take a look at this? It was introduced after your patch.
Yes, it looks like in set_field we don't reset the mirror in case of
tunnel metadata, but do for other metadata fields. I can prepare a
patch for that.
-M
>
> > 1、ovs version: 3.2.1
> >
> > 2、dpcls
> > [root@localhost ~]# ovs-appctl dpctl/dump-flows -m
> > flow-dump from pmd on cpu core: 1
> > ufid:bc206dc0-dcfb-4396-894a-695656ffe6cc,
> > recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(tapVm72),skb_mark(0/0),ct_state(0/0x20),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),packet_type(ns=0,id=0),eth(src=52:54:00:9a:bf:ed,dst=52:54:00:67:d5:61/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=1.1.70.2,dst=1.1.70.1/0.0.0.0,proto=1,tos=0/0,ttl=64/0,frag=no),icmp(type=0/0,code=0/0),
> > packets:449, bytes:44002, used:0.086s, dp:ovs,
> > actions:mitapVm72,ct(zone=6),recirc(0xe), dp-extra-info:miniflow_bits(5,2)
> > ufid:fbd3db23-9e12-4f6b-826e-5672d33856ae,
> > recirc_id(0),dp_hash(0/0),skb_priority(0/0),in_port(tapVm71),skb_mark(0/0),ct_state(0/0x3f),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),packet_type(ns=0,id=0),eth(src=52:54:00:67:d5:61,dst=52:54:00:9a:bf:ed),eth_type(0x0800),ipv4(src=1.1.70.1,dst=1.1.70.2/255.255.192.0,proto=1,tos=0/0,ttl=64/0,frag=no),icmp(type=8/0,code=0/0),
> > packets:449, bytes:44002, used:0.086s, dp:ovs,
> > actions:ct(zone=6),recirc(0xd), dp-extra-info:miniflow_bits(5,2)
> > ufid:63728c39-8203-4dd3-8d48-1dd1a2ec999e,
> > recirc_id(0xe),dp_hash(0/0),skb_priority(0/0),in_port(tapVm72),skb_mark(0/0),ct_state(0x2a/0x3f),ct_zone(0x6/0),ct_mark(0/0),ct_label(0/0),ct_tuple4(src=1.1.70.1/0.0.0.0,dst=1.1.70.2/0.0.0.0,proto=1/0,tp_src=8/0,tp_dst=0/0),packet_type(ns=0,id=0),eth(src=52:54:00:9a:bf:ed/00:00:00:00:00:00,dst=52:54:00:67:d5:61),eth_type(0x0800),ipv4(src=1.1.70.2/0.0.0.0,dst=1.1.70.1/255.255.192.0,proto=1,tos=0/0,ttl=64/0,frag=no),icmp(type=0/0,code=0/0),
> > packets:449, bytes:44002, used:0.086s, dp:ovs, actions:mitapVm72,tapVm71,
> > dp-extra-info:miniflow_bits(5,2)
> > ufid:54af771a-7c46-4b33-b207-93b219d014a4,
> > recirc_id(0xd),dp_hash(0/0),skb_priority(0/0),in_port(tapVm71),skb_mark(0/0),ct_state(0x22/0xf),ct_zone(0x6/0),ct_mark(0/0),ct_label(0/0),ct_tuple4(src=1.1.70.1/0.0.0.0,dst=1.1.70.2/0.0.0.0,proto=1/0,tp_src=8/0,tp_dst=0/0),packet_type(ns=0,id=0),eth(src=52:54:00:67:d5:61/00:00:00:00:00:00,dst=52:54:00:9a:bf:ed/00:00:00:00:00:00),eth_type(0x0800),ipv4(src=1.1.70.1/0.0.0.0,dst=1.1.70.2/0.0.0.0,proto=1/0,tos=0/0,ttl=64/0,frag=no),icmp(type=8/0,code=0/0),
> > packets:448, bytes:43904, used:0.086s, dp:ovs, actions:tapVm72,mitapVm72,
> > dp-extra-info:miniflow_bits(4,1)
> >
> > 3、Bridge
> > [root@localhost openvswitch-3.2.1]# ovs-vsctl show
> > Bridge vds1-br
> > Controller "tcp:172.20.66.228:6633"
> > is_connected: true
> > Controller "tcp:172.20.66.229:6633"
> > is_connected: true
> > fail_mode: secure
> > datapath_type: netdev
> > Port vxlan_vds1-br
> > Interface vxlan_vds1-br
> > type: vxlan
> > options: {key=flow, local_ip="3.3.3.70", remote_ip=flow,
> > tos=inherit}
> > Port tapVm72
> > tag: 0
> > Interface tapVm72
> > type: dpdkvhostuserclient
> > options: {vhost-server-path="/var/run/openvswitch/tapVm72"}
> > Port mitapVm72
> > Interface mitapVm72
> > Port tapVm71
> > tag: 0
> > Interface tapVm71
> > type: dpdkvhostuserclient
> > options: {vhost-server-path="/var/run/openvswitch/tapVm71"}
> > Port vds1-br
> > Interface vds1-br
> > type: internal
> > ovs_version: "3.2.1"
> >
> > 4、packets
> > [root@localhost ~]# ovs-tcpdump -i tapVm72
> > dropped privs to tcpdump
> > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> > listening on mitapVm72, link-type EN10MB (Ethernet), capture size 262144
> > bytes
> >
> > 10:03:30.173595 IP 1.1.70.1 > 1.1.70.2: ICMP echo request, id 16616, seq 1,
> > length 64
> > 10:03:30.173696 IP 1.1.70.2 > 1.1.70.1: ICMP echo reply, id 16616, seq 1,
> > length 64
> > 10:03:30.173697 IP 1.1.70.2 > 1.1.70.1: ICMP echo reply, id 16616, seq 1,
> > length 64
> >
> > 5、track
> > [root@localhost~]# ovs-appctl ofproto/trace vds1-br in_port=tapVm72
> > 52540067d5615254009abfed080045000054f9ba00004001f2e901014602010146010000a0761bcd0001c487ea6500000000cbfa0a0000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363752540067d5615254009abfed080045000054f9ba00004001f2e901014602010146010000a0761bcd0001c487ea6500000000cbfa0a0000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637
> > Flow:
> > icmp,in_port=6,vlan_tci=0x0000,dl_src=52:54:00:9a:bf:ed,dl_dst=52:54:00:67:d5:61,nw_src=1.1.70.2,nw_dst=1.1.70.1,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=0,icmp_code=0
> >
> > bridge("vds1-br")
> > -----------------
> > 0. priority 0, cookie 0x1000000000000001
> > goto_table:5
> > 5. priority 0, cookie 0x1000000000000001
> > group:553648133
> > bucket 0
> > resubmit(,166)
> > 166. No match.
> > drop
> > goto_table:12
> > 12. priority 0, cookie 0x1000000000000001
> > goto_table:15
> > 15. in_port=6, priority 30000, cookie 0x1000000000000000
> > write_metadata:0x500000000
> > goto_table:17
> > 17. ip,metadata=0x500000000/0x80ffffff0fff0000,in_port=6, priority 30000,
> > cookie 0x1000000000000000
> > load:0x1->NXM_NX_REG4[0..30]
> > goto_table:20
> > 20. priority 0, cookie 0x1000000000000001
> > goto_table:21
> > 21. priority 0, cookie 0x1000000000000001
> > goto_table:25
> > 25. priority 0, cookie 0x1000000000000001
> > goto_table:30
> > 30.
> > ip,metadata=0x500000000/0x80ffffff0fff0000,in_port=6,dl_src=52:54:00:9a:bf:ed,nw_src=1.1.70.2,
> > priority 30000, cookie 0x1000000000000000
> > goto_table:35
> > 35. metadata=0/0x8000000000000000, priority 30000, cookie 0x1000000000000000
> >
> > learn(table=151,hard_timeout=300,priority=10000,OXM_OF_METADATA[32..55],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:NXM_OF_IN_PORT[]->OXM_OF_METADATA[0..15],load:OXM_OF_METADATA[16..27]->OXM_OF_METADATA[16..27])
> > -> table=151
> > metadata=0x500000000/0xffffff00000000,dl_dst=52:54:00:9a:bf:ed
> > priority=10000 hard=300
> > actions=load:0x6->OXM_OF_METADATA[0..15],load:0->OXM_OF_METADATA[16..27]
> > goto_table:40
> > 40. ip, priority 30000, cookie 0x1000000000000000
> > goto_table:75
> > 75. ct_state=-trk,ip,metadata=0x500000000/0x80ffffff0fff0000,in_port=6,
> > priority 30000, cookie 0x1020000000000000
> > ct(table=76,zone=6)
> > drop
> > -> A clone of the packet is forked to recirculate. The forked pipeline
> > will be resumed at table 76.
> > -> Sets the packet to an untracked state, and clears all the conntrack
> > fields.
> >
> > Final flow:
> > icmp,reg4=0x1,metadata=0x500000000,in_port=6,vlan_tci=0x0000,dl_src=52:54:00:9a:bf:ed,dl_dst=52:54:00:67:d5:61,nw_src=1.1.70.2,nw_dst=1.1.70.1,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=0,icmp_code=0
> > Megaflow:
> > recirc_id=0,ct_state=-trk,eth,icmp,in_port=6,dl_src=52:54:00:9a:bf:ed,nw_src=1.1.70.2,nw_frag=no
> > Datapath actions: 5,ct(zone=6),recirc(0xe)
> >
> > ===============================================================================
> > recirc(0xe) - resume conntrack with default ct_state=trk|new (use --ct-next
> > to customize)
> > ===============================================================================
> >
> > Flow:
> > recirc_id=0xe,ct_state=new|trk,ct_zone=6,eth,icmp,reg4=0x1,metadata=0x500000000,in_port=6,vlan_tci=0x0000,dl_src=52:54:00:9a:bf:ed,dl_dst=52:54:00:67:d5:61,nw_src=1.1.70.2,nw_dst=1.1.70.1,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=0,icmp_code=0
> >
> > bridge("vds1-br")
> > -----------------
> > thaw
> > Resuming from table 76
> > 76. ct_state=+new-est,ip,metadata=0x500000000/0x80ffffff0fff0000,in_port=6,
> > priority 50000, cookie 0x1020000000000000
> > ct(commit,zone=6)
> > drop
> > -> Sets the packet to an untracked state, and clears all the conntrack
> > fields.
> > goto_table:78
> > 78. priority 0, cookie 0x1000000000000001
> > goto_table:80
> > 80. priority 0, cookie 0x1000000000000001
> > goto_table:85
> > 85. priority 0, cookie 0x1000000000000001
> > goto_table:87
> > 87. priority 0, cookie 0x1000000000000000
> > group:553648129
> > bucket 0
> > resubmit(,160)
> > 160. No match.
> > drop
> > bucket 1
> > resubmit(,145)
> > 145. No match.
> > drop
> > goto_table:90
> > 90. ip,metadata=0x500000000/0x80ffffff0fff0000,in_port=6, priority 30000,
> > cookie 0x1000000000000000
> > load:0x1->NXM_NX_REG4[0..30]
> > goto_table:92
> > 92. ip,reg4=0x1,nw_dst=1.1.0.0/16, priority 30000, cookie 0x1000000000000000
> > load:0x1->NXM_NX_REG4[31]
> > goto_table:100
> > 100. priority 0, cookie 0x1000000000000001
> > goto_table:105
> > 105. priority 0, cookie 0x1000000000000001
> > goto_table:110
> > 110. dl_dst=00:00:00:00:00:00/01:00:00:00:00:00, priority 30000, cookie
> > 0x1000000000000000
> > goto_table:115
> > 115. priority 0, cookie 0x1000000000000001
> > goto_table:120
> > 120. priority 0, cookie 0x1000000000000001
> > goto_table:130
> > 130. priority 0, cookie 0x1000000000000001
> > goto_table:140
> > 140. priority 0, cookie 0x1000000000000001
> > goto_table:150
> > 150. metadata=0x500000000/0xffffff00000000,dl_dst=52:54:00:67:d5:61,
> > priority 30000, cookie 0x1000000000000000
> > write_metadata:0x500000003/0xffffff0fffffff
> > goto_table:230
> > 230. priority 0, cookie 0x1000000000000001
> > goto_table:231
> > 231. priority 0, cookie 0x1000000000000001
> > goto_table:233
> > 233. priority 0, cookie 0x1000000000000001
> > goto_table:235
> > 235. priority 0, cookie 0x1000000000000001
> > goto_table:240
> > 240. priority 0, cookie 0x1000000000000001
> > goto_table:245
> > 245. priority 0, cookie 0x1000000000000000
> > group:553648130
> > bucket 0
> > resubmit(,162)
> > 162. No match.
> > drop
> > bucket 1
> > resubmit(,171)
> > 171. No match.
> > drop
> > bucket 2
> > resubmit(,145)
> > 145. No match.
> > drop
> > goto_table:249
> > 249. metadata=0x500000003/0xffffff0fffffff, priority 30000, cookie
> > 0x1000000000000000
> > output:3
> >
> > Final flow:
> > recirc_id=0xe,eth,icmp,reg4=0x80000001,metadata=0x500000003,in_port=6,vlan_tci=0x0000,dl_src=52:54:00:9a:bf:ed,dl_dst=52:54:00:67:d5:61,nw_src=1.1.70.2,nw_dst=1.1.70.1,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,icmp_type=0,icmp_code=0
> > Megaflow:
> > recirc_id=0xe,ct_state=+new-est-rel-rpl-inv+trk,eth,icmp,in_port=6,dl_dst=52:54:00:67:d5:61,nw_dst=1.1.64.0/18,nw_frag=no
> > Datapath actions: 5,ct(commit,zone=6),4
> >
> > [root@localhost ~]# ovs-appctl dpif/show
> > netdev@ovs-netdev: hit:14940 missed:34
> > vds1-br:
> > mitapVm72 5/5: (system)
> > tapVm71 3/4: (dpdkvhostuserclient: configured_rx_queues=1,
> > configured_tx_queues=1, mtu=1500, requested_rx_queues=1,
> > requested_tx_queues=1)
> > tapVm72 6/6: (dpdkvhostuserclient: configured_rx_queues=1,
> > configured_tx_queues=1, mtu=1500, requested_rx_queues=1,
> > requested_tx_queues=1)
> > vds1-br 65534/2: (tap)
> > vxlan_vds1-br 2/7: (vxlan: key=flow, local_ip=3.3.3.70, remote_ip=flow,
> > tos=inherit)
> >
> >
> > Best regards, WeiweiZhang
> >
> > -------------------------------------------------------------------------------------------------------------------------------------
> > 本邮件及其附件含有新华三集团的保密信息，仅限于发送给上面地址中列出
> > 的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
> > 或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
> > 邮件！
> > This e-mail and its attachments contain confidential information from New
> > H3C, which is
> > intended only for the person or entity whose address is listed above. Any
> > use of the
> > information contained herein in any way (including, but not limited to,
> > total or partial
> > disclosure, reproduction, or dissemination) by persons other than the
> > intended
> > recipient(s) is prohibited. If you receive this e-mail in error, please
> > notify the sender
> > by phone or email immediately and delete it!
>
> > _______________________________________________
> > discuss mailing list
> > disc...@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
>
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
