On 12/22/23 17:33, Celyn Birkinshaw via discuss wrote: > Hi, > > I've made a topology of four OVS (version 2.17.8) switches in a diamond, > two endpoints, and I'm using a Ryu controller. Installed in br1 is a group > and a flow entry to direct packets out via buckets, toward br2 and br3. > The group is set via the Ryu API in an OpenFlow v1.5 OFPGroupMod message. > The problem is that I cannot trace packets through the buckets since there > is 'no live bucket' in the group. However, if I check the group using > ovs-ofctl, it looks ok.
It is an issue of the trace output, it should say something like "selection method in use: dp_hash, recirculating" instead of "no live bucket". You can see the actual datapath actions at the end of all your traces: Datapath actions: hash(l4(0)),recirc(0x1b) OpenFlow layer doesn't know what the hash will be, so it can't know to which bucket the packet will be directed to. It instructs the datapath to calculate the hash and recirculate the packet, once the hash is known the packet will be passed through the OpenFlow tables again and the bucket will be selected. You may add dp_hash=0xabcd to the flow definition of the trace command to see to which bucket the packet with a particular hash will go. But ovs-vswitchd in general doesn't know what the hash value will be, because it depends on datapath implementation. Actual traffic should work fine, it's only a trace output that is misleading. > Also, if I remove it and reinstall it via ovs-ofctl, > I can trace a packet through a bucket but I have to set the group selection > method to 'hash'. 'hash' is showing the trace because in this case the hash is calculated in userspace, so OpenFlow layer can calculate the hash right away and figure out which bucket it will go. > I'm looking at how to include the selection method in the > properties of an OFPGroupMod message, or how to set the default selection > method in the switches, or any other suggestions. If you still want to use the 'selection_method' property, it is a Netronome extension NTRT_SELECTION_METHOD. See more details here: https://raw.githubusercontent.com/openvswitch/ovs/master/Documentation/group-selection-method-property.txt Best regards, Ilya Maximets. > Here is a set of commands to show what I'm seeing in OVS (initial group > installed via Ryu's API): > > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ./groups.sh > br1 > OFPST_GROUP_DESC reply (OF1.5) (xid=0x2): > > group_id=1,type=select,bucket=bucket_id:0,weight:10,actions=output:"vp1",bucket=bucket_id:1,weight:10,actions=output:"vp2" > > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ./flows.sh > br1 > cookie=0x1, duration=8.464s, table=0, n_packets=0, n_bytes=0, idle_age=8, > priority=1,in_port="fd4352db8bf24_l",dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12 > actions=group:1 > > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ sudo ovs-appctl ofproto/trace br1 > in_port=3,tcp,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3 > Flow: > tcp,in_port=3,vlan_tci=0x0000,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3,nw_tos=0,nw_ecn=0,nw_ttl=0,nw_frag=no,tp_src=0,tp_dst=0,tcp_flags=0 > > bridge("br1") > ------------- > 0. in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12, priority 1, > cookie 0x1 > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: > recirc_id=0,eth,ip,in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=0.0.0.0/2,nw_dst=0.0.0.0/2,nw_frag=no > <http://0.0.0.0/2,nw_dst=0.0.0.0/2,nw_frag=no> > Datapath actions: hash(l4(0)),recirc(0x1b) > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl del-groups br1 > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl add-group br1 > group_id=1,type=select,bucket=bucket_id:0,weight:10,actions=output:"vp1",bucket=bucket_id:1,weight:10,actions=output:"vp2" > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl add-flow br1 > in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,actions=group:1 > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ sudo ovs-appctl ofproto/trace br1 > in_port=3,tcp,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3 > Flow: > tcp,in_port=3,vlan_tci=0x0000,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3,nw_tos=0,nw_ecn=0,nw_ttl=0,nw_frag=no,tp_src=0,tp_dst=0,tcp_flags=0 > > bridge("br1") > ------------- > 0. in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12, priority > 32768 > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: > recirc_id=0,eth,ip,in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=0.0.0.0/2,nw_dst=0.0.0.0/2,nw_frag=no > <http://0.0.0.0/2,nw_dst=0.0.0.0/2,nw_frag=no> > Datapath actions: hash(l4(0)),recirc(0x1c) > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl del-groups br1 > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl add-group br1 > group_id=1,type=select,selection_method=dp_hash,bucket=bucket_id:0,weight:10,actions=output:"vp1",bucket=bucket_id:1,weight:10,actions=output:"vp2" > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl add-flow br1 > in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,actions=group:1 > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ sudo ovs-appctl ofproto/trace br1 > in_port=3,tcp,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3 > Flow: > tcp,in_port=3,vlan_tci=0x0000,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3,nw_tos=0,nw_ecn=0,nw_ttl=0,nw_frag=no,tp_src=0,tp_dst=0,tcp_flags=0 > > bridge("br1") > ------------- > 0. in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12, priority > 32768 > group:1 > -> no live bucket > > Final flow: unchanged > Megaflow: > recirc_id=0,eth,ip,in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=0.0.0.0/2,nw_dst=0.0.0.0/2,nw_frag=no > <http://0.0.0.0/2,nw_dst=0.0.0.0/2,nw_frag=no> > Datapath actions: hash(l4(0)),recirc(0x1d) > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl del-groups br1 > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl add-group br1 > group_id=1,type=select,selection_method=hash,bucket=bucket_id:0,weight:10,actions=output:"vp1",bucket=bucket_id:1,weight:10,actions=output:"vp2" > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ ovs-ofctl add-flow br1 > in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,actions=group:1 > ubuntu@ip-172-31-18-197:~/sdn/ovs_scripts$ sudo ovs-appctl ofproto/trace br1 > in_port=3,tcp,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3 > Flow: > tcp,in_port=3,vlan_tci=0x0000,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12,nw_src=10.2.0.2,nw_dst=10.2.0.3,nw_tos=0,nw_ecn=0,nw_ttl=0,nw_frag=no,tp_src=0,tp_dst=0,tcp_flags=0 > > bridge("br1") > ------------- > 0. in_port=3,dl_src=de:9c:43:99:7c:46,dl_dst=ce:5c:1e:cd:2b:12, priority > 32768 > group:1 > -> bucket 0: score 377590 > -> bucket 1: score 6450 > -> using bucket 0 > bucket 0 > output:1 > > Thanks, > > Celyn _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
