I want to report progress I've made tonight on an extended form of the
OVS sandbox that I expect to be useful for testing OVN. The main point
of the extension is to make it easy to start up multiple sandboxed
instances of OVS and then connect them together.
What I have is at:
https://github.com/blp/ovs-reviews/tree/ovn-sandbox
Please don't bother to review it; the code and its history is a mess.
But I want to demonstrate some of the potential. If you check this out
and build, then run a command like this (assuming that you keep your
build directory as a subdirectory of your source directory as I do;
otherwise adjust the path on --init):
utilities/ovs-sandbox --ovn --init=../ovn/sbtest
Before I show what this does, take a look at sbtest. It's a shell
script that uses a mix of OVS/OVN commands and some extra commands
provided by ovs-sandbox:
# Create a pair of sandboxes.
sb_add hv0
sb_add hv1
# Create a network for connecting sandboxes together,
# then attach our sandboxes to it with specified IP addresses,
# via a new bridge in each of hv0 and hv1 called "br-phys":
net_add n1
ovn_attach n1 hv0 br-phys 192.168.0.1
ovn_attach n1 hv1 br-phys 192.168.0.2
# Create logical ports and networks:
ovn-nbctl lswitch-add br0
ovn-nbctl lport-add br0 lp0
ovn-nbctl lport-set-macs lp0 f0:00:00:00:00:00
ovn-nbctl lport-add br0 lp1
ovn-nbctl lport-set-macs lp1 f0:11:11:11:11:11
# Create ports on hv0 and hv1 that instantiate lp0 and lp1
# (the syntax "as $sandbox" means to run the following command
# in the context of $sandbox":
as hv0 ovs-vsctl add-port br-int vif0 -- set Interface vif0 type=dummy
external-ids:iface-id=lp0
as hv1 ovs-vsctl add-port br-int vif1 -- set Interface vif1 type=dummy
external-ids:iface-id=lp1
# Run once for ARP resolution.
as hv0 ovn-trace vif0 'f01111111111f000000000001234'
# Run again for end-to-end test.
as hv0 ovn-trace vif0 'f01111111111f000000000001234'
The interesting part is the pair of ovn-trace calls at the end. This
injects a packet into the (simulated) system at the specified interface,
then waits 1 second, then greps all the system's logs to find out what
happened to the packet and dumps out the details. So the first trace
shows us how the first packet gets encapsulated in a Geneve tunnel, and
then that Geneve packet gets dropped when hv0 finds that it doesn't have
the MAC address of hv1 and sends out an ARP:
----------------------------------------------------------------------
---- 0x80000000 ---- /home/blp/nicira/ovs/_build/sandbox/hv0/ovs-vswitchd.log
----
received on vif0:
pkt_mark=0x80000000,in_port=ANY,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
---- 0x80000000 ---- /home/blp/nicira/ovs/_build/sandbox/hv0/ovs-vswitchd.log
----
Bridge: br-int
Flow:
pkt_mark=0x80000000,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=0 cookie=0 priority=100,in_port=2
OpenFlow actions=set_field:0x1->metadata,set_field:0x1->reg6,resubmit(,16)
Resubmitted flow:
pkt_mark=0x80010000,reg6=0x1,metadata=0x1,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0
reg6=0x1 reg7=0x0
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_type=0x1234
Rule: table=16 cookie=0 priority=50,reg6=0x1,metadata=0x1
OpenFlow actions=resubmit(,17)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0
reg5=0x0 reg6=0x1 reg7=0x0
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=17 cookie=0
priority=50,metadata=0x1,dl_dst=f0:11:11:11:11:11
OpenFlow actions=set_field:0x2->reg7,resubmit(,18)
Resubmitted flow:
pkt_mark=0x80010000,reg6=0x1,reg7=0x2,metadata=0x1,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0
reg4=0x0 reg5=0x0 reg6=0x1 reg7=0x2
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=18 cookie=0 priority=0,metadata=0x1
OpenFlow actions=resubmit(,19)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0
reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x1 reg7=0x2
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=19 cookie=0
priority=50,reg7=0x2,metadata=0x1
OpenFlow actions=resubmit(,64)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x0 reg1=0x0
reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x1 reg7=0x2
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=64 cookie=0
priority=50,reg7=0x2
OpenFlow
actions=set_field:0x2->tun_id,output:1
output to native tunnel
tunneling to 192.168.0.2 via br-phys
ARP cache miss for 192.168.0.2 on
bridge br-phys, sending ARP request
Final flow:
pkt_mark=0x80010000,reg6=0x1,reg7=0x2,tun_id=0x2,metadata=0x1,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,tun_id=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Datapath actions: set(skb_mark(0x80010000/0xffff0000))
This flow is handled by the userspace slow path because it:
- Triggers tracing.
----------------------------------------------------------------------
When the second packet is sent, ARP resolution is complete, so we can
follow the path of the packet all the way from vif0 to vif1:
----------------------------------------------------------------------
---- 0x80000000 ---- /home/blp/nicira/ovs/_build/sandbox/hv0/ovs-vswitchd.log
----
received on vif0:
pkt_mark=0x80000000,in_port=ANY,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
---- 0x80000000 ---- /home/blp/nicira/ovs/_build/sandbox/hv0/ovs-vswitchd.log
----
Bridge: br-int
Flow:
pkt_mark=0x80000000,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=0 cookie=0 priority=100,in_port=2
OpenFlow actions=set_field:0x1->metadata,set_field:0x1->reg6,resubmit(,16)
Resubmitted flow:
pkt_mark=0x80010000,reg6=0x1,metadata=0x1,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0
reg6=0x1 reg7=0x0
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_type=0x1234
Rule: table=16 cookie=0 priority=50,reg6=0x1,metadata=0x1
OpenFlow actions=resubmit(,17)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0
reg5=0x0 reg6=0x1 reg7=0x0
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=17 cookie=0
priority=50,metadata=0x1,dl_dst=f0:11:11:11:11:11
OpenFlow actions=set_field:0x2->reg7,resubmit(,18)
Resubmitted flow:
pkt_mark=0x80010000,reg6=0x1,reg7=0x2,metadata=0x1,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0
reg4=0x0 reg5=0x0 reg6=0x1 reg7=0x2
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=18 cookie=0 priority=0,metadata=0x1
OpenFlow actions=resubmit(,19)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0
reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x1 reg7=0x2
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=19 cookie=0
priority=50,reg7=0x2,metadata=0x1
OpenFlow actions=resubmit(,64)
Resubmitted flow: unchanged
Resubmitted regs: reg0=0x0 reg1=0x0
reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x1 reg7=0x2
Resubmitted odp: drop
Resubmitted megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=64 cookie=0
priority=50,reg7=0x2
OpenFlow
actions=set_field:0x2->tun_id,output:1
output to native tunnel
tunneling to 192.168.0.2 via br-phys
tunneling from fa:a8:6b:4c:97:43
192.168.0.1 to 7a:ca:d9:a0:59:42 192.168.0.2
Final flow:
pkt_mark=0x80010000,reg6=0x1,reg7=0x2,tun_id=0x2,metadata=0x1,in_port=2,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Megaflow:
pkt_mark=0x80000000/0xffff0000,recirc_id=0,reg6=0,reg7=0,tun_id=0,metadata=0,in_port=2,vlan_tci=0x0000/0x1000,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Datapath actions:
set(skb_mark(0x80010000/0xffff0000)),tnl_push(tnl_port(6081),header(size=50,type=5,eth(dst=7a:ca:d9:a0:59:42,src=fa:a8:6b:4c:97:43,dl_type=0x0800),ipv4(src=192.168.0.1,dst=192.168.0.2,proto=17,tos=0,ttl=64,frag=0x40),udp(src=0,dst=6081,csum=0x0),geneve(vni=0x2)),out_port(1))
This flow is handled by the userspace slow path because it:
- Triggers tracing.
---- 0x80010000 ---- /home/blp/nicira/ovs/_build/sandbox/hv0/ovs-vswitchd.log
----
Bridge: br-phys
Flow:
pkt_mark=0x80010000,udp,in_port=LOCAL,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
Rule: table=0 cookie=0 priority=0
OpenFlow actions=NORMAL
forwarding to learned port
Final flow:
pkt_mark=0x80020000,udp,in_port=LOCAL,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
Megaflow:
pkt_mark=0x80010000/0xffff0000,recirc_id=0,ip,in_port=LOCAL,vlan_tci=0x0000/0x1fff,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_frag=no
Datapath actions: set(skb_mark(0x80020000/0xffff0000)),101
This flow is handled by the userspace slow path because it:
- Triggers tracing.
---- 0x80020000 ----
/home/blp/nicira/ovs/_build/sandbox/sandbox/ovs-vswitchd.log ----
received on hv0_br-phys:
pkt_mark=0x80020000,udp,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
---- 0x80020000 ----
/home/blp/nicira/ovs/_build/sandbox/sandbox/ovs-vswitchd.log ----
Bridge: n1
Flow:
pkt_mark=0x80020000,udp,in_port=1,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
Rule: table=0 cookie=0 priority=0
OpenFlow actions=NORMAL
forwarding to learned port
Final flow:
pkt_mark=0x80030000,udp,in_port=1,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
Megaflow:
pkt_mark=0x80020000/0xffff0000,recirc_id=0,ip,in_port=1,vlan_tci=0x0000/0x1fff,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_frag=no
Datapath actions: set(skb_mark(0x80030000/0xffff0000)),3
This flow is handled by the userspace slow path because it:
- Triggers tracing.
---- 0x80020000 ---- /home/blp/nicira/ovs/_build/sandbox/hv0/ovs-vswitchd.log
----
sent on br-phys_n1:
pkt_mark=0x80020000,udp,in_port=1,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
---- 0x80030000 ----
/home/blp/nicira/ovs/_build/sandbox/sandbox/ovs-vswitchd.log ----
sent on hv1_br-phys:
pkt_mark=0x80030000,udp,in_port=2,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
---- 0x80030000 ---- /home/blp/nicira/ovs/_build/sandbox/hv1/ovs-vswitchd.log
----
received on br-phys_n1:
pkt_mark=0x80030000,udp,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
---- 0x80030000 ---- /home/blp/nicira/ovs/_build/sandbox/hv1/ovs-vswitchd.log
----
Bridge: br-phys
Flow:
pkt_mark=0x80030000,udp,in_port=1,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
Rule: table=0 cookie=0 priority=0
OpenFlow actions=NORMAL
forwarding to learned port
Final flow:
pkt_mark=0x80040000,udp,in_port=1,vlan_tci=0x0000,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,nw_dst=192.168.0.2,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=32805,tp_dst=6081
Megaflow:
pkt_mark=0x80030000/0xffff0000,recirc_id=0,udp,in_port=1,vlan_tci=0x0000/0x1fff,dl_src=fa:a8:6b:4c:97:43,dl_dst=7a:ca:d9:a0:59:42,nw_src=192.168.0.1,tp_dst=6081
Datapath actions: set(skb_mark(0x80040000/0xffff0000)),tnl_pop(6081)
This flow is handled by the userspace slow path because it:
- Triggers tracing.
---- 0x80040000 ---- /home/blp/nicira/ovs/_build/sandbox/hv1/ovs-vswitchd.log
----
Bridge: br-int
Flow:
pkt_mark=0x80040000,tun_id=0x2,tun_src=192.168.0.1,tun_dst=192.168.0.2,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=64,key,in_port=1,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Rule: table=0 cookie=0 priority=50,tun_id=0x2
OpenFlow actions=output:2
Final flow:
pkt_mark=0x80050000,tun_id=0x2,tun_src=192.168.0.1,tun_dst=192.168.0.2,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=64,key,in_port=1,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
Megaflow:
pkt_mark=0x80040000,recirc_id=0,tun_id=0x2,tun_src=192.168.0.1,tun_dst=192.168.0.2,tun_tos=0,tun_ttl=64,key,in_port=1,dl_type=0x1234
Datapath actions: set(skb_mark(0x80050000)),3
This flow is handled by the userspace slow path because it:
- Triggers tracing.
---- 0x80050000 ---- /home/blp/nicira/ovs/_build/sandbox/hv1/ovs-vswitchd.log
----
sent on vif1:
pkt_mark=0x80050000,tun_id=0x2,tun_src=192.168.0.1,tun_dst=192.168.0.2,tun_gbp_id=0,tun_gbp_flags=0,tun_tos=0,tun_ttl=64,key,in_port=6081,vlan_tci=0x0000,dl_src=f0:00:00:00:00:00,dl_dst=f0:11:11:11:11:11,dl_type=0x1234
----------------------------------------------------------------------
Anyway, there's cleanup to do, and some of the stuff I had to do to get
the above traces to work might require rethinking, but I think that the
basic idea is sound, and comments on the ideas are welcome.
Thanks,
Ben.
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
