On Sat, Aug 13, 2016 at 3:20 AM, Yong Wang <yongw...@vmware.com> wrote: > Just a quick update that a fix has been sent to dpdk.org for review. This > turns out to be a problem with vmxnet3 dpdk pmd driver when queue number are > reconfigured. Thanks Daniele for the reproduction, code review and test. > > Sothy, please let us know if applying the patch to dpdk get your problem > resolved (you are already copied on the review request to dpdk.org). > > Yong
Hello Yong, I applied patches to existing dpdk 16.04 version. Then compiled dpdk and ovs (2.5.90). I created two ports in ovs with dpdk. It works! if u need any info, I can provide. I didnt go into details of the patches. > > On 7/28/16, 12:43 AM, "sothy shan" <sothy....@gmail.com> wrote: > > On Thu, Jul 28, 2016 at 2:29 AM, Yong Wang <yongw...@vmware.com> wrote: > > > >> On Jul 27, 2016, at 12:25 AM, sothy shan <sothy....@gmail.com> wrote: > >> > >> On Tue, Jul 26, 2016 at 6:32 PM, Mauricio Vasquez > >> <mauricio.vasq...@polito.it> wrote: > >>> Hello Sothy, > >>> > >>> Sorry for the delay in answering, some busy days here. > >>> > >>> > >>> On 07/25/2016 12:03 PM, sothy shan wrote: > >>>> > >>>> Hello Mauricio, > >>>> > >>>> On Sun, Jul 24, 2016 at 12:17 PM, Mauricio Vasquez > >>>> <mauricio.vasq...@polito.it> wrote: > >>>>> > >>>>> Hi Sothy, > >>>>> > >>>>> > >>>>> On 07/21/2016 11:55 AM, sothy shan wrote: > >>>>>> > >>>>>> On Thu, Jul 21, 2016 at 11:43 AM, Mauricio Vasquez > >>>>>> <mauricio.vasq...@polito.it> wrote: > >>>>>>> > >>>>>>> > >>>>>>> On 07/21/2016 10:51 AM, sothy shan wrote: > >>>>>>>> > >>>>>>>> On Thu, Jul 21, 2016 at 10:41 AM, sothy shan > <sothy....@gmail.com> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> On Wed, Jul 20, 2016 at 9:02 PM, Mauricio Vasquez > >>>>>>>>> <mauricio.vasq...@polito.it> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi Sothy, > >>>>>>>>>> > >>>>>>>>>> On 07/20/2016 07:40 PM, Aaron Conole wrote: > >>>>>>>>>>> > >>>>>>>>>>> sothy shan <sothy....@gmail.com> writes: > >>>>>>>>>>> > >>>>>>>>>>>> Hello all, > >>>>>>>>>>> > >>>>>>>>>>> Hi Sothy, > >>>>>>>>>>> > >>>>>>>>>>>> I followed the installion guide provided in the web site > >>>>>>>>>>>> > >>>>>>>>>>>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitch_ovs_blob_master_INSTALL.DPDK.md-23ovstc&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=ojiXK7Tj1ANsHE_AlchVROP4R7LrgLQqVqTx7BjJQas&e= > >>>>>>>>>>>> to install dpdk OVS. > >>>>>>>>>>>> > >>>>>>>>>>>> I am rurnning Ubuntu 14.04 (kernel 3.13). It is a VM running > >>>>>>>>>>>> VMware. > >>>>>>>>>>>> One intereface is enabled by VMXNET3. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> It is not clear for me, are you trying to install OvS within > the VM? > >>>>>>>>>> > >>>>>>>>>>>> I used dpdk 16.04 > >>>>>>>>>>>> Today I made clone of OVS (committ > >>>>>>>>>>>> :bf32e3e2c029da18b5d6fd9210cb0ea12a1d0383). > >>>>>>>>>>>> > >>>>>>>>>>>> The following are different configuration compared to above > link. > >>>>>>>>>>>> > >>>>>>>>>>>> sysctl -w vm.nr_hugepages=2048 > >>>>>>>>>>>> Binded interface via igb_uio. > >>>>>>>>>>>> > >>>>>>>>>>>> Then followed same configuration to start ovs-vsswitch with > dpdk. > >>>>>>>>>>> > >>>>>>>>>>> Can you please post the steps exactly that you followed? > Which > >>>>>>>>>>> user > >>>>>>>>>>> you > >>>>>>>>>>> were running as, etc? Can you also post how you expect > packets to > >>>>>>>>>>> arrive into the vm? > >>>>>>>>>>> > >>>>>>>>>>>> I am able to creatre bridge. When I create a port. It gives > error > >>>>>>>>>>>> message. > >>>>>>>>>>> > >>>>>>>>>>> Please show the steps for this, too. The exact commands you > used. > >>>>>>>>>>> > >>>>>>>>>>>> When I check ovs-vswitchd.log file, > >>>>>>>>>>>> > >>>>>>>>>>>> There is warning that dpif_netlink|WARN|genric netlink > family, > >>>>>>>>>>>> ovs_datapath does not exist. The openvswitch kernel module > may be > >>>>>>>>>>>> not > >>>>>>>>>>>> loaded. > >>>>>>>>>>> > >>>>>>>>>>> This is definitely a problem. How did you start the > ovs-vswitchd > >>>>>>>>>>> daemon? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> This is not a problem Aaron, the ovs kernel module is not > needed > >>>>>>>>>> when > >>>>>>>>>> ovs is > >>>>>>>>>> compiled with DPDK. > >>>>>>>>>>>> > >>>>>>>>>>>> Then ERR: ERR|interface dpdk0 start error:operation not > permitted. > >>>>>>>>>>>> > >>>>>>>>>>>> The corresponding port is loaded by dpdk driver as shown in > >>>>>>>>>>>> dpdk_nic_bind --status. > >>>>>>>>>>>> > >>>>>>>>>>>> After that I did sudo ovs-vsctl show. it show br and port > details > >>>>>>>>>>>> as > >>>>>>>>>>>> well. > >>>>>>>>>>>> I am able to add one flow rule as well. But ovs bridge didnt > get > >>>>>>>>>>>> packets from the interface. > >>>>>>>>>>>> > >>>>>>>>>>>> Do you know why this error is popup? anyway to solve the > problem? > >>>>>>>>>>>> > >>>>>>>>>>>> Thank you very much > >>>>>>>>>>>> > >>>>>>>>>>>> Best regars > >>>>>>>>>>>> Sothy > >>>>>>>>>>> > >>>>>>>>>>> _______________________________________________ > >>>>>>>>>>> discuss mailing list > >>>>>>>>>>> discuss@openvswitch.org > >>>>>>>>>>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_discuss&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=DZpyuDSP-5BYH4bvfNHk68XRFB9-M-M43FxZCCpZKho&e= > >>>>>>>>>> > >>>>>>>>>> As Aaron wrote, having the exact commands is important to help > you. > >>>>>>>>> > >>>>>>>>> My configuration and log info: > >>>>>>>>> > >>>>>>>>> sudo sysctl vm.nr_hugepages=2048 > >>>>>>>>> sudo mkdir -p /dev/hugepages > >>>>>>>>> sudo mount -t hugetlbfs hugetlbfs /dev/hugepages > >>>>>>>>> sudo modprobe uio > >>>>>>>>> sudo insmod $DPDK_BUILD/kmod/igb_uio.ko > >>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py --status > >>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py -b igb_uio 0000:0b:00.0 > >>>>>>>>> sudo ./dpdk-16.04/tools/dpdk_nic_bind.py --status > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Network devices using DPDK-compatible driver > >>>>>>>>> ============================================ > >>>>>>>>> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused= > >>>>>>>>> 0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused= > >>>>>>>>> > >>>>>>>>> Network devices using kernel driver > >>>>>>>>> =================================== > >>>>>>>>> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 > >>>>>>>>> unused=igb_uio *Active* > >>>>>>>>> > >>>>>>>>> Other network devices > >>>>>>>>> ===================== > >>>>>>>>> <none> > >>>>>>>>> > >>>>>>>>> .........Then...................... > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> sudo ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 > type=dpdk > >>>>>>>>> sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev > >>>>>>> > >>>>>>> These two commands are inverted. > >>>>>>> The bridge has to be created first and then ports have to be > added to > >>>>>>> it. > >>>>>>> > >>>>>> It is my mistake when coping. > >>>>>>>>> > >>>>>>>>> Log file/ > >>>>>>>>> 2016-07-21T07:53:35.527Z|00001|vlog|INFO|opened log file > >>>>>>>>> /var/log/openvswitch/ovs-vswitchd.log > >>>>>>>>> 2016-07-21T07:53:35.529Z|00002|ovs_numa|INFO|Discovered 2 CPU > cores > >>>>>>>>> on > >>>>>>>>> NUMA node 0 > >>>>>>>>> 2016-07-21T07:53:35.529Z|00003|ovs_numa|INFO|Discovered 1 NUMA > nodes > >>>>>>>>> and 2 CPU cores > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > 2016-07-21T07:53:35.529Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: > >>>>>>>>> connecting... > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > 2016-07-21T07:53:35.529Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: > >>>>>>>>> connected > >>>>>>>>> 2016-07-21T07:53:35.533Z|00006|dpdk|INFO|DPDK Enabled, > initializing > >>>>>>>>> 2016-07-21T07:53:35.533Z|00007|dpdk|INFO|No vhost-sock-dir > provided - > >>>>>>>>> defaulting to /usr/local/var/run/openvswitch > >>>>>>>>> 2016-07-21T07:53:35.533Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd > >>>>>>>>> --socket-mem 1024,0 -c 0x00000001 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00009|timeval|WARN|Unreasonably long > 1076ms > >>>>>>>>> poll interval (80ms user, 875ms system) > >>>>>>>>> 2016-07-21T07:53:36.606Z|00010|timeval|WARN|faults: 3306 minor, > 0 > >>>>>>>>> major > >>>>>>>>> 2016-07-21T07:53:36.606Z|00011|timeval|WARN|context switches: 3 > >>>>>>>>> voluntary, 101 involuntary > >>>>>>>>> 2016-07-21T07:53:36.606Z|00012|coverage|INFO|Event coverage, > avg rate > >>>>>>>>> over last: 5 seconds, last minute, last hour, hash=398361de: > >>>>>>>>> 2016-07-21T07:53:36.606Z|00013|coverage|INFO|bridge_reconfigure > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00014|coverage|INFO|cmap_expand > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00015|coverage|INFO|miniflow_malloc > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00016|coverage|INFO|hmap_pathological > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00017|coverage|INFO|hmap_expand > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 640 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00018|coverage|INFO|txn_unchanged > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 > >>>>>>>>> 2016-07-21T07:53:36.606Z|00019|coverage|INFO|txn_incomplete > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00020|coverage|INFO|poll_create_node > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 35 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00021|coverage|INFO|seq_change > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 38 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00022|coverage|INFO|pstream_open > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00023|coverage|INFO|stream_open > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00024|coverage|INFO|util_xalloc > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 11049 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00025|coverage|INFO|netdev_get_hwaddr > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00026|coverage|INFO|netlink_received > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00027|coverage|INFO|netlink_sent > >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>>>>> 2016-07-21T07:53:36.607Z|00028|coverage|INFO|86 events never hit > >>>>>>>>> 2016-07-21T07:53:36.607Z|00029|bridge|INFO|ovs-vswitchd (Open > >>>>>>>>> vSwitch) > >>>>>>>>> 2.5.90 > >>>>>>>>> 2016-07-21T07:54:50.861Z|00030|memory|INFO|18980 kB peak > resident set > >>>>>>>>> size after 75.3 seconds > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath supports recirculation > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> MPLS label stack length probed as 3 > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath supports truncate action > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath supports unique flow ids > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath does not support ct_state > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath does not support ct_zone > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath does not support ct_mark > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath does not support ct_label > >>>>>>>>> > 2016-07-21T07:54:50.866Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>>>>> Datapath does not support ct_state_nat > >>>>>>>>> 2016-07-21T07:54:50.875Z|00040|bridge|INFO|bridge br0: added > >>>>>>>>> interface > >>>>>>>>> br0 on port 65534 > >>>>>>>>> 2016-07-21T07:54:50.887Z|00041|dpif_netlink|WARN|Generic Netlink > >>>>>>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel > module > >>>>>>>>> is probably not loaded. > >>>>>>>>> 2016-07-21T07:54:50.887Z|00042|bridge|INFO|bridge br0: using > datapath > >>>>>>>>> ID 0000c664fbd86348 > >>>>>>>>> 2016-07-21T07:54:50.887Z|00043|connmgr|INFO|br0: added service > >>>>>>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt" > >>>>>>>>> 2016-07-21T07:58:18.796Z|00044|dpdk|INFO|Port 0: > 00:0c:29:b5:9f:5b > >>>>>>>>> 2016-07-21T07:58:18.810Z|00045|dpdk|ERR|Interface dpdk0 start > error: > >>>>>>>>> Operation not permitted > >>>>>>>>> 2016-07-21T07:58:18.810Z|00046|dpif|WARN|netdev@ovs-netdev: > failed to > >>>>>>>>> add dpdk0 as port: Operation not permitted > >>>>>>>>> 2016-07-21T08:10:35.440Z|00047|connmgr|INFO|br0<->unix: 1 > flow_mods > >>>>>>>>> in > >>>>>>>>> the last 0 s (1 adds) > >>>>>>> > >>>>>>> Could you also provide the full terminal output when you run > >>>>>>> ovs-vswitchd?, > >>>>>>> I would like to see the DPDK output > >>>>>> > >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vswitchd > unix:$DB_SOCK > >>>>>> --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log > >>>>>> 2016-07-21T09:33:35Z|00001|vlog|INFO|opened log file > >>>>>> /var/log/openvswitch/ovs-vswitchd.log > >>>>>> 2016-07-21T09:33:35Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on > NUMA > >>>>>> node 0 > >>>>>> 2016-07-21T09:33:35Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes > and 2 > >>>>>> CPU > >>>>>> cores > >>>>>> > >>>>>> > >>>>>> > 2016-07-21T09:33:35Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: > >>>>>> connecting... > >>>>>> > >>>>>> > >>>>>> > 2016-07-21T09:33:35Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: > >>>>>> connected > >>>>>> 2016-07-21T09:33:35Z|00006|dpdk|INFO|DPDK Enabled, initializing > >>>>>> 2016-07-21T09:33:35Z|00007|dpdk|INFO|No vhost-sock-dir provided - > >>>>>> defaulting to /usr/local/var/run/openvswitch > >>>>>> 2016-07-21T09:33:35Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd > >>>>>> --socket-mem 1024,0 -c 0x00000001 > >>>>>> EAL: Detected lcore 0 as core 0 on socket 0 > >>>>>> EAL: Detected lcore 1 as core 0 on socket 0 > >>>>>> EAL: Support maximum 128 logical core(s) by configuration. > >>>>>> EAL: Detected 2 lcore(s) > >>>>>> EAL: Probing VFIO support... > >>>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file > or > >>>>>> directory) > >>>>>> EAL: VFIO modules not loaded, skipping VFIO support... > >>>>>> EAL: Setting up physically contiguous memory... > >>>>>> EAL: Ask a virtual area of 0x5e400000 bytes > >>>>>> EAL: Virtual area found at 0x7fa618400000 (size = 0x5e400000) > >>>>>> EAL: Ask a virtual area of 0x200000 bytes > >>>>>> EAL: Virtual area found at 0x7fa618000000 (size = 0x200000) > >>>>>> EAL: Ask a virtual area of 0xa1800000 bytes > >>>>>> EAL: Virtual area found at 0x7fa576600000 (size = 0xa1800000) > >>>>>> EAL: Ask a virtual area of 0x200000 bytes > >>>>>> EAL: Virtual area found at 0x7fa576200000 (size = 0x200000) > >>>>>> EAL: Requesting 512 pages of size 2MB from socket 0 > >>>>>> EAL: TSC frequency is ~2799213 KHz > >>>>>> EAL: Master lcore 0 is ready (tid=78ad7b80;cpuset=[0]) > >>>>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1 > >>>>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > >>>>>> EAL: Not managed by a supported kernel driver, skipped > >>>>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1 > >>>>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > >>>>>> EAL: PCI memory mapped at 0x7fa658400000 > >>>>>> EAL: PCI memory mapped at 0x7fa658401000 > >>>>>> EAL: PCI memory mapped at 0x7fa658402000 > >>>>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1 > >>>>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > >>>>>> EAL: PCI memory mapped at 0x7fa658404000 > >>>>>> EAL: PCI memory mapped at 0x7fa658405000 > >>>>>> EAL: PCI memory mapped at 0x7fa658406000 > >>>>>> Zone 0: name:<RG_MP_log_history>, phys:0x82bfdec0, len:0x2080, > >>>>>> virt:0x7fa6583fdec0, socket_id:0, flags:0 > >>>>>> Zone 1: name:<MP_log_history>, phys:0x82973d40, len:0x28a0c0, > >>>>>> virt:0x7fa658173d40, socket_id:0, flags:0 > >>>>>> Zone 2: name:<rte_eth_dev_data>, phys:0x82943200, len:0x2f800, > >>>>>> virt:0x7fa658143200, socket_id:0, flags:0 > >>>>>> 2016-07-21T09:33:36Z|00009|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath supports recirculation > >>>>>> 2016-07-21T09:33:36Z|00010|ofproto_dpif|INFO|netdev@ovs-netdev: > MPLS > >>>>>> label stack length probed as 3 > >>>>>> 2016-07-21T09:33:36Z|00011|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath supports truncate action > >>>>>> 2016-07-21T09:33:36Z|00012|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath supports unique flow ids > >>>>>> 2016-07-21T09:33:36Z|00013|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath does not support ct_state > >>>>>> 2016-07-21T09:33:36Z|00014|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath does not support ct_zone > >>>>>> 2016-07-21T09:33:36Z|00015|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath does not support ct_mark > >>>>>> 2016-07-21T09:33:36Z|00016|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath does not support ct_label > >>>>>> 2016-07-21T09:33:36Z|00017|ofproto_dpif|INFO|netdev@ovs-netdev: > >>>>>> Datapath does not support ct_state_nat > >>>>>> 2016-07-21T09:33:36Z|00018|bridge|INFO|bridge br0: added interface > br0 > >>>>>> on port 65534 > >>>>>> 2016-07-21T09:33:36Z|00019|dpif_netlink|WARN|Generic Netlink family > >>>>>> 'ovs_datapath' does not exist. The Open vSwitch kernel module is > >>>>>> probably not loaded. > >>>>>> 2016-07-21T09:33:36Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b > >>>>>> 2016-07-21T09:33:36Z|00021|dpdk|ERR|Interface dpdk0 start error: > >>>>>> Operation not permitted > >>>>>> 2016-07-21T09:33:36Z|00022|bridge|INFO|bridge br0: using datapath > ID > >>>>>> 0000c664fbd86348 > >>>>>> 2016-07-21T09:33:36Z|00023|connmgr|INFO|br0: added service > controller > >>>>>> "punix:/usr/local/var/run/openvswitch/br0.mgmt" > >>>>>> 2016-07-21T09:33:36Z|00024|timeval|WARN|Unreasonably long 1244ms > poll > >>>>>> interval (233ms user, 880ms system) > >>>>>> 2016-07-21T09:33:36Z|00025|timeval|WARN|faults: 3902 minor, 0 major > >>>>>> 2016-07-21T09:33:36Z|00026|timeval|WARN|context switches: 10 > >>>>>> voluntary, 214 involuntary > >>>>>> 2016-07-21T09:33:36Z|00027|coverage|INFO|Event coverage, avg rate > over > >>>>>> last: 5 seconds, last minute, last hour, hash=75cfd2fc: > >>>>>> 2016-07-21T09:33:36Z|00028|coverage|INFO|bridge_reconfigure > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00029|coverage|INFO|ofproto_flush > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00030|coverage|INFO|ofproto_update_port > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00031|coverage|INFO|rev_flow_table > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00032|coverage|INFO|cmap_expand > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 22 > >>>>>> 2016-07-21T09:33:36Z|00033|coverage|INFO|cmap_shrink > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 7 > >>>>>> 2016-07-21T09:33:36Z|00034|coverage|INFO|dpif_port_add > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 > >>>>>> 2016-07-21T09:33:36Z|00035|coverage|INFO|dpif_flow_flush > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 > >>>>>> 2016-07-21T09:33:36Z|00036|coverage|INFO|dpif_flow_get > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 5 > >>>>>> 2016-07-21T09:33:36Z|00037|coverage|INFO|dpif_flow_put > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 10 > >>>>>> 2016-07-21T09:33:36Z|00038|coverage|INFO|dpif_flow_del > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 5 > >>>>>> 2016-07-21T09:33:36Z|00039|coverage|INFO|dpif_execute > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3 > >>>>>> 2016-07-21T09:33:36Z|00040|coverage|INFO|flow_extract > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4 > >>>>>> 2016-07-21T09:33:36Z|00041|coverage|INFO|miniflow_malloc > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 16 > >>>>>> 2016-07-21T09:33:36Z|00042|coverage|INFO|hmap_pathological > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00043|coverage|INFO|hmap_expand > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 653 > >>>>>> 2016-07-21T09:33:36Z|00044|coverage|INFO|netdev_get_stats > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00045|coverage|INFO|poll_create_node > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 15 > >>>>>> 2016-07-21T09:33:36Z|00046|coverage|INFO|poll_zero_timeout > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00047|coverage|INFO|seq_change > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 575 > >>>>>> 2016-07-21T09:33:36Z|00048|coverage|INFO|pstream_open > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3 > >>>>>> 2016-07-21T09:33:36Z|00049|coverage|INFO|stream_open > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00050|coverage|INFO|util_xalloc > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 11820 > >>>>>> 2016-07-21T09:33:36Z|00051|coverage|INFO|netdev_set_policing > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00052|coverage|INFO|netdev_get_ifindex > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00053|coverage|INFO|netdev_get_hwaddr > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 > >>>>>> 2016-07-21T09:33:36Z|00054|coverage|INFO|netdev_set_hwaddr > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00055|coverage|INFO|netdev_get_ethtool > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 > >>>>>> 2016-07-21T09:33:36Z|00056|coverage|INFO|netlink_received > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 6 > >>>>>> 2016-07-21T09:33:36Z|00057|coverage|INFO|netlink_recv_jumbo > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 > >>>>>> 2016-07-21T09:33:36Z|00058|coverage|INFO|netlink_sent > >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 5 > >>>>>> 2016-07-21T09:33:36Z|00059|coverage|INFO|70 events never hit > >>>>>> 2016-07-21T09:33:36Z|00060|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b > >>>>>> 2016-07-21T09:33:36Z|00061|dpdk|ERR|Interface dpdk0 start error: > >>>>>> Operation not permitted > >>>>> > >>>>> The PMD start function is failing, unfortunately that error does > not tell > >>>>> that much information. > >>>>> > >>>>> I would suggest you to enable the debug of the vmxnet3 PMD in DPDK, > set > >>>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y in the config/common_base > file of > >>>>> DPDK, then recompile DPDK and OvS. > >>>>> > >>>>> Please try once again and send the terminal output, I hope we can > get a > >>>>> more > >>>>> information of what is going on with that log. > >>>> > >>>> Based on your request, I reproduce the log with > >>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y > >>>> sudo ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock > >>>> --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile --detach > >>>> 2016-07-25T09:24:17Z|00001|vlog|INFO|opened log file > >>>> /var/log/openvswitch/ovs-vswitchd.log > >>>> 2016-07-25T09:24:17Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on > NUMA > >>>> node 0 > >>>> 2016-07-25T09:24:17Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and > 2 CPU > >>>> cores > >>>> > >>>> > 2016-07-25T09:24:17Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: > >>>> connecting... > >>>> > >>>> > 2016-07-25T09:24:17Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: > >>>> connected > >>>> 2016-07-25T09:24:17Z|00006|dpdk|INFO|DPDK Enabled, initializing > >>>> 2016-07-25T09:24:17Z|00007|dpdk|INFO|No vhost-sock-dir provided - > >>>> defaulting to /usr/local/var/run/openvswitch > >>>> 2016-07-25T09:24:17Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd > >>>> --socket-mem 1024,0 -c 0x00000001 > >>>> EAL: Detected lcore 0 as core 0 on socket 0 > >>>> EAL: Detected lcore 1 as core 0 on socket 0 > >>>> EAL: Support maximum 128 logical core(s) by configuration. > >>>> EAL: Detected 2 lcore(s) > >>>> EAL: Probing VFIO support... > >>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or > >>>> directory) > >>>> EAL: VFIO modules not loaded, skipping VFIO support... > >>>> EAL: Setting up physically contiguous memory... > >>>> EAL: Ask a virtual area of 0x200000 bytes > >>>> EAL: Virtual area found at 0x7fdea6000000 (size = 0x200000) > >>>> EAL: Ask a virtual area of 0x5e000000 bytes > >>>> EAL: Virtual area found at 0x7fde47e00000 (size = 0x5e000000) > >>>> EAL: Ask a virtual area of 0x200000 bytes > >>>> EAL: Virtual area found at 0x7fde47a00000 (size = 0x200000) > >>>> EAL: Ask a virtual area of 0x200000 bytes > >>>> EAL: Virtual area found at 0x7fde47600000 (size = 0x200000) > >>>> EAL: Ask a virtual area of 0xa1800000 bytes > >>>> EAL: Virtual area found at 0x7fdda5c00000 (size = 0xa1800000) > >>>> EAL: Ask a virtual area of 0x200000 bytes > >>>> EAL: Virtual area found at 0x7fdda5800000 (size = 0x200000) > >>>> EAL: Requesting 512 pages of size 2MB from socket 0 > >>>> EAL: TSC frequency is ~2799209 KHz > >>>> EAL: Master lcore 0 is ready (tid=a855eb80;cpuset=[0]) > >>>> PMD: rte_vmxnet3_pmd_init(): >> > >>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1 > >>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > >>>> EAL: Not managed by a supported kernel driver, skipped > >>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1 > >>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > >>>> EAL: PCI memory mapped at 0x7fdea6200000 > >>>> EAL: PCI memory mapped at 0x7fdea6201000 > >>>> EAL: PCI memory mapped at 0x7fdea6202000 > >>>> PMD: eth_vmxnet3_dev_init(): >> > >>>> PMD: eth_vmxnet3_dev_init(): Hardware version : 1 > >>>> PMD: eth_vmxnet3_dev_init(): UPT hardware version : 1 > >>>> PMD: eth_vmxnet3_dev_init(): MAC Address : 00:0c:29:b5:9f:5b > >>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1 > >>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd > >>>> EAL: Not managed by a supported kernel driver, skipped > >>>> Zone 0: name:<RG_MP_log_history>, phys:0x555fdec0, len:0x2080, > >>>> virt:0x7fdea61fdec0, socket_id:0, flags:0 > >>>> Zone 1: name:<MP_log_history>, phys:0x95375f00, len:0x28a0c0, > >>>> virt:0x7fde87975f00, socket_id:0, flags:0 > >>>> Zone 2: name:<rte_eth_dev_data>, phys:0x555cd300, len:0x2f800, > >>>> virt:0x7fdea61cd300, socket_id:0, flags:0 > >>>> > >>>> PMD: vmxnet3_dev_configure(): >> > >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> > >>>> PMD: vmxnet3_dev_rx_queue_setup(): >> > >>>> PMD: vmxnet3_dev_start(): >> > >>>> PMD: vmxnet3_rss_configure(): >> > >>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b > >>>> PMD: vmxnet3_disable_intr(): >> > >>>> PMD: vmxnet3_dev_rxtx_init(): >> > >>>> 2016-07-25T09:40:23Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b > >>>> PMD: vmxnet3_dev_stop(): >> > >>>> PMD: vmxnet3_disable_intr(): >> > >>>> PMD: vmxnet3_dev_stop(): Device reset. > >>>> PMD: vmxnet3_dev_clear_queues(): >> > >>>> PMD: vmxnet3_dev_configure(): >> > >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> > >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> > >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> > >>>> PMD: vmxnet3_dev_rx_queue_setup(): >> > >>>> PMD: vmxnet3_dev_start(): >> > >>>> PMD: vmxnet3_rss_configure(): >> > >>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b > >>>> PMD: vmxnet3_dev_start(): Device activation: UNSUCCESSFUL > >>> > >>> > >>> Unfortunately it is the most I can help you, it is strange for me why > >>> test-pmd is working and why OvS is not. > >>> I think it is a DPDK issue, then the next step is to post on the DPDK > >>> mailing (d...@dpdk.org, add also in CC Yong Wang > <yongw...@vmware.com> that > >>> is the vmxnet3 maintainer), send a link to this thread in order to be > clear > >>> what you have tested. > >> > >> > >> Hello Yong Wang, > >> > >> I am running fedora VM in ESXi 5.5.0 environment. I tried to install > >> OVS and DPDK. OVS couldnt start DPDK device. > >> You can see the error in the mail below. Do you know why? IF you need > >> more information. I can provide more information. > >> > >> Good news, when I tried in Fedora 23 VM in ESXi 5.5.0, it works. It is > >> possible to bind an dpdk device and get packets on port. > >> > >> Thanks Mauricio for your help and reply. > >> > >> Best regards > >> Sothy > > > > Sothy, > > > > Hello, > > You mentioned that on Fedora 23, it works. Then on some version Fedora > that it failed? > > Initially, I started working on Ubuntu 14.04. It produce error. > Therefore, I changed in Fedora. I worked directly on Fedora 23. > For fedora, I only tried on Fedora 23. It works. I didnt tried on > other version. All Vms are in ESX 5.5. The problem is in Ubuntu vm > ONLY, > > > From your previous post, you mentioned that it’s a Ubuntu 14.04 VM? > Can you clarify? Other than the changes you >mentioned already, are there > any other changes (such as ring buffer size, number of rx queues, etc) that > have been >changed? > I didnt do any changes. I just followed basic commands given in OVDK > install.md and > >Is there any updates installed on your ESX 5.5? > > no > >If yes, what’s the exact update version? Can you also send me a copy of > your vmkernel.log offline? > > I will send you offline. > > > > Also copying Daniele as he is trying to help reproduce the reported > issue. > > > >> > >>> > >>>> I deleted exisiting bridge and recreated:::::: > >>>> > >>>> > >>>> Then I added bridge. After I try to create bridge. It gives error. > The > >>>> erros is same as shown in log file. > >>>> > >>>> 2016-07-25T09:29:43.248Z|00020|bridge|INFO|bridge br0: added > interface > >>>> br0 on port 65534 > >>>> 2016-07-25T09:29:43.253Z|00021|dpif_netlink|WARN|Generic Netlink > >>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module > >>>> is probably not loaded. > >>>> 2016-07-25T09:29:43.253Z|00022|bridge|INFO|bridge br0: using datapath > >>>> ID 0000e6aac740584b > >>>> 2016-07-25T09:29:43.253Z|00023|connmgr|INFO|br0: added service > >>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt" > >>>> 2016-07-25T09:30:54.008Z|00024|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b > >>>> 2016-07-25T09:30:54.021Z|00025|dpdk|ERR|Interface dpdk0 start error: > >>>> Operation not permitted > >>>> 2016-07-25T09:30:54.021Z|00026|dpif|WARN|netdev@ovs-netdev: failed to > >>>> add dpdk0 as port: Operation not permitted > >>>> > >>>> > >>>> > >>>>> Another thing to do would be to bind the NICs to the > uio_pci_generic and > >>>>> VFIO modules, just to be sure that the issue is also present there. > >>>> > >>>> I tried in uio and uio_pci_genric also. Both are giving same error. > >>>> I suspect I cannt activate VFIO in VM since it is BIOS suport also. > AM I > >>>> right? > >>>> > >>>> Thanks > >>>> > >>>>> > >>>>>> in command line some information for your verification: > >>>>>> > >>>>>> sudo ovs-vsctl show > >>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2 > >>>>>> Bridge "br0" > >>>>>> Port "dpdk0" > >>>>>> Interface "dpdk0" > >>>>>> type: dpdk > >>>>>> Port "br0" > >>>>>> Interface "br0" > >>>>>> type: internal > >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl del-br br0 > >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-br br0 -- > >>>>>> set bridge br0 datapath_type=netdev > >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-port br0 > >>>>>> dpdk0 -- set Interface dpdk0 type=dpdk > >>>>>> ovs-vsctl: Error detected while setting up 'dpdk0'. See > ovs-vswitchd > >>>>>> log for details. > >>>>>> > >>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>>>>>>>> sudo ovs-vsctl show > >>>>>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2 > >>>>>>>>> Bridge "br0" > >>>>>>>>> Port "dpdk0" > >>>>>>>>> Interface "dpdk0" > >>>>>>>>> type: dpdk > >>>>>>>>> Port "br0" > >>>>>>>>> Interface "br0" > >>>>>>>>> type: internal > >>>>>>>>> > >>>>>>>>> sudo ovs-ofctl dump-flows br0 > >>>>>>>>> NXST_FLOW reply (xid=0x4): > >>>>>>>>> cookie=0x0, duration=487.638s, table=0, n_packets=0, > n_bytes=0, > >>>>>>>>> idle_age=487, priority=0 actions=NORMAL > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> In addition I would recommend to use the test-pmd [1] > application > >>>>>>>>>> from > >>>>>>>>>> DPDK > >>>>>>>>>> to check if the issue is also present in that case. > >>>>>>>> > >>>>>>>> I run testpmd command.The result is > >>>>>>>> > >>>>>>>> Configuring Port 0 (socket 0) > >>>>>>>> Port 0: 00:0C:29:B5:9F:5B > >>>>>>>> Configuring Port 1 (socket 0) > >>>>>>>> Port 1: 00:0C:29:B5:9F:65 > >>>>>>>> Checking link statuses... > >>>>>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex > >>>>>>>> Port 1 Link Up - speed 10000 Mbps - full-duplex > >>>>>>>> Done > >>>>>>>> No commandline core given, start packet forwarding > >>>>>>>> io packet forwarding - CRC stripping disabled - > packets/burst=32 > >>>>>>>> nb forwarding cores=1 - nb forwarding ports=2 > >>>>>>>> RX queues=1 - RX desc=128 - RX free threshold=0 > >>>>>>>> RX threshold registers: pthresh=0 hthresh=0 wthresh=0 > >>>>>>>> TX queues=1 - TX desc=512 - TX free threshold=0 > >>>>>>>> TX threshold registers: pthresh=0 hthresh=0 wthresh=0 > >>>>>>>> TX RS bit threshold=0 - TXQ flags=0x200 > >>>>>>>> Press enter to exit > >>>>>>>> > >>>>>>>> Telling cores to stop... > >>>>>>>> Waiting for lcores to finish... > >>>>>>>> > >>>>>>>> ---------------------- Forward statistics for port 0 > >>>>>>>> ---------------------- > >>>>>>>> RX-packets: 88 RX-dropped: 0 > RX-total: 88 > >>>>>>>> TX-packets: 2 TX-dropped: 0 > TX-total: 2 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > ---------------------------------------------------------------------------- > >>>>>>>> > >>>>>>>> ---------------------- Forward statistics for port 1 > >>>>>>>> ---------------------- > >>>>>>>> RX-packets: 2 RX-dropped: 0 > RX-total: 2 > >>>>>>>> TX-packets: 88 TX-dropped: 0 > TX-total: 88 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > ---------------------------------------------------------------------------- > >>>>>>>> > >>>>>>>> +++++++++++++++ Accumulated forward statistics for all > >>>>>>>> ports+++++++++++++++ > >>>>>>>> RX-packets: 90 RX-dropped: 0 > RX-total: 90 > >>>>>>>> TX-packets: 90 TX-dropped: 0 > TX-total: 90 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >>>>>>>> > >>>>>>>> make sense? > >>>>>>> > >>>>>>> > >>>>>>> Yes, it makes sense. Ports are being recognized by DPDK and some > >>>>>>> packets > >>>>>>> were exchanged. > >>>>>>> > >>>>>>> > >>>>>>>>>> [1] > https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides-2D16.04_testpmd-5Fapp-5Fug_index.html&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=JnLeHoyKuiecYI95-fcfFiuev5g2CDweT2MnyUiKWzM&e= > >>>>>>>>>> > >>>>>>>>>> Mauricio Vasquez, > >>>>>>> > >>>>>>> > >>>>> Mauricio V, > >>> > >>> > > > > _______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
