On Thu, Jul 28, 2016 at 2:29 AM, Yong Wang <yongw...@vmware.com> wrote: > >> On Jul 27, 2016, at 12:25 AM, sothy shan <sothy....@gmail.com> wrote: >> >> On Tue, Jul 26, 2016 at 6:32 PM, Mauricio Vasquez >> <mauricio.vasq...@polito.it> wrote: >>> Hello Sothy, >>> >>> Sorry for the delay in answering, some busy days here. >>> >>> >>> On 07/25/2016 12:03 PM, sothy shan wrote: >>>> >>>> Hello Mauricio, >>>> >>>> On Sun, Jul 24, 2016 at 12:17 PM, Mauricio Vasquez >>>> <mauricio.vasq...@polito.it> wrote: >>>>> >>>>> Hi Sothy, >>>>> >>>>> >>>>> On 07/21/2016 11:55 AM, sothy shan wrote: >>>>>> >>>>>> On Thu, Jul 21, 2016 at 11:43 AM, Mauricio Vasquez >>>>>> <mauricio.vasq...@polito.it> wrote: >>>>>>> >>>>>>> >>>>>>> On 07/21/2016 10:51 AM, sothy shan wrote: >>>>>>>> >>>>>>>> On Thu, Jul 21, 2016 at 10:41 AM, sothy shan <sothy....@gmail.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> On Wed, Jul 20, 2016 at 9:02 PM, Mauricio Vasquez >>>>>>>>> <mauricio.vasq...@polito.it> wrote: >>>>>>>>>> >>>>>>>>>> Hi Sothy, >>>>>>>>>> >>>>>>>>>> On 07/20/2016 07:40 PM, Aaron Conole wrote: >>>>>>>>>>> >>>>>>>>>>> sothy shan <sothy....@gmail.com> writes: >>>>>>>>>>> >>>>>>>>>>>> Hello all, >>>>>>>>>>> >>>>>>>>>>> Hi Sothy, >>>>>>>>>>> >>>>>>>>>>>> I followed the installion guide provided in the web site >>>>>>>>>>>> >>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitch_ovs_blob_master_INSTALL.DPDK.md-23ovstc&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=ojiXK7Tj1ANsHE_AlchVROP4R7LrgLQqVqTx7BjJQas&e= >>>>>>>>>>>> to install dpdk OVS. >>>>>>>>>>>> >>>>>>>>>>>> I am rurnning Ubuntu 14.04 (kernel 3.13). It is a VM running >>>>>>>>>>>> VMware. >>>>>>>>>>>> One intereface is enabled by VMXNET3. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> It is not clear for me, are you trying to install OvS within the VM? >>>>>>>>>> >>>>>>>>>>>> I used dpdk 16.04 >>>>>>>>>>>> Today I made clone of OVS (committ >>>>>>>>>>>> :bf32e3e2c029da18b5d6fd9210cb0ea12a1d0383). >>>>>>>>>>>> >>>>>>>>>>>> The following are different configuration compared to above link. >>>>>>>>>>>> >>>>>>>>>>>> sysctl -w vm.nr_hugepages=2048 >>>>>>>>>>>> Binded interface via igb_uio. >>>>>>>>>>>> >>>>>>>>>>>> Then followed same configuration to start ovs-vsswitch with dpdk. >>>>>>>>>>> >>>>>>>>>>> Can you please post the steps exactly that you followed? Which >>>>>>>>>>> user >>>>>>>>>>> you >>>>>>>>>>> were running as, etc? Can you also post how you expect packets to >>>>>>>>>>> arrive into the vm? >>>>>>>>>>> >>>>>>>>>>>> I am able to creatre bridge. When I create a port. It gives error >>>>>>>>>>>> message. >>>>>>>>>>> >>>>>>>>>>> Please show the steps for this, too. The exact commands you used. >>>>>>>>>>> >>>>>>>>>>>> When I check ovs-vswitchd.log file, >>>>>>>>>>>> >>>>>>>>>>>> There is warning that dpif_netlink|WARN|genric netlink family, >>>>>>>>>>>> ovs_datapath does not exist. The openvswitch kernel module may be >>>>>>>>>>>> not >>>>>>>>>>>> loaded. >>>>>>>>>>> >>>>>>>>>>> This is definitely a problem. How did you start the ovs-vswitchd >>>>>>>>>>> daemon? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This is not a problem Aaron, the ovs kernel module is not needed >>>>>>>>>> when >>>>>>>>>> ovs is >>>>>>>>>> compiled with DPDK. >>>>>>>>>>>> >>>>>>>>>>>> Then ERR: ERR|interface dpdk0 start error:operation not permitted. >>>>>>>>>>>> >>>>>>>>>>>> The corresponding port is loaded by dpdk driver as shown in >>>>>>>>>>>> dpdk_nic_bind --status. >>>>>>>>>>>> >>>>>>>>>>>> After that I did sudo ovs-vsctl show. it show br and port details >>>>>>>>>>>> as >>>>>>>>>>>> well. >>>>>>>>>>>> I am able to add one flow rule as well. But ovs bridge didnt get >>>>>>>>>>>> packets from the interface. >>>>>>>>>>>> >>>>>>>>>>>> Do you know why this error is popup? anyway to solve the problem? >>>>>>>>>>>> >>>>>>>>>>>> Thank you very much >>>>>>>>>>>> >>>>>>>>>>>> Best regars >>>>>>>>>>>> Sothy >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> discuss mailing list >>>>>>>>>>> discuss@openvswitch.org >>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_discuss&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=DZpyuDSP-5BYH4bvfNHk68XRFB9-M-M43FxZCCpZKho&e= >>>>>>>>>> >>>>>>>>>> As Aaron wrote, having the exact commands is important to help you. >>>>>>>>> >>>>>>>>> My configuration and log info: >>>>>>>>> >>>>>>>>> sudo sysctl vm.nr_hugepages=2048 >>>>>>>>> sudo mkdir -p /dev/hugepages >>>>>>>>> sudo mount -t hugetlbfs hugetlbfs /dev/hugepages >>>>>>>>> sudo modprobe uio >>>>>>>>> sudo insmod $DPDK_BUILD/kmod/igb_uio.ko >>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py --status >>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py -b igb_uio 0000:0b:00.0 >>>>>>>>> sudo ./dpdk-16.04/tools/dpdk_nic_bind.py --status >>>>>>>>> >>>>>>>>> >>>>>>>>> Network devices using DPDK-compatible driver >>>>>>>>> ============================================ >>>>>>>>> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused= >>>>>>>>> 0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused= >>>>>>>>> >>>>>>>>> Network devices using kernel driver >>>>>>>>> =================================== >>>>>>>>> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 >>>>>>>>> unused=igb_uio *Active* >>>>>>>>> >>>>>>>>> Other network devices >>>>>>>>> ===================== >>>>>>>>> <none> >>>>>>>>> >>>>>>>>> .........Then...................... >>>>>>>>> >>>>>>>>> >>>>>>>>> sudo ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk >>>>>>>>> sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev >>>>>>> >>>>>>> These two commands are inverted. >>>>>>> The bridge has to be created first and then ports have to be added to >>>>>>> it. >>>>>>> >>>>>> It is my mistake when coping. >>>>>>>>> >>>>>>>>> Log file/ >>>>>>>>> 2016-07-21T07:53:35.527Z|00001|vlog|INFO|opened log file >>>>>>>>> /var/log/openvswitch/ovs-vswitchd.log >>>>>>>>> 2016-07-21T07:53:35.529Z|00002|ovs_numa|INFO|Discovered 2 CPU cores >>>>>>>>> on >>>>>>>>> NUMA node 0 >>>>>>>>> 2016-07-21T07:53:35.529Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes >>>>>>>>> and 2 CPU cores >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2016-07-21T07:53:35.529Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: >>>>>>>>> connecting... >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2016-07-21T07:53:35.529Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: >>>>>>>>> connected >>>>>>>>> 2016-07-21T07:53:35.533Z|00006|dpdk|INFO|DPDK Enabled, initializing >>>>>>>>> 2016-07-21T07:53:35.533Z|00007|dpdk|INFO|No vhost-sock-dir provided - >>>>>>>>> defaulting to /usr/local/var/run/openvswitch >>>>>>>>> 2016-07-21T07:53:35.533Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd >>>>>>>>> --socket-mem 1024,0 -c 0x00000001 >>>>>>>>> 2016-07-21T07:53:36.606Z|00009|timeval|WARN|Unreasonably long 1076ms >>>>>>>>> poll interval (80ms user, 875ms system) >>>>>>>>> 2016-07-21T07:53:36.606Z|00010|timeval|WARN|faults: 3306 minor, 0 >>>>>>>>> major >>>>>>>>> 2016-07-21T07:53:36.606Z|00011|timeval|WARN|context switches: 3 >>>>>>>>> voluntary, 101 involuntary >>>>>>>>> 2016-07-21T07:53:36.606Z|00012|coverage|INFO|Event coverage, avg rate >>>>>>>>> over last: 5 seconds, last minute, last hour, hash=398361de: >>>>>>>>> 2016-07-21T07:53:36.606Z|00013|coverage|INFO|bridge_reconfigure >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.606Z|00014|coverage|INFO|cmap_expand >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9 >>>>>>>>> 2016-07-21T07:53:36.606Z|00015|coverage|INFO|miniflow_malloc >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 9 >>>>>>>>> 2016-07-21T07:53:36.606Z|00016|coverage|INFO|hmap_pathological >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.606Z|00017|coverage|INFO|hmap_expand >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 640 >>>>>>>>> 2016-07-21T07:53:36.606Z|00018|coverage|INFO|txn_unchanged >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 >>>>>>>>> 2016-07-21T07:53:36.606Z|00019|coverage|INFO|txn_incomplete >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.607Z|00020|coverage|INFO|poll_create_node >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 35 >>>>>>>>> 2016-07-21T07:53:36.607Z|00021|coverage|INFO|seq_change >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 38 >>>>>>>>> 2016-07-21T07:53:36.607Z|00022|coverage|INFO|pstream_open >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.607Z|00023|coverage|INFO|stream_open >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.607Z|00024|coverage|INFO|util_xalloc >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 11049 >>>>>>>>> 2016-07-21T07:53:36.607Z|00025|coverage|INFO|netdev_get_hwaddr >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.607Z|00026|coverage|INFO|netlink_received >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3 >>>>>>>>> 2016-07-21T07:53:36.607Z|00027|coverage|INFO|netlink_sent >>>>>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>>>>> 2016-07-21T07:53:36.607Z|00028|coverage|INFO|86 events never hit >>>>>>>>> 2016-07-21T07:53:36.607Z|00029|bridge|INFO|ovs-vswitchd (Open >>>>>>>>> vSwitch) >>>>>>>>> 2.5.90 >>>>>>>>> 2016-07-21T07:54:50.861Z|00030|memory|INFO|18980 kB peak resident set >>>>>>>>> size after 75.3 seconds >>>>>>>>> 2016-07-21T07:54:50.866Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath supports recirculation >>>>>>>>> 2016-07-21T07:54:50.866Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> MPLS label stack length probed as 3 >>>>>>>>> 2016-07-21T07:54:50.866Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath supports truncate action >>>>>>>>> 2016-07-21T07:54:50.866Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath supports unique flow ids >>>>>>>>> 2016-07-21T07:54:50.866Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath does not support ct_state >>>>>>>>> 2016-07-21T07:54:50.866Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath does not support ct_zone >>>>>>>>> 2016-07-21T07:54:50.866Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath does not support ct_mark >>>>>>>>> 2016-07-21T07:54:50.866Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath does not support ct_label >>>>>>>>> 2016-07-21T07:54:50.866Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>>>>> Datapath does not support ct_state_nat >>>>>>>>> 2016-07-21T07:54:50.875Z|00040|bridge|INFO|bridge br0: added >>>>>>>>> interface >>>>>>>>> br0 on port 65534 >>>>>>>>> 2016-07-21T07:54:50.887Z|00041|dpif_netlink|WARN|Generic Netlink >>>>>>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module >>>>>>>>> is probably not loaded. >>>>>>>>> 2016-07-21T07:54:50.887Z|00042|bridge|INFO|bridge br0: using datapath >>>>>>>>> ID 0000c664fbd86348 >>>>>>>>> 2016-07-21T07:54:50.887Z|00043|connmgr|INFO|br0: added service >>>>>>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt" >>>>>>>>> 2016-07-21T07:58:18.796Z|00044|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b >>>>>>>>> 2016-07-21T07:58:18.810Z|00045|dpdk|ERR|Interface dpdk0 start error: >>>>>>>>> Operation not permitted >>>>>>>>> 2016-07-21T07:58:18.810Z|00046|dpif|WARN|netdev@ovs-netdev: failed to >>>>>>>>> add dpdk0 as port: Operation not permitted >>>>>>>>> 2016-07-21T08:10:35.440Z|00047|connmgr|INFO|br0<->unix: 1 flow_mods >>>>>>>>> in >>>>>>>>> the last 0 s (1 adds) >>>>>>> >>>>>>> Could you also provide the full terminal output when you run >>>>>>> ovs-vswitchd?, >>>>>>> I would like to see the DPDK output >>>>>> >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vswitchd unix:$DB_SOCK >>>>>> --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log >>>>>> 2016-07-21T09:33:35Z|00001|vlog|INFO|opened log file >>>>>> /var/log/openvswitch/ovs-vswitchd.log >>>>>> 2016-07-21T09:33:35Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA >>>>>> node 0 >>>>>> 2016-07-21T09:33:35Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 >>>>>> CPU >>>>>> cores >>>>>> >>>>>> >>>>>> 2016-07-21T09:33:35Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: >>>>>> connecting... >>>>>> >>>>>> >>>>>> 2016-07-21T09:33:35Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: >>>>>> connected >>>>>> 2016-07-21T09:33:35Z|00006|dpdk|INFO|DPDK Enabled, initializing >>>>>> 2016-07-21T09:33:35Z|00007|dpdk|INFO|No vhost-sock-dir provided - >>>>>> defaulting to /usr/local/var/run/openvswitch >>>>>> 2016-07-21T09:33:35Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd >>>>>> --socket-mem 1024,0 -c 0x00000001 >>>>>> EAL: Detected lcore 0 as core 0 on socket 0 >>>>>> EAL: Detected lcore 1 as core 0 on socket 0 >>>>>> EAL: Support maximum 128 logical core(s) by configuration. >>>>>> EAL: Detected 2 lcore(s) >>>>>> EAL: Probing VFIO support... >>>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or >>>>>> directory) >>>>>> EAL: VFIO modules not loaded, skipping VFIO support... >>>>>> EAL: Setting up physically contiguous memory... >>>>>> EAL: Ask a virtual area of 0x5e400000 bytes >>>>>> EAL: Virtual area found at 0x7fa618400000 (size = 0x5e400000) >>>>>> EAL: Ask a virtual area of 0x200000 bytes >>>>>> EAL: Virtual area found at 0x7fa618000000 (size = 0x200000) >>>>>> EAL: Ask a virtual area of 0xa1800000 bytes >>>>>> EAL: Virtual area found at 0x7fa576600000 (size = 0xa1800000) >>>>>> EAL: Ask a virtual area of 0x200000 bytes >>>>>> EAL: Virtual area found at 0x7fa576200000 (size = 0x200000) >>>>>> EAL: Requesting 512 pages of size 2MB from socket 0 >>>>>> EAL: TSC frequency is ~2799213 KHz >>>>>> EAL: Master lcore 0 is ready (tid=78ad7b80;cpuset=[0]) >>>>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1 >>>>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd >>>>>> EAL: Not managed by a supported kernel driver, skipped >>>>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1 >>>>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd >>>>>> EAL: PCI memory mapped at 0x7fa658400000 >>>>>> EAL: PCI memory mapped at 0x7fa658401000 >>>>>> EAL: PCI memory mapped at 0x7fa658402000 >>>>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1 >>>>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd >>>>>> EAL: PCI memory mapped at 0x7fa658404000 >>>>>> EAL: PCI memory mapped at 0x7fa658405000 >>>>>> EAL: PCI memory mapped at 0x7fa658406000 >>>>>> Zone 0: name:<RG_MP_log_history>, phys:0x82bfdec0, len:0x2080, >>>>>> virt:0x7fa6583fdec0, socket_id:0, flags:0 >>>>>> Zone 1: name:<MP_log_history>, phys:0x82973d40, len:0x28a0c0, >>>>>> virt:0x7fa658173d40, socket_id:0, flags:0 >>>>>> Zone 2: name:<rte_eth_dev_data>, phys:0x82943200, len:0x2f800, >>>>>> virt:0x7fa658143200, socket_id:0, flags:0 >>>>>> 2016-07-21T09:33:36Z|00009|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath supports recirculation >>>>>> 2016-07-21T09:33:36Z|00010|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS >>>>>> label stack length probed as 3 >>>>>> 2016-07-21T09:33:36Z|00011|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath supports truncate action >>>>>> 2016-07-21T09:33:36Z|00012|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath supports unique flow ids >>>>>> 2016-07-21T09:33:36Z|00013|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath does not support ct_state >>>>>> 2016-07-21T09:33:36Z|00014|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath does not support ct_zone >>>>>> 2016-07-21T09:33:36Z|00015|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath does not support ct_mark >>>>>> 2016-07-21T09:33:36Z|00016|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath does not support ct_label >>>>>> 2016-07-21T09:33:36Z|00017|ofproto_dpif|INFO|netdev@ovs-netdev: >>>>>> Datapath does not support ct_state_nat >>>>>> 2016-07-21T09:33:36Z|00018|bridge|INFO|bridge br0: added interface br0 >>>>>> on port 65534 >>>>>> 2016-07-21T09:33:36Z|00019|dpif_netlink|WARN|Generic Netlink family >>>>>> 'ovs_datapath' does not exist. The Open vSwitch kernel module is >>>>>> probably not loaded. >>>>>> 2016-07-21T09:33:36Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b >>>>>> 2016-07-21T09:33:36Z|00021|dpdk|ERR|Interface dpdk0 start error: >>>>>> Operation not permitted >>>>>> 2016-07-21T09:33:36Z|00022|bridge|INFO|bridge br0: using datapath ID >>>>>> 0000c664fbd86348 >>>>>> 2016-07-21T09:33:36Z|00023|connmgr|INFO|br0: added service controller >>>>>> "punix:/usr/local/var/run/openvswitch/br0.mgmt" >>>>>> 2016-07-21T09:33:36Z|00024|timeval|WARN|Unreasonably long 1244ms poll >>>>>> interval (233ms user, 880ms system) >>>>>> 2016-07-21T09:33:36Z|00025|timeval|WARN|faults: 3902 minor, 0 major >>>>>> 2016-07-21T09:33:36Z|00026|timeval|WARN|context switches: 10 >>>>>> voluntary, 214 involuntary >>>>>> 2016-07-21T09:33:36Z|00027|coverage|INFO|Event coverage, avg rate over >>>>>> last: 5 seconds, last minute, last hour, hash=75cfd2fc: >>>>>> 2016-07-21T09:33:36Z|00028|coverage|INFO|bridge_reconfigure >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00029|coverage|INFO|ofproto_flush >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00030|coverage|INFO|ofproto_update_port >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00031|coverage|INFO|rev_flow_table >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00032|coverage|INFO|cmap_expand >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 22 >>>>>> 2016-07-21T09:33:36Z|00033|coverage|INFO|cmap_shrink >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 7 >>>>>> 2016-07-21T09:33:36Z|00034|coverage|INFO|dpif_port_add >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 >>>>>> 2016-07-21T09:33:36Z|00035|coverage|INFO|dpif_flow_flush >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 >>>>>> 2016-07-21T09:33:36Z|00036|coverage|INFO|dpif_flow_get >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 5 >>>>>> 2016-07-21T09:33:36Z|00037|coverage|INFO|dpif_flow_put >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 10 >>>>>> 2016-07-21T09:33:36Z|00038|coverage|INFO|dpif_flow_del >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 5 >>>>>> 2016-07-21T09:33:36Z|00039|coverage|INFO|dpif_execute >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3 >>>>>> 2016-07-21T09:33:36Z|00040|coverage|INFO|flow_extract >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 4 >>>>>> 2016-07-21T09:33:36Z|00041|coverage|INFO|miniflow_malloc >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 16 >>>>>> 2016-07-21T09:33:36Z|00042|coverage|INFO|hmap_pathological >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00043|coverage|INFO|hmap_expand >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 653 >>>>>> 2016-07-21T09:33:36Z|00044|coverage|INFO|netdev_get_stats >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00045|coverage|INFO|poll_create_node >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 15 >>>>>> 2016-07-21T09:33:36Z|00046|coverage|INFO|poll_zero_timeout >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00047|coverage|INFO|seq_change >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 575 >>>>>> 2016-07-21T09:33:36Z|00048|coverage|INFO|pstream_open >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 3 >>>>>> 2016-07-21T09:33:36Z|00049|coverage|INFO|stream_open >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00050|coverage|INFO|util_xalloc >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 11820 >>>>>> 2016-07-21T09:33:36Z|00051|coverage|INFO|netdev_set_policing >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00052|coverage|INFO|netdev_get_ifindex >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00053|coverage|INFO|netdev_get_hwaddr >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 >>>>>> 2016-07-21T09:33:36Z|00054|coverage|INFO|netdev_set_hwaddr >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00055|coverage|INFO|netdev_get_ethtool >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 2 >>>>>> 2016-07-21T09:33:36Z|00056|coverage|INFO|netlink_received >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 6 >>>>>> 2016-07-21T09:33:36Z|00057|coverage|INFO|netlink_recv_jumbo >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 1 >>>>>> 2016-07-21T09:33:36Z|00058|coverage|INFO|netlink_sent >>>>>> 0.0/sec 0.000/sec 0.0000/sec total: 5 >>>>>> 2016-07-21T09:33:36Z|00059|coverage|INFO|70 events never hit >>>>>> 2016-07-21T09:33:36Z|00060|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b >>>>>> 2016-07-21T09:33:36Z|00061|dpdk|ERR|Interface dpdk0 start error: >>>>>> Operation not permitted >>>>> >>>>> The PMD start function is failing, unfortunately that error does not tell >>>>> that much information. >>>>> >>>>> I would suggest you to enable the debug of the vmxnet3 PMD in DPDK, set >>>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y in the config/common_base file of >>>>> DPDK, then recompile DPDK and OvS. >>>>> >>>>> Please try once again and send the terminal output, I hope we can get a >>>>> more >>>>> information of what is going on with that log. >>>> >>>> Based on your request, I reproduce the log with >>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y >>>> sudo ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock >>>> --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile --detach >>>> 2016-07-25T09:24:17Z|00001|vlog|INFO|opened log file >>>> /var/log/openvswitch/ovs-vswitchd.log >>>> 2016-07-25T09:24:17Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA >>>> node 0 >>>> 2016-07-25T09:24:17Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU >>>> cores >>>> >>>> 2016-07-25T09:24:17Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: >>>> connecting... >>>> >>>> 2016-07-25T09:24:17Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock: >>>> connected >>>> 2016-07-25T09:24:17Z|00006|dpdk|INFO|DPDK Enabled, initializing >>>> 2016-07-25T09:24:17Z|00007|dpdk|INFO|No vhost-sock-dir provided - >>>> defaulting to /usr/local/var/run/openvswitch >>>> 2016-07-25T09:24:17Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd >>>> --socket-mem 1024,0 -c 0x00000001 >>>> EAL: Detected lcore 0 as core 0 on socket 0 >>>> EAL: Detected lcore 1 as core 0 on socket 0 >>>> EAL: Support maximum 128 logical core(s) by configuration. >>>> EAL: Detected 2 lcore(s) >>>> EAL: Probing VFIO support... >>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or >>>> directory) >>>> EAL: VFIO modules not loaded, skipping VFIO support... >>>> EAL: Setting up physically contiguous memory... >>>> EAL: Ask a virtual area of 0x200000 bytes >>>> EAL: Virtual area found at 0x7fdea6000000 (size = 0x200000) >>>> EAL: Ask a virtual area of 0x5e000000 bytes >>>> EAL: Virtual area found at 0x7fde47e00000 (size = 0x5e000000) >>>> EAL: Ask a virtual area of 0x200000 bytes >>>> EAL: Virtual area found at 0x7fde47a00000 (size = 0x200000) >>>> EAL: Ask a virtual area of 0x200000 bytes >>>> EAL: Virtual area found at 0x7fde47600000 (size = 0x200000) >>>> EAL: Ask a virtual area of 0xa1800000 bytes >>>> EAL: Virtual area found at 0x7fdda5c00000 (size = 0xa1800000) >>>> EAL: Ask a virtual area of 0x200000 bytes >>>> EAL: Virtual area found at 0x7fdda5800000 (size = 0x200000) >>>> EAL: Requesting 512 pages of size 2MB from socket 0 >>>> EAL: TSC frequency is ~2799209 KHz >>>> EAL: Master lcore 0 is ready (tid=a855eb80;cpuset=[0]) >>>> PMD: rte_vmxnet3_pmd_init(): >> >>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1 >>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd >>>> EAL: Not managed by a supported kernel driver, skipped >>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1 >>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd >>>> EAL: PCI memory mapped at 0x7fdea6200000 >>>> EAL: PCI memory mapped at 0x7fdea6201000 >>>> EAL: PCI memory mapped at 0x7fdea6202000 >>>> PMD: eth_vmxnet3_dev_init(): >> >>>> PMD: eth_vmxnet3_dev_init(): Hardware version : 1 >>>> PMD: eth_vmxnet3_dev_init(): UPT hardware version : 1 >>>> PMD: eth_vmxnet3_dev_init(): MAC Address : 00:0c:29:b5:9f:5b >>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1 >>>> EAL: probe driver: 15ad:7b0 rte_vmxnet3_pmd >>>> EAL: Not managed by a supported kernel driver, skipped >>>> Zone 0: name:<RG_MP_log_history>, phys:0x555fdec0, len:0x2080, >>>> virt:0x7fdea61fdec0, socket_id:0, flags:0 >>>> Zone 1: name:<MP_log_history>, phys:0x95375f00, len:0x28a0c0, >>>> virt:0x7fde87975f00, socket_id:0, flags:0 >>>> Zone 2: name:<rte_eth_dev_data>, phys:0x555cd300, len:0x2f800, >>>> virt:0x7fdea61cd300, socket_id:0, flags:0 >>>> >>>> PMD: vmxnet3_dev_configure(): >> >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> >>>> PMD: vmxnet3_dev_rx_queue_setup(): >> >>>> PMD: vmxnet3_dev_start(): >> >>>> PMD: vmxnet3_rss_configure(): >> >>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b >>>> PMD: vmxnet3_disable_intr(): >> >>>> PMD: vmxnet3_dev_rxtx_init(): >> >>>> 2016-07-25T09:40:23Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b >>>> PMD: vmxnet3_dev_stop(): >> >>>> PMD: vmxnet3_disable_intr(): >> >>>> PMD: vmxnet3_dev_stop(): Device reset. >>>> PMD: vmxnet3_dev_clear_queues(): >> >>>> PMD: vmxnet3_dev_configure(): >> >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> >>>> PMD: vmxnet3_dev_tx_queue_setup(): >> >>>> PMD: vmxnet3_dev_rx_queue_setup(): >> >>>> PMD: vmxnet3_dev_start(): >> >>>> PMD: vmxnet3_rss_configure(): >> >>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b >>>> PMD: vmxnet3_dev_start(): Device activation: UNSUCCESSFUL >>> >>> >>> Unfortunately it is the most I can help you, it is strange for me why >>> test-pmd is working and why OvS is not. >>> I think it is a DPDK issue, then the next step is to post on the DPDK >>> mailing (d...@dpdk.org, add also in CC Yong Wang <yongw...@vmware.com> that >>> is the vmxnet3 maintainer), send a link to this thread in order to be clear >>> what you have tested. >> >> >> Hello Yong Wang, >> >> I am running fedora VM in ESXi 5.5.0 environment. I tried to install >> OVS and DPDK. OVS couldnt start DPDK device. >> You can see the error in the mail below. Do you know why? IF you need >> more information. I can provide more information. >> >> Good news, when I tried in Fedora 23 VM in ESXi 5.5.0, it works. It is >> possible to bind an dpdk device and get packets on port. >> >> Thanks Mauricio for your help and reply. >> >> Best regards >> Sothy > > Sothy, >
Hello, > You mentioned that on Fedora 23, it works. Then on some version Fedora that > it failed? Initially, I started working on Ubuntu 14.04. It produce error. Therefore, I changed in Fedora. I worked directly on Fedora 23. For fedora, I only tried on Fedora 23. It works. I didnt tried on other version. All Vms are in ESX 5.5. The problem is in Ubuntu vm ONLY, > From your previous post, you mentioned that it’s a Ubuntu 14.04 VM? Can you > clarify? Other than the changes you >mentioned already, are there any other > changes (such as ring buffer size, number of rx queues, etc) that have been > >changed? I didnt do any changes. I just followed basic commands given in OVDK install.md and >Is there any updates installed on your ESX 5.5? no >If yes, what’s the exact update version? Can you also send me a copy of your >vmkernel.log offline? I will send you offline. > > Also copying Daniele as he is trying to help reproduce the reported issue. > >> >>> >>>> I deleted exisiting bridge and recreated:::::: >>>> >>>> >>>> Then I added bridge. After I try to create bridge. It gives error. The >>>> erros is same as shown in log file. >>>> >>>> 2016-07-25T09:29:43.248Z|00020|bridge|INFO|bridge br0: added interface >>>> br0 on port 65534 >>>> 2016-07-25T09:29:43.253Z|00021|dpif_netlink|WARN|Generic Netlink >>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module >>>> is probably not loaded. >>>> 2016-07-25T09:29:43.253Z|00022|bridge|INFO|bridge br0: using datapath >>>> ID 0000e6aac740584b >>>> 2016-07-25T09:29:43.253Z|00023|connmgr|INFO|br0: added service >>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt" >>>> 2016-07-25T09:30:54.008Z|00024|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b >>>> 2016-07-25T09:30:54.021Z|00025|dpdk|ERR|Interface dpdk0 start error: >>>> Operation not permitted >>>> 2016-07-25T09:30:54.021Z|00026|dpif|WARN|netdev@ovs-netdev: failed to >>>> add dpdk0 as port: Operation not permitted >>>> >>>> >>>> >>>>> Another thing to do would be to bind the NICs to the uio_pci_generic and >>>>> VFIO modules, just to be sure that the issue is also present there. >>>> >>>> I tried in uio and uio_pci_genric also. Both are giving same error. >>>> I suspect I cannt activate VFIO in VM since it is BIOS suport also. AM I >>>> right? >>>> >>>> Thanks >>>> >>>>> >>>>>> in command line some information for your verification: >>>>>> >>>>>> sudo ovs-vsctl show >>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2 >>>>>> Bridge "br0" >>>>>> Port "dpdk0" >>>>>> Interface "dpdk0" >>>>>> type: dpdk >>>>>> Port "br0" >>>>>> Interface "br0" >>>>>> type: internal >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl del-br br0 >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-br br0 -- >>>>>> set bridge br0 datapath_type=netdev >>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-port br0 >>>>>> dpdk0 -- set Interface dpdk0 type=dpdk >>>>>> ovs-vsctl: Error detected while setting up 'dpdk0'. See ovs-vswitchd >>>>>> log for details. >>>>>> >>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>>>> sudo ovs-vsctl show >>>>>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2 >>>>>>>>> Bridge "br0" >>>>>>>>> Port "dpdk0" >>>>>>>>> Interface "dpdk0" >>>>>>>>> type: dpdk >>>>>>>>> Port "br0" >>>>>>>>> Interface "br0" >>>>>>>>> type: internal >>>>>>>>> >>>>>>>>> sudo ovs-ofctl dump-flows br0 >>>>>>>>> NXST_FLOW reply (xid=0x4): >>>>>>>>> cookie=0x0, duration=487.638s, table=0, n_packets=0, n_bytes=0, >>>>>>>>> idle_age=487, priority=0 actions=NORMAL >>>>>>>>> >>>>>>>>> >>>>>>>>>> In addition I would recommend to use the test-pmd [1] application >>>>>>>>>> from >>>>>>>>>> DPDK >>>>>>>>>> to check if the issue is also present in that case. >>>>>>>> >>>>>>>> I run testpmd command.The result is >>>>>>>> >>>>>>>> Configuring Port 0 (socket 0) >>>>>>>> Port 0: 00:0C:29:B5:9F:5B >>>>>>>> Configuring Port 1 (socket 0) >>>>>>>> Port 1: 00:0C:29:B5:9F:65 >>>>>>>> Checking link statuses... >>>>>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex >>>>>>>> Port 1 Link Up - speed 10000 Mbps - full-duplex >>>>>>>> Done >>>>>>>> No commandline core given, start packet forwarding >>>>>>>> io packet forwarding - CRC stripping disabled - packets/burst=32 >>>>>>>> nb forwarding cores=1 - nb forwarding ports=2 >>>>>>>> RX queues=1 - RX desc=128 - RX free threshold=0 >>>>>>>> RX threshold registers: pthresh=0 hthresh=0 wthresh=0 >>>>>>>> TX queues=1 - TX desc=512 - TX free threshold=0 >>>>>>>> TX threshold registers: pthresh=0 hthresh=0 wthresh=0 >>>>>>>> TX RS bit threshold=0 - TXQ flags=0x200 >>>>>>>> Press enter to exit >>>>>>>> >>>>>>>> Telling cores to stop... >>>>>>>> Waiting for lcores to finish... >>>>>>>> >>>>>>>> ---------------------- Forward statistics for port 0 >>>>>>>> ---------------------- >>>>>>>> RX-packets: 88 RX-dropped: 0 RX-total: 88 >>>>>>>> TX-packets: 2 TX-dropped: 0 TX-total: 2 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ---------------------------------------------------------------------------- >>>>>>>> >>>>>>>> ---------------------- Forward statistics for port 1 >>>>>>>> ---------------------- >>>>>>>> RX-packets: 2 RX-dropped: 0 RX-total: 2 >>>>>>>> TX-packets: 88 TX-dropped: 0 TX-total: 88 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ---------------------------------------------------------------------------- >>>>>>>> >>>>>>>> +++++++++++++++ Accumulated forward statistics for all >>>>>>>> ports+++++++++++++++ >>>>>>>> RX-packets: 90 RX-dropped: 0 RX-total: 90 >>>>>>>> TX-packets: 90 TX-dropped: 0 TX-total: 90 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>>>>>> >>>>>>>> make sense? >>>>>>> >>>>>>> >>>>>>> Yes, it makes sense. Ports are being recognized by DPDK and some >>>>>>> packets >>>>>>> were exchanged. >>>>>>> >>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides-2D16.04_testpmd-5Fapp-5Fug_index.html&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=JnLeHoyKuiecYI95-fcfFiuev5g2CDweT2MnyUiKWzM&e= >>>>>>>>>> >>>>>>>>>> Mauricio Vasquez, >>>>>>> >>>>>>> >>>>> Mauricio V, >>> >>> > _______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
