This provides a location to include all testing-related aspects of documentation.
Signed-off-by: Stephen Finucane <step...@that.guru> --- Documentation/index.rst | 2 + Documentation/install-guide/dpdk.rst | 197 +--------------- Documentation/install-guide/general.rst | 348 +---------------------------- Documentation/test-guide/code-analysis.rst | 45 ++++ Documentation/test-guide/datapath.rst | 138 ++++++++++++ Documentation/test-guide/dpdk.rst | 226 +++++++++++++++++++ Documentation/test-guide/index.rst | 45 ++++ Documentation/test-guide/oftest.rst | 87 ++++++++ Documentation/test-guide/ryu.rst | 62 +++++ Documentation/test-guide/travis.rst | 64 ++++++ Documentation/test-guide/unit.rst | 106 +++++++++ 11 files changed, 780 insertions(+), 540 deletions(-) create mode 100644 Documentation/test-guide/code-analysis.rst create mode 100644 Documentation/test-guide/datapath.rst create mode 100644 Documentation/test-guide/dpdk.rst create mode 100644 Documentation/test-guide/index.rst create mode 100644 Documentation/test-guide/oftest.rst create mode 100644 Documentation/test-guide/ryu.rst create mode 100644 Documentation/test-guide/travis.rst create mode 100644 Documentation/test-guide/unit.rst diff --git a/Documentation/index.rst b/Documentation/index.rst index e6b9c44..7eb262a 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -27,6 +27,8 @@ Welcome to Open vSwitch's documentation .. include:: install-guide/index.rst +.. include:: test-guide/index.rst + Bugs ---- diff --git a/Documentation/install-guide/dpdk.rst b/Documentation/install-guide/dpdk.rst index 50e2872..b51716f 100644 --- a/Documentation/install-guide/dpdk.rst +++ b/Documentation/install-guide/dpdk.rst @@ -57,9 +57,6 @@ Detailed system requirements can be found at `DPDK requirements`_, while more detailed install information can be found in the `advanced installation guide <../../INSTALL.DPDK-advanced.md>`__. -.. _DPDK supported NIC: http://dpdk.org/doc/nics -.. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html - Installing ---------- @@ -86,8 +83,6 @@ DPDK $ export DPDK_TARGET=x86_64-ivshmem-linuxapp-gcc -.. _DPDK sources: http://dpdk.org/browse/dpdk/refs/ - Install OVS ~~~~~~~~~~~ @@ -98,8 +93,6 @@ has to be configured with DPDK support (``--with-dpdk``). This section focuses on generic recipe that suits most cases. For distribution specific instructions, refer to one of the more relevant guides. -.. _OVS sources: http://openvswitch.org/releases/ - 1. Ensure the standard OVS requirements, described in :ref:`general-build-reqs` and :ref:`general-install-reqs`, are installed. @@ -390,192 +383,6 @@ Setup huge pages and DPDK devices using UIO:: lspci | grep Ethernet -Testing -------- - -Below are few testcases and the list of steps to be followed. Before beginning, -ensure a userspace bridge has been created and two DPDK ports added:: - - $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev - $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk - $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk - -PHY-PHY -~~~~~~~ - -Add test flows to forward packets betwen DPDK port 0 and port 1:: - - # Clear current flows - $ ovs-ofctl del-flows br0 - - # Add flows between port 1 (dpdk0) to port 2 (dpdk1) - $ ovs-ofctl add-flow br0 in_port=1,action=output:2 - $ ovs-ofctl add-flow br0 in_port=2,action=output:1 - -Transmit traffic into either port. You should see it returned via the other. - -PHY-VM-PHY (vhost loopback) -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Add two ``dpdkvhostuser`` ports to bridge ``br0``:: - - $ ovs-vsctl add-port br0 dpdkvhostuser0 \ - -- set Interface dpdkvhostuser0 type=dpdkvhostuser - $ ovs-vsctl add-port br0 dpdkvhostuser1 \ - -- set Interface dpdkvhostuser1 type=dpdkvhostuser - -Add test flows to forward packets betwen DPDK devices and VM ports:: - - # Clear current flows - $ ovs-ofctl del-flows br0 - - # Add flows - $ ovs-ofctl add-flow br0 in_port=1,action=output:3 - $ ovs-ofctl add-flow br0 in_port=3,action=output:1 - $ ovs-ofctl add-flow br0 in_port=4,action=output:2 - $ ovs-ofctl add-flow br0 in_port=2,action=output:4 - - # Dump flows - $ ovs-ofctl dump-flows br0 - -Create a VM using the following configuration: - -+----------------------+--------+-----------------+ -| configuration | values | comments | -+----------------------+--------+-----------------+ -| qemu version | 2.2.0 | n/a | -| qemu thread affinity | core 5 | taskset 0x20 | -| memory | 4GB | n/a | -| cores | 2 | n/a | -| Qcow2 image | CentOS7| n/a | -| mrg_rxbuf | off | n/a | -+----------------------+--------+-----------------+ - -You can do this directly with QEMU via the ``qemu-system-x86_64`` application:: - - $ export VM_NAME=vhost-vm - $ export GUEST_MEM=3072M - $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 - $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch - - $ taskset 0x20 qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ - -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ - -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ - -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ - -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ - -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ - -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ - -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ - -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off - -Alternatively, you can configure the guest using libvirt. Below is an XML -configuration for a 'demovm' guest that can be instantiated using `virsh`:: - - <domain type='kvm'> - <name>demovm</name> - <uuid>4a9b3f53-fa2a-47f3-a757-dd87720d9d1d</uuid> - <memory unit='KiB'>4194304</memory> - <currentMemory unit='KiB'>4194304</currentMemory> - <memoryBacking> - <hugepages> - <page size='2' unit='M' nodeset='0'/> - </hugepages> - </memoryBacking> - <vcpu placement='static'>2</vcpu> - <cputune> - <shares>4096</shares> - <vcpupin vcpu='0' cpuset='4'/> - <vcpupin vcpu='1' cpuset='5'/> - <emulatorpin cpuset='4,5'/> - </cputune> - <os> - <type arch='x86_64' machine='pc'>hvm</type> - <boot dev='hd'/> - </os> - <features> - <acpi/> - <apic/> - </features> - <cpu mode='host-model'> - <model fallback='allow'/> - <topology sockets='2' cores='1' threads='1'/> - <numa> - <cell id='0' cpus='0-1' memory='4194304' unit='KiB' memAccess='shared'/> - </numa> - </cpu> - <on_poweroff>destroy</on_poweroff> - <on_reboot>restart</on_reboot> - <on_crash>destroy</on_crash> - <devices> - <emulator>/usr/bin/qemu-kvm</emulator> - <disk type='file' device='disk'> - <driver name='qemu' type='qcow2' cache='none'/> - <source file='/root/CentOS7_x86_64.qcow2'/> - <target dev='vda' bus='virtio'/> - </disk> - <disk type='dir' device='disk'> - <driver name='qemu' type='fat'/> - <source dir='/usr/src/dpdk-16.07'/> - <target dev='vdb' bus='virtio'/> - <readonly/> - </disk> - <interface type='vhostuser'> - <mac address='00:00:00:00:00:01'/> - <source type='unix' path='/usr/local/var/run/openvswitch/dpdkvhostuser0' mode='client'/> - <model type='virtio'/> - <driver queues='2'> - <host mrg_rxbuf='off'/> - </driver> - </interface> - <interface type='vhostuser'> - <mac address='00:00:00:00:00:02'/> - <source type='unix' path='/usr/local/var/run/openvswitch/dpdkvhostuser1' mode='client'/> - <model type='virtio'/> - <driver queues='2'> - <host mrg_rxbuf='off'/> - </driver> - </interface> - <serial type='pty'> - <target port='0'/> - </serial> - <console type='pty'> - <target type='serial' port='0'/> - </console> - </devices> - </domain> - -Once the guest is configured and booted, configure DPDK packet forwarding -within the guest. To accomplish this, DPDK and testpmd application have to -be first compiled on the VM as described in :ref:`dpdk-guest-setup`. Once -compiled, run the ``test-pmd`` application:: - - $ cd $DPDK_DIR/app/test-pmd; - $ ./testpmd -c 0x3 -n 4 --socket-mem 1024 -- \ - --burst=64 -i --txqflags=0xf00 --disable-hw-vlan - $ set fwd mac retry - $ start - -When you finish testing, bind the vNICs back to kernel. - - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 - $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 - -.. note:: - Appropriate PCI IDs to be passed in above example. The PCI IDs can be - retrieved like so:: - - $ $DPDK_DIR/tools/dpdk-devbind.py --status - -.. note:: - More information on the dpdkvhostuser ports can be found in the `advanced - installation guide <../../INSTALL.DPDK-ADVANCED.md>`__. - -PHY-VM-PHY (IVSHMEM loopback) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Refer to the `advanced installation guide <../../INSTALL.DPDK-ADVANCED.md>`__. - Limitations ------------ @@ -591,4 +398,8 @@ Limitations The latest list of validated firmware versions can be found in the `DPDK release notes`_. +.. _DPDK supported NIC: http://dpdk.org/doc/nics +.. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html .. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_16.07.html +.. _DPDK sources: http://dpdk.org/browse/dpdk/refs/ +.. _OVS sources: http://openvswitch.org/releases/ diff --git a/Documentation/install-guide/general.rst b/Documentation/install-guide/general.rst index e1866f8..e003563 100644 --- a/Documentation/install-guide/general.rst +++ b/Documentation/install-guide/general.rst @@ -342,7 +342,7 @@ Building see unusual warnings when you use both together, consider disabling ccache. -2. Consider running the testsuite. Refer to :ref:`general-testing` for +2. Consider running the testsuite. Refer to :doc:`../test-guide/index` for instructions. 3. Run ``make install`` to install the executables and manpages into the @@ -522,349 +522,3 @@ minimum. The ovs-ctl utility's ``force-reload-kmod`` function does all of the above, but also replaces the old kernel module with the new one. Open vSwitch startup scripts for Debian, XenServer and RHEL use ovs-ctl's functions and it is recommended that these functions be used for other software platforms too. - -.. _general-testing: - -Testing -------- - -This section describe Open vSwitch's built-in support for various test -suites. You must bootstrap, configure and build Open vSwitch (steps are -in "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD" -above) before you run the tests described here. You do not need to -install Open vSwitch or to build or load the kernel module to run these -test suites. You do not need supervisor privilege to run these test -suites. - -Unit Tests -~~~~~~~~~~ - -Open vSwitch includes a suite of self-tests. Before you submit patches -upstream, we advise that you run the tests and ensure that they pass. If you -add new features to Open vSwitch, then adding tests for those features will -ensure your features don't break as developers modify other areas of Open -vSwitch. - -To run all the unit tests in Open vSwitch, one at a time, run:: - - $ make check - -This takes under 5 minutes on a modern desktop system. - -To run all the unit tests in Open vSwitch in parallel, run:: - - $ make check TESTSUITEFLAGS=-j8 - -You can run up to eight threads. This takes under a minute on a modern 4-core -desktop system. - -To see a list of all the available tests, run: - - $ make check TESTSUITEFLAGS=--list - -To run only a subset of tests, e.g. test 123 and tests 477 through 484, run:: - - $ make check TESTSUITEFLAGS='123 477-484' - -Tests do not have inter-dependencies, so you may run any subset. - -To run tests matching a keyword, e.g. ``ovsdb``, run:: - - $ make check TESTSUITEFLAGS='-k ovsdb' - -To see a complete list of test options, run:: - - $ make check TESTSUITEFLAGS=--help - -The results of a testing run are reported in ``tests/testsuite.log``. Report -report test failures as bugs and include the ``testsuite.log`` in your report. - -.. note:: - Sometimes a few tests may fail on some runs but not others. This is usually a - bug in the testsuite, not a bug in Open vSwitch itself. If you find that a - test fails intermittently, please report it, since the developers may not - have noticed. You can make the testsuite automatically rerun tests that fail, - by adding ``RECHECK=yes`` to the ``make`` command line, e.g.:: - - $ make check TESTSUITEFLAGS=-j8 RECHECK=yes - -Coverage -++++++++ - -If the build was configured with ``--enable-coverage`` and the ``lcov`` utility -is installed, you can run the testsuite and generate a code coverage report by -using the ``check-lcoc`` target:: - - $ make check-lcov - -All the same options are avaiable via TESTSUITEFLAGS. For example:: - - $ make check-lcov TESTSUITEFLAGS=-j8 -k ovn - -Profiling -+++++++++ - -If you have ``valgrind`` installed, you can run the testsuite under -valgrind by using the ``check-valgrind`` target:: - - $ make check-valgrind - -When you do this, the "valgrind" results for test ``<N>`` are reported in files -named ``tests/testsuite.dir/<N>/valgrind.*``. - -All the same options are available via TESTSUITEFLAGS. - -.. hint:: - You may find that the valgrind results are easier to interpret if you put - ``-q`` in ``~/.valgrindrc``, since that reduces the amount of output. - -.. _general-oftest: - -OFTest -~~~~~~ - -OFTest is an OpenFlow protocol testing suite. Open vSwitch includes a Makefile -target to run OFTest with Open vSwitch in "dummy mode". In this mode of -testing, no packets travel across physical or virtual networks. Instead, Unix -domain sockets stand in as simulated networks. This simulation is imperfect, -but it is much easier to set up, does not require extra physical or virtual -hardware, and does not require supervisor privileges. - -To run OFTest with Open vSwitch, first read and follow the instructions under -:ref:`general-testing` above. Second, obtain a copy of OFTest and install its -prerequisites. You need a copy of OFTest that includes commit 406614846c5 (make -ovs-dummy platform work again). This commit was merged into the OFTest -repository on Feb 1, 2013, so any copy of OFTest more recent than that should -work. Testing OVS in dummy mode does not require root privilege, so you may -ignore that requirement. - -Optionally, add the top-level OFTest directory (containing the ``oft`` program) -to your ``$PATH``. This slightly simplifies running OFTest later. - -To run OFTest in dummy mode, run the following command from your Open vSwitch -build directory:: - - $ make check-oftest OFT=<oft-binary> - -where ``<oft-binary>`` is the absolute path to the ``oft`` program in OFTest. -If you added "oft" to your $PATH, you may omit the OFT variable -assignment - -By default, ``check-oftest`` passes ``oft`` just enough options to enable dummy -mode. You can use ``OFTFLAGS`` to pass additional options. For example, to run -just the ``basic.Echo`` test instead of all tests (the default) and enable -verbose logging, run:: - - $ make check-oftest OFT=<oft-binary> OFTFLAGS='--verbose -T basic.Echo' - -If you use OFTest that does not include commit 4d1f3eb2c792 (oft: change -default port to 6653), merged into the OFTest repository in October 2013, then -you need to add an option to use the IETF-assigned controller port:: - - $ make check-oftest OFT=<oft-binary> OFTFLAGS='--port=6653' - -Interpret OFTest results cautiously. Open vSwitch can fail a given test in -OFTest for many reasons, including bugs in Open vSwitch, bugs in OFTest, bugs -in the "dummy mode" integration, and differing interpretations of the OpenFlow -standard and other standards. - -.. note:: - Open vSwitch has not been validated against OFTest. Report test failures that - you believe to represent bugs in Open vSwitch. Include the precise versions - of Open vSwitch and OFTest in your bug report, plus any other information - needed to reproduce the problem. - -Ryu -~~~ - -Ryu is an OpenFlow controller written in Python that includes an extensive -OpenFlow testsuite. Open vSwitch includes a Makefile target to run Ryu in -"dummy mode". See :ref:`general-oftest` above for an explanation of dummy mode. - -To run Ryu tests with Open vSwitch, first read and follow the instructions -under :ref:`general-testing` above. Second, obtain a copy of Ryu, install its -prerequisites, and build it. You do not need to install Ryu (some of the tests -do not get installed, so it does not help). - -To run Ryu tests, run the following command from your Open vSwitch build -directory:: - - $ make check-ryu RYUDIR=<ryu-source-dir>`` - -where ``<ryu-source-dir>`` is the absolute path to the root of the Ryu source -distribution. The default ``<ryu-source-dir>`` is ``$srcdir/../ryu`` -where ``$srcdir`` is your Open vSwitch source directory. If this is correct, -omit ``RYUDIR`` - -.. note:: - Open vSwitch has not been validated against Ryu. Report test failures that - you believe to represent bugs in Open vSwitch. Include the precise versions - of Open vSwitch and Ryu in your bug report, plus any other information - needed to reproduce the problem. - -Datapath testing -~~~~~~~~~~~~~~~~ - -Open vSwitch includes a suite of tests specifically for datapath functionality, -which can be run against the userspace or kernel datapaths. If you are -developing datapath features, it is recommended that you use these tests and -build upon them to verify your implementation. - -The datapath tests make some assumptions about the environment. They must be -run under root privileges on a Linux system with support for network -namespaces. For ease of use, the OVS source tree includes a vagrant box to -invoke these tests. Running the tests inside Vagrant provides kernel isolation, -protecting your development host from kernel panics or configuration conflicts -in the testsuite. If you wish to run the tests without using the vagrant box, -there are further instructions below. - -Vagrant -+++++++ - -.. important:: - - Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor - -.. note:: - You must :ref:`bootstrap <general-bootstrapping>` and - :ref:`configure<general-configuring>` the sources before you run the steps - described here. - -A Vagrantfile is provided allowing to compile and provision the source tree as -found locally in a virtual machine using the following command:: - - $ vagrant up - -This will bring up a Fedora 23 VM by default. If you wish to use a different -box or a vagrant backend not supported by the default box, the ``Vagrantfile`` -can be modified to use a different box as base. - -The VM can be reprovisioned at any time:: - - $ vagrant provision - -OVS out-of-tree compilation environment can be set up with:: - - $ ./boot.sh - $ vagrant provision --provision-with configure_ovs,build_ovs - -This will set up an out-of-tree build environment inside the VM in -``/root/build``. The source code can be found in ``/vagrant``. - -To recompile and reinstall OVS in the VM using RPM:: - - $ ./boot.sh - $ vagrant provision --provision-with configure_ovs,install_rpm - -Two provisioners are included to run system tests with the OVS kernel module or -with a userspace datapath. This tests are different from the self-tests -mentioned above. To run them:: - - $ ./boot.sh - $ vagrant provision --provision-with \ - configure_ovs,test_ovs_kmod,test_ovs_system_userspace - -The results of the testsuite reside in the VM root user's home directory:: - - $ vagrant ssh - $ sudo -s - $ cd /root/build - $ ls tests/system* - -Native -++++++ - -The datapath testsuite as invoked by Vagrant above may also be run manually on -a Linux system with root privileges. These tests may take several minutes to -complete, and cannot be run in parallel. - -Userspace datapath -''''''''''''''''''' - -To invoke the datapath testsuite with the userspace datapath, run:: - - $ make check-system-userspace - -The results of the testsuite are in ``tests/system-userspace-traffic.dir``. - -Kernel datapath -''''''''''''''' - -Make targets are also provided for testing the Linux kernel module. Note that -these tests operate by inserting modules into the running Linux kernel, so if -the tests are able to trigger a bug in the OVS kernel module or in the upstream -kernel then the kernel may panic. - -To run the testsuite against the kernel module which is currently installed on -your system, run:: - - $ make check-kernel - -To install the kernel module from the current build directory and run the -testsuite against that kernel module:: - - $ make check-kmod - -The results of the testsuite are in ``tests/system-kmod-traffic.dir``. - -Continuous Integration with Travis-CI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A .travis.yml file is provided to automatically build Open vSwitch with various -build configurations and run the testsuite using travis-ci. Builds will be -performed with gcc, sparse and clang with the -Werror compiler flag included, -therefore the build will fail if a new warning has been introduced. - -The CI build is triggered via git push (regardless of the specific branch) or -pull request against any Open vSwitch GitHub repository that is linked to -travis-ci. - -Instructions to setup travis-ci for your GitHub repository: - -1. Go to http://travis-ci.org/ and sign in using your GitHub ID. -2. Go to the "Repositories" tab and enable the ovs repository. You may disable - builds for pushes or pull requests. -3. In order to avoid forks sending build failures to the upstream mailing list, - the notification email recipient is encrypted. If you want to receive email - notification for build failures, replace the the encrypted string: - - 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis - 2. In your Open vSwitch repository: travis encrypt myl...@mydomain.org - 3. Add/replace the notifications section in .travis.yml and fill in the - secure string as returned by travis encrypt:: - - notifications: - email: - recipients: - - secure: "....." - - .. note:: - You may remove/omit the notifications section to fall back to default - notification behaviour which is to send an email directly to the author and - committer of the failing commit. Note that the email is only sent if the - author/committer have commit rights for the particular GitHub repository. - -4. Pushing a commit to the repository which breaks the build or the - testsuite will now trigger a email sent to myl...@mydomain.org - -Static Code Analysis -~~~~~~~~~~~~~~~~~~~~ - -Static Analysis is a method of debugging Software by examining code rather than -actually executing it. This can be done through 'scan-build' commandline -utility which internally uses clang (or) gcc to compile the code and also -invokes a static analyzer to do the code analysis. At the end of the build, the -reports are aggregated in to a common folder and can later be analyzed using -'scan-view'. - -Open vSwitch includes a Makefile target to trigger static code analysis:: - - $ ./boot.sh - $ ./configure CC=clang # clang - # or - $ ./configure CC=gcc CFLAGS="-std=gnu99" # gcc - $ make clang-analyze - -You should invoke scan-view to view analysis results. The last line of output -from ``clang-analyze`` will list the command (containing results directory) -that you should invoke to view the results on a browser. diff --git a/Documentation/test-guide/code-analysis.rst b/Documentation/test-guide/code-analysis.rst new file mode 100644 index 0000000..a4fb96d --- /dev/null +++ b/Documentation/test-guide/code-analysis.rst @@ -0,0 +1,45 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +==================== +Static Code Analysis +==================== + +Static Analysis is a method of debugging Software by examining code rather than +actually executing it. This can be done through 'scan-build' commandline +utility which internally uses clang (or) gcc to compile the code and also +invokes a static analyzer to do the code analysis. At the end of the build, the +reports are aggregated in to a common folder and can later be analyzed using +'scan-view'. + +Open vSwitch includes a Makefile target to trigger static code analysis:: + + $ ./boot.sh + $ ./configure CC=clang # clang + # or + $ ./configure CC=gcc CFLAGS="-std=gnu99" # gcc + $ make clang-analyze + +You should invoke scan-view to view analysis results. The last line of output +from ``clang-analyze`` will list the command (containing results directory) +that you should invoke to view the results on a browser. diff --git a/Documentation/test-guide/datapath.rst b/Documentation/test-guide/datapath.rst new file mode 100644 index 0000000..ba93720 --- /dev/null +++ b/Documentation/test-guide/datapath.rst @@ -0,0 +1,138 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============== +Datapath Tests +============== + +Open vSwitch includes a suite of tests specifically for datapath functionality, +which can be run against the userspace or kernel datapaths. If you are +developing datapath features, it is recommended that you use these tests and +build upon them to verify your implementation. + +The datapath tests make some assumptions about the environment. They must be +run under root privileges on a Linux system with support for network +namespaces. For ease of use, the OVS source tree includes a Vagrant box to +invoke these tests. Running the tests inside Vagrant provides kernel isolation, +protecting your development host from kernel panics or configuration conflicts +in the testsuite. If you wish to run the tests without using the vagrant box, +there are further instructions below. + +Vagrant +------- + +.. important:: + + Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor + +.. note:: + You must :ref:`bootstrap <general-bootstrapping>` and + :ref:`configure<general-configuring>` the sources before you run the steps + described here. + +A Vagrantfile is provided allowing to compile and provision the source tree as +found locally in a virtual machine using the following command:: + + $ vagrant up + +This will bring up a Fedora 23 VM by default. If you wish to use a different +box or a vagrant backend not supported by the default box, the ``Vagrantfile`` +can be modified to use a different box as base. + +The VM can be reprovisioned at any time:: + + $ vagrant provision + +OVS out-of-tree compilation environment can be set up with:: + + $ ./boot.sh + $ vagrant provision --provision-with configure_ovs,build_ovs + +This will set up an out-of-tree build environment inside the VM in +``/root/build``. The source code can be found in ``/vagrant``. + +To recompile and reinstall OVS in the VM using RPM:: + + $ ./boot.sh + $ vagrant provision --provision-with configure_ovs,install_rpm + +Two provisioners are included to run system tests with the OVS kernel module or +with a userspace datapath. This tests are different from the self-tests +mentioned above. To run them:: + + $ ./boot.sh + $ vagrant provision --provision-with \ + configure_ovs,test_ovs_kmod,test_ovs_system_userspace + +The results of the testsuite reside in the VM root user's home directory:: + + $ vagrant ssh + $ sudo -s + $ cd /root/build + $ ls tests/system* + +Native +------ + +The datapath testsuite as invoked by Vagrant above may also be run manually on +a Linux system with root privileges. These tests may take several minutes to +complete, and cannot be run in parallel. + +.. note:: + + You must build and install Open vSwitch before you can run any of the tests + described below. Refer to the :doc:`../install-guide/general` for more + information. + +Userspace datapath +~~~~~~~~~~~~~~~~~~~ + +Make targets are provided for testing datapath functionality. + +To invoke the datapath testsuite with the userspace datapath, run:: + + $ make check-system-userspace + +The results of the testsuite are in ``tests/system-userspace-traffic.dir``. + +Kernel datapath +~~~~~~~~~~~~~~~~ + +Make targets are also provided for testing the Linux kernel module. + +.. warning:: + These tests operate by inserting modules into the running Linux kernel, so if + the tests are able to trigger a bug in the OVS kernel module or in the + upstream kernel then the kernel may panic. + +To run the testsuite against the kernel module which is currently installed on +your system, run:: + + $ make check-kernel + +To install the kernel module from the current build directory and run the +testsuite against that kernel module:: + + $ make check-kmod + +The results of the testsuite are in ``tests/system-kmod-traffic.dir``. diff --git a/Documentation/test-guide/dpdk.rst b/Documentation/test-guide/dpdk.rst new file mode 100644 index 0000000..e4296b6 --- /dev/null +++ b/Documentation/test-guide/dpdk.rst @@ -0,0 +1,226 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=================== +DPDK Datapath Tests +=================== + +When building Open vSwitch with DPDK, it may be helpful to validate correct +behavior of the DPDK bridges and ports. Below are a list of scenarios that can +be used to validate behavior. + +.. note:: + + You must build and install Open vSwitch with DPDK before you can run any of + the tests described below. Refer to the :doc:`../install-guide/dpdk` for more + information. + +Pre-requesites +--------------- + +Before beginning, ensure a userspace bridge has been created and two DPDK ports +added:: + + $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev + $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk + $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk + +PHY-PHY +------- + +Add test flows to forward packets betwen DPDK port 0 and port 1:: + + # Clear current flows + $ ovs-ofctl del-flows br0 + + # Add flows between port 1 (dpdk0) to port 2 (dpdk1) + $ ovs-ofctl add-flow br0 in_port=1,action=output:2 + $ ovs-ofctl add-flow br0 in_port=2,action=output:1 + +Transmit traffic into either port. You should see it returned via the other. + +PHY-VM-PHY (vhost loopback) +--------------------------- + +The vHost loopback "bunny ears" test can be used to validate correct behavior +of the vHost user interfaces. + +To start, add two ``dpdkvhostuser`` ports to bridge ``br0``. These should exist +in addition to the two ports previously created:: + + $ ovs-vsctl add-port br0 dpdkvhostuser0 \ + -- set Interface dpdkvhostuser0 type=dpdkvhostuser + $ ovs-vsctl add-port br0 dpdkvhostuser1 \ + -- set Interface dpdkvhostuser1 type=dpdkvhostuser + +Add test flows to forward packets betwen DPDK devices and VM ports:: + + # Clear current flows + $ ovs-ofctl del-flows br0 + + # Add flows + $ ovs-ofctl add-flow br0 in_port=1,action=output:3 + $ ovs-ofctl add-flow br0 in_port=3,action=output:1 + $ ovs-ofctl add-flow br0 in_port=4,action=output:2 + $ ovs-ofctl add-flow br0 in_port=2,action=output:4 + + # Dump flows + $ ovs-ofctl dump-flows br0 + +Create a VM using the following configuration: + ++----------------------+--------+-----------------+ +| configuration | values | comments | ++----------------------+--------+-----------------+ +| qemu version | 2.2.0 | n/a | +| qemu thread affinity | core 5 | taskset 0x20 | +| memory | 4GB | n/a | +| cores | 2 | n/a | +| Qcow2 image | CentOS7| n/a | +| mrg_rxbuf | off | n/a | ++----------------------+--------+-----------------+ + +You can do this directly with QEMU via the ``qemu-system-x86_64`` application:: + + $ export VM_NAME=vhost-vm + $ export GUEST_MEM=3072M + $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 + $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch + + $ taskset 0x20 qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ + -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ + -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ + -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ + -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ + -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ + -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off + +Alternatively, you can configure the guest using libvirt. Below is an XML +configuration for a 'demovm' guest that can be instantiated using `virsh`:: + + <domain type='kvm'> + <name>demovm</name> + <uuid>4a9b3f53-fa2a-47f3-a757-dd87720d9d1d</uuid> + <memory unit='KiB'>4194304</memory> + <currentMemory unit='KiB'>4194304</currentMemory> + <memoryBacking> + <hugepages> + <page size='2' unit='M' nodeset='0'/> + </hugepages> + </memoryBacking> + <vcpu placement='static'>2</vcpu> + <cputune> + <shares>4096</shares> + <vcpupin vcpu='0' cpuset='4'/> + <vcpupin vcpu='1' cpuset='5'/> + <emulatorpin cpuset='4,5'/> + </cputune> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <features> + <acpi/> + <apic/> + </features> + <cpu mode='host-model'> + <model fallback='allow'/> + <topology sockets='2' cores='1' threads='1'/> + <numa> + <cell id='0' cpus='0-1' memory='4194304' unit='KiB' memAccess='shared'/> + </numa> + </cpu> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-kvm</emulator> + <disk type='file' device='disk'> + <driver name='qemu' type='qcow2' cache='none'/> + <source file='/root/CentOS7_x86_64.qcow2'/> + <target dev='vda' bus='virtio'/> + </disk> + <disk type='dir' device='disk'> + <driver name='qemu' type='fat'/> + <source dir='/usr/src/dpdk-16.07'/> + <target dev='vdb' bus='virtio'/> + <readonly/> + </disk> + <interface type='vhostuser'> + <mac address='00:00:00:00:00:01'/> + <source type='unix' path='/usr/local/var/run/openvswitch/dpdkvhostuser0' mode='client'/> + <model type='virtio'/> + <driver queues='2'> + <host mrg_rxbuf='off'/> + </driver> + </interface> + <interface type='vhostuser'> + <mac address='00:00:00:00:00:02'/> + <source type='unix' path='/usr/local/var/run/openvswitch/dpdkvhostuser1' mode='client'/> + <model type='virtio'/> + <driver queues='2'> + <host mrg_rxbuf='off'/> + </driver> + </interface> + <serial type='pty'> + <target port='0'/> + </serial> + <console type='pty'> + <target type='serial' port='0'/> + </console> + </devices> + </domain> + +Once the guest is configured and booted, configure DPDK packet forwarding +within the guest. To accomplish this, DPDK and testpmd application have to +be first compiled on the VM as described in :ref:`dpdk-guest-setup`. Once +compiled, run the ``test-pmd`` application:: + + $ cd $DPDK_DIR/app/test-pmd; + $ ./testpmd -c 0x3 -n 4 --socket-mem 1024 -- \ + --burst=64 -i --txqflags=0xf00 --disable-hw-vlan + $ set fwd mac retry + $ start + +When you finish testing, bind the vNICs back to kernel. + + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 + +.. note:: + Appropriate PCI IDs to be passed in above example. The PCI IDs can be + retrieved like so:: + + $ $DPDK_DIR/tools/dpdk-devbind.py --status + +.. note:: + More information on the dpdkvhostuser ports can be found in the `advanced + installation guide <../../INSTALL.DPDK-ADVANCED.md>`__. + +PHY-VM-PHY (IVSHMEM loopback) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the `advanced installation guide <../../INSTALL.DPDK-ADVANCED.md>`__. diff --git a/Documentation/test-guide/index.rst b/Documentation/test-guide/index.rst new file mode 100644 index 0000000..3228386 --- /dev/null +++ b/Documentation/test-guide/index.rst @@ -0,0 +1,45 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +Testing Guides +-------------- + +Once installed, there are a variety of methods available for testing Open +vSwitch. Some of these require different configurations of Open vSwitch. + +.. toctree:: + :maxdepth: 1 + + /test-guide/unit + /test-guide/datapath + /test-guide/oftest + /test-guide/ryu + /test-guide/travis + /test-guide/code-analysis + +Tests that require specific Open vSwitch configurations. + +.. toctree:: + :maxdepth: 1 + + /test-guide/dpdk diff --git a/Documentation/test-guide/oftest.rst b/Documentation/test-guide/oftest.rst new file mode 100644 index 0000000..e5afe81 --- /dev/null +++ b/Documentation/test-guide/oftest.rst @@ -0,0 +1,87 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +====== +OFTest +====== + +OFTest is an OpenFlow protocol testing suite. Open vSwitch includes a Makefile +target to run OFTest with Open vSwitch in "dummy mode". In this mode of +testing, no packets travel across physical or virtual networks. Instead, Unix +domain sockets stand in as simulated networks. This simulation is imperfect, +but it is much easier to set up, does not require extra physical or virtual +hardware, and does not require supervisor privileges. + +.. note:: + + You must build and install Open vSwitch before you can run any of the tests + described below. Refer to the :doc:`../install-guide/general` for more + information. + +Installation +------------ + +To configure OFTest for testing with Open vSwitch, obtain a copy of OFTest and +install its prerequisites. OFTest will be run in dummy mode, meaning root +privileges are not required. + +.. note:: + OFTest must includes commit 406614846c5 (make ovs-dummy platform work again). + This commit was merged into the OFTest repository on Feb 1, 2013, so any copy + of OFTest more recent than that should work. If you use a version of OFTest + that does not include this commit then then you need to add an option to use + the IETF-assigned controller port. You can do this by setting ``OFTFLAGS`` + like so:: + + $ make check-oftest OFT=<oft-binary> OFTFLAGS='--port=6653' + +.. hint:: + You may wish to add the top-level OFTest directory (containing the ``oft`` + program) to your ``$PATH``. This slightly simplifies running OFTest later. + +To run OFTest in dummy mode, run the following command from your Open vSwitch +build directory:: + + $ make check-oftest OFT=<oft-binary> + +where ``<oft-binary>`` is the absolute path to the ``oft`` program in OFTest. +If you added "oft" to your $PATH, you may omit the OFT variable +assignment + +By default, ``check-oftest`` passes ``oft`` just enough options to enable dummy +mode. You can use ``OFTFLAGS`` to pass additional options. For example, to run +just the ``basic.Echo`` test instead of all tests (the default) and enable +verbose logging, run:: + + $ make check-oftest OFT=<oft-binary> OFTFLAGS='--verbose -T basic.Echo' + +Interpret OFTest results cautiously. Open vSwitch can fail a given test in +OFTest for many reasons, including bugs in Open vSwitch, bugs in OFTest, bugs +in the "dummy mode" integration, and differing interpretations of the OpenFlow +standard and other standards. + +.. note:: + Open vSwitch has not been validated against OFTest. Report test failures that + you believe to represent bugs in Open vSwitch. Include the precise versions + of Open vSwitch and OFTest in your bug report, plus any other information + needed to reproduce the problem. diff --git a/Documentation/test-guide/ryu.rst b/Documentation/test-guide/ryu.rst new file mode 100644 index 0000000..0f36b24 --- /dev/null +++ b/Documentation/test-guide/ryu.rst @@ -0,0 +1,62 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +=== +Ryu +=== + +Ryu is an OpenFlow controller written in Python that includes an extensive +OpenFlow testsuite. Open vSwitch includes a Makefile target to run Ryu in +"dummy mode". See :doc:`oftest` for an explanation of dummy mode. + +.. note:: + + You must build and install Open vSwitch before you can run any of the tests + described below. Refer to the :doc:`../install-guide/general` for more + information. + +Installation +------------ + +To configure RYU for testing with Open vSwitch, obtain a copy of Ryu, install +its prerequisites, and build it. You do not need to install Ryu: some of the +tests do not get installed, so it does not help. + +Running Tests +------------- + +To run Ryu tests, run the following command from your Open vSwitch build +directory:: + + $ make check-ryu RYUDIR=<ryu-source-dir>`` + +where ``<ryu-source-dir>`` is the absolute path to the root of the Ryu source +distribution. The default ``<ryu-source-dir>`` is ``$srcdir/../ryu`` +where ``$srcdir`` is your Open vSwitch source directory. If this is correct, +omit ``RYUDIR`` + +.. note:: + Open vSwitch has not been validated against Ryu. Report test failures that + you believe to represent bugs in Open vSwitch. Include the precise versions + of Open vSwitch and Ryu in your bug report, plus any other information + needed to reproduce the problem. diff --git a/Documentation/test-guide/travis.rst b/Documentation/test-guide/travis.rst new file mode 100644 index 0000000..9ba3d40 --- /dev/null +++ b/Documentation/test-guide/travis.rst @@ -0,0 +1,64 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +===================================== +Continuous Integration with Travis-CI +===================================== + +A .travis.yml file is provided to automatically build Open vSwitch with various +build configurations and run the testsuite using travis-ci. Builds will be +performed with gcc, sparse and clang with the -Werror compiler flag included, +therefore the build will fail if a new warning has been introduced. + +The CI build is triggered via git-push (regardless of the specific branch) or +pull request against any Open vSwitch GitHub repository that is linked to +travis-ci. + +You can use Travis to build your fork of the Open vSwitch repository on GitHub. +To do so: + +1. Go to http://travis-ci.org/ and sign in using your GitHub ID. +2. Go to the "Repositories" tab and enable the ovs repository. You may disable + builds for pushes or pull requests. +3. In order to avoid forks sending build failures to the upstream mailing list, + the notification email recipient is encrypted. If you want to receive email + notification for build failures, replace the the encrypted string: + + 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis + 2. In your Open vSwitch repository: travis encrypt myl...@mydomain.org + 3. Add/replace the notifications section in .travis.yml and fill in the + secure string as returned by travis encrypt:: + + notifications: + email: + recipients: + - secure: "....." + + .. note:: + You may remove/omit the notifications section to fall back to default + notification behaviour which is to send an email directly to the author and + committer of the failing commit. Note that the email is only sent if the + author/committer have commit rights for the particular GitHub repository. + +4. Pushing a commit to the repository which breaks the build or the + testsuite will now trigger a email sent to myl...@mydomain.org diff --git a/Documentation/test-guide/unit.rst b/Documentation/test-guide/unit.rst new file mode 100644 index 0000000..153352b --- /dev/null +++ b/Documentation/test-guide/unit.rst @@ -0,0 +1,106 @@ +.. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +========== +Unit Tests +========== + +Open vSwitch includes a suite of self-tests. Before you submit patches +upstream, we advise that you run the tests and ensure that they pass. If you +add new features to Open vSwitch, then adding tests for those features will +ensure your features don't break as developers modify other areas of Open +vSwitch. + +To run all the unit tests in Open vSwitch, one at a time, run:: + + $ make check + +This takes under 5 minutes on a modern desktop system. + +To run all the unit tests in Open vSwitch in parallel, run:: + + $ make check TESTSUITEFLAGS=-j8 + +You can run up to eight threads. This takes under a minute on a modern 4-core +desktop system. + +To see a list of all the available tests, run: + + $ make check TESTSUITEFLAGS=--list + +To run only a subset of tests, e.g. test 123 and tests 477 through 484, run:: + + $ make check TESTSUITEFLAGS='123 477-484' + +Tests do not have inter-dependencies, so you may run any subset. + +To run tests matching a keyword, e.g. ``ovsdb``, run:: + + $ make check TESTSUITEFLAGS='-k ovsdb' + +To see a complete list of test options, run:: + + $ make check TESTSUITEFLAGS=--help + +The results of a testing run are reported in ``tests/testsuite.log``. Report +report test failures as bugs and include the ``testsuite.log`` in your report. + +.. note:: + Sometimes a few tests may fail on some runs but not others. This is usually a + bug in the testsuite, not a bug in Open vSwitch itself. If you find that a + test fails intermittently, please report it, since the developers may not + have noticed. You can make the testsuite automatically rerun tests that fail, + by adding ``RECHECK=yes`` to the ``make`` command line, e.g.:: + + $ make check TESTSUITEFLAGS=-j8 RECHECK=yes + +Coverage +-------- + +If the build was configured with ``--enable-coverage`` and the ``lcov`` utility +is installed, you can run the testsuite and generate a code coverage report by +using the ``check-lcoc`` target:: + + $ make check-lcov + +All the same options are avaiable via TESTSUITEFLAGS. For example:: + + $ make check-lcov TESTSUITEFLAGS=-j8 -k ovn + +Profiling +--------- + +If you have ``valgrind`` installed, you can run the testsuite under +valgrind by using the ``check-valgrind`` target:: + + $ make check-valgrind + +When you do this, the "valgrind" results for test ``<N>`` are reported in files +named ``tests/testsuite.dir/<N>/valgrind.*``. + +All the same options are available via TESTSUITEFLAGS. + +.. hint:: + You may find that the valgrind results are easier to interpret if you put + ``-q`` in ``~/.valgrindrc``, since that reduces the amount of output. -- 2.7.4 _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
