> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Wenzhuo Lu
> Sent: Monday, June 06, 2016 6:48 AM
> To: dev at dpdk.org
> Cc: Lu, Wenzhuo
> Subject: [dpdk-dev] [PATCH] examples: add a new example for link reset
>
> Add a new example to show when the PF is down and up,
> VF port can be reset and recover.
Do we really need a totally new example for it?
Can't we put it in one of already existing ones?
Let say we have l3fwd-vf... wouldn't that suit your needs?
Konstantin
>
> Signed-off-by: Wenzhuo Lu <wenzhuo.lu at intel.com>
> ---
> MAINTAINERS | 4 +
> doc/guides/sample_app_ug/link_reset.rst | 177 ++++++++
> examples/link_reset/Makefile | 50 +++
> examples/link_reset/main.c | 769
> ++++++++++++++++++++++++++++++++
> 4 files changed, 1000 insertions(+)
> create mode 100644 doc/guides/sample_app_ug/link_reset.rst
> create mode 100644 examples/link_reset/Makefile
> create mode 100644 examples/link_reset/main.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 3e8558f..76879c3 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -650,3 +650,7 @@ F: examples/tep_termination/
> F: examples/vmdq/
> F: examples/vmdq_dcb/
> F: doc/guides/sample_app_ug/vmdq_dcb_forwarding.rst
> +
> +M: Wenzhuo Lu <wenzhuo.lu at intel.com>
> +F: examples/link_reset/
> +F: doc/guides/sample_app_ug/link_reset.rst
> diff --git a/doc/guides/sample_app_ug/link_reset.rst
> b/doc/guides/sample_app_ug/link_reset.rst
> new file mode 100644
> index 0000000..fecae6d
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/link_reset.rst
> @@ -0,0 +1,177 @@
> +.. BSD LICENSE
> + Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
> + All rights reserved.
> +
> + Redistribution and use in source and binary forms, with or without
> + modification, are permitted provided that the following conditions
> + are met:
> +
> + * Redistributions of source code must retain the above copyright
> + notice, this list of conditions and the following disclaimer.
> + * Redistributions in binary form must reproduce the above copyright
> + notice, this list of conditions and the following disclaimer in
> + the documentation and/or other materials provided with the
> + distribution.
> + * Neither the name of Intel Corporation nor the names of its
> + contributors may be used to endorse or promote products derived
> + from this software without specific prior written permission.
> +
> + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +Link Reset Sample Application (in Virtualized Environments)
> +===========================================================
> +
> +The Link Reset sample application is a simple example of VF traffic recovery
> +using the Data Plane Development Kit (DPDK) which also takes advantage of
> Single
> +Root I/O Virtualization (SR-IOV) features in a virtualized environment.
> +
> +Overview
> +--------
> +
> +The Link Reset sample application, which should operate in virtualized
> +environments, performs L2 forwarding for each packet that is received on an
> +RX_PORT.
> +This example is extended from the L2 forwarding example. Please reference the
> +example of L2 forwarding in virtualized environments for more details and
> +explanation about the behavior of forwarding and how to setup the test.
> +The purpose of this example is to show when the PF port is down and up, the
> VF
> +port can recover and the traffic can recover too.
> +
> +Virtual Function Setup Instructions
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +This application can use the virtual function available in the system and
> +therefore can be used in a virtual machine without passing through
> +the whole Network Device into a guest machine in a virtualized scenario.
> +The virtual functions can be enabled in the host machine or the hypervisor
> +with the respective physical function driver.
> +
> +For example, in a Linux* host machine, it is possible to enable a virtual
> +function using the following command:
> +
> +.. code-block:: console
> +
> + modprobe ixgbe max_vfs=2,2
> +
> +This command enables two Virtual Functions on each of Physical Function of
> the
> +NIC, with two physical ports in the PCI configuration space.
> +It is important to note that enabled Virtual Function 0 and 2 would belong to
> +Physical Function 0 and Virtual Function 1 and 3 would belong to Physical
> +Function 1, in this case enabling a total of four Virtual Functions.
> +
> +Compiling the Application
> +-------------------------
> +
> +#. Go to the example directory:
> +
> + .. code-block:: console
> +
> + export RTE_SDK=/path/to/rte_sdk
> + cd ${RTE_SDK}/examples/link_reset
> +
> +#. Set the target (a default target is used if not specified). For example:
> +
> + .. code-block:: console
> +
> + export RTE_TARGET=x86_64-native-linuxapp-gcc
> +
> + *See the DPDK Getting Started Guide* for possible RTE_TARGET values.
> +
> +#. Build the application:
> +
> + .. code-block:: console
> +
> + make
> +
> +Running the Application
> +-----------------------
> +
> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> + ./build/link_reset [EAL options] -- -p PORTMASK [-q NQ]
> +
> +where,
> +
> +* p PORTMASK: A hexadecimal bitmask of the ports to configure
> +
> +* q NQ: A number of queues (=ports) per lcore (default is 1)
> +
> +To run the application in linuxapp environment with 4 lcores, 16 ports and 8
> RX
> +queues per lcore, issue the command:
> +
> +.. code-block:: console
> +
> + $ ./build/link_reset -c f -n 4 -- -q 8 -p ffff
> +
> +Refer to the *DPDK Getting Started Guide* for general information on running
> applications
> +and the Environment Abstraction Layer (EAL) options.
> +
> +Explanation
> +-----------
> +
> +Handle VF link reset event
> +~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In main function, when initialising each port, register a callback for reset
> +event.
> +
> +.. code-block:: c
> +
> + /* Initialise each port */
> + for (portid = 0; portid < nb_ports; portid++) {
> + ......
> + /* register reset interrupt callback */
> + rte_eth_dev_callback_register(portid,
> + RTE_ETH_EVENT_INTR_RESET, reset_event_callback,
> NULL);
> + ......
> + }
> +
> +The callback function *reset_event_callback* will be executed in the
> +interruption thread. But we want the event to be handled in the management
> +thread. So in the callback function only a reset flag *stop_forwarding* is
> set.
> +
> +.. code-block:: c
> +
> + static void
> + reset_event_callback(uint8_t port_id, enum rte_eth_event_type type,
> void *param)
> + {
> + RTE_SET_USED(param);
> +
> + printf("\n\nIn registered callback...\n");
> + printf("Event type: %s on port %d\n",
> + type == RTE_ETH_EVENT_INTR_RESET ? "RESET interrupt"
> :
> + "unknown event", port_id);
> + reset_port = port_id;
> + rte_compiler_barrier(); /* prevent compiler reordering */
> + stop_forwarding = 1;
> + }
> +
> +The management thread keeps checking the reset flag *stop_forwarding* to see
> +if VF port reset is needed. If so, it should call the API *rte_eth_dev_reset*
> +to reset VF port. After that, the traffic will recover.
> +
> +.. code-block:: c
> +
> + while (1) {
> + rte_delay_ms(1000);
> + printf("..");
> + if (stop_forwarding == 1) {
> + printf("\nreset\n");
> + rte_eth_dev_reset(reset_port);
> + stop_forwarding = 0;
> + }
> + if (force_quit)
> + break;
> + }
> diff --git a/examples/link_reset/Makefile b/examples/link_reset/Makefile
> new file mode 100644
> index 0000000..6a63256
> --- /dev/null
> +++ b/examples/link_reset/Makefile
> @@ -0,0 +1,50 @@
> +# BSD LICENSE
> +#
> +# Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
> +# All rights reserved.
> +#
> +# Redistribution and use in source and binary forms, with or without
> +# modification, are permitted provided that the following conditions
> +# are met:
> +#
> +# * Redistributions of source code must retain the above copyright
> +# notice, this list of conditions and the following disclaimer.
> +# * Redistributions in binary form must reproduce the above copyright
> +# notice, this list of conditions and the following disclaimer in
> +# the documentation and/or other materials provided with the
> +# distribution.
> +# * Neither the name of Intel Corporation nor the names of its
> +# contributors may be used to endorse or promote products derived
> +# from this software without specific prior written permission.
> +#
> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +ifeq ($(RTE_SDK),)
> +$(error "Please define RTE_SDK environment variable")
> +endif
> +
> +# Default target, can be overridden by command line or environment
> +RTE_TARGET ?= x86_64-native-linuxapp-gcc
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +# binary name
> +APP = link_reset
> +
> +# all source are stored in SRCS-y
> +SRCS-y := main.c
> +
> +CFLAGS += -O3
> +CFLAGS += $(WERROR_FLAGS)
> +
> +include $(RTE_SDK)/mk/rte.extapp.mk
> diff --git a/examples/link_reset/main.c b/examples/link_reset/main.c
> new file mode 100644
> index 0000000..f151a83
> --- /dev/null
> +++ b/examples/link_reset/main.c
> @@ -0,0 +1,769 @@
> +/*-
> + * BSD LICENSE
> + *
> + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
> + * All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in
> + * the documentation and/or other materials provided with the
> + * distribution.
> + * * Neither the name of Intel Corporation nor the names of its
> + * contributors may be used to endorse or promote products derived
> + * from this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <stdint.h>
> +#include <inttypes.h>
> +#include <sys/types.h>
> +#include <sys/queue.h>
> +#include <netinet/in.h>
> +#include <setjmp.h>
> +#include <stdarg.h>
> +#include <ctype.h>
> +#include <errno.h>
> +#include <getopt.h>
> +#include <signal.h>
> +#include <stdbool.h>
> +
> +#include <rte_common.h>
> +#include <rte_log.h>
> +#include <rte_malloc.h>
> +#include <rte_memory.h>
> +#include <rte_memcpy.h>
> +#include <rte_memzone.h>
> +#include <rte_eal.h>
> +#include <rte_per_lcore.h>
> +#include <rte_launch.h>
> +#include <rte_atomic.h>
> +#include <rte_cycles.h>
> +#include <rte_prefetch.h>
> +#include <rte_lcore.h>
> +#include <rte_per_lcore.h>
> +#include <rte_branch_prediction.h>
> +#include <rte_interrupts.h>
> +#include <rte_pci.h>
> +#include <rte_random.h>
> +#include <rte_debug.h>
> +#include <rte_ether.h>
> +#include <rte_ethdev.h>
> +#include <rte_ring.h>
> +#include <rte_mempool.h>
> +#include <rte_mbuf.h>
> +
> +static volatile bool force_quit;
> +
> +#define RTE_LOGTYPE_L2FWD RTE_LOGTYPE_USER1
> +
> +#define NB_MBUF 8192
> +
> +#define MAX_PKT_BURST 32
> +#define BURST_TX_DRAIN_US 100 /* TX drain every ~100us */
> +
> +/*
> + * Configurable number of RX/TX ring descriptors
> + */
> +#define RTE_TEST_RX_DESC_DEFAULT 128
> +#define RTE_TEST_TX_DESC_DEFAULT 512
> +static uint16_t nb_rxd = RTE_TEST_RX_DESC_DEFAULT;
> +static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
> +
> +/* ethernet addresses of ports */
> +static struct ether_addr l2fwd_ports_eth_addr[RTE_MAX_ETHPORTS];
> +
> +/* mask of enabled ports */
> +static uint32_t l2fwd_enabled_port_mask;
> +
> +/* list of enabled ports */
> +static uint32_t l2fwd_dst_ports[RTE_MAX_ETHPORTS];
> +
> +static unsigned int l2fwd_rx_queue_per_lcore = 1;
> +
> +#define MAX_RX_QUEUE_PER_LCORE 16
> +#define MAX_TX_QUEUE_PER_PORT 16
> +struct lcore_queue_conf {
> + unsigned n_rx_port;
> + unsigned rx_port_list[MAX_RX_QUEUE_PER_LCORE];
> +} __rte_cache_aligned;
> +struct lcore_queue_conf lcore_queue_conf[RTE_MAX_LCORE];
> +
> +static struct rte_eth_dev_tx_buffer *tx_buffer[RTE_MAX_ETHPORTS];
> +
> +static const struct rte_eth_conf port_conf = {
> + .rxmode = {
> + .split_hdr_size = 0,
> + .header_split = 0, /**< Header Split disabled */
> + .hw_ip_checksum = 0, /**< IP checksum offload disabled */
> + .hw_vlan_filter = 0, /**< VLAN filtering disabled */
> + .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
> + .hw_strip_crc = 0, /**< CRC stripped by hardware */
> + .lock_mode = 1, /**< Use lock mode in RX */
> + },
> + .txmode = {
> + .mq_mode = ETH_MQ_TX_NONE,
> + .lock_mode = 1, /**< Use lock mode in TX */
> + },
> +};
> +
> +struct rte_mempool *l2fwd_pktmbuf_pool;
> +
> +/* Per-port statistics struct */
> +struct l2fwd_port_statistics {
> + uint64_t tx;
> + uint64_t rx;
> + uint64_t dropped;
> +} __rte_cache_aligned;
> +struct l2fwd_port_statistics port_statistics[RTE_MAX_ETHPORTS];
> +
> +/* A tsc-based timer responsible for triggering statistics printout */
> +#define TIMER_MILLISECOND 2000000ULL /* around 1ms at 2 Ghz */
> +#define MAX_TIMER_PERIOD 86400 /* 1 day max */
> +/* default period is 10 seconds */
> +static int64_t timer_period = 10 * TIMER_MILLISECOND * 1000;
> +
> +static uint32_t stop_forwarding;
> +static uint8_t reset_port;
> +
> +/* Print out statistics on packets dropped */
> +static void
> +print_stats(void)
> +{
> + uint64_t total_packets_dropped, total_packets_tx, total_packets_rx;
> + unsigned portid;
> +
> + total_packets_dropped = 0;
> + total_packets_tx = 0;
> + total_packets_rx = 0;
> +
> + const char clr[] = { 27, '[', '2', 'J', '\0' };
> + const char topLeft[] = { 27, '[', '1', ';', '1', 'H', '\0' };
> +
> + /* Clear screen and move to top left */
> + printf("%s%s", clr, topLeft);
> +
> + printf("\nPort statistics ====================================");
> +
> + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++) {
> + /* skip disabled ports */
> + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
> + continue;
> + printf("\nStatistics for port %u ------------------------------"
> + "\nPackets sent: %24"PRIu64
> + "\nPackets received: %20"PRIu64
> + "\nPackets dropped: %21"PRIu64,
> + portid,
> + port_statistics[portid].tx,
> + port_statistics[portid].rx,
> + port_statistics[portid].dropped);
> +
> + total_packets_dropped += port_statistics[portid].dropped;
> + total_packets_tx += port_statistics[portid].tx;
> + total_packets_rx += port_statistics[portid].rx;
> + }
> + printf("\nAggregate statistics ==============================="
> + "\nTotal packets sent: %18"PRIu64
> + "\nTotal packets received: %14"PRIu64
> + "\nTotal packets dropped: %15"PRIu64,
> + total_packets_tx,
> + total_packets_rx,
> + total_packets_dropped);
> + printf("\n====================================================\n");
> +}
> +
> +static void
> +l2fwd_simple_forward(struct rte_mbuf *m, unsigned portid)
> +{
> + struct ether_hdr *eth;
> + void *tmp;
> + unsigned dst_port;
> + int sent;
> + struct rte_eth_dev_tx_buffer *buffer;
> +
> + dst_port = l2fwd_dst_ports[portid];
> + eth = rte_pktmbuf_mtod(m, struct ether_hdr *);
> +
> + /* 02:00:00:00:00:xx */
> + tmp = ð->d_addr.addr_bytes[0];
> + *((uint64_t *)tmp) = 0x000000000002 + ((uint64_t)dst_port << 40);
> +
> + /* src addr */
> + ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], ð->s_addr);
> +
> + buffer = tx_buffer[dst_port];
> + sent = rte_eth_tx_buffer(dst_port, 0, buffer, m);
> + if (sent)
> + port_statistics[dst_port].tx += sent;
> +}
> +
> +/* main processing loop */
> +static void
> +l2fwd_main_loop(void)
> +{
> + struct rte_mbuf *pkts_burst[MAX_PKT_BURST];
> + struct rte_mbuf *m;
> + int sent;
> + unsigned lcore_id;
> + uint64_t prev_tsc, diff_tsc, cur_tsc, timer_tsc;
> + unsigned i, j, portid, nb_rx;
> + struct lcore_queue_conf *qconf;
> + const uint64_t drain_tsc = (rte_get_tsc_hz() + US_PER_S - 1) /
> + US_PER_S * BURST_TX_DRAIN_US;
> + struct rte_eth_dev_tx_buffer *buffer;
> +
> + prev_tsc = 0;
> + timer_tsc = 0;
> +
> + lcore_id = rte_lcore_id();
> + qconf = &lcore_queue_conf[lcore_id];
> +
> + if (qconf->n_rx_port == 0) {
> + RTE_LOG(INFO, L2FWD, "lcore %u has nothing to do\n", lcore_id);
> + return;
> + }
> +
> + RTE_LOG(INFO, L2FWD, "entering main loop on lcore %u\n", lcore_id);
> +
> + for (i = 0; i < qconf->n_rx_port; i++) {
> +
> + portid = qconf->rx_port_list[i];
> + RTE_LOG(INFO, L2FWD, " -- lcoreid=%u portid=%u\n", lcore_id,
> + portid);
> +
> + }
> +
> + while (!force_quit) {
> +
> + cur_tsc = rte_rdtsc();
> +
> + /*
> + * TX burst queue drain
> + */
> + diff_tsc = cur_tsc - prev_tsc;
> + if (unlikely(diff_tsc > drain_tsc)) {
> +
> + for (i = 0; i < qconf->n_rx_port; i++) {
> + portid =
> + l2fwd_dst_ports[qconf->rx_port_list[i]];
> + buffer = tx_buffer[portid];
> +
> + sent = rte_eth_tx_buffer_flush(portid,
> + 0,
> + buffer);
> + if (sent)
> + port_statistics[portid].tx += sent;
> + }
> +
> + /* if timer is enabled */
> + if (timer_period > 0) {
> +
> + /* advance the timer */
> + timer_tsc += diff_tsc;
> +
> + /* if timer has reached its timeout */
> + if (unlikely(timer_tsc >=
> + (uint64_t) timer_period)) {
> + /* do this only on master core */
> + if (lcore_id ==
> + rte_get_master_lcore()) {
> + print_stats();
> + /* reset the timer */
> + timer_tsc = 0;
> + }
> + }
> + }
> +
> + prev_tsc = cur_tsc;
> + }
> +
> + /*
> + * Read packet from RX queues
> + */
> + for (i = 0; i < qconf->n_rx_port; i++) {
> +
> + portid = qconf->rx_port_list[i];
> + nb_rx = rte_eth_rx_burst((uint8_t) portid, 0,
> + pkts_burst, MAX_PKT_BURST);
> +
> + port_statistics[portid].rx += nb_rx;
> +
> + for (j = 0; j < nb_rx; j++) {
> + m = pkts_burst[j];
> + rte_prefetch0(rte_pktmbuf_mtod(m, void *));
> + l2fwd_simple_forward(m, portid);
> + }
> + }
> + }
> +}
> +
> +static int
> +l2fwd_launch_one_lcore(__attribute__((unused)) void *dummy)
> +{
> + l2fwd_main_loop();
> + return 0;
> +}
> +
> +/* display usage */
> +static void
> +l2fwd_usage(const char *prgname)
> +{
> + printf("%s [EAL options] -- -p PORTMASK [-q NQ]\n"
> + " -p PORTMASK: hexadecimal bitmask of ports to configure\n"
> + " -q NQ: number of queue (=ports) per lcore (default is 1)\n"
> + " -T PERIOD: statistics will be refreshed each PERIOD
> seconds (0 to disable, 10 default, 86400 maximum)\n",
> + prgname);
> +}
> +
> +static int
> +l2fwd_parse_portmask(const char *portmask)
> +{
> + char *end = NULL;
> + unsigned long pm;
> +
> + /* parse hexadecimal string */
> + pm = strtoul(portmask, &end, 16);
> + if ((portmask[0] == '\0') || (end == NULL) || (*end != '\0'))
> + return -1;
> +
> + if (pm == 0)
> + return -1;
> +
> + return pm;
> +}
> +
> +static unsigned int
> +l2fwd_parse_nqueue(const char *q_arg)
> +{
> + char *end = NULL;
> + unsigned long n;
> +
> + /* parse hexadecimal string */
> + n = strtoul(q_arg, &end, 10);
> + if ((q_arg[0] == '\0') || (end == NULL) || (*end != '\0'))
> + return 0;
> + if (n == 0)
> + return 0;
> + if (n >= MAX_RX_QUEUE_PER_LCORE)
> + return 0;
> +
> + return n;
> +}
> +
> +static int
> +l2fwd_parse_timer_period(const char *q_arg)
> +{
> + char *end = NULL;
> + int n;
> +
> + /* parse number string */
> + n = strtol(q_arg, &end, 10);
> + if ((q_arg[0] == '\0') || (end == NULL) || (*end != '\0'))
> + return -1;
> + if (n >= MAX_TIMER_PERIOD)
> + return -1;
> +
> + return n;
> +}
> +
> +/* Parse the argument given in the command line of the application */
> +static int
> +l2fwd_parse_args(int argc, char **argv)
> +{
> + int opt, ret;
> + char **argvopt;
> + int option_index;
> + char *prgname = argv[0];
> + static struct option lgopts[] = {
> + {NULL, 0, 0, 0}
> + };
> +
> + argvopt = argv;
> +
> + while ((opt = getopt_long(argc, argvopt, "p:q:T:",
> + lgopts, &option_index)) != EOF) {
> +
> + switch (opt) {
> + /* portmask */
> + case 'p':
> + l2fwd_enabled_port_mask = l2fwd_parse_portmask(optarg);
> + if (l2fwd_enabled_port_mask == 0) {
> + printf("invalid portmask\n");
> + l2fwd_usage(prgname);
> + return -1;
> + }
> + break;
> +
> + /* nqueue */
> + case 'q':
> + l2fwd_rx_queue_per_lcore = l2fwd_parse_nqueue(optarg);
> + if (l2fwd_rx_queue_per_lcore == 0) {
> + printf("invalid queue number\n");
> + l2fwd_usage(prgname);
> + return -1;
> + }
> + break;
> +
> + /* timer period */
> + case 'T':
> + timer_period = l2fwd_parse_timer_period(optarg) *
> + 1000 * TIMER_MILLISECOND;
> + if (timer_period < 0) {
> + printf("invalid timer period\n");
> + l2fwd_usage(prgname);
> + return -1;
> + }
> + break;
> +
> + /* long options */
> + case 0:
> + l2fwd_usage(prgname);
> + return -1;
> +
> + default:
> + l2fwd_usage(prgname);
> + return -1;
> + }
> + }
> +
> + if (optind >= 0)
> + argv[optind-1] = prgname;
> +
> + ret = optind-1;
> + optind = 0; /* reset getopt lib */
> + return ret;
> +}
> +
> +/* Check the link status of all ports in up to 9s, and print them finally */
> +static void
> +check_all_ports_link_status(uint8_t port_num, uint32_t port_mask)
> +{
> +#define CHECK_INTERVAL 100 /* 100ms */
> +#define MAX_CHECK_TIME 90 /* 9s (90 * 100ms) in total */
> + uint8_t portid, count, all_ports_up, print_flag = 0;
> + struct rte_eth_link link;
> +
> + printf("\nChecking link status");
> + fflush(stdout);
> + for (count = 0; count <= MAX_CHECK_TIME; count++) {
> + if (force_quit)
> + return;
> + all_ports_up = 1;
> + for (portid = 0; portid < port_num; portid++) {
> + if (force_quit)
> + return;
> + if ((port_mask & (1 << portid)) == 0)
> + continue;
> + memset(&link, 0, sizeof(link));
> + rte_eth_link_get_nowait(portid, &link);
> + /* print link status if flag set */
> + if (print_flag == 1) {
> + if (link.link_status)
> + printf("Port %d Link Up - speed %u "
> + "Mbps - %s\n", (uint8_t)portid,
> + (unsigned)link.link_speed,
> + (link.link_duplex == ETH_LINK_FULL_DUPLEX) ?
> + ("full-duplex") : ("half-duplex\n"));
> + else
> + printf("Port %d Link Down\n",
> + (uint8_t)portid);
> + continue;
> + }
> + /* clear all_ports_up flag if any link down */
> + if (link.link_status == ETH_LINK_DOWN) {
> + all_ports_up = 0;
> + break;
> + }
> + }
> + /* after finally printing all link status, get out */
> + if (print_flag == 1)
> + break;
> +
> + if (all_ports_up == 0) {
> + printf(".");
> + fflush(stdout);
> + rte_delay_ms(CHECK_INTERVAL);
> + }
> +
> + /* set the print_flag if all ports up or timeout */
> + if (all_ports_up == 1 || count == (MAX_CHECK_TIME - 1)) {
> + print_flag = 1;
> + printf("done\n");
> + }
> + }
> +}
> +
> +static void
> +signal_handler(int signum)
> +{
> + if (signum == SIGINT || signum == SIGTERM) {
> + printf("\n\nSignal %d received, preparing to exit...\n",
> + signum);
> + force_quit = true;
> + }
> +}
> +
> +static void
> +reset_event_callback(uint8_t port_id, enum rte_eth_event_type type, void
> *param)
> +{
> + RTE_SET_USED(param);
> +
> + printf("\n\nIn registered callback...\n");
> + printf("Event type: %s on port %d\n",
> + type == RTE_ETH_EVENT_INTR_RESET ? "RESET interrupt" :
> + "unknown event", port_id);
> + reset_port = port_id;
> + rte_compiler_barrier(); /* prevent compiler reordering */
> + stop_forwarding = 1;
> +}
> +
> +int
> +main(int argc, char **argv)
> +{
> + struct lcore_queue_conf *qconf;
> + struct rte_eth_dev_info dev_info;
> + int ret;
> + uint8_t nb_ports;
> + uint8_t nb_ports_available;
> + uint8_t portid, last_port;
> + unsigned lcore_id, rx_lcore_id;
> + unsigned nb_ports_in_mask = 0;
> +
> + /* init EAL */
> + ret = rte_eal_init(argc, argv);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n");
> + argc -= ret;
> + argv += ret;
> +
> + force_quit = false;
> + signal(SIGINT, signal_handler);
> + signal(SIGTERM, signal_handler);
> +
> + /* parse application arguments (after the EAL ones) */
> + ret = l2fwd_parse_args(argc, argv);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE, "Invalid L2FWD arguments\n");
> +
> + /* create the mbuf pool */
> + l2fwd_pktmbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NB_MBUF, 32,
> + 0, RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());
> + if (l2fwd_pktmbuf_pool == NULL)
> + rte_exit(EXIT_FAILURE, "Cannot init mbuf pool\n");
> +
> + nb_ports = rte_eth_dev_count();
> + if (nb_ports == 0)
> + rte_exit(EXIT_FAILURE, "No Ethernet ports - bye\n");
> +
> + if (nb_ports > RTE_MAX_ETHPORTS)
> + nb_ports = RTE_MAX_ETHPORTS;
> +
> + /* reset l2fwd_dst_ports */
> + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++)
> + l2fwd_dst_ports[portid] = 0;
> + last_port = 0;
> +
> + /*
> + * Each logical core is assigned a dedicated TX queue on each port.
> + */
> + for (portid = 0; portid < nb_ports; portid++) {
> + /* skip ports that are not enabled */
> + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
> + continue;
> +
> + if (nb_ports_in_mask % 2) {
> + l2fwd_dst_ports[portid] = last_port;
> + l2fwd_dst_ports[last_port] = portid;
> + } else
> + last_port = portid;
> +
> + nb_ports_in_mask++;
> +
> + rte_eth_dev_info_get(portid, &dev_info);
> + }
> + if (nb_ports_in_mask % 2) {
> + printf("Notice: odd number of ports in portmask.\n");
> + l2fwd_dst_ports[last_port] = last_port;
> + }
> +
> + rx_lcore_id = 1;
> + qconf = NULL;
> +
> + /* Initialize the port/queue configuration of each logical core */
> + for (portid = 0; portid < nb_ports; portid++) {
> + /* skip ports that are not enabled */
> + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
> + continue;
> +
> + /* get the lcore_id for this port */
> + while (rte_lcore_is_enabled(rx_lcore_id) == 0 ||
> + lcore_queue_conf[rx_lcore_id].n_rx_port ==
> + l2fwd_rx_queue_per_lcore) {
> + rx_lcore_id++;
> + if (rx_lcore_id >= RTE_MAX_LCORE)
> + rte_exit(EXIT_FAILURE, "Not enough cores\n");
> + }
> +
> + if (qconf != &lcore_queue_conf[rx_lcore_id])
> + /* Assigned a new logical core in the loop above. */
> + qconf = &lcore_queue_conf[rx_lcore_id];
> +
> + qconf->rx_port_list[qconf->n_rx_port] = portid;
> + qconf->n_rx_port++;
> + printf("Lcore %u: RX port %u\n",
> + rx_lcore_id,
> + (unsigned) portid);
> + }
> +
> + nb_ports_available = nb_ports;
> +
> + /* Initialise each port */
> + for (portid = 0; portid < nb_ports; portid++) {
> + /* skip ports that are not enabled */
> + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0) {
> + printf("Skipping disabled port %u\n",
> + (unsigned) portid);
> + nb_ports_available--;
> + continue;
> + }
> + /* init port */
> + printf("Initializing port %u... ", (unsigned) portid);
> + fflush(stdout);
> + ret = rte_eth_dev_configure(portid, 1, 1, &port_conf);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE,
> + "Cannot configure device: err=%d, port=%u\n",
> + ret, (unsigned) portid);
> +
> + /* register reset interrupt callback */
> + rte_eth_dev_callback_register(portid,
> + RTE_ETH_EVENT_INTR_RESET, reset_event_callback, NULL);
> +
> + rte_eth_macaddr_get(portid, &l2fwd_ports_eth_addr[portid]);
> +
> + /* init one RX queue */
> + fflush(stdout);
> + ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd,
> + rte_eth_dev_socket_id(portid),
> + NULL,
> + l2fwd_pktmbuf_pool);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE,
> + "rte_eth_rx_queue_setup:err=%d, port=%u\n",
> + ret, (unsigned) portid);
> +
> + /* init one TX queue on each port */
> + fflush(stdout);
> + ret = rte_eth_tx_queue_setup(portid, 0, nb_txd,
> + rte_eth_dev_socket_id(portid),
> + NULL);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE,
> + "rte_eth_tx_queue_setup:err=%d, port=%u\n",
> + ret, (unsigned) portid);
> +
> + /* Initialize TX buffers */
> + tx_buffer[portid] = rte_zmalloc_socket("tx_buffer",
> + RTE_ETH_TX_BUFFER_SIZE(MAX_PKT_BURST), 0,
> + rte_eth_dev_socket_id(portid));
> + if (tx_buffer[portid] == NULL)
> + rte_exit(EXIT_FAILURE,
> + "Cannot allocate buffer for tx on port %u\n",
> + (unsigned) portid);
> +
> + rte_eth_tx_buffer_init(tx_buffer[portid], MAX_PKT_BURST);
> +
> + ret = rte_eth_tx_buffer_set_err_callback(tx_buffer[portid],
> + rte_eth_tx_buffer_count_callback,
> + &port_statistics[portid].dropped);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE, "Cannot set error callback for "
> + "tx buffer on port %u\n", (unsigned) portid);
> +
> + /* Start device */
> + ret = rte_eth_dev_start(portid);
> + if (ret < 0)
> + rte_exit(EXIT_FAILURE,
> + "rte_eth_dev_start:err=%d, port=%u\n",
> + ret, (unsigned) portid);
> +
> + printf("done:\n");
> +
> + rte_eth_promiscuous_enable(portid);
> +
> + printf("Port %u, MAC address: "
> + "%02X:%02X:%02X:%02X:%02X:%02X\n\n",
> + (unsigned) portid,
> + l2fwd_ports_eth_addr[portid].addr_bytes[0],
> + l2fwd_ports_eth_addr[portid].addr_bytes[1],
> + l2fwd_ports_eth_addr[portid].addr_bytes[2],
> + l2fwd_ports_eth_addr[portid].addr_bytes[3],
> + l2fwd_ports_eth_addr[portid].addr_bytes[4],
> + l2fwd_ports_eth_addr[portid].addr_bytes[5]);
> +
> + /* initialize port stats */
> + memset(&port_statistics, 0, sizeof(port_statistics));
> + }
> +
> + if (!nb_ports_available) {
> + rte_exit(EXIT_FAILURE,
> + "All available ports are disabled. Please set
> portmask.\n");
> + }
> +
> + check_all_ports_link_status(nb_ports, l2fwd_enabled_port_mask);
> +
> + ret = 0;
> + /* launch per-lcore init on every lcore */
> + rte_eal_mp_remote_launch(l2fwd_launch_one_lcore, NULL, SKIP_MASTER);
> +
> + printf("\nwaiting..");
> + while (1) {
> + rte_delay_ms(1000);
> + printf("..");
> + if (stop_forwarding == 1) {
> + printf("\nreset\n");
> + rte_eth_dev_reset(reset_port);
> + stop_forwarding = 0;
> + }
> + if (force_quit)
> + break;
> + }
> +
> + RTE_LCORE_FOREACH_SLAVE(lcore_id) {
> + if (rte_eal_wait_lcore(lcore_id) < 0) {
> + ret = -1;
> + break;
> + }
> + }
> +
> + for (portid = 0; portid < nb_ports; portid++) {
> + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
> + continue;
> + printf("Closing port %d...", portid);
> + rte_eth_dev_stop(portid);
> + rte_eth_dev_close(portid);
> + printf(" Done\n");
> + }
> + printf("Bye...\n");
> +
> + return ret;
> +}
> --
> 1.9.3
