[ovs-dev] PATCH 1/1 : netdev-dpdk / add dpdk rings to netdev-dpdk
This patch enables the client dpdk rings within the netdev-dpdk. It adds
a new
dpdk device called dpdkr (other naming suggestions?). This allows for the
use
of shared memory to communicate with other dpdk applications, on the host
or
within a virtual machine. Instructions for use are in INSTALL.DPDK.
This has been tested on Intel multi-core platforms and with the client
application
within the host.
Signed-off-by: Gerald Rogers
Signed-off-by: Gerald Rogers
diff --git a/INSTALL.DPDK b/INSTALL.DPDK
index 3e0247a..ec1b814 100644
--- a/INSTALL.DPDK
+++ b/INSTALL.DPDK
@@ -74,6 +74,44 @@ Once first DPDK port is added vswitchd, it creates
Polling thread and
polls dpdk device in continuous loop. Therefore CPU utilization
for that thread is always 100%.
+DPDK Rings :
+
+Following the steps above to create a bridge, you can now add dpdk rings
+as a port to the vswitch. OVS will expect the DPDK ring device name to
+start with dpdkr and end with portid.
+
+ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr
+
+DPDK rings client test application
+
+Included in the test directory is a sample DPDK application for testing
+the rings. This is from the base dpdk directory and modified to work
+with the ring naming used within ovs.
+
+location tests/ovs_client
+
+To run the client in tests directory :
+
+ovsclient -c 1 -n 4 --proc-type=secondary -- --n "port id you gave dpdkr"
+
+It is essential to have --proc-type=secondary
+
+The application simply receives an mbuf on the receive queue of the
+ethernet ring and then places that same mbuf on the transmit ring of
+the ethernet ring. It is a trivial loopback application.
+
+In addition to executing the client in the host, you can execute it
within
+a guest VM. To do so you will need a patched qemu. You can download the
+patch and getting started guid at :
+
+https://01.org/packet-processing/downloads
+
+A general rule of thumb for better performance is that the client
+application shouldn't be assigned the same dpdk core mask "-c" as
+the vswitchd.
+
+
+
Restrictions:
-
@@ -86,6 +124,11 @@ Restrictions:
device queues configured.
- Work with 1500 MTU, needs few changes in DPDK lib to fix this issue.
- Currently DPDK port does not make use any offload functionality.
+ ivshmem
+ - The shared memory is currently restricted to the use of a 1GB
+ huge pages.
+ - All huge pages are shared amongst the host, clients, virtual
+ machines etc.
Bug Reporting:
--
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index fd991ab..a90913f 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -148,6 +148,23 @@ struct dpdk_tx_queue {
struct rte_mbuf *burst_pkts[MAX_TX_QUEUE_LEN];
};
+/* dpdk has no way to remove dpdk ring ethernet devices
+ so we have to keep them around once they've been created
+*/
+
+static struct list dpdk_ring_list OVS_GUARDED_BY(dpdk_mutex)
+= LIST_INITIALIZER(&dpdk_ring_list);
+
+struct dpdk_ring {
+/* For the client rings */
+struct rte_ring * cring_tx;
+struct rte_ring * cring_rx;
+int port_id; /* dpdkr port id */
+int eth_port_id; /* ethernet device port id */
+struct list list_node OVS_GUARDED_BY(mutex);
+
+};
+
struct netdev_dpdk {
struct netdev up;
int port_id;
@@ -167,9 +184,14 @@ struct netdev_dpdk {
uint8_t hwaddr[ETH_ADDR_LEN];
enum netdev_flags flags;
+
struct rte_eth_link link;
int link_reset_cnt;
+/* If a shared memory ring */
+
+struct dpdk_ring *ivshmem;
+
/* In dpdk_list. */
struct list list_node OVS_GUARDED_BY(dpdk_mutex);
};
@@ -571,6 +593,7 @@ dpdk_queue_flush(struct netdev_dpdk *dev, int qid)
if (txq->count == 0) {
return;
}
+
rte_spinlock_lock(&txq->tx_lock);
nb_tx = rte_eth_tx_burst(dev->port_id, qid, txq->burst_pkts,
txq->count);
if (nb_tx != txq->count) {
@@ -593,6 +616,7 @@ netdev_dpdk_rxq_recv(struct netdev_rxq *rxq_, struct
ofpbuf **packets, int *c)
dpdk_queue_flush(dev, rxq_->queue_id);
+
nb_rx = rte_eth_rx_burst(rx->port_id, rxq_->queue_id,
(struct rte_mbuf **) packets,
MAX_RX_QUEUE_LEN);
if (!nb_rx) {
@@ -1170,6 +1194,200 @@ static struct netdev_class netdev_dpdk_class = {
NULL, /* rxq_drain */
};
+/* Client Rings */
+
+static int
+dpdk_classr_init(void)
+{
+ VLOG_INFO("Initialized dpdk client handlers:\n");
+ return 0;
+}
+
+static int
+netdev_dpdkr_construct(struct netdev *netdev_)
+{
+struct netdev_dpdk *netdev = netdev_dpdk_cast(netdev_);
+struct dpdk_ring *ivshmem;
+unsigned int port_no;
+char *cport;
+int err=0;
+int found=0;
+char name[10];
+
+
+if (rte_eal_init_ret) {
+return rte_eal_init_ret;
+}
+
+ovs_mutex_lock(&dpdk_mutex);
+cport = netdev_->name + 5; /* Names always start with "dpdkr" */
+
+if (strncmp(netdev_->name, "dpdkr", 5)) {
+VLOG_ERR("Not a valid dpdkr name %s:\n"
Re: [ovs-dev] PATCH 1/1 : netdev-dpdk / add dpdk rings to netdev-dpdk
Pravin,
On the 1GB page question. The patch simply allocates the rings and
creates a port. You can allocate more than a single 1 GB page, the
restriction is that it would need 1GB pages. The entire Huge Page
allocated by DPDK is passed to each Virtual Machine, and all VM¹s share
this memory. There exists an API in DPDK to store the memory utilized by
the ring so that you can use pages smaller than 1GB. Using this API, you
can allocate to a single VM only the memory associated with the ring and
the rings mbuf. However, when using this API any traffic too and from the
VM will need to incur a copy. The reason is that each VM would be
isolated from the others by which memory in the Huge Page Table mapping
they are given. If you use just 1GB pages, then you can communicate
between VM¹s without incurring a copy penalty. I chose not to implement
it using the API because I wanted the ivshmem to obtain the best
performance (ie. No copies). A patch we are working to release in a
couple of weeks will support vHost within DPDK, thus eliminating the
isolation (ie. Shared memory) issues with ivshmem. This vHost patch will
incur a performance penalty since copies are involved. The vHost
implementation also won¹t have the 1GB page limitation.
As for the formatting issues, is there a ³c² pretty print formatter I can
run on ovs_client.c to make it conform to coding standards?
Thanks,
Gerald
On 5/6/14, 2:49 PM, "Pravin Shelar" wrote:
>Thanks for patch, It looks good, I have comments mostly related to coding
>style.
>
>Subject line should follow following format
>[PATCH /] :
>
>On Thu, May 1, 2014 at 3:58 AM, Gerald Rogers
>wrote:
>> This patch enables the client dpdk rings within the netdev-dpdk. It
>>adds
>> a new dpdk device called dpdkr (other naming suggestions?). This allows
>> for the use of shared memory to communicate with other dpdk
>>applications,
>> on the host or within a virtual machine. Instructions for use are in
>> INSTALL.DPDK.
>>
>> This has been tested on Intel multi-core platforms and with the client
>> application within the host.
>>
>>
>> Signed-off-by: Gerald Rogers
>> Signed-off-by: Gerald Rogers
>>
>Duplicate signed-off line.
>
>> diff --git a/INSTALL.DPDK b/INSTALL.DPDK
>> index 3e0247a..ec1b814 100644
>> --- a/INSTALL.DPDK
>> +++ b/INSTALL.DPDK
>> @@ -74,6 +74,44 @@ Once first DPDK port is added vswitchd, it creates
>>Polling thread and
>> polls dpdk device in continuous loop. Therefore CPU utilization
>> for that thread is always 100%.
>>
>> +DPDK Rings :
>> +
>> +Following the steps above to create a bridge, you can now add dpdk
>>rings
>> +as a port to the vswitch. OVS will expect the DPDK ring device name to
>> +start with dpdkr and end with portid.
>> +
>> +ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr
>> +
>> +DPDK rings client test application
>> +
>> +Included in the test directory is a sample DPDK application for testing
>> +the rings. This is from the base dpdk directory and modified to work
>> +with the ring naming used within ovs.
>> +
>> +location tests/ovs_client
>> +
>> +To run the client :
>> +
>> +ovsclient -c 1 -n 4 --proc-type=secondary -- --n "port id you gave
>>dpdkr"
>> +
>> +It is essential to have --proc-type=secondary
>> +
>> +The application simply receives an mbuf on the receive queue of the
>> +ethernet ring and then places that same mbuf on the transmit ring of
>> +the ethernet ring. It is a trivial loopback application.
>> +
>> +In addition to executing the client in the host, you can execute it
>>within
>> +a guest VM. To do so you will need a patched qemu. You can download
>>the
>> +patch and getting started guid at :
>> +
>> +https://01.org/packet-processing/downloads
>> +
>> +A general rule of thumb for better performance is that the client
>> +application shouldn't be assigned the same dpdk core mask "-c" as
>> +the vswitchd.
>> +
>> +
>> +
>> Restrictions:
>> -
>>
>> @@ -86,6 +124,11 @@ Restrictions:
>> device queues configured.
>>- Work with 1500 MTU, needs few changes in DPDK lib to fix this
>>issue.
>>- Currently DPDK port does not make use any offload functionality.
>> + ivshmem
>> + - The shared memory is currently restricted to the use of a 1GB
>> + huge pages.
>I am not sure why do we have this limit.
>
>> + - All huge pages are shared amongst the host, clients, virtual
>> + machines etc.
>>
>> Bug Reporting:
>> --
>> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
>> index fd991ab..a90913f 100644
>> --- a/lib/netdev-dpdk.c
>> +++ b/lib/netdev-dpdk.c
>> @@ -148,6 +148,23 @@ struct dpdk_tx_queue {
>> struct rte_mbuf *burst_pkts[MAX_TX_QUEUE_LEN];
>> };
>>
>> +/* dpdk has no way to remove dpdk ring ethernet devices
>> + so we have to keep them around once they've been created
>> +*/
>> +
>> +static struct list dpdk_ring_list OVS_GUARDED_BY(dpdk_mutex)
>> += LIST_INITIALIZER(&dpdk_ring_list);
>> +
>> +struct dpdk_ring {
>> +/* For the clie
Re: [ovs-dev] PATCH 1/1 : netdev-dpdk / add dpdk rings to netdev-dpdk
Ben, I used the following indent options to format the file : indent -bad -bap -bbb -br -blf -brs -cdw -ce -fca -i4 -l79 -lc79 -nbfda -nut -saf -sai -saw -sbi4 -sc -sob -st -cdb -pi4 Gerald On 5/7/14, 11:25 AM, "Ben Pfaff" wrote: >On Wed, May 07, 2014 at 11:21:37AM -0700, Pravin Shelar wrote: >> On Wed, May 7, 2014 at 8:49 AM, Rogers, Gerald >> wrote: >> > As for the formatting issues, is there a ??c?? pretty print formatter >>I can >> > run on ovs_client.c to make it conform to coding standards? >> >> I do not know such tool, I think you have to do it manually. > >I guess that the "indent" program can come pretty close. It would >take a few minutes adjusting its options. Gerald, if you happen to >figure out some good options for that, then please post them so that >we can add them as advice to the CodingStyle file. ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
Re: [ovs-dev] netdev-dpdk / add dpdk rings to netdev-dpdk
Pravin,
Which version of DPDK are you using?
In mine the rte_config.h is located in the DPDK directory
include/rte_config.h
Gerald
On 6/24/14, 12:11 PM, "Pravin Shelar" wrote:
>Hi Gerald,
>
>Thanks for updating patch, here are few comments:
>
>I got compilation erorr without --with-dpdk config option:
>
>tests/ovs_client/ovs_client.c:38:24: fatal error: rte_config.h: No
>such file or directory
>#include
>
>
>
>On Mon, Jun 23, 2014 at 4:55 AM, wrote:
>> Shared memory ring patch
>>
>> This patch enables the client dpdk rings within the netdev-dpdk. It
>>adds
>> a new dpdk device called dpdkr (other naming suggestions?). This allows
>> for the use of shared memory to communicate with other dpdk
>>applications,
>> on the host or within a virtual machine. Instructions for use are in
>> INSTALL.DPDK.
>>
>> This has been tested on Intel multi-core platforms and with the client
>> application within the host.
>>
>> Signed-off-by: Gerald Rogers
>>
>> diff --git a/INSTALL.DPDK b/INSTALL.DPDK
>> index 2a6d7ef..6a8b565 100644
>> --- a/INSTALL.DPDK
>> +++ b/INSTALL.DPDK
>> @@ -190,6 +190,44 @@ The core 23 is left idle, which allows core 7 to
>>run at full rate.
>>
>> Future changes may change the need for cpu core affinitization.
>>
>> +DPDK Rings :
>> +
>> +Following the steps above to create a bridge, you can now add dpdk
>>rings
>> +as a port to the vswitch. OVS will expect the DPDK ring device name to
>> +start with dpdkr and end with portid.
>> +
>> +ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr
>> +
>> +DPDK rings client test application
>> +
>> +Included in the test directory is a sample DPDK application for testing
>> +the rings. This is from the base dpdk directory and modified to work
>> +with the ring naming used within ovs.
>> +
>> +location tests/ovs_client
>> +
>> +To run the client :
>> +
>> +ovsclient -c 1 -n 4 --proc-type=secondary -- --n "port id you gave
>>dpdkr"
>> +
>> +It is essential to have --proc-type=secondary
>> +
>> +The application simply receives an mbuf on the receive queue of the
>> +ethernet ring and then places that same mbuf on the transmit ring of
>> +the ethernet ring. It is a trivial loopback application.
>> +
>> +In addition to executing the client in the host, you can execute it
>>within
>> +a guest VM. To do so you will need a patched qemu. You can download
>>the
>> +patch and getting started guid at :
>> +
>> +https://01.org/packet-processing/downloads
>> +
>> +A general rule of thumb for better performance is that the client
>> +application shouldn't be assigned the same dpdk core mask "-c" as
>> +the vswitchd.
>> +
>> +
>> +
>> Restrictions:
>> -
>>
>> @@ -202,6 +240,11 @@ Restrictions:
>> device queues configured.
>>- Work with 1500 MTU, needs few changes in DPDK lib to fix this
>>issue.
>>- Currently DPDK port does not make use any offload functionality.
>> + ivshmem
>> + - The shared memory is currently restricted to the use of a 1GB
>> + huge pages.
>> + - All huge pages are shared amongst the host, clients, virtual
>> + machines etc.
>>
>> Bug Reporting:
>> --
>> diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
>> index 6c281fe..aecf379 100644
>> --- a/lib/dpif-netdev.c
>> +++ b/lib/dpif-netdev.c
>> @@ -68,7 +68,7 @@ VLOG_DEFINE_THIS_MODULE(dpif_netdev);
>> /* By default, choose a priority in the middle. */
>> #define NETDEV_RULE_PRIORITY 0x8000
>>
>> -#define NR_THREADS 1
>> +#define NR_THREADS 4
>Alex is working on threading improvements. So there is no need to
>increase it in this patch.
>
>> /* Use per thread recirc_depth to prevent recirculation loop. */
>> #define MAX_RECIRC_DEPTH 5
>> DEFINE_STATIC_PER_THREAD_DATA(uint32_t, recirc_depth, 0)
>> diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
>> index fbdb6b3..3208ad6 100644
>> --- a/lib/netdev-dpdk.c
>> +++ b/lib/netdev-dpdk.c
>> @@ -148,6 +148,22 @@ struct dpdk_tx_queue {
>> struct rte_mbuf *burst_pkts[MAX_TX_QUEUE_LEN];
>> };
>>
>> +/* dpdk has no way to remove dpdk ring ethernet devices
>> + so we have to keep them around once they've been created
>> +*/
>> +
>> +static struct list dpdk_ring_list OVS_GUARDED_BY(dpdk_mutex)
>> += LIST_INITIALIZER(&dpdk_ring_list);
>> +
>> +struct dpdk_ring {
>> +/* For the client rings */
>> +struct rte_ring *cring_tx;
>
>> +struct rte_ring *cring_rx;
>> +int port_id; /* dpdkr port id */
>> +int eth_port_id; /* ethernet device port id */
>> +struct list list_node OVS_GUARDED_BY(mutex);
>s/mutex/dpdk_mutex/
>
>> +};
>> +
>> struct netdev_dpdk {
>> struct netdev up;
>> int port_id;
>> @@ -167,6 +183,7 @@ struct netdev_dpdk {
>> uint8_t hwaddr[ETH_ADDR_LEN];
>> enum netdev_flags flags;
>>
>> +
>> struct rte_eth_link link;
>> int link_reset_cnt;
>>
>extra blank line.
>
>> @@ -179,6 +196,11 @@ struct netdev_rxq_dpdk {
>> int port_id;
>> };
>>
>> +struct dpdk_class {
>> +const char *dp
Re: [ovs-dev] netdev-dpdk / add dpdk rings to netdev-dpdk
Ben, Thanks, I didn¹t catch that. I will fix the makefile to exclude it. Gerald On 6/24/14, 12:35 PM, "Ben Pfaff" wrote: >I think you misread Pravin's message. That's what he gets without >using --with-dpdk. Without --with-dpdk, OVS shouldn't need the DPDK >headers. > >On Tue, Jun 24, 2014 at 07:29:29PM +, Rogers, Gerald wrote: >> Pravin, >> >> Which version of DPDK are you using? >> >> In mine the rte_config.h is located in the DPDK directory >> include/rte_config.h >> >> Gerald >> >> On 6/24/14, 12:11 PM, "Pravin Shelar" wrote: >> >> >Hi Gerald, >> > >> >Thanks for updating patch, here are few comments: >> > >> >I got compilation erorr without --with-dpdk config option: >> > >> >tests/ovs_client/ovs_client.c:38:24: fatal error: rte_config.h: No >> >such file or directory >> >#include >> > >> > >> > >> >On Mon, Jun 23, 2014 at 4:55 AM, wrote: >> >> Shared memory ring patch >> >> >> >> This patch enables the client dpdk rings within the netdev-dpdk. It >> >>adds >> >> a new dpdk device called dpdkr (other naming suggestions?). This >>allows >> >> for the use of shared memory to communicate with other dpdk >> >>applications, >> >> on the host or within a virtual machine. Instructions for use are in >> >> INSTALL.DPDK. >> >> >> >> This has been tested on Intel multi-core platforms and with the >>client >> >> application within the host. >> >> >> >> Signed-off-by: Gerald Rogers >> >> >> >> diff --git a/INSTALL.DPDK b/INSTALL.DPDK >> >> index 2a6d7ef..6a8b565 100644 >> >> --- a/INSTALL.DPDK >> >> +++ b/INSTALL.DPDK >> >> @@ -190,6 +190,44 @@ The core 23 is left idle, which allows core 7 to >> >>run at full rate. >> >> >> >> Future changes may change the need for cpu core affinitization. >> >> >> >> +DPDK Rings : >> >> + >> >> +Following the steps above to create a bridge, you can now add dpdk >> >>rings >> >> +as a port to the vswitch. OVS will expect the DPDK ring device >>name to >> >> +start with dpdkr and end with portid. >> >> + >> >> +ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr >> >> + >> >> +DPDK rings client test application >> >> + >> >> +Included in the test directory is a sample DPDK application for >>testing >> >> +the rings. This is from the base dpdk directory and modified to >>work >> >> +with the ring naming used within ovs. >> >> + >> >> +location tests/ovs_client >> >> + >> >> +To run the client : >> >> + >> >> +ovsclient -c 1 -n 4 --proc-type=secondary -- --n "port id you gave >> >>dpdkr" >> >> + >> >> +It is essential to have --proc-type=secondary >> >> + >> >> +The application simply receives an mbuf on the receive queue of the >> >> +ethernet ring and then places that same mbuf on the transmit ring of >> >> +the ethernet ring. It is a trivial loopback application. >> >> + >> >> +In addition to executing the client in the host, you can execute it >> >>within >> >> +a guest VM. To do so you will need a patched qemu. You can download >> >>the >> >> +patch and getting started guid at : >> >> + >> >> +https://01.org/packet-processing/downloads >> >> + >> >> +A general rule of thumb for better performance is that the client >> >> +application shouldn't be assigned the same dpdk core mask "-c" as >> >> +the vswitchd. >> >> + >> >> + >> >> + >> >> Restrictions: >> >> - >> >> >> >> @@ -202,6 +240,11 @@ Restrictions: >> >> device queues configured. >> >>- Work with 1500 MTU, needs few changes in DPDK lib to fix this >> >>issue. >> >>- Currently DPDK port does not make use any offload functionality. >> >> + ivshmem >> >> + - The shared memory is currently restricted to the use of a 1GB >> >> + huge pages. >> >> + - All huge pages are shared amongst the host, clients, vir
Re: [ovs-dev] [PATCH RFC 1/1] netdev-dpdk: add dpdk vhost-user ports
Pravin, There are some deployments that are using or require older OS distributions using qemu versions prior to version 2.1, thus would require the support for vHost-cuse. Most general reason is qualification cycles for new OS versions. The patch provides compile time support for choosing which vHost version to use. The support for vhost-cuse should be kept for at least several releases with documentation indicating it will be eventually deprecated. Gerald On 4/2/15, 10:31 PM, "Pravin Shelar" wrote: >On Thu, Apr 2, 2015 at 8:37 AM, Traynor, Kevin >wrote: >> >>> -Original Message- >>> From: dev [mailto:dev-boun...@openvswitch.org] On Behalf Of Pravin >>>Shelar >>> Sent: Friday, March 27, 2015 6:08 PM >>> To: Flavio Leitner >>> Cc: dev@openvswitch.org >>> Subject: Re: [ovs-dev] [PATCH RFC 1/1] netdev-dpdk: add dpdk >>>vhost-user ports >>> >>> On Fri, Mar 27, 2015 at 6:57 AM, Flavio Leitner >>>wrote: >>> > On Fri, 20 Mar 2015 10:53:14 -0700 >>> > Pravin Shelar wrote: >>> > >>> >> On Thu, Mar 19, 2015 at 11:48 AM, Ciara Loftus >>> >> wrote: >>> >> > This patch adds support for a new port type to the userspace >>> >> > datapath called dpdkvhostuser. It adds to the existing >>> >> > infrastructure of vhost-cuse, however disables vhost-cuse ports in >>> >> > favour of vhost-user ports. >>> >> > >>> >> > A new dpdkvhostuser port will create a unix domain socket which >>>when >>> >> > provided to QEMU is used to facilitate communication between the >>> >> > virtio-net device on the VM and the OVS port. >>> >> > >>> >> Thanks for the patch. I have pushed OVS DPDK vHost cuse patch. Once >>>we >>> >> add support for vhost user, vhost-cuse support should be dropped. >>> > >>> > Do you mean to literally take vhost-cuse out of the code? If so, yes, >>> > I think that makes sense. >>> > >>> >>> Yes. thats what I meant. >> >> One issue with removing is that vhost-user is only supported from QEMU >>2.1.0 >> onwards. If anyone is using an older version of QEMU they would not be >>able >> to use vhost-user, whereas vhost-cuse will support older versions of >>QEMU. >> > >Are there distribution still using older QEMU? ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
Re: [ovs-dev] [dpdk-dev] [PATCH RFC] dpif-netdev: Add support Intel DPDK based ports.
Prashant, IVShm is supported by the Intel DPDK client rings, and a patched QEMU/KVM from the OVDK work on 01.org (https://01.org/packet-processing). The main part being the patched QEMU/KVM to map the Intel DPDK Huge Page Tables (with Release of Intel DPDK 1.6 the requirement to map 1 GB huge pages has been removed, but still supported) into the QEMU/KVM ivshm device (the Client rings are standard in Intel DPDK releases). My suggestion to add this functionality is similar to the way different interfaces are supported under Linux (tap, socket, etc.). Basically add another Intel DPDK Netdev type for client rings. Once the rings are instantiated (upon Intel DPDK initialization), then it is simple enough to have the OpenVSwitch control to initialize them into the polling method (just like the physical ports are done today). Structures like mac address, MTU, etc. for the would need to be considered since the client rings are not really thought of as ³Ethernet² ports within DPDK. If you want to make them virtual ³Ethernet² ports, then assigning the Ethernet parameters a static value upon initialization would be acceptable. Thoughts from the community are much welcomed on this whole topic. As Pravin mentioned in one of his previous e-mails, the support for IVShmem, Intel DPDK QOS, vxLan etc. wasn¹t added (and in some cases doesn¹t exist as an Intel DPDK library, ie. vxLan) to simplify the patch. It will be worked on for subsequent patches. Sincerely, Gerald On 1/30/14, 3:15 AM, "Prashant Upadhyaya" wrote: >Hi Pravin, > >Request you to please validate atleast one method to interface VM's with >your innovative dpdk port on the OVS. >Preferably IVSHM. >Please do publish the steps for that too. > >We really need the above for huge acceptance. > >Regards >-Prashant > > >-Original Message- >From: Pravin Shelar [mailto:pshe...@nicira.com] >Sent: Thursday, January 30, 2014 3:00 AM >To: Prashant Upadhyaya >Cc: dev@openvswitch.org; d...@dpdk.org; dpdk-...@lists.01.org; Gerald >Rogers >Subject: Re: [dpdk-dev] [PATCH RFC] dpif-netdev: Add support Intel DPDK >based ports. > >On Wed, Jan 29, 2014 at 12:56 AM, Prashant Upadhyaya > wrote: >> Hi Pravin, >> >> I think your stuff is on the brink of a creating a mini revolution :) >> >> Some questions inline below -- >> +ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk >> What do you mean by portid here, do you mean the physical interface id >>like eth0 which I have bound to igb_uio now ? >> If I have multiple interfaces I have assigned igb_uio to, eg. eth0, >>eth1, eth2 etc., what is the id mapping for those ? >> >Port id is id assigned by DPDK. DPDK interface takes this port id as >argument. Currently you need to look at pci id to figure out the device >mapping to port id. I know it is clean and I am exploring better >interface so that we can specify device names to ovs-vsctl. > >> If I have VM's running, then typically how to interface those VM's to >>this OVS in user space now, do I use the same classical 'tap' interface >>and add it to the OVS above. > >tap device will work, but you would not get performance primarily due to >scheduling delay and memcopy. >DPDK has multiple drivers to create interface with KVM guests OS. >those should perform better. I have no tried it yet. > >> What is the actual path the data takes from the VM now all the way to >>the switch, wouldn't it be hypervisor to kernel to OVS switch in user >>space to other VM/Network ? > >Depends on method you use. e.g. Memnic bypass hypervisor and host kernel >entirely. > >> I think if we can solve the VM to OVS port connectivity remaining in >>userspace only, then we have a great thing at our hand. Kindly comment >>on this. >> >right, performance looks pretty good. Still DPDK needs constant polling >which consumes more power. RFC ovs-dkdp patch has simple polling which >need tweaking for better power usage. > >Thanks, >Pravin. > > > >> Regards >> -Prashant >> >> > > > > >== >= >Please refer to http://www.aricent.com/legal/email_disclaimer.html >for important disclosures regarding this electronic communication. >== >= ___ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
