[dpdk-dev] Query regarding multiple processes in DPDK

[Jeff Venable, Sr.]

The memory configuration will persist while the files describing them do.
By default, these are located in /var/run/.rte_config and
/var/run/.rte_hugepage_info.  Even if you terminate all processes, you can
still launch new secondary processes which will find those files and
attach, and unless something is messing with your huge pages the memory
state will be as it was left before.  The logic for these file locations
is in lib/librte_eal/linuxapp/eal/include/eal_filesystem.h if you want to
review the code.  You can search the rest of the code for calls to these
functions to locate the logic regarding what?s stored in the files.

Jeff

On 11/25/13, 5:57 AM, "Prashant Upadhyaya"
 wrote:

>Hi Bruce,
>
>Thanks, this was very useful information.
>
>Regards
>-Prashant
>
>
>-Original Message-
>From: Richardson, Bruce [mailto:bruce.richardson at intel.com]
>Sent: Monday, November 25, 2013 2:59 PM
>To: Prashant Upadhyaya; dev at dpdk.org
>Subject: RE: Query regarding multiple processes in DPDK
>
>If the primary process dies:
>a) The memory does not go away, so the second process can still use it
>b) When restarting the primary process, you should restart it as a
>secondary one, to ensure it reattaches to memory properly instead of
>trying to re-initialize it.
>
>Regards
>/Bruce
>
>> -Original Message-
>> From: Prashant Upadhyaya [mailto:prashant.upadhyaya at aricent.com]
>> Sent: Monday, November 25, 2013 4:08 AM
>> To: Richardson, Bruce; dev at dpdk.org
>> Subject: RE: Query regarding multiple processes in DPDK
>>
>> Hi Bruce,
>>
>> One more question --
>>
>> Suppose the first instance comes up as primary and creates the mbuf
>> pool and rings etc. [ok] Now, the second instance comes up as
>> secondary and does the corresponding lookup functions [ok] Now the
>> primary exits -- at this point can the secondary still run with all
>> the memory to which it had done the lookup intact, or does the fact
>> that primary died will lead to all the memory also taken away with it
>> so that the secondary can no longer function now ?
>>
>> Regards
>> -Prashant
>>
>>
>> -Original Message-
>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Prashant
>> Upadhyaya
>> Sent: Friday, November 22, 2013 7:16 PM
>> To: Richardson, Bruce; dev at dpdk.org
>> Subject: Re: [dpdk-dev] Query regarding multiple processes in DPDK
>>
>> Thanks Bruce, I think your suggested example of multi_process answers
>> my questions.
>>
>> Regards
>> -Prashant
>>
>>
>> -Original Message-
>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Prashant
>> Upadhyaya
>> Sent: Friday, November 22, 2013 7:10 PM
>> To: Richardson, Bruce; dev at dpdk.org
>> Subject: Re: [dpdk-dev] Query regarding multiple processes in DPDK
>>
>> Hi Bruce,
>>
>> Thanks.
>>
>> Regarding your comment --
>> [BR] It will depend upon the application, but in most cases you
>> probably want to have slightly different code paths for primary and
>> secondary instances. For example, if a process is running as primary
>> instance, it will probably call rte_mempool_create or rte_ring_create.
>> A secondary instance which wants to use these should instead call
>> rte_mempool_lookup and rte_ring_lookup instead.
>> For an example of how to write the one binary to be used as both
>> primary and secondary process, I suggest looking at the symmetric_mp
>> example application in the examples/multi_process/ directory.
>>
>> I was really hoping that the --proc-type=auto, would make the DPDK
>> libraries internally resolving all this stuff, is that not the case ?
>> I have not started reading the code for all this yet.
>> I must launch the same executable twice in my usecase. Even if the
>> executable code has to make different calls when it comes up as
>> secondary, is there a way for the usercode to know that it has really
>> come up as secondary when the --proc-type=auto is used ?
>>
>> Regards
>> -Prashant
>>
>> -Original Message-
>> From: Richardson, Bruce [mailto:bruce.richardson at intel.com]
>> Sent: Friday, November 22, 2013 7:02 PM
>> To: Prashant Upadhyaya; dev at dpdk.org
>> Subject: RE: Query regarding multiple processes in DPDK
>>
>> Hi Prashant
>>
>> > ===
>> > The EAL also supports an auto-detection mode (set by EAL
>> > --proc-type=auto flag), whereby an Intel(r) DPDK process is started
>> > as a secondary instance if a primary instance is already running.
>> > ===
>> >
>> > So does this mean that if I have a DPDK exe foo.out, then when I run
>> > the first instance of foo.out with -proc-type = auto, then foo.out
>> > will run as a primary process and when I spawn the second instance
>> > of foo.out (with first already running) again with -proc-type=auto,
>> > then this second instance automatically becomes secondary ?
>> [BR] Yes, that is the idea.
>>
>> >
>> > Also is there any user code initialization change required or
>> > exactly the same code will work for both the processes ?
>> [BR] It will depend upon the application, but in m

[dpdk-dev] NRMK build break

[Jeff Venable, Sr.]

For those who leave the office before verifying they didn?t break the build,
a public flogging! :)

/usr/bin/ld: cannot find -lipinfo

collect2: error: ld returned 1 exit status

make: *** [/home/jeff/Colossus/nrmk/dataplane/P2P/obj/debug/libP2P.so] Error
1

make: *** Waiting for unfinished jobs


You marked libipinfo.so as an external dependency.  You need to make it a
makefile dependency.  The fix:

(jeff at colossus ~/Colossus/nrmk)$ git diff dataplane/P2P/Rules.mk

diff --git a/nrmk/dataplane/P2P/Rules.mk b/nrmk/dataplane/P2P/Rules.mk

index 2ecc396..093b7ac 100644

--- a/nrmk/dataplane/P2P/Rules.mk

+++ b/nrmk/dataplane/P2P/Rules.mk

@@ -34,6 +34,7 @@ libP2P.so_DEPS := \

 P2PConfig.o \

 $(TARGETS_$(TOP)/lib/json) \

 $(TARGETS_$(TOP)/lib/notes) \

+$(TARGETS_$(TOP)/lib/ipinfo)



 LIBDIRS_$(d) = $(TOP)/../lib



@@ -41,5 +42,4 @@ libP2P.so_LIBS = \

 -lcolossuscommon \

 -lcolossuscommand \

 -lcolossustrace \

--lcolossusutilities \

--lipinfo

+-lcolossusutilities


Pushing the fix.

Thanks for moving all your algorithms into NRMK! :)

Jeff

[dpdk-dev] ASN.1 codec

[Jeff Venable, Sr.]

For those who are interested, I?ve pushed an ASN.1 codec API sufficient for
Kerberos parsing to nrmk/include/codec/asn.1.

The unit-test is in nrmk/lib/codec/asn.1/test if you want to see how it can
be used.

I?ll be working on the Kerberos 5 codec next leveraging the BER/DER APIs, so
be patient with me. :)

Jeff

[dpdk-dev] processor/core count issue

[Jeff Venable, Sr.]

In a VM Fusion environment configured with 2 processors with 1 core each I get 
the following:

PANIC in rte_eal_init():
Cannot init logs
4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) 
[0x7f9fd3bdaec5]]
3: 
[/usr/lib/dpdk-2.0.0/librte_eal.so.1(rte_eal_init+0x123a)
 [0x7f9fd4f462fa]]
2: 
[/usr/lib/dpdk-2.0.0/librte_eal.so.1(__rte_panic+0xc3) 
[0x7f9fd4f449e8]]
1: 
[/usr/lib/dpdk-2.0.0/librte_eal.so.1(rte_dump_stack+0x16)
 [0x7f9fd4f4daa6]]
Aborted (core dumped)

rte_socket_id() was returning 0x here:

(gdb) bt
#0  0x7797ae95 in rte_malloc_socket () from 
/usr/lib/dpdk-2.0.0/librte_malloc.so.1
#1  0x7797af4e in rte_zmalloc_socket () from 
/usr/lib/dpdk-2.0.0/librte_malloc.so.1
#2  0x77573ef8 in rte_ring_create () from 
/usr/lib/dpdk-2.0.0/librte_ring.so.1
#3  0x770d in rte_mempool_xmem_create () from 
/usr/lib/dpdk-2.0.0/librte_mempool.so.1
#4  0x7b97 in rte_mempool_create () from 
/usr/lib/dpdk-2.0.0/librte_mempool.so.1
#5  0x77b90899 in ?? () from /usr/lib/dpdk-2.0.0/librte_eal.so.1
#6  0x77b88d34 in ?? () from /usr/lib/dpdk-2.0.0/librte_eal.so.1
#7  0x77b84950 in rte_eal_init () from 
/usr/lib/dpdk-2.0.0/librte_eal.so.1

When I reconfigured the VM to 1 processor with 2 cores, everything works as 
expected.

This was not a problem in 1.6.

Jeff

[dpdk-dev] Why only rx queue "0" can receive network packet by i40e NIC

[Jeff Venable, Sr.]

Is the I40E incapable of operating RSS with ETH_RSS_IP (i.e. hashing without L4 
ports)?

Thanks,

Jeff

-Original Message-
From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Zhang, Helin
Sent: Saturday, February 28, 2015 6:34 AM
To: lhffjzh; 'Thomas Monjalon'
Cc: dev at dpdk.org; maintainers at dpdk.org
Subject: Re: [dpdk-dev] Why only rx queue "0" can receive network packet by 
i40e NIC

Good to know that!

> -Original Message-
> From: lhffjzh [mailto:lhffjzh at 126.com]
> Sent: Saturday, February 28, 2015 12:34 PM
> To: Zhang, Helin; 'Thomas Monjalon'
> Cc: dev at dpdk.org; maintainers at dpdk.org
> Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> packet by i40e NIC
> 
> Hi Helin,
> 
> Thanks a lot for your great help, all of rx queue received network 
> packet after I update rss_hf from "ETH_RSS_IP" to " ETH_RSS_PROTO_MASK ".
> 
> static struct rte_eth_conf port_conf = {
> .rxmode = {
> .mq_mode= ETH_MQ_RX_RSS,
> .max_rx_pkt_len = ETHER_MAX_LEN,
> .split_hdr_size = 0,
> .header_split   = 0, /**< Header Split disabled */
> .hw_ip_checksum = 1, /**< IP checksum offload enabled */
> .hw_vlan_filter = 0, /**< VLAN filtering disabled */
> .jumbo_frame= 0, /**< Jumbo Frame Support disabled */
> .hw_strip_crc   = 0, /**< CRC stripped by hardware */
> },
> .rx_adv_conf = {
> .rss_conf = {
> .rss_key = NULL,
> .rss_hf = ETH_RSS_PROTO_MASK,
> },
> },
> .txmode = {
> .mq_mode = ETH_MQ_TX_NONE,
> },
> .fdir_conf.mode = RTE_FDIR_MODE_SIGNATURE, };
> 
> 
> Regards,
> Haifeng
> 
> -Original Message-
> From: Zhang, Helin [mailto:helin.zhang at intel.com]
> Sent: Saturday, February 28, 2015 11:18 AM
> To: lhffjzh; 'Thomas Monjalon'
> Cc: dev at dpdk.org; maintainers at dpdk.org
> Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> packet by i40e NIC
> 
> Hi Haifeng
> 
> > -Original Message-
> > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of lhffjzh
> > Sent: Saturday, February 28, 2015 9:48 AM
> > To: 'Thomas Monjalon'
> > Cc: dev at dpdk.org; maintainers at dpdk.org
> > Subject: Re: [dpdk-dev] Why only rx queue "0" can receive network 
> > packet
> by
> > i40e NIC
> >
> > Hi Thomas,
> >
> > Thanks very much for your reminder, you give me many help in this 
> > mail
> list.
> >
> > The issue with detailed information just as below. but I don't know 
> > who is
> the
> > dpdk i40e maintainers? is maintainers at dpdk.org?
> >
> > Hardware list:
> > 2 i40e 40G NICs
> > Xeon E5-2670 v2(10 cores)
> > 32G memory
> >
> > I loopback 2 i40e NICs by QSFP cable, one NIC send UDP network 
> > packet by DPDK, and another for receiving. I bind 4 processor's 
> > logical cores with 4
> rx
> > queue "0,1,2,3" on receiving NIC, when I start to send packet, only 
> > rx
> queue
> > "0"
> > can receive
> > the UDP packet, the others queue always receive nothing. but it is 
> > work
> well on
> > ixgbe 10G NICs, I can receive network packet from all rx queues. 
> > does
> anyone
> > kindly know why?
> Could you help to list the DPDK version you are using now?
> Two possible reasons:
> 1. UDP rss is not enabled on your board correctly.
>   I40e has different rss flags from ixgbe, so I am wondering if you use 
> it correctly.
>   In addition, this will be unified from 2.0. So I care about the DPDK 
> version.
> 2. The UDP stream is occasionally hit the hash key of queue 0.
>   You'd better to try to send your UDP stream with random 5-tuples, to 
> get the
>   hash value hit different queues randomly.
> 
> Regards,
> Helin
> 
> >
> >
> > Regards,
> > Haifeng
> >
> > -Original Message-
> > From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> > Sent: Friday, February 27, 2015 6:55 PM
> > To: lhffjzh
> > Cc: dev at dpdk.org
> > Subject: Re: Why only rx queue "0" can receive network packet by 
> > i40e NIC
> >
> > 2015-02-27 16:47, lhffjzh:
> > > Hi All,
> > >
> > > We use 4 cores loop 4 rx queues on one i40e port, but only rx queue "0"
> > can
> > > receive network packet, do anyone kindly know why? BTW, all of 
> > > network packet has same destination ip address but has more than 
> > > 200 different source ip address.
> >
> > It's possible that you don't have any answer for 2 reasons:
> > - you replied in a thread dedicated to Cisco enic questions
> > - you didn't describe your usage enough to understand your problem
> >
> > I suggest to use the button "new email" instead of "reply all" to 
> > start a new question with enough details.
> >
> > Did you noticed you put some Cisco guys in CC instead of putting the 
> > Intel responsible for i40e (see MAINTAINERS file)?
> >
> 

[dpdk-dev] Why only rx queue "0" can receive network packet by i40e NIC

[Jeff Venable, Sr.]

Hi Helin,

We do not want RSS to include L4 ports in the hash because packet fragments 
would get routed to queue #0 and would be more difficult to work with.  We are 
using the model where multiple CPUs are pulling from the NIC queues 
independently with no shared state, so each 'pipeline' has private fragment 
reassembly state for the sessions it is managing.

Getting RSS Toeplitz hash to work on { source_ip, dest_ip } tuples only using a 
symmetric rss-key is important.  This works properly with all other Intel NICs 
in the DPDK thus far that we have tested until the i40E PMD with the Intel 
X710-DA4.  The Microsoft RSS specification allows for this.

With the i40E PMD, we have been unsuccessful at enabling this RSS 
configuration.  From the source code and XL710 controller datasheet, we cannot 
find any reference to the flags for this RSS mode.  Unless we can achieve 
feature parity with the other Intel NICs, we don't want to write special case 
code for this one driver which makes the XL710 controller unusable for us and 
seems contrary to the intent of the DPDK APIs which are abstracting this 
behavior.

Do you have any suggestions?

Thanks kindly,

Jeff

-Original Message-
From: Zhang, Helin [mailto:helin.zh...@intel.com] 
Sent: Wednesday, July 22, 2015 5:56 PM
To: Jeff Venable, Sr. ; lhffjzh ; 'Thomas Monjalon' 
Cc: dev at dpdk.org
Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network packet by 
i40e NIC



> -----Original Message-
> From: Jeff Venable, Sr. [mailto:jeff at vectranetworks.com]
> Sent: Wednesday, July 22, 2015 5:47 PM
> To: Zhang, Helin; lhffjzh; 'Thomas Monjalon'
> Cc: dev at dpdk.org
> Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> packet by i40e NIC
> 
> Is the I40E incapable of operating RSS with ETH_RSS_IP (i.e. hashing 
> without L4 ports)?
Why do you think like this? Sorry, I am a bit confused.
ETH_RSS_IP is a super set of all IP based rss types. Please see the rss types 
listed in rte_ethdev.h.
The supports rss types of each NIC can be queried via 'struct rte_eth_dev_info' 
of field 'flow_type_rss_offloads'.

Regards,
Helin

> 
> Thanks,
> 
> Jeff
> 
> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin
> Sent: Saturday, February 28, 2015 6:34 AM
> To: lhffjzh; 'Thomas Monjalon'
> Cc: dev at dpdk.org; maintainers at dpdk.org
> Subject: Re: [dpdk-dev] Why only rx queue "0" can receive network 
> packet by i40e NIC
> 
> Good to know that!
> 
> > -Original Message-
> > From: lhffjzh [mailto:lhffjzh at 126.com]
> > Sent: Saturday, February 28, 2015 12:34 PM
> > To: Zhang, Helin; 'Thomas Monjalon'
> > Cc: dev at dpdk.org; maintainers at dpdk.org
> > Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> > packet by i40e NIC
> >
> > Hi Helin,
> >
> > Thanks a lot for your great help, all of rx queue received network 
> > packet after I update rss_hf from "ETH_RSS_IP" to " ETH_RSS_PROTO_MASK ".
> >
> > static struct rte_eth_conf port_conf = {
> > .rxmode = {
> > .mq_mode= ETH_MQ_RX_RSS,
> > .max_rx_pkt_len = ETHER_MAX_LEN,
> > .split_hdr_size = 0,
> > .header_split   = 0, /**< Header Split disabled */
> > .hw_ip_checksum = 1, /**< IP checksum offload enabled */
> > .hw_vlan_filter = 0, /**< VLAN filtering disabled */
> > .jumbo_frame= 0, /**< Jumbo Frame Support disabled */
> > .hw_strip_crc   = 0, /**< CRC stripped by hardware */
> > },
> > .rx_adv_conf = {
> > .rss_conf = {
> > .rss_key = NULL,
> > .rss_hf = ETH_RSS_PROTO_MASK,
> > },
> > },
> > .txmode = {
> > .mq_mode = ETH_MQ_TX_NONE,
> > },
> > .fdir_conf.mode = RTE_FDIR_MODE_SIGNATURE, };
> >
> >
> > Regards,
> > Haifeng
> >
> > -Original Message-
> > From: Zhang, Helin [mailto:helin.zhang at intel.com]
> > Sent: Saturday, February 28, 2015 11:18 AM
> > To: lhffjzh; 'Thomas Monjalon'
> > Cc: dev at dpdk.org; maintainers at dpdk.org
> > Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> > packet by i40e NIC
> >
> > Hi Haifeng
> >
> > > -Original Message-
> > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of lhffjzh
> > > Sent: Saturday, February 28, 2015 9:48 AM
> > > To: 'Thomas Monjalon'
> > > Cc: dev at dpdk.org; maintainers at dpdk.o