ipsec on dpdk
Hi I am interested to do ipsec encoding and decoding in my dpdk application From my readings, i understand ipsec can be done one time in the nic (inline ipsec) or with multiple calls (rte_cryptodev_enqueue_burst, rte_cryptodev_dequeue_burst) 1. If ipsec is done by nic I only need to call rte_ipsec_pkt_process(...) without other functions? I use rte_eth_rx_burst to read from nic. 1. Where do I see list of nic that support nic inline ipsec? I believe not all dpdk nic support it. 2. How much does it impact performance ? is there a table of performance per nic? 3. My application is multi process, I can see in documentation : “Currently, the security library does not support the case of multi-process. It will be updated in the future releases.” From https://doc.dpdk.org/guides/prog_guide/rte_security.html So ipsec also is not supported for multi process application? Even if done inline by the nic? And what about non inline ipsec for multi process applications? 1. Is ip sec also supported in multi queue with rte flow in the inline ipsec ?
crc stripping for vf on same pf
I have 2 pods running on same worker Pod1 send to pod2 Pod2 receive with 4 bytes less at end of packet This problem happens only if the 2 nic are on the same pf, If different pf, the problem doesn’t occurs I tried with dpdk21 and dpdk22 The code is using driver net_iavf nic e810c driver: ice firmware-version: 4.00 0x800139bc 21.5.9 Who does the stripping? The dpdk code or the card? Why is is different behavior for same pf and different pf ? What should i change or check? port_conf.rxmode.offloads |= RTE_ETH_RX_OFFLOAD_KEEP_CRC; //Don't strip CRC port_conf.rxmode.offloads &= pi_devInfo.rx_offload_capa; int ret = rte_eth_dev_configure(pi_nPort, nRxQueues, nTxQueues, &port_conf); struct rte_eth_rxconf rx_conf; rx_conf.offloads = RTE_ETH_RX_OFFLOAD_KEEP_CRC; int ret = rte_eth_rx_queue_setup( pi_nPort, nQueue, nRxRingSize, socket, performanceMode?NULL:&rx_conf, pool);
Rss hash on mellanox 100G card
I am using Mellanox Technologies MT27800 Family [ConnectX-5], using dpdk 19 with multi rx queue with rss port_conf.rx_adv_conf.rss_conf.rss_hf=(ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP) I analyze traffic and need all packet of same session to arrive to the same process ( session for now can be ip+port) So Packet that have the same ip + port arrive to the same queue. But If some packet are ip fragmented, packet arrive to different process. It is a problem! How can i calculate the hash value in the c++ code, like it is done in the card, so i can reassemble packets and send them to the same process like the non fragmented packets?
mellanox connect x 5 drops when cache full
Hi I am using multiqueue with RSS to read from MT27800 Family [ConnectX-5] 1017 My application receive traffic, and write some data to disk. As a result I/O write is cached in linux memory. When the server memory is completely cache ( I know it is still available) I start seeing drops at nic. If I delete the data from disk every minute the cached memory is released to free and and no drops at nic. How can I avoid the drops at nic? totalusedfree shared buff/cache available Mem: 125G 77G325M 29M 47G 47G Swap: 8.0G256K8.0G
rxq_cq_process_v high cpu
I moved from dpdk 19 to dpdk 21 I read with multiqueue from Mellanox 100G card with dpdk function rte_eth_rx_burst I can see in perf top very high cpu on function rxq_cq_process_v How can i fix it? This function wasn't in my perf top with dpdk 19 version 53.12% libdpdkutils.so [.] rxq_cq_process_v 16.25% libdpdkutils.so [.] mlx5_rx_burst_vec 5.50% libcaptureimpl.so [.] CrPacketReceiver::ReadFromPort 2.06% libdpdkutils.so [.] CrDpdkDeviceUtil::GetPacketDetails 2.04% libhpadecoders.so [.] CrReorderMbuf::GetTimedOut 1.55% libhpahwswmsg.so [.] CrNetworkPacket::FindEthernetTypeAndPayload 1.25% libcaptureimpl.so [.] CrPacketReceiver::FullPollMode 1.17% libdpdkutils.so [.] rte_eth_dev_is_valid_port 0.96% libhpartpdecoderimpl.so [.] CrRtpProcessing::GetSession 0.77% libdpdkutils.so [.] CrDpdkDeviceUtil::InitPacketDetailsFromBuff 0.66% libhpartpdecoderimpl.so [.] VQmonStreamIndicatePacket 0.63% libhpartpdecoderimpl.so [.] VQmonJitterBufferIndicatePacket
[dpdk-dev] imissed drop with mellanox connectx5
Hi We try to read from 100G NIC Mellanox ConnectX-5 without drop at nic. All thread are with core pinning and cpu isolation. We use dpdk 19.11 I tried to apply all configuration that are in https://fast.dpdk.org/doc/perf/DPDK_19_08_Mellanox_NIC_performance_report.pdf We have a strange behavior, 1 thread can receive receive 20 Gbps/12 Mpps and free mbuf without dropps, but when trying to pass these mbuf to another thread that only free them there are drops, even when trying to work with more threads. When running 1 thread that only read from port (no multi queue) and free mbuf in the same thread, there are no dropp with traffic up to 21 Gbps 12.4 Mpps. When running 6 thread that only read from port (with multi queue) and free mbuf in the same threads, there are no dropp with traffic up to 21 Gbps 12.4 Mpps. When running 1 to 6 thread that only read from port and pass them to another 6 thread that only read from ring and free mbuf, there are dropp in nic (imissed counter) with traffic over to 10 Gbps 5.2 Mpps.(Here receive thread were pinned to cpu 1-6 and additional thread from 7-12 each thread on a single cpu) Each receive thread send to one thread that free the buffer. Configurations: We use rings of size 32768 between the threads. Ring are initialized with SP/SC, Write are done with bulk of 512 with rte_ring_enqueue_burst. Port is initialized with rte_eth_rx_queue_setup nb_rx_desc=8192 rte_eth_rxconf - rx_conf.rx_thresh.pthresh = DPDK_NIC_RX_PTHRESH; //ring prefetch threshold rx_conf.rx_thresh.hthresh = DPDK_NIC_RX_HTHRESH; //ring host threshold rx_conf.rx_thresh.wthresh = DPDK_NIC_RX_WTHRESH; //ring writeback threshold rx_conf.rx_free_thresh = DPDK_NIC_RX_FREE_THRESH; rss -> ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP; We tried to work with and without hyperthreading. Network devices using kernel driver === :37:00.0 'MT27800 Family [ConnectX-5] 1017' if=ens2f0 drv=mlx5_core unused=igb_uio :37:00.1 'MT27800 Family [ConnectX-5] 1017' if=ens2f1 drv=mlx5_core unused=igb_uio ethtool -i ens2f0 driver: mlx5_core version: 5.3-1.0.0 firmware-version: 16.30.1004 (HPE09) expansion-rom-version: bus-info: :37:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes uname -a Linux localhost.localdomain 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux lscpu | grep -e Socket -e Core -e Thread Thread(s) per core:1 Core(s) per socket:24 Socket(s): 2 cat /sys/devices/system/node/node0/cpulist 0-23 From /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 85 model name : Intel(R) Xeon(R) Gold 5220R CPU @ 2.20GHz stepping: 7 microcode : 0x5003003 cpu MHz : 2200.000 python /home/cpu_layout.py == Core and Socket Information (as reported by '/sys/devices/system/cpu') == cores = [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 16, 17, 18, 19, 20, 21, 25, 26, 27, 28, 29, 24] sockets = [0, 1] Socket 0Socket 1 Core 0 [0] [24] Core 1 [1] [25] Core 2 [2] [26] Core 3 [3] [27] Core 4 [4] [28] Core 5 [5] [29] Core 6 [6] [30] Core 8 [7] Core 9 [8] [31] Core 10 [9] [32] Core 11 [10][33] Core 12 [11][34] Core 13 [12][35] Core 16 [13][36] Core 17 [14][37] Core 18 [15][38] Core 19 [16][39] Core 20 [17][40] Core 21 [18][41] Core 25 [19][43] Core 26 [20][44] Core 27 [21][45] Core 28 [22][46] Core 29 [23][47] Core 24 [42]
Re: [dpdk-dev] imissed drop with mellanox connectx5
Hi Matan We work with mbuf in all threads and lcores, We pass them from one thread to another through the dpdk ring before releasing them. There are drops in 10K to 100K pps, we can't stay with these drops. The drops are in the imissed counter from rte_eth_stats_get, so I thought that the drops are at the port level and not drop at mempool level From what I see number of mbuf in pool is stable( and close to the total/original number of mbuf in pool), the rings are empty, Traffic is well balanced between threads, All threads are running in pool from port and from ring. And from perf top profiler there doesn't seem to be any unexpected function taking cpu. So the only possible architecture would be to implement all logic in the threads that read from port, and to launch hundreds of threads in multiqueue mode that read from port? I don't think this is a viable solution ( In the following link for example they show an example of application that pass packet from one core/thread to another https://doc.dpdk.org/guides-16.04/sample_app_ug/qos_scheduler.html ) Thank you answer -Original Message- From: Matan Azrad Sent: Thursday, July 22, 2021 8:19 AM To: Yaron Illouz ; us...@dpdk.org Cc: dev@dpdk.org Subject: RE: imissed drop with mellanox connectx5 Hi Yaron Freeing mbufs from a different lcore than the original lcore allocated them causes cache miss in the mempool cache of the original lcore per mbuf allocation - all the time the PMD will get non-hot mbufs to work with. It can be one of the reasons for the earlier drops you see. Matan From: Yaron Illouz > Hi > > We try to read from 100G NIC Mellanox ConnectX-5 without drop at nic. > All thread are with core pinning and cpu isolation. > We use dpdk 19.11 > I tried to apply all configuration that are in > https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffast > .dpdk.org%2Fdoc%2Fperf%2FDPDK_19_08_Mellanox_NIC_performance_r&dat > a=04%7C01%7C%7Cdcbb2d8246be4dc456c508d94cd038a7%7C0eb9e2d98763412e9709 > 3f539e9e25bc%7C0%7C0%7C637625279453292671%7CUnknown%7CTWFpbGZsb3d8eyJW > IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000& > amp;sdata=KMBFyIMEFV4B0JqxQE%2BiMXJ2p9qE8lEOpUWRsFhD0gM%3D&reserve > d=0 > eport.pdf > > We have a strange behavior, 1 thread can receive receive 20 Gbps/12 > Mpps and free mbuf without dropps, but when trying to pass these mbuf > to another thread that only free them there are drops, even when > trying to work with more threads. > > When running 1 thread that only read from port (no multi queue) and > free mbuf in the same thread, there are no dropp with traffic up to 21 > Gbps 12.4 Mpps. > When running 6 thread that only read from port (with multi queue) and > free mbuf in the same threads, there are no dropp with traffic up to > 21 Gbps 12.4 Mpps. > > When running 1 to 6 thread that only read from port and pass them to > another 6 thread that only read from ring and free mbuf, there are > dropp in nic (imissed counter) with traffic over to 10 Gbps 5.2 > Mpps.(Here receive thread were pinned to cpu 1-6 and additional thread > from 7-12 each thread on a single cpu) Each receive thread send to one thread > that free the buffer. > > Configurations: > > We use rings of size 32768 between the threads. Ring are initialized > with SP/SC, Write are done with bulk of 512 with rte_ring_enqueue_burst. > Port is initialized with rte_eth_rx_queue_setup nb_rx_desc=8192 > rte_eth_rxconf - rx_conf.rx_thresh.pthresh = DPDK_NIC_RX_PTHRESH; > //ring prefetch threshold > rx_conf.rx_thresh.hthresh = > DPDK_NIC_RX_HTHRESH; //ring host threshold > rx_conf.rx_thresh.wthresh = > DPDK_NIC_RX_WTHRESH; //ring writeback threshold > rx_conf.rx_free_thresh = > DPDK_NIC_RX_FREE_THRESH; rss - > > ETH_RSS_IP | ETH_RSS_UDP | ETH_RSS_TCP; > > > We tried to work with and without hyperthreading. > > > > Network devices using kernel driver > === > :37:00.0 'MT27800 Family [ConnectX-5] 1017' if=ens2f0 > drv=mlx5_core unused=igb_uio > :37:00.1 'MT27800 Family [ConnectX-5] 1017' if=ens2f1 > drv=mlx5_core unused=igb_uio > > > > ethtool -i ens2f0 > driver: mlx5_core > version: 5.3-1.0.0 > firmware-version: 16.30.1004 (HPE09) > expansion-rom-version: > bus-info: :37:00.0 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: yes > > > > uname -a > Linux
rss calculation as the nic
I am using mellanox Connectx6, dpdk 22
'MT2892 Family [ConnectX-6 Dx] 101d' if=ens5f1 drv=mlx5_core unused=igb_uio
I configure port with multiqueue and split traffic according to ip+port
I want to calculate the hash as the nic do, to be able to load balance traffic
( from another card ) - the information is inside the packet and not in in ip
and transport layer. For this purpose i need to be able to calculate the hash
value as the nic do for the first nic.
Here is the code i use to split traffic to rx queues
/*rte flow*/
const int MAX_PATTERN_IN_FLOW = 10;
const int MAX_ACTIONS_IN_FLOW = 10;
struct rte_flow_attr attr;
struct rte_flow_item pattern[MAX_PATTERN_IN_FLOW];
struct rte_flow_action actions[MAX_ACTIONS_IN_FLOW];
struct rte_flow *flow;
struct rte_flow_error error;
memset(pattern, 0, sizeof(pattern));
memset(actions, 0, sizeof(actions));
/* Set the rule attribute, only ingress packets will be checked. 8< */
memset(&attr, 0, sizeof(struct rte_flow_attr));
attr.ingress = 1;
pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH;
pattern[0].spec = NULL;
pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV4;
pattern[1].spec = NULL;
pattern[2].type = RTE_FLOW_ITEM_TYPE_GRE;
pattern[2].spec = NULL;
pattern[3].type = RTE_FLOW_ITEM_TYPE_ETH;
pattern[3].spec = NULL;
pattern[4].type = RTE_FLOW_ITEM_TYPE_IPV4;
pattern[4].spec = NULL;
pattern[5].type = RTE_FLOW_ITEM_TYPE_UDP;
pattern[5].spec = NULL;
// end the pattern array
pattern[6].type = RTE_FLOW_ITEM_TYPE_END;
struct rte_flow_action_rss rss_conf;
uint16_t queues[pi_nNumRxQueues];
rss_conf.func = RTE_ETH_HASH_FUNCTION_DEFAULT;
uint64_t hf = RTE_ETH_RSS_IP | RTE_ETH_RSS_TCP | RTE_ETH_RSS_UDP |
RTE_ETH_RSS_SCTP;
hf &= pi_devInfo.flow_type_rss_offloads;
rss_conf.types = hf;
rss_conf.queue_num = pi_nNumRxQueues;
for (int nqQueueIndex= 0; nqQueueIndex < pi_nNumRxQueues;
nqQueueIndex++)
queues[nqQueueIndex] = nqQueueIndex;
rss_conf.queue = queues;
rss_conf.key_len = 0;
rss_conf.key = NULL;
rss_conf.level = 2;
// create the drop action
actions[0].type = RTE_FLOW_ACTION_TYPE_RSS;
actions[0].conf = &rss_conf;
actions[1].type = RTE_FLOW_ACTION_TYPE_END;
// validate and create the flow rule
if (rte_flow_validate(pi_nPort, &attr, pattern, actions, &error)==0)
{
flow = rte_flow_create(pi_nPort, &attr, pattern, actions, &error);
if(flow){//success}
else{//error}
}
else {error}
}
And this is how i tried to get the hash value saved in the mbuf, but failed
uint8_t rss_hash_default_key[] = {
0x2c, 0xc6, 0x81, 0xd1,
0x5b, 0xdb, 0xf4, 0xf7,
0xfc, 0xa2, 0x83, 0x19,
0xdb, 0x1a, 0x3e, 0x94,
0x6b, 0x9e, 0x38, 0xd9,
0x2c, 0x9c, 0x03, 0xd1,
0xad, 0x99, 0x44, 0xa7,
0xd9, 0x56, 0x3d, 0x59,
0x06, 0x3c, 0x25, 0xf3,
0xfc, 0x1f, 0xdc, 0x2a,
};
static inline uint32_t
do_softrss(struct rte_mbuf *m)
{
uint32_t input_len;
struct rte_ipv4_tuple ipv4_tuple;
char * pRawPacket = static_cast(rte_pktmbuf_mtod(pi_mbuf, void* ));
IpHeader * pIpHeader = (IpHeader *)(pRawPacket + offsetOfIp);
if(pIpHeader->GetVersion()==4)
{
ipv4_tuple.src_addr =
rte_be_to_cpu_32(pIpHeader->dwSrcAddressBigEndian);
ipv4_tuple.dst_addr =
rte_be_to_cpu_32(pIpHeader->dwDstAddressBigEndian);
ipv4_tuple.sport = *(uint16_t*)(pRawPacket + transportLayerOffset);
ipv4_tuple.dport = *(uint16_t*)(pRawPacket + transportLayerOffset+2);
input_len = RTE_THASH_V4_L3_LEN;
return rte_softrss_be((uint32_t *)&ipv4_tuple, input_len, rss_key_be);
}
return 0;
}
new_rss = do_softrss(mbuf_pointer);
std::cout<< std::hex << mbuf_pointer->hash.rss << " -> " << new_rss << std::dec
<< std::endl;
And i get a different value than the mbuf_pointer->hash.rss
5ed28a5c -> 33eb33eb
974c1896 -> 24e224e2
1edf1638 -> 21752175
8a54c19 -> 80638063
459a6f76 -> 1b351b35
1cdf1d1c -> e53be53b
I understand it is possible to do it, but i don't get the same value ***
ethtool -i ens5f0
driver: mlx5_core
version: 5.8-3.0.7
firmware-version: 22.32.2004 (MT_000437)
expansion-rom-version:
bus-info: :83:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
RE: rss calculation as the nic
Hi Ori
Thank you for your answer it helped me
But I can see the load balancing is not done to the same instance for both
directions
I would like to have ip + port pairs arrive to same instance
Now ip1+port1 -> ip2+port2 arrive to instance x and ip2+port2 -> ip1+port1
arrive to another instance
How can I solve this?
From: Ori Kam
Sent: Wednesday, February 14, 2024 6:07 PM
To: Yaron Illouz ; dev@dpdk.org; 'us...@dpdk.org'
Subject: RE: rss calculation as the nic
EXTERNAL EMAIL: Do not click links or attachments unless you recognize the
sender and know the content is safe
Hi Yaron,
I can see some possible issues with your code, please see below.
From: Yaron Illouz mailto:yar...@radcom.com>>
Sent: Saturday, February 3, 2024 7:03 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; 'us...@dpdk.org'
mailto:us...@dpdk.org>>
Subject: rss calculation as the nic
[Snip]
static inline uint32_t
do_softrss(struct rte_mbuf *m)
{
uint32_t input_len;
struct rte_ipv4_tuple ipv4_tuple;
char * pRawPacket = static_cast(rte_pktmbuf_mtod(pi_mbuf, void* ));
IpHeader * pIpHeader = (IpHeader *)(pRawPacket + offsetOfIp);
if(pIpHeader->GetVersion()==4)
{
ipv4_tuple.src_addr =
rte_be_to_cpu_32(pIpHeader->dwSrcAddressBigEndian);
ipv4_tuple.dst_addr =
rte_be_to_cpu_32(pIpHeader->dwDstAddressBigEndian);
[Ori] here you are converting BE to CPU
ipv4_tuple.sport = *(uint16_t*)(pRawPacket + transportLayerOffset);
ipv4_tuple.dport = *(uint16_t*)(pRawPacket + transportLayerOffset+2);
[Ori] here you keep the values as BE
input_len = RTE_THASH_V4_L3_LEN;
return rte_softrss_be((uint32_t *)&ipv4_tuple, input_len, rss_key_be);
[Ori] the above function expects to get everything in BE and you should also
call rte_convert_rss_key before the use of the above function.
}
return 0;
}
new_rss = do_softrss(mbuf_pointer);
std::cout<< std::hex << mbuf_pointer->hash.rss << " -> " << new_rss << std::dec
<< std::endl;
And i get a different value than the mbuf_pointer->hash.rss
5ed28a5c -> 33eb33eb
974c1896 -> 24e224e2
1edf1638 -> 21752175
8a54c19 -> 80638063
459a6f76 -> 1b351b35
1cdf1d1c -> e53be53b
I understand it is possible to do it, but i don't get the same value ***
ethtool -i ens5f0
driver: mlx5_core
version: 5.8-3.0.7
firmware-version: 22.32.2004 (MT_000437)
expansion-rom-version:
bus-info: :83:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
rte flow for arp packet
Hi Is there a way using rte_flow to send arp and ndp packet to specific rx queue with dpdk
RE: rte flow for arp packet
In the rte_flow_item_type I don’t see an entry for arp or ndp Is there another way to do it ? For ipv4 I did the following way pattern[0].type = RTE_FLOW_ITEM_TYPE_ETH; pattern[0].spec = NULL; pattern[1].type = RTE_FLOW_ITEM_TYPE_IPV4; pattern[1].spec = NULL; From: Ori Kam Sent: Wednesday, March 16, 2022 3:11 PM To: Yaron Illouz ; dev@dpdk.org Subject: RE: rte flow for arp packet EXTERNAL EMAIL: Do not click links or attachments unless you recognize the sender and know the content is safe Hi You need to create a rule that matches the arp/ndp packets and use the queue action. From: Yaron Illouz mailto:yar...@radcom.com>> Sent: Wednesday, March 16, 2022 2:42 PM To: dev@dpdk.org<mailto:dev@dpdk.org> Cc: Ori Kam mailto:or...@nvidia.com>> Subject: rte flow for arp packet Hi Is there a way using rte_flow to send arp and ndp packet to specific rx queue with dpdk
dpdk rte rss flow with sriov
Hi Is this possible to use dpdk rte rss flow with a vf sriov, or is it an option available only for real interface? I use dpdk 19.11 and a Mellanox card 100G with mlx5
RE: dpdk rte rss flow with sriov
Perhaps I didn’t ask correctly My vm have a vf sriov, The dpdk application will create x rx queues on the vf as it should have done for a pf I expect traffic to be spreaded between the rx queues according to rte_flow rss defined (RTE_FLOW_ACTION_TYPE_QUEUE or RTE_FLOW_ACTION_TYPE_RSS) Asaf: I am not asking about the RTE_FLOW_ACTION_TYPE_VF – When you said Mellanox support where you referring to RTE_FLOW_ACTION_TYPE_VF? From: Asaf Penso Sent: Tuesday, March 29, 2022 4:40 PM To: Yaron Illouz ; us...@dpdk.org; dev@dpdk.org Subject: RE: dpdk rte rss flow with sriov EXTERNAL EMAIL: Do not click links or attachments unless you recognize the sender and know the content is safe Mellanox NICs support RSS for PF/VF/SF. Regards, Asaf Penso From: Yaron Illouz mailto:yar...@radcom.com>> Sent: Tuesday, March 29, 2022 4:35 PM To: us...@dpdk.org<mailto:us...@dpdk.org>; dev@dpdk.org<mailto:dev@dpdk.org> Subject: dpdk rte rss flow with sriov Hi Is this possible to use dpdk rte rss flow with a vf sriov, or is it an option available only for real interface? I use dpdk 19.11 and a Mellanox card 100G with mlx5
sriov net_mlx5 error - Unexpected error in DR drop action support detection
Hi I am using dpd19.11 and having error when trying to use sriov vf of a mlx5 100G card net_mlx5: Unexpected error in DR drop action support detection The same code work for pf I am not trying to do filtering on nic, but just to read from the port, 1 rx queue, 1 tx queue Network devices using kernel driver === :0b:00.0 'MT27800 Family [ConnectX-5 Virtual Function] 1018' if=ens192 drv=mlx5_core unused=igb_uio :13:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens224 drv=vmxnet3 unused=igb_uio *Active* :1b:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens256 drv=vmxnet3 unused=igb_uio *Active* ethtool -i ens192 driver: mlx5_core version: 5.0-0 firmware-version: 16.29.1016 (HPE09) expansion-rom-version: bus-info: :0b:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes Here is a part of the initialization code struct rte_eth_conf port_conf; memset(&port_conf, 0, sizeof(port_conf)); port_conf.link_speeds = ETH_LINK_SPEED_AUTONEG; port_conf.rxmode.mq_mode = (pi_nNumRxQueues > 1 ? ETH_MQ_RX_RSS : ETH_MQ_RX_NONE); port_conf.rxmode.max_rx_pkt_len = nMaxPacketLen; //only used with jumbo_frame. For now, only allow packets which fit in mbufs port_conf.rxmode.offloads |= DEV_RX_OFFLOAD_JUMBO_FRAME; //Jumbo Frame Support enabled if(pi_sDriverName != "net_ixgbe_vf")//if not sriov port_conf.rxmode.offloads |= DEV_RX_OFFLOAD_KEEP_CRC; //Don't strip CRC port_conf.rxmode.offloads &= pi_devInfo.rx_offload_capa; //turn off unsupported offloads port_conf.rx_adv_conf.rss_conf.rss_key = NULL; //use default RSS key port_conf.rx_adv_conf.rss_conf.rss_hf = (pi_nNumRxQueues > 1) ? GetRSSTypes(pi_sDriverName) : 0; port_conf.fdir_conf.mode = RTE_FDIR_MODE_NONE; //flow director off port_conf.intr_conf.lsc = bLSC ? 1 : 0; //enable link status change interrupt. Without this int ret = rte_eth_dev_configure(pi_nPort, nRxQueues, nTxQueues, &port_conf); if (ret < 0) //failure //Set MTU - needed for some NICs (at least mlx5 and vmxnet3) to receive jumbo packets ret = rte_eth_dev_set_mtu(pi_nPort, nMaxPacketLen); if (ret < 0) //failure struct rte_eth_rxconf rx_conf; memset(&rx_conf, 0, sizeof(rx_conf)); rx_conf.rx_thresh.pthresh = DPDK_NIC_RX_PTHRESH; //ring prefetch threshold rx_conf.rx_thresh.hthresh = DPDK_NIC_RX_HTHRESH; //ring host threshold rx_conf.rx_thresh.wthresh = DPDK_NIC_RX_WTHRESH; //ring writeback threshold rx_conf.rx_free_thresh = DPDK_NIC_RX_FREE_THRESH; for(RU32 nQueue = 0; nQueue
RE: dpdk rte rss flow with sriov
]) EAL: lcore 10 is ready (tid=70478700;cpuset=[10]) EAL: lcore 14 is ready (tid=7fffee474700;cpuset=[14]) EAL: lcore 2 is ready (tid=74480700;cpuset=[2]) EAL: lcore 3 is ready (tid=73c7f700;cpuset=[3]) EAL: lcore 4 is ready (tid=7347e700;cpuset=[4]) EAL: lcore 15 is ready (tid=7fffedc73700;cpuset=[15]) EAL: lcore 12 is ready (tid=7fffef476700;cpuset=[12]) EAL: lcore 9 is ready (tid=70c79700;cpuset=[9]) EAL: lcore 7 is ready (tid=71c7b700;cpuset=[7]) EAL: lcore 8 is ready (tid=7147a700;cpuset=[8]) EAL: lcore 5 is ready (tid=72c7d700;cpuset=[5]) EAL: lcore 13 is ready (tid=7fffeec75700;cpuset=[13]) EAL: lcore 6 is ready (tid=7247c700;cpuset=[6]) EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 1024MB EAL: PCI device :0b:00.0 on NUMA socket -1 EAL: probe driver: 15b3:1018 net_mlx5 EAL: Mem event callback 'MLX5_MEM_EVENT_CB:(nil)' registered net_mlx5: checking device "mlx5_0" net_mlx5: PCI information matches for device "mlx5_0" net_mlx5: no E-Switch support detected net_mlx5: naming Ethernet device "0b:00.0" net_mlx5: DevX is NOT supported net_mlx5: enhanced MPW is supported net_mlx5: SWP support: 0 net_mlx5: min_single_stride_log_num_of_bytes: 6 net_mlx5: max_single_stride_log_num_of_bytes: 13 net_mlx5: min_single_wqe_log_num_of_strides: 9 net_mlx5: max_single_wqe_log_num_of_strides: 16 net_mlx5: supported_qpts: 256 net_mlx5: device supports Multi-Packet RQ net_mlx5: tunnel offloading is supported net_mlx5: MPLS over GRE/UDP tunnel offloading is not supported net_mlx5: checksum offloading is supported net_mlx5: maximum Rx indirection table size is 512 net_mlx5: VLAN stripping is supported net_mlx5: FCS stripping configuration is supported net_mlx5: enhanced MPS is enabled net_mlx5: port 0 MAC address is 00:50:56:a9:a9:3e net_mlx5: port 0 MTU is 2040 net_mlx5: port 0 forcing Ethernet interface up net_mlx5: min tx inline configured: 0 net_mlx5: Hash list with mlx5_0_flow_table size 0x1000 is created. net_mlx5: Hash list with mlx5_0_tags size 0x2000 is created. net_mlx5: port 0 flow maximum priority: 3 net_mlx5: metadata mode 0 net_mlx5: metadata MARK mask 00FF net_mlx5: metadata META mask net_mlx5: metadata reg_c0 mask net_mlx5: port 0 extensive metadata register is not supported net_mlx5: Unexpected error in DR drop action support detection EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) Succeeded to init eal net_mlx5: port 0 Tx queues number update: 0 -> 1 net_mlx5: port 0 Rx queues number update: 0 -> 1 Succeeded to configure port net_mlx5: port 0 adapter MTU set to 2040 Succeeded to set mtu port net_mlx5: port 0 configuring Rx queue 0 for 512 descriptors net_mlx5: port 0 maximum number of segments per packet: 1 net_mlx5: port 0 CRC stripping is disabled, 4 bytes will be subtracted from incoming frames to hide it net_mlx5: port 0 adding Rx queue 0 to list Succeeded to rx queue setup queue 0 testReceiver.cpp 103 net_mlx5: port 0 configuring queue 0 for 512 descriptors net_mlx5: port 0 adding Tx queue 0 to list Succeeded to tx queue setup queue 0 net_mlx5: port 0 starting device net_mlx5: port 0 Rx queues number update: 1 -> 1 net_mlx5: port 0 Tx queue 0 allocated and configured 512 WRs net_mlx5: port 0: uar_mmap_offset 0x306000 net_mlx5: port 0 Rx queue 0 registering mp pool having 1 chunks net_mlx5: port 0 creating a MR using address (0x15a9fd6c0) net_mlx5: device mlx5_0 inserting MR(0x15a9f4ec0) to global cache net_mlx5: inserted B-tree(0x17ffdcb60)[1], [0x14000, 0x18000) lkey=0x7a0d0100 net_mlx5: inserted B-tree(0x15a9fbfec)[1], [0x14000, 0x18000) lkey=0x7a0d0100 net_mlx5: port 0 Rx queue 0 allocated and configured 512 segments (max 512 packets) net_mlx5: port 0 device_attr.max_qp_wr is 32768 net_mlx5: port 0 device_attr.max_sge is 30 net_mlx5: port 0 rxq 0 updated with 0x7fffd688 net_mlx5: port 0 failed to set defaults flows net_mlx5: port 0 Rx queue 0 freeing WRs DPDK: Error starting or stopping port -12 From: Asaf Penso Sent: Friday, April 1, 2022 9:39 AM To: Yaron Illouz ; us...@dpdk.org; dev@dpdk.org Subject: Re: dpdk rte rss flow with sriov I understand your scenario. I want referring to action VF. Can you share the testpmd commands line you use in the VM? Regards, Asaf Penso From: Yaron Illouz mailto:yar...@radcom.com>> Sent: Tuesday, March 29, 2022 5:33:39 PM To: Asaf Penso mailto:as...@nvidia.com>>; us...@dpdk.org<mailto:us...@dpdk.org> mailto:us...@dpdk.org>>; dev@dpdk.org<mailto:dev@dpdk.org> mailto:dev@dpdk.org>> Subject: RE: dpdk rte rss flow with sriov Perhaps I didn’t ask correctly My vm have a vf sriov, The dpdk application will create x rx queues on the v
[dpdk-dev] compile dpdk with no avx
Hi I use dpdk in my product. I can't change my dpdk version. I compile it on a server that have avx. Most of the time, it run on machine that have avx, but not always. When running the product on a machine with no avx, it crash on illegal instruction. The current solution is to build it on a computer with no avx. Is there a compilation flag to disable avx? So it doesn't matter on which computer I compiled it. Compile flag defconfig_x86_64-default-linuxapp-gcc Dpdk version PACKAGE_VERSION_MAJOR_NUMBER=1 PACKAGE_VERSION_MINOR_NUMBER=6 PACKAGE_VERSION_PATCH_NUMBER=0 PACKAGE_VERSION_BUILD_NUMBER=18 Gcc version gcc (GCC) 4.4.6 20120305 (Red Hat 4.4.6-4) OS CentOS release 6.3 (Final)
[dpdk-dev] vlan id from ixgbevf
Hi I am using the dpdk igb_uio in vm instead of ixgbevf To get the mac address from code I call function rte_eth_macaddr_get How do I get the vlan id assigned to the vf I am using, is there a similar function?
[dpdk-dev] patch for dpdk-devbind.py support ipv6
Hi
dpdk-devbind.py support only ipv4 interface, when run with -s parameter which
is status, it display all ipv6 interface as non Active
It set interface as Active according to "ip -o route" output
Here is a change that can support ipv6
@@ -269,6 +269,18 @@
# filter out all lines for 169.254 routes
route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
route.decode().splitlines()))
+
+routev6 = check_output(["ip", "-6", "-o", "route"])
+# filter out all lines for 169.254 routes
+routev6 = "\n".join(filter(lambda ln: not ln.startswith("unreachable") and
not ln.startswith("fe80::/64"),
+ routev6.decode().splitlines()))
+
+if len(routev6):
+if len(route):
+route = route+"\n"+routev6
+else:
+route = routev6
+
rt_info = route.split()
for i in range(len(rt_info) - 1):
if rt_info[i] == "dev":
@@ -654,3 +666,4 @@
Yaron Illouz
[dpdk-dev] dpdk kni ping answer
Hi I want to give an ip to a dpdk port, because I need to receive traffic by gre replication. I though I could do it with dpdk kni (http://dpdk.org/doc/guides/prog_guide/kernel_nic_interface.html) I am able to give an ip to interface vEth0, but I can't ping or ssh to the ip address. I set the ip with the following command: ifconfig vEth0 192.168.69.4 netmask 255.255.255.0 up Is this something possible to do with dpdk kni? If not, what could be another solution for this. Regards
[dpdk-dev] dpdk kni ping answer
The source of my problem seems to be that Ethtool is not supported in VMs (VF). Is there another solution to give an ip to a dpdk port? -Original Message- From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Yaron Illouz Sent: Monday, March 23, 2015 2:08 PM To: dev at dpdk.org; dev Subject: [dpdk-dev] dpdk kni ping answer Hi I want to give an ip to a dpdk port, because I need to receive traffic by gre replication. I though I could do it with dpdk kni (http://dpdk.org/doc/guides/prog_guide/kernel_nic_interface.html) I am able to give an ip to interface vEth0, but I can't ping or ssh to the ip address. I set the ip with the following command: ifconfig vEth0 192.168.69.4 netmask 255.255.255.0 up Is this something possible to do with dpdk kni? If not, what could be another solution for this. Regards
[dpdk-dev] ixgbevf missed stats
Hi There is no statistics of missed packet for ixgbevf in a vm. How can I get these stats? In ixgbe it is called imissed
[dpdk-dev] ixgbevf missed stats
So I can't know if there where dropps!!! Is this the same for dpdk with virtio? Can I know if there where dropps? -Original Message- From: Lu, Wenzhuo [mailto:wenzhuo...@intel.com] Sent: Thursday, March 10, 2016 2:40 PM To: Yaron Illouz ; dev at dpdk.org Subject: RE: ixgbevf missed stats Hi Yaron, The stats of PF is done by HW. The bad news is I don' find anything for VF. I also doubt if there's corresponding stats for VF. > -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Yaron Illouz > Sent: Thursday, March 10, 2016 8:27 PM > To: dev at dpdk.org > Subject: [dpdk-dev] ixgbevf missed stats > > Hi > > There is no statistics of missed packet for ixgbevf in a vm. > How can I get these stats? > In ixgbe it is called imissed
[dpdk-dev] Is mbuf detached from pool?
Hi I had a check in code for mbuf double release I used to check if the reference count <= 0 Since the following change i can't check anymore this way since the reference count is 1 even after it is release to the pool http://mails.dpdk.org/archives/dev/2017-January/056188.html I there a way to check if the mbuf is detached(in the pool or out of the pool) instead of looking at ref count?
