[dpdk-dev] [PATCH v2] crypto/mrvl: rename PMD to mvsam

[Tomasz Duszynski]

Picking a company stock ticker for a PMD name might not be a best approach
in a long run since name is too generic.

This patch addresses that and renames mrvl to mvsam.

Signed-off-by: Natalie Samsonov 
Signed-off-by: Tomasz Duszynski 
---
v2:
 * fix test-build script
 * use proper define in test_cryptodev

 MAINTAINERS|  6 +++---
 config/common_base |  4 ++--
 devtools/test-build.sh |  2 +-
 .../cryptodevs/features/{mrvl.ini => mvsam.ini}|  2 +-
 doc/guides/cryptodevs/index.rst|  2 +-
 doc/guides/cryptodevs/{mrvl.rst => mvsam.rst}  | 24 +++---
 doc/guides/rel_notes/release_17_11.rst |  2 +-
 doc/guides/tools/cryptoperf.rst|  2 +-
 drivers/crypto/Makefile|  2 +-
 drivers/crypto/meson.build |  2 +-
 drivers/crypto/{mrvl => mvsam}/Makefile|  8 
 drivers/crypto/{mrvl => mvsam}/meson.build |  0
 drivers/crypto/{mrvl => mvsam}/rte_mrvl_compat.h   |  0
 drivers/crypto/{mrvl => mvsam}/rte_mrvl_pmd.c  |  0
 drivers/crypto/{mrvl => mvsam}/rte_mrvl_pmd_ops.c  |  0
 .../crypto/{mrvl => mvsam}/rte_mrvl_pmd_private.h  |  2 +-
 .../rte_pmd_mvsam_version.map} |  0
 mk/rte.app.mk  |  2 +-
 test/test/meson.build  |  2 +-
 test/test/test_cryptodev.c | 10 -
 test/test/test_cryptodev.h |  2 +-
 21 files changed, 37 insertions(+), 37 deletions(-)
 rename doc/guides/cryptodevs/features/{mrvl.ini => mvsam.ini} (93%)
 rename doc/guides/cryptodevs/{mrvl.rst => mvsam.rst} (88%)
 rename drivers/crypto/{mrvl => mvsam}/Makefile (80%)
 rename drivers/crypto/{mrvl => mvsam}/meson.build (100%)
 rename drivers/crypto/{mrvl => mvsam}/rte_mrvl_compat.h (100%)
 rename drivers/crypto/{mrvl => mvsam}/rte_mrvl_pmd.c (100%)
 rename drivers/crypto/{mrvl => mvsam}/rte_mrvl_pmd_ops.c (100%)
 rename drivers/crypto/{mrvl => mvsam}/rte_mrvl_pmd_private.h (98%)
 rename drivers/crypto/{mrvl/rte_pmd_mrvl_version.map => 
mvsam/rte_pmd_mvsam_version.map} (100%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 645bd52..57460d0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -735,9 +735,9 @@ M: Tomasz Duszynski 
 M: Dmitri Epshtein 
 M: Natalie Samsonov 
 M: Jianbo Liu 
-F: drivers/crypto/mrvl/
-F: doc/guides/cryptodevs/mrvl.rst
-F: doc/guides/cryptodevs/features/mrvl.ini
+F: drivers/crypto/mvsam/
+F: doc/guides/cryptodevs/mvsam.rst
+F: doc/guides/cryptodevs/features/mvsam.ini

 Null Crypto
 M: Declan Doherty 
diff --git a/config/common_base b/config/common_base
index 7e45412..f5e04a4 100644
--- a/config/common_base
+++ b/config/common_base
@@ -560,8 +560,8 @@ CONFIG_RTE_LIBRTE_PMD_CCP_CPU_AUTH=n
 #
 # Compile PMD for Marvell Crypto device
 #
-CONFIG_RTE_LIBRTE_PMD_MRVL_CRYPTO=n
-CONFIG_RTE_LIBRTE_PMD_MRVL_CRYPTO_DEBUG=n
+CONFIG_RTE_LIBRTE_PMD_MVSAM_CRYPTO=n
+CONFIG_RTE_LIBRTE_PMD_MVSAM_CRYPTO_DEBUG=n

 #
 # Compile generic security library
diff --git a/devtools/test-build.sh b/devtools/test-build.sh
index 12d4ce6..ebc426c 100755
--- a/devtools/test-build.sh
+++ b/devtools/test-build.sh
@@ -202,7 +202,7 @@ config () #   
sed -ri 's,(BBDEV_TURBO_SW=)n,\1y,' $1/.config
sed -ri   's,(SCHED_.*=)n,\1y,' $1/.config
test -z "$LIBMUSDK_PATH" || \
-   sed -ri's,(PMD_MRVL_CRYPTO=)n,\1y,' $1/.config
+   sed -ri's,(PMD_MVSAM_CRYPTO=)n,\1y,' $1/.config
test -z "$LIBMUSDK_PATH" || \
sed -ri  's,(MVPP2_PMD=)n,\1y,' $1/.config
build_config_hook $1 $2 $3
diff --git a/doc/guides/cryptodevs/features/mrvl.ini 
b/doc/guides/cryptodevs/features/mvsam.ini
similarity index 93%
rename from doc/guides/cryptodevs/features/mrvl.ini
rename to doc/guides/cryptodevs/features/mvsam.ini
index 6d2fe6a..b7c105a 100644
--- a/doc/guides/cryptodevs/features/mrvl.ini
+++ b/doc/guides/cryptodevs/features/mvsam.ini
@@ -1,4 +1,4 @@
-; Supported features of the 'mrvl' crypto driver.
+; Supported features of the 'mvsam' crypto driver.
 ;
 ; Refer to default.ini for the full list of available PMD features.
 ;
diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst
index 0529583..e9928a4 100644
--- a/doc/guides/cryptodevs/index.rst
+++ b/doc/guides/cryptodevs/index.rst
@@ -18,7 +18,7 @@ Crypto Device Drivers
 dpaa_sec
 kasumi
 openssl
-mrvl
+mvsam
 null
 scheduler
 snow3g
diff --git a/doc/guides/cryptodevs/mrvl.rst b/doc/guides/cryptodevs/mvsam.rst
similarity index 88%
rename from doc/guides/cryptodevs/mrvl.rst
rename to doc/guides/cryptodevs/mvsam.rst
index 443ebcd..fd418c2 100644
--- a/doc/guides/cryptodevs/mrvl.rst
+++ b/doc/guides/cryptodevs/mvsam.rst
@@ -29,10 +29,10 @@
 (INCLUDING NEGLIGENCE OR OTHERWISE) 

Re: [dpdk-dev] [PATCH v7 1/2] mbuf: support attaching external buffer to mbuf

[Andrew Rybchenko]

On 04/27/2018 03:01 AM, Yongseok Koh wrote:

This patch introduces a new way of attaching an external buffer to a mbuf.

Attaching an external buffer is quite similar to mbuf indirection in
replacing buffer addresses and length of a mbuf, but a few differences:
   - When an indirect mbuf is attached, refcnt of the direct mbuf would be
 2 as long as the direct mbuf itself isn't freed after the attachment.
 In such cases, the buffer area of a direct mbuf must be read-only. But
 external buffer has its own refcnt and it starts from 1. Unless
 multiple mbufs are attached to a mbuf having an external buffer, the
 external buffer is writable.
   - There's no need to allocate buffer from a mempool. Any buffer can be
 attached with appropriate free callback.
   - Smaller metadata is required to maintain shared data such as refcnt.

Signed-off-by: Yongseok Koh 
Acked-by: Konstantin Ananyev 
Acked-by: Olivier Matz 


Many thanks,
Acked-by: Andrew Rybchenko 

Re: [dpdk-dev] [PATCH v3] net/i40e:fix missing jumbo frame offload capability

[Zhang, Helin]

> -Original Message-
> From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Yanglong Wu
> Sent: Thursday, April 19, 2018 10:19 AM
> To: dev@dpdk.org
> Cc: Zhang, Qi Z; Dai, Wei; Yao, Lei A; Wu, Yanglong
> Subject: [dpdk-dev] [PATCH v3] net/i40e:fix missing jumbo frame offload
> capability
> 
> JUMBO_FRAME offload capability should
> be exposed since i40e does support it.
> 
> Fixes: fbf959fe61e7 ("net/i40e: convert to new Rx offloads API")
> Signed-off-by: Yanglong Wu 
> Acked-by: Qi Zhang 
Applied to dpdk-next-net-intel, thanks!

/Helin

[dpdk-dev] [PATCH] net/sfc: add missing case to FW variant mapping on get

[Andrew Rybchenko]

Fixes: 7e2dc79b4be7 ("net/sfc: support DPDK firmware variant")

Signed-off-by: Andrew Rybchenko 
Reviewed-by: Ivan Malov 
---
 drivers/net/sfc/sfc.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/net/sfc/sfc.c b/drivers/net/sfc/sfc.c
index d1abf62a2..6690053f2 100644
--- a/drivers/net/sfc/sfc.c
+++ b/drivers/net/sfc/sfc.c
@@ -867,6 +867,10 @@ sfc_get_fw_variant(struct sfc_adapter *sa, 
efx_fw_variant_t *efv)
*efv = EFX_FW_VARIANT_PACKED_STREAM;
break;
 
+   case EFX_RXDP_DPDK_FW_ID:
+   *efv = EFX_FW_VARIANT_DPDK;
+   break;
+
default:
/*
 * Other firmware variants are not considered, since they are
-- 
2.14.1

Re: [dpdk-dev] [PATCH v3 4/4] ethdev: Add metadata flow and action items support

[Xueming(Steven) Li]

Hi Declan,

Is there a mapping between maetadata and mbuf field(s), or the lifecycle of 
metadata supposed to 
be in middle of flows?

There might be another use case that set metadata to a constant, is this 
covered?

Thanks,
Xueming

> -Original Message-
> From: dev  On Behalf Of Declan Doherty
> Sent: Friday, April 6, 2018 8:24 PM
> To: dev@dpdk.org
> Cc: Declan Doherty 
> Subject: [dpdk-dev] [PATCH v3 4/4] ethdev: Add metadata flow and action items 
> support
> 
> Introduces a new action type RTE_FLOW_ACTION_TYPE_METADATA which enables 
> metadata extraction from a
> packet into a specified metadata container for consumption on further 
> pipeline stages or for
> propagation to the host interface.
> 
> As complementary function to the new metadata action type this patch also 
> introduces a new flow item
> type which enables flow patterns to specify a specific metadata container as 
> a matching criteria for a
> flow rule.
> 
> Signed-off-by: Declan Doherty 
> ---
>  doc/guides/prog_guide/rte_flow.rst | 85 
> ++
>  lib/librte_ether/rte_flow.h| 42 +++
>  2 files changed, 127 insertions(+)
> 
> diff --git a/doc/guides/prog_guide/rte_flow.rst 
> b/doc/guides/prog_guide/rte_flow.rst
> index 2f0a47a..9b2b0e3 100644
> --- a/doc/guides/prog_guide/rte_flow.rst
> +++ b/doc/guides/prog_guide/rte_flow.rst
> @@ -1580,6 +1580,91 @@ group on that device.
> +--+-+
> 
> 
> +
> +Action: ``METADATA``
> +
> +
> +Action extracts data from packet into user specified metadata field in
> +pipeline for use in downstream processing stages or for propagation to host 
> interface.
> +
> +The pattern mask is used to define the data which is to be extracted
> +from the packet. The mask pattern types defined in the action metadata
> +pattern must match the flow pattern definitions up to the last flow
> +item from which data is to be extracted.
> +
> +- Non-terminating by default.
> +
> +.. _table_rte_flow_action_metadata:
> +
> +.. table:: METADATA
> +
> +   +--+---+
> +   | Field| Value |
> +   +==+===+
> +   | ``id``   | Metadata field Identifier |
> +   +--+---+
> +   | ``pattern``  | Extraction mask pattern   |
> +   +--+---+
> +
> +The example below demonstrates how the extraction mask to extract the
> +source/ destination IPv4 address, the UDP destination port and and the
> +VxLAN VNI can be specified.
> +
> +.. _table_rte_flow_action_metadata_example:
> +
> +.. table:: IPv4 VxLAN metadata extraction
> +
> +   +---+--+---+
> +   | Index | Flow Item Type   | Flow Mask |
> +   +===+==+===+
> +   | 0 | RTE_FLOW_ITEM_TYPE_ETH   | .dst = "\x00\x00\x00\x00\x00\x00" |
> +   |   |  +---+
> +   |   |  | .src = "\x00\x00\x00\x00\x00\x00" |
> +   |   |  +---+
> +   |   |  | .type = RTE_BE16(0x0) |
> +   +---+--+---+
> +   | 1 | RTE_FLOW_ITEM_TYPE_IPV4  | .src_addr = RTE_BE32(0x)  |
> +   |   |  +---+
> +   |   |  | .dst_addr = RTE_BE32(0x)  |
> +   +---+--+---+
> +   | 2 | RTE_FLOW_ITEM_TYPE_UDP   | .src_port = RTE_BE16(0x0) |
> +   |   |  +---+
> +   |   |  | .dst_port = RTE_BE16(0x)  |
> +   +---+--+---+
> +   | 3 | RTE_FLOW_ITEM_TYPE_VXLAN | .vni = "\xff\xff\xff" |
> +   +---+--+---+
> +   | 4 | RTE_FLOW_ITEM_TYPE_END   | NULL  |
> +
> + +---+--+--
> + -+
> +
> +If only the VxLAN VNI extraction was required then the extraction mask
> +would be as follows.
> +
> +.. _table_rte_flow_action_metadata_example_2:
> +
> +.. table::  VxLAN VNI metadata extraction
> +
> +   +---+--+---+
> +   | Index | Flow Item Type   | Flow Mask |
> +   +===+==+===+
> +   | 0 | RTE_FLOW_ITEM_TYPE_ETH   | .dst = "\x00\x00\x00\x00\x00\x00" |
> +   |   |  +---

Re: [dpdk-dev] [PATCH v7 2/2] app/testpmd: conserve offload flags of mbuf

[Andrew Rybchenko]

On 04/27/2018 03:01 AM, Yongseok Koh wrote:

This patch is to accommodate an experimental feature of mbuf - external
buffer attachment. If mbuf is attached to an external buffer, its ol_flags
will have EXT_ATTACHED_MBUF set. Without enabling/using the feature,
everything remains same.

If PMD delivers Rx packets with non-direct mbuf, ol_flags should not be
overwritten. For mlx5 PMD, if Multi-Packet RQ is enabled, Rx packets could
be carried with externally attached mbufs.

Signed-off-by: Yongseok Koh 
---
  app/test-pmd/csumonly.c | 3 +++
  app/test-pmd/macfwd.c   | 3 +++
  app/test-pmd/macswap.c  | 3 +++
  3 files changed, 9 insertions(+)

diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c
index 53b98412a..4a82bbc92 100644
--- a/app/test-pmd/csumonly.c
+++ b/app/test-pmd/csumonly.c
@@ -851,6 +851,9 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
m->l4_len = info.l4_len;
m->tso_segsz = info.tso_segsz;
}
+   if (!RTE_MBUF_DIRECT(m))
+   tx_ol_flags |= m->ol_flags &
+   (IND_ATTACHED_MBUF | EXT_ATTACHED_MBUF);


1. I see no point to check !RTE_MBUF_DIRECT(m). Just inherit flags.
2. Consider to do it when tx_ol_flags are initialized above as 0, i.e.
   tx_ol_flags = m->ol_flags & (IND_ATTACHED_MBUF | EXT_ATTACHED_MBUF);


m->ol_flags = tx_ol_flags;
  
  		/* Do split & copy for the packet. */

diff --git a/app/test-pmd/macfwd.c b/app/test-pmd/macfwd.c
index 2adce7019..ba0021194 100644
--- a/app/test-pmd/macfwd.c
+++ b/app/test-pmd/macfwd.c
@@ -96,6 +96,9 @@ pkt_burst_mac_forward(struct fwd_stream *fs)
ð_hdr->d_addr);
ether_addr_copy(&ports[fs->tx_port].eth_addr,
ð_hdr->s_addr);
+   if (!RTE_MBUF_DIRECT(mb))
+   ol_flags |= mb->ol_flags &
+   (IND_ATTACHED_MBUF | EXT_ATTACHED_MBUF);


1. Do not update ol_flags which is global and applied for all mbufs in 
the burst.
2. There is no point to check for  (!RTE_MBUF_DIRECT(mb). Just inherit 
these flags.



mb->ol_flags = ol_flags;
mb->l2_len = sizeof(struct ether_hdr);
mb->l3_len = sizeof(struct ipv4_hdr);
diff --git a/app/test-pmd/macswap.c b/app/test-pmd/macswap.c
index e2cc4812c..b8d15f6ba 100644
--- a/app/test-pmd/macswap.c
+++ b/app/test-pmd/macswap.c
@@ -127,6 +127,9 @@ pkt_burst_mac_swap(struct fwd_stream *fs)
ether_addr_copy(ð_hdr->s_addr, ð_hdr->d_addr);
ether_addr_copy(&addr, ð_hdr->s_addr);
  
+		if (!RTE_MBUF_DIRECT(mb))

+   ol_flags |= mb->ol_flags &
+   (IND_ATTACHED_MBUF | EXT_ATTACHED_MBUF);


Same as above.k


mb->ol_flags = ol_flags;
mb->l2_len = sizeof(struct ether_hdr);
mb->l3_len = sizeof(struct ipv4_hdr);


With above fixed:
Acked-by: Andrew Rybchenko 

[dpdk-dev] [PATCH v2] net/i40e: fix checking offloading fail for Tx

[Yanglong Wu]

Missing "return -ENOTSUP" will alays leads to
illegal offload passing through offload checking.

Fixes: 7497d3e2f777 ("net/i40e: convert to new Tx offloads API")
Signed-off-by: Yanglong Wu 
---
v2:
rework fix line 
---
 drivers/net/i40e/i40e_rxtx.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index bc660596b..2ce930827 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -2038,7 +2038,9 @@ i40e_dev_tx_queue_setup(struct rte_eth_dev *dev,
" or supported offloads 0x%" PRIx64,
(void *)dev, tx_conf->offloads,
dev->data->dev_conf.txmode.offloads,
-   dev_info.tx_offload_capa); }
+   dev_info.tx_offload_capa);
+   return -ENOTSUP;
+   }
 
if (hw->mac.type == I40E_MAC_VF || hw->mac.type == I40E_MAC_X722_VF) {
vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
-- 
2.11.0

[dpdk-dev] [PATCH v2] net/i40e: fix missing some offload capabilities

[Yanglong Wu]

MULTI_SEGS and JUMBO_FRAME offload capability
should be exposed both VF and PF since i40e
does support it.

Fixes: c3ac7c5b0b8a ("net/i40e: convert to new Rx offloads API")
Fixes: 7497d3e2f777 ("net/i40e: convert to new Tx offloads API")
Signed-off-by: Yanglong Wu 
---
v2:
rework fix line
---
 drivers/net/i40e/i40e_ethdev.c| 3 ++-
 drivers/net/i40e/i40e_ethdev_vf.c | 6 --
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index c3bd254ae..d069a3ebe 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -3244,7 +3244,8 @@ i40e_dev_info_get(struct rte_eth_dev *dev, struct 
rte_eth_dev_info *dev_info)
DEV_TX_OFFLOAD_VXLAN_TNL_TSO |
DEV_TX_OFFLOAD_GRE_TNL_TSO |
DEV_TX_OFFLOAD_IPIP_TNL_TSO |
-   DEV_TX_OFFLOAD_GENEVE_TNL_TSO;
+   DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
+   DEV_TX_OFFLOAD_MULTI_SEGS;
dev_info->hash_key_size = (I40E_PFQF_HKEY_MAX_INDEX + 1) *
sizeof(uint32_t);
dev_info->reta_size = pf->hash_lut_size;
diff --git a/drivers/net/i40e/i40e_ethdev_vf.c 
b/drivers/net/i40e/i40e_ethdev_vf.c
index 031c70680..bf9e14ee3 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -2200,7 +2200,8 @@ i40evf_dev_info_get(struct rte_eth_dev *dev, struct 
rte_eth_dev_info *dev_info)
DEV_RX_OFFLOAD_TCP_CKSUM |
DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM |
DEV_RX_OFFLOAD_CRC_STRIP |
-   DEV_RX_OFFLOAD_SCATTER;
+   DEV_RX_OFFLOAD_SCATTER |
+   DEV_RX_OFFLOAD_JUMBO_FRAME;
 
dev_info->tx_queue_offload_capa = 0;
dev_info->tx_offload_capa =
@@ -2215,7 +2216,8 @@ i40evf_dev_info_get(struct rte_eth_dev *dev, struct 
rte_eth_dev_info *dev_info)
DEV_TX_OFFLOAD_VXLAN_TNL_TSO |
DEV_TX_OFFLOAD_GRE_TNL_TSO |
DEV_TX_OFFLOAD_IPIP_TNL_TSO |
-   DEV_TX_OFFLOAD_GENEVE_TNL_TSO;
+   DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
+   DEV_TX_OFFLOAD_MULTI_SEGS;
 
dev_info->default_rxconf = (struct rte_eth_rxconf) {
.rx_thresh = {
-- 
2.11.0

Re: [dpdk-dev] [PATCH] net/sfc: add missing case to FW variant mapping on get

[Ferruh Yigit]

On 4/27/2018 8:31 AM, Andrew Rybchenko wrote:
> Fixes: 7e2dc79b4be7 ("net/sfc: support DPDK firmware variant")
> 
> Signed-off-by: Andrew Rybchenko 
> Reviewed-by: Ivan Malov 

Squashed into relevant commit in next-net, thanks.

Re: [dpdk-dev] [PATCH v4 0/4] lib/rib: Add Routing Information Base library

[Vladimir Medvedkin]

Hi Stephen,


2018-04-27 1:24 GMT+03:00 Stephen Hemminger :

> On Fri, 27 Apr 2018 01:03:30 +0300
> Medvedkin Vladimir  wrote:
>
> > This patch series introduces new library librte_rib which potentially
> could
> > replace librte_lpm.
> >
> > RIB is an alternative to current LPM library.
> > It solves the following problems
> >  - Increases the speed of control plane operations against lpm such as
> >adding/deleting routes
> >  - Adds abstraction from dataplane algorithms, so it is possible to add
> >different ip route lookup algorythms such as DXR/poptrie/lpc-trie/etc
> >in addition to current dir24_8
> >  - It is possible to keep user defined application specific additional
> >information in struct rte_rib_node which represents route entry.
> >It can be next hop/set of next hops (i.e. active and feasible),
> >pointers to link rte_rib_node based on some criteria (i.e. next_hop),
> >plenty of additional control plane information.
> >
> > v4:
> >   fix various bugs
> >   make struct rte_rib opaque
> >   make inline functions instead of macro
> >   remove RTE_RIB_MALLOC node allocation type
> >   add new lookup functions
> >   remove rte_dir24_8_lookup()
> >   add rte_dir24_8_get_lookup()
> >   add meson support
> >   add fib configuration
> >
> >
> > Medvedkin Vladimir (4):
> >   Add RIB library
> >   Add dir24_8 implementation for rib library
> >   Add autotests for RIB library
>
> The existing DPDK LPM code does need more work it does trade space for
> time.
>
> It does have some advantages though:
> * LPM lookup table is independent of number of routes.
>
To be honest it depends on number of prefixes longer 24. But despite this
my implementation is independent of number of routes too.

> * LPM lookup table can be lock free reader safe using RCU.
>
As I see you are using rcu only for tbl8's growing. This feature was out of
scope and could be added if needed. But I'm afraid about license
constraints.

>
> The existing slowness of add and delete was fixed at Vyatta/Brocade by
> replacing list with red-black tree. Patches were submitted but never
> merged.
>
Could you please share lpm_perf_autotest results for your version?
Here my results for lpm_perf_autotest and rib_perf_autotest (sizeof key ==
sizeof(struct rte_lpm_tbl_entry) so memory footprint is the same)
LPM:
Unique added entries = 1037026
Used table 24 entries = 14680064 (87.5%)
64 byte Cache entries used = 458753 (29360192 bytes)
Average LPM Add: 306566 cycles
Average LPM Lookup: 29.7 cycles (fails = 0.0%)
BULK LPM Lookup: 29.4 cycles (fails = 0.0%)
LPM LookupX4: 27.1 cycles (fails = 0.0%)
Average LPM Delete: 201360 cycles
Test OK
RIB:
Unique added entries = 1076816
Average RIB Add: 3149.62 cycles
BULK RIB Lookup: 23.8 cycles (fails = 0.0%)
Average RIB Delete: 2949.62 cycles
Test OK

As you can see for 1M+ routes I have 100 times faster add and 70 times
faster delete. Lookup speed is faster too.
It was achieved due to:
1. routes are kept in compressed binary tree so rule insertion/deletion are
cheap comparing to flat array in LPM. And yes, rb_tree implementation fixes
that problem too.
2. routes are kept in compressed binary tree so you can get less specific
prefix (parent node) for a given prefix (you want add/del to) and compare
it's next hop to a given next hop. There is no need to touch fib table if
they are equal.
3. routes are kept in compressed binary tree so you can traverse on some
part of that tree to get subprefixes for a given prefix (you want add/del
to). Hence you can get gaps between retrieved more specific routes and
based on this gaps unconditionally rewrite part of fib table. Comparing to
LPM where every write to tbl24 or tbl8 is accompanied by a comparison of
depth. In addition there is no need to keep depth in fib anymore.
4. LPM tbl8_alloc is expensive due to memory scan through tbl8 to find a
free tbl8 group. More effective to keep free tbl8 indexes in bitmap.

-- 
Regards,
Vladimir

Re: [dpdk-dev] [PATCH 0/4] support for write combining

[Rafał Kozik]

Hello Bruce,

As from my last post passed ten days I would kindly ask if there are
any more comments?

Best regards,
Rafal Kozik

Re: [dpdk-dev] [PATCH v2] net/i40e: fix checking offloading fail for Tx

[Zhang, Helin]

> -Original Message-
> From: Wu, Yanglong
> Sent: Friday, April 27, 2018 4:14 PM
> To: dev@dpdk.org
> Cc: Zhang, Helin; Zhang, Qi Z; Dai, Wei; Wu, Yanglong
> Subject: [PATCH v2] net/i40e: fix checking offloading fail for Tx
> 
> Missing "return -ENOTSUP" will alays leads to illegal offload passing through
> offload checking.
> 
> Fixes: 7497d3e2f777 ("net/i40e: convert to new Tx offloads API")
> Signed-off-by: Yanglong Wu 
Acked-by: Helin Zhang 

Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after freeing

[Akhil Goyal]

Hi Pablo,

On 4/26/2018 8:39 PM, Pablo de Lara wrote:

When freeing memory, pointers should be set to NULL,
to avoid memory corruption/segmentation faults.


Shouldn't this be handled in the rte_free itself. A lot of other driver 
are also not setting null after rte_free.
This would require change at a lot of places if this is not handled in 
rte_free.


Thanks,
Akhil

[dpdk-dev] [PATCH 1/1] doc: update doc and release notes for szedata2 driver

[Matej Vido]

New version of the packages with dependencies for the szedata2
driver is needed due to the new API of the libsze2 library which
is used in the driver.
The documentation and the release notes are updated to contain
the information about the required versions.

Signed-off-by: Matej Vido 
Acked-by: Jan Remes 
---
 doc/guides/nics/szedata2.rst   | 9 +
 doc/guides/rel_notes/release_18_05.rst | 4 
 2 files changed, 13 insertions(+)

diff --git a/doc/guides/nics/szedata2.rst b/doc/guides/nics/szedata2.rst
index 96fed30..a34fffc 100644
--- a/doc/guides/nics/szedata2.rst
+++ b/doc/guides/nics/szedata2.rst
@@ -54,6 +54,15 @@ separately:
 Information about getting the dependencies can be found `here
 `_.
 
+Versions of the packages
+~~~
+
+The minimum version of the provided packages:
+
+* for DPDK from 18.05: **4.4.1**
+
+* for DPDK up to 18.02 (including): **3.0.5**
+
 Configuration
 -
 
diff --git a/doc/guides/rel_notes/release_18_05.rst 
b/doc/guides/rel_notes/release_18_05.rst
index 3d18298..cce82ba 100644
--- a/doc/guides/rel_notes/release_18_05.rst
+++ b/doc/guides/rel_notes/release_18_05.rst
@@ -81,6 +81,10 @@ New Features
 * **Updated szedata2 PMD.**
 
   Added support for new NFB-200G2QL card.
+  New API was introduced in the libsze2 library which the szedata2 PMD depends
+  on thus the new version of the library was needed.
+  New versions of the packages are available and the minimum required version
+  is 4.4.1.
 
 * **Added support for Broadcom NetXtreme-S (BCM58800) family of controllers 
(aka Stingray)**
 
-- 
2.7.4

[dpdk-dev] [PATCH] net: fix meson build

[Ferruh Yigit]

librte_net started to use experimental APIs, this explicity allowed by
make build system, but not for meson.

Fixes: b0aae17c3e4a ("net: export IPv6 header extensions skip function")
Cc: didier.pall...@6wind.com

Signed-off-by: Ferruh Yigit 
---
 lib/librte_net/meson.build | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/librte_net/meson.build b/lib/librte_net/meson.build
index 78c0f03e5..d3ea1feb5 100644
--- a/lib/librte_net/meson.build
+++ b/lib/librte_net/meson.build
@@ -2,6 +2,7 @@
 # Copyright(c) 2017 Intel Corporation
 
 version = 1
+allow_experimental_apis = true
 headers = files('rte_ip.h',
'rte_tcp.h',
'rte_udp.h',
-- 
2.14.3

[dpdk-dev] [PATCH] ethdev: fix meson build

[Ferruh Yigit]

kvargs dependency added to ehtdev which is not updated in meson build

Fixes: 9d5c2725ed8c ("ethdev: add common devargs parser")
Cc: remy.hor...@intel.com

Signed-off-by: Ferruh Yigit 
---
 lib/librte_ethdev/meson.build | 2 +-
 lib/meson.build   | 3 +--
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build
index 12bdb6b61..aed5d2265 100644
--- a/lib/librte_ethdev/meson.build
+++ b/lib/librte_ethdev/meson.build
@@ -24,4 +24,4 @@ headers = files('rte_ethdev.h',
'rte_tm.h',
'rte_tm_driver.h')
 
-deps += ['net']
+deps += ['net', 'kvargs']
diff --git a/lib/meson.build b/lib/meson.build
index 0d58c556f..166905c1c 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -9,10 +9,9 @@
 # given as a dep, no need to mention ring. This is especially true for the
 # core libs which are widely reused, so their deps are kept to a minimum.
 libraries = [ 'compat', # just a header, used for versioning
-   'eal', 'ring', 'mempool', 'mbuf', 'net', 'ethdev', 'pci', # core
+   'eal', 'ring', 'mempool', 'mbuf', 'net', 'kvargs', 'ethdev', 'pci', # 
core
'metrics', # bitrate/latency stats depends on this
'hash',# efd depends on this
-   'kvargs',  # cryptodev depends on this
'timer',   # eventdev depends on this
'acl', 'bbdev', 'bitratestats', 'cfgfile',
'cmdline', 'cryptodev',
-- 
2.14.3

Re: [dpdk-dev] [PATCH 0/2] fix compilation on FreeBSD

[Maxime Coquelin]

On 04/27/2018 04:43 AM, Thomas Monjalon wrote:

The compilation was broken on FreeBSD due to the commit
for auxv, implemented only for Linux in the common files.

Thomas Monjalon (2):
   eal: fix build with glibc < 2.16
   eal: fix build on FreeBSD

  lib/librte_eal/bsdapp/eal/Makefile |  1 +
  lib/librte_eal/bsdapp/eal/eal_cpuflags.c   | 21 ++
  lib/librte_eal/bsdapp/eal/meson.build  |  1 +
  lib/librte_eal/common/eal_common_cpuflags.c| 79 --
  lib/librte_eal/linuxapp/eal/Makefile   |  1 +
  .../eal/eal_cpuflags.c}| 49 +-
  lib/librte_eal/linuxapp/eal/meson.build|  1 +
  7 files changed, 27 insertions(+), 126 deletions(-)
  create mode 100644 lib/librte_eal/bsdapp/eal/eal_cpuflags.c
  copy lib/librte_eal/{common/eal_common_cpuflags.c => 
linuxapp/eal/eal_cpuflags.c} (60%)



For the series:
Acked-by: Maxime Coquelin 

Maxime

[dpdk-dev] [PATCH] vhost/crypto: fix build issue with GCC 4.7.2

[Maxime Coquelin]

Build error has been reported by Intel build system:
SUSE12SP3_64 / Linux 3.7.10-1 / GCC 4.7.2
lib/librte_vhost/vhost_crypto.c: In function ‘rte_vhost_crypto_set_zero_copy’:
lib/librte_vhost/vhost_crypto.c:1192:2: error: comparison of unsigned 
expression < 0 is always false [-Werror=type-limits]

As enums can be either signed or unsigned, this patch removes
the negative check and cast to unsigned the upper limit check.

Fixes: 939066d96563 ("vhost/crypto: add public function implementation")

Signed-off-by: Maxime Coquelin 
---
 lib/librte_vhost/vhost_crypto.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index c38eb3bb5..c6fb2fe5f 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -1189,8 +1189,8 @@ rte_vhost_crypto_set_zero_copy(int vid, enum 
rte_vhost_crypto_zero_copy option)
return -EINVAL;
}
 
-   if (unlikely(option < 0 || option >=
-   RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
+   if (unlikely((uint32_t)option >=
+   RTE_VHOST_CRYPTO_MAX_ZERO_COPY_OPTIONS)) {
VC_LOG_ERR("Invalid option %i", option);
return -EINVAL;
}
-- 
2.14.3

Re: [dpdk-dev] [PATCH] ethdev: fix meson build

[Thomas Monjalon]

27/04/2018 11:01, Ferruh Yigit:
> kvargs dependency added to ehtdev which is not updated in meson build
> 
> Fixes: 9d5c2725ed8c ("ethdev: add common devargs parser")
> Cc: remy.hor...@intel.com
> 
> Signed-off-by: Ferruh Yigit 

Ack
I guess you want to squash it in next-net?

Re: [dpdk-dev] [PATCH 0/2] fix compilation on FreeBSD

[Thomas Monjalon]

27/04/2018 11:03, Maxime Coquelin:
> 
> On 04/27/2018 04:43 AM, Thomas Monjalon wrote:
> > The compilation was broken on FreeBSD due to the commit
> > for auxv, implemented only for Linux in the common files.
> > 
> > Thomas Monjalon (2):
> >eal: fix build with glibc < 2.16
> >eal: fix build on FreeBSD
> > 
> >   lib/librte_eal/bsdapp/eal/Makefile |  1 +
> >   lib/librte_eal/bsdapp/eal/eal_cpuflags.c   | 21 ++
> >   lib/librte_eal/bsdapp/eal/meson.build  |  1 +
> >   lib/librte_eal/common/eal_common_cpuflags.c| 79 
> > --
> >   lib/librte_eal/linuxapp/eal/Makefile   |  1 +
> >   .../eal/eal_cpuflags.c}| 49 +-
> >   lib/librte_eal/linuxapp/eal/meson.build|  1 +
> >   7 files changed, 27 insertions(+), 126 deletions(-)
> >   create mode 100644 lib/librte_eal/bsdapp/eal/eal_cpuflags.c
> >   copy lib/librte_eal/{common/eal_common_cpuflags.c => 
> > linuxapp/eal/eal_cpuflags.c} (60%)
> > 
> 
> For the series:
> Acked-by: Maxime Coquelin 

Applied

[dpdk-dev] [PATCH] ehdev: fix redundancy in meson file

[Ferruh Yigit]

Meson "name" variable required for the case library name is not same as
the library folder, this was the case for ehtdev.
After ethdev library folder name updated this field no more required.

Fixes: b3238506199f ("ethdev: rename folder to library name")
Cc: ferruh.yi...@intel.com

Signed-off-by: Ferruh Yigit 
---
 lib/librte_ethdev/meson.build | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lib/librte_ethdev/meson.build b/lib/librte_ethdev/meson.build
index aed5d2265..b7474d55d 100644
--- a/lib/librte_ethdev/meson.build
+++ b/lib/librte_ethdev/meson.build
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-name = 'ethdev'
 version = 9
 allow_experimental_apis = true
 sources = files('ethdev_profile.c',
-- 
2.14.3

Re: [dpdk-dev] [PATCH] doc: update eal option usage for FreeBSD

[Burakov, Anatoly]

On 26-Apr-18 4:48 PM, Reshma Pattan wrote:

EAL option -m is supported in FreeBSD,
so move it under supported heading from non
supported heading.

Signed-off-by: Reshma Pattan 
---
  doc/guides/freebsd_gsg/build_sample_apps.rst | 8 
  1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/guides/freebsd_gsg/build_sample_apps.rst 
b/doc/guides/freebsd_gsg/build_sample_apps.rst
index 90f050232..ac92260f1 100644
--- a/doc/guides/freebsd_gsg/build_sample_apps.rst
+++ b/doc/guides/freebsd_gsg/build_sample_apps.rst
@@ -128,6 +128,10 @@ The EAL options for FreeBSD are as follows:
  *   ``--proc-type``:
  The type of process instance.
  
+*   ``-m MB``:

+Memory to allocate from hugepages, regardless of processor socket.
+It is recommended that ``--socket-mem`` be used instead of this option.
+


--socket-mem is not supported on FreeBSD (because no NUMA support), so 
the comment about recommending it is wrong.


--
Thanks,
Anatoly

Re: [dpdk-dev] [PATCH] ethdev: fix meson build

[Ferruh Yigit]

On 4/27/2018 10:10 AM, Thomas Monjalon wrote:
> 27/04/2018 11:01, Ferruh Yigit:
>> kvargs dependency added to ehtdev which is not updated in meson build
>>
>> Fixes: 9d5c2725ed8c ("ethdev: add common devargs parser")
>> Cc: remy.hor...@intel.com
>>
>> Signed-off-by: Ferruh Yigit 
> 
> Ack
> I guess you want to squash it in next-net?
Squashed into relevant commit in next-net, thanks.

Re: [dpdk-dev] [PATCH] doc: update eal option usage for FreeBSD

[Pattan, Reshma]

> -Original Message-
> From: Burakov, Anatoly
> Sent: Friday, April 27, 2018 10:21 AM
> To: Pattan, Reshma ; Mcnamara, John
> ; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] doc: update eal option usage for FreeBSD
> 
> On 26-Apr-18 4:48 PM, Reshma Pattan wrote:
> > EAL option -m is supported in FreeBSD, so move it under supported
> > heading from non supported heading.
> >
> > Signed-off-by: Reshma Pattan 
> > ---
> >   doc/guides/freebsd_gsg/build_sample_apps.rst | 8 
> >   1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/doc/guides/freebsd_gsg/build_sample_apps.rst
> > b/doc/guides/freebsd_gsg/build_sample_apps.rst
> > index 90f050232..ac92260f1 100644
> > --- a/doc/guides/freebsd_gsg/build_sample_apps.rst
> > +++ b/doc/guides/freebsd_gsg/build_sample_apps.rst
> > @@ -128,6 +128,10 @@ The EAL options for FreeBSD are as follows:
> >   *   ``--proc-type``:
> >   The type of process instance.
> >
> > +*   ``-m MB``:
> > +Memory to allocate from hugepages, regardless of processor socket.
> > +It is recommended that ``--socket-mem`` be used instead of this option.
> > +
> 
> --socket-mem is not supported on FreeBSD (because no NUMA support), so
> the comment about recommending it is wrong.

Ohh yes, my bad :-( I will recorrect it.

> 
> --
> Thanks,
> Anatoly

Re: [dpdk-dev] [PATCH] net: fix meson build

[Ferruh Yigit]

On 4/27/2018 10:01 AM, Ferruh Yigit wrote:
> librte_net started to use experimental APIs, this explicity allowed by
> make build system, but not for meson.
> 
> Fixes: b0aae17c3e4a ("net: export IPv6 header extensions skip function")
> Cc: didier.pall...@6wind.com
> 
> Signed-off-by: Ferruh Yigit 

Squashed into relevant commit in next-net, thanks.

Re: [dpdk-dev] [PATCH] vhost/crypto: fix build issue with GCC 4.7.2

[Thomas Monjalon]

27/04/2018 11:04, Maxime Coquelin:
> Build error has been reported by Intel build system:
> SUSE12SP3_64 / Linux 3.7.10-1 / GCC 4.7.2
> lib/librte_vhost/vhost_crypto.c: In function ‘rte_vhost_crypto_set_zero_copy’:
> lib/librte_vhost/vhost_crypto.c:1192:2: error: comparison of unsigned 
> expression < 0 is always false [-Werror=type-limits]
> 
> As enums can be either signed or unsigned, this patch removes
> the negative check and cast to unsigned the upper limit check.
> 
> Fixes: 939066d96563 ("vhost/crypto: add public function implementation")
> 
> Signed-off-by: Maxime Coquelin 

Applied, thanks

Re: [dpdk-dev] [PATCH] ehdev: fix redundancy in meson file

[Ferruh Yigit]

On 4/27/2018 10:19 AM, Ferruh Yigit wrote:
> Meson "name" variable required for the case library name is not same as
> the library folder, this was the case for ehtdev.
> After ethdev library folder name updated this field no more required.
> 
> Fixes: b3238506199f ("ethdev: rename folder to library name")
> Cc: ferruh.yi...@intel.com
> 
> Signed-off-by: Ferruh Yigit 

Squashed into relevant commit in next-net, thanks.

Re: [dpdk-dev] [PATCH] net/virtio-user: fix hugepage files enumeration

[Burakov, Anatoly]

On 26-Apr-18 4:34 PM, Jianfeng Tan wrote:

After the commit 2a04139f66b4 ("eal: add single file segments option"),
one hugepage file could contain multiple hugepages which are further
mapped to different memory regions.

Original enumeration implementation cannot handle this situation.

This patch filters out the duplicated files; and adjust the size after
the enumeration.

Fixes: 6a84c37e3975 ("net/virtio-user: add vhost-user adapter layer")

Signed-off-by: Jianfeng Tan 
---
  .../howto/virtio_user_for_container_networking.rst |  3 ++-
  drivers/net/virtio/virtio_user/vhost_user.c| 28 --
  2 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/doc/guides/howto/virtio_user_for_container_networking.rst 
b/doc/guides/howto/virtio_user_for_container_networking.rst
index aa68b53..476ce3a 100644
--- a/doc/guides/howto/virtio_user_for_container_networking.rst
+++ b/doc/guides/howto/virtio_user_for_container_networking.rst
@@ -109,7 +109,8 @@ We have below limitations in this solution:
   * Cannot work with --no-huge option. Currently, DPDK uses anonymous mapping
 under this option which cannot be reopened to share with vhost backend.
   * Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) 
hugepages.
-   In another word, do not use 2MB hugepage so far.
+   If you have more regions (especially when 2MB hugepages are used), the 
option,
+   --single-file-segments, can help to reduce the number of shared files.
   * Applications should not use file name like HUGEFILE_FMT ("%smap_%d"). That
 will bring confusion when sharing hugepage files with backend by name.
   * Root privilege is a must. DPDK resolves physical addresses of hugepages
diff --git a/drivers/net/virtio/virtio_user/vhost_user.c 
b/drivers/net/virtio/virtio_user/vhost_user.c
index a6df97a..01201c9 100644
--- a/drivers/net/virtio/virtio_user/vhost_user.c
+++ b/drivers/net/virtio/virtio_user/vhost_user.c
@@ -138,12 +138,13 @@ struct hugepage_file_info {
  static int
  get_hugepage_file_info(struct hugepage_file_info huges[], int max)
  {
-   int idx;
+   int idx, k, exist;
FILE *f;
char buf[BUFSIZ], *tmp, *tail;
char *str_underline, *str_start;
int huge_index;
uint64_t v_start, v_end;
+   struct stat stats;
  
  	f = fopen("/proc/self/maps", "r");

if (!f) {
@@ -183,16 +184,39 @@ get_hugepage_file_info(struct hugepage_file_info huges[], 
int max)
if (sscanf(str_start, "map_%d", &huge_index) != 1)
continue;
  
+		/* skip duplicated file which is mapped to different regions */

+   for (k = 0, exist = -1; k < idx; ++k) {
+   if (!strcmp(huges[k].path, tmp)) {
+   exist = k;
+   break;
+   }
+   }
+   if (exist >= 0)
+   continue;
+
if (idx >= max) {
PMD_DRV_LOG(ERR, "Exceed maximum of %d", max);
goto error;
}
+
huges[idx].addr = v_start;
-   huges[idx].size = v_end - v_start;
+   huges[idx].size = v_end - v_start; /* To be corrected later */
snprintf(huges[idx].path, PATH_MAX, "%s", tmp);
idx++;
}
  
+	/* correct the size for files who have many regions */

+   for (k = 0; k < idx; ++k) {
+   if (stat(huges[k].path, &stats) < 0) {
+   PMD_DRV_LOG(ERR, "Failed to stat %s, %s\n",
+   huges[k].path, strerror(errno));
+   continue;
+   }
+   huges[k].size = stats.st_size;
+   PMD_DRV_LOG(INFO, "file %s, size %"PRIx64"\n",
+   huges[k].path, huges[k].size);
+   }
+
fclose(f);
return idx;
  



That sounds like potentially a lot of strcmp()'s (quadratic?). Can't it 
be sped up somehow? Maybe use rte_hash for storing this data?


--
Thanks,
Anatoly

Re: [dpdk-dev] [PATCH v9 09/10] eal: replace rte_panic instances in init sequence

[Kevin Traynor]

On 04/26/2018 11:08 PM, Arnon Warshavsky wrote:
> Hi Kevin
> 
> > + if (rte_config_init() != 0)
> > + return -1;
> > +
> 
> I'm confused, is this deliberate? there is now two rte_config_init()
> calls. 
> 
> 
> Please note that there are 2 eal.c files. One under linux and one under bsd
> There is a copy of rte_config_init() in each.
>  
> 
> Aaron's comments on v4 about rte_eal_init_alert() are missed? and
> it's missing the clear once and error number like the other returns.
> 
> 
> These comments were for code that I reverted to call panic again.
> As Aaron pointed out, the partial treatment of v4 did not only
> not-improve things, rather made them worst.
> This code along with other panic instances thrown from within threads
> will have a dedicated patchset.
> 

I was referencing the code above (but as it's a duplicate call, I don't
know if it was merge leftovers from a previous version). Other returns
from rte_eal_init() have something like:
rte_eal_init_alert("Cannot init logging.");
rte_errno = ENOMEM;
rte_atomic32_clear(&run_once);
return -1;

> Thanks
> /Arnon
> 

Re: [dpdk-dev] [PATCH v9 10/10] devtools: prevent new instances of rte_panic and rte_exit

[Kevin Traynor]

On 04/26/2018 10:57 PM, Arnon Warshavsky wrote:
> 
> 
> On Thu, Apr 26, 2018 at 7:08 PM, Kevin Traynor  > wrote:
> 
> I think the first line of the commit msg ^^^ should change as it no
> longer prevents but just warns
> 
> Actually the return value did not change, only the warning text did.

So will it fail checkpatch in patchwork? I agree with Aaron and Anatoly
that patches with rte_panic/exit should flag some warning message, but
the maintainer should have final say. I don't think failing checkpatch
is the solution for that.

> I would rather keep the word 'prevents' as an intention declaration that
> puts the fact that a panic is undesired first, and the technical ability
> of a maintainer to allow it second.

It's only words, but to me 'prevents' does not indicate that it is
undesirable - it indicates it is not allowed.

> This is just in terms of how one positions the usage of panic.
> 
>  
> 
> > 
> 
> Seems to drop the patch name if it finds a match
> 
> 
> Not sure I understand what you meant here. Can you please elaborate?
> 

Sure, I made a patch with an rte_panic and ran checkpatch on a bunch of
patches. It gave me the name of all the patches except the one with the
rte_panic.

> Thanks
> /Arnon
> 

Re: [dpdk-dev] [PATCH v7 01/11] crypto/dpaa: replace rte_panic instances in crypto/dpaa driver

[Kevin Traynor]

On 04/26/2018 10:28 PM, Arnon Warshavsky wrote:
> 
> > - if (cryptodev->data->dev_private == NULL)
> > - rte_panic("Cannot allocate memzone for private "
> > -   "device data");
> > + if (cryptodev->data->dev_private == NULL) {
> > + DPAA_SEC_ERR("%s() Cannot allocate memzone for 
> private device data",
> > + __func__);
> > + return -ENOMEM;
> 
> I'm not familiar with the code but there was a successful allocate
> already, so it seems you should jump to the cleanup section at the end
> of the function before returning.
> 
> Hi Kevin,
> The purpose of this patchset is not to offer a recoverable alternative
> for panic,
> rather allow the process to abort in an orderly manner.
> It does not cover in this version all the panic instances on the init
> sequence.
> Other than in places where it seemed straight forward I tend not to
> perform in this patchset
> partial resource release where panic was before.
> 

Ok, I understand the intention better now.

> Thanks
> /Arnon
> 
> 
>  

Re: [dpdk-dev] [PATCH 0/4] support for write combining

[Bruce Richardson]

On Fri, Apr 27, 2018 at 10:27:02AM +0200, Rafał Kozik wrote:
> Hello Bruce,
> 
> As from my last post passed ten days I would kindly ask if there are
> any more comments?
> 
> Best regards,
> Rafal Kozik

Hi Rafal,

sorry for not responding before. The set makes a lot more sense to me now.
However, just one more question? If igb_uio is passed the flag to make
mappings wc, what happens if you have a mix of drivers that support or
don't support wc at the driver, not the hardware level. For example,
support there is one ena adapter and one i40e adapter. The hardware on both
should be mappable as WC, but one driver can use that, the other can't. How
is it handled?

/Bruce

Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after freeing

[De Lara Guarch, Pablo]

Hi Akhil,

> -Original Message-
> From: Akhil Goyal [mailto:akhil.go...@nxp.com]
> Sent: Friday, April 27, 2018 9:47 AM
> To: De Lara Guarch, Pablo ; Zhang, Roy Fan
> 
> Cc: dev@dpdk.org; sta...@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after
> freeing
> 
> Hi Pablo,
> 
> On 4/26/2018 8:39 PM, Pablo de Lara wrote:
> > When freeing memory, pointers should be set to NULL, to avoid memory
> > corruption/segmentation faults.
> 
> Shouldn't this be handled in the rte_free itself. A lot of other driver are 
> also not
> setting null after rte_free.
> This would require change at a lot of places if this is not handled in 
> rte_free.
> 

The glibc function "free" works the same way. Users are responsible for
setting to NULL these pointers (because sometimes, it is not necessary to do 
such thing).

Anyway, in case we still wanted to change it, we would need to pass a pointer
to a pointer in rte_free, which would imply an API breakage.

Thanks,
Pablo

> Thanks,
> Akhil

Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after freeing

[Akhil Goyal]

Hi Pablo,

On 4/27/2018 5:06 PM, De Lara Guarch, Pablo wrote:

Hi Akhil,


-Original Message-
From: Akhil Goyal [mailto:akhil.go...@nxp.com]
Sent: Friday, April 27, 2018 9:47 AM
To: De Lara Guarch, Pablo ; Zhang, Roy Fan

Cc: dev@dpdk.org; sta...@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after
freeing

Hi Pablo,

On 4/26/2018 8:39 PM, Pablo de Lara wrote:

When freeing memory, pointers should be set to NULL, to avoid memory
corruption/segmentation faults.


Shouldn't this be handled in the rte_free itself. A lot of other driver are 
also not
setting null after rte_free.
This would require change at a lot of places if this is not handled in rte_free.



The glibc function "free" works the same way. Users are responsible for
setting to NULL these pointers (because sometimes, it is not necessary to do 
such thing).
Yes it is correct but rte_free is custom free API in DPDK which can be 
modified or we can have a safer API rte_free_safe which can set the 
pointer to null.


Anyway, in case we still wanted to change it, we would need to pass a pointer
to a pointer in rte_free, which would imply an API breakage.

I think if the community agrees, we can add this change may be in next 
releases.



Thanks,
Pablo


Thanks,
Akhil

Re: [dpdk-dev] [PATCH v7 3/4] ethdev: add mark flow item to rte_flow_item_types

[Ferruh Yigit]

On 4/26/2018 7:52 PM, Thomas Monjalon wrote:
> 26/04/2018 19:29, Declan Doherty:
>> Introduces a new action type RTE_FLOW_ITEM_TYPE_MARK which enables
>> flow patterns to specify arbitrary integer values to match aginst
>> set by the RTE_FLOW_ACTION_TYPE_MARK action in previously matched
>> flows.
> 
> In the title, I don't think you need specify rte_flow_item_types
> (which doesn't exist BTW).
> I suggest:
>   ethdev: add mark flow item

Updated on next-net.

Re: [dpdk-dev] [PATCH v7 4/4] ethdev: add shared counter support to rte_flow

[Ferruh Yigit]

On 4/26/2018 7:55 PM, Thomas Monjalon wrote:
> 26/04/2018 19:29, Declan Doherty:
>> Add rte_flow_action_count action data structure to enable shared
>> counters across multiple flows on a single port or across multiple
>> flows on multiple ports within the same switch domain. Also this enables
>> multiple count actions to be specified in a single flow action.
>>
>> This patch also modifies the existing rte_flow_query API to take the
>> rte_flow_action structure as an input parameter instead of the
>> rte_flow_action_type enumeration to allow querying a specific action
>> from a flow rule when multiple actions of the same type are specified.
>>
>> This patch also contains updates for the bonding, failsafe and mlx5 PMDs
>> and testpmd application which are affected by this API change.
> 
> The API changes must be notified in the release notes (there is a section
> for API changes).

Hi Declan,

If you can send the update as a separate patch, I can squash it in next-net.

Re: [dpdk-dev] [PATCH v7 2/4] ethdev: Add group JUMP action

[Ferruh Yigit]

On 4/26/2018 7:54 PM, Thomas Monjalon wrote:
> About title, we use uppercase only for acronyms.
> So it should be:
>   ethdev: add group jump action
Updated on next-net.

Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after freeing

[Van Haaren, Harry]

> From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Akhil Goyal
> Sent: Friday, April 27, 2018 12:59 PM
> To: De Lara Guarch, Pablo ; Akhil Goyal
> ; Zhang, Roy Fan 
> Cc: dev@dpdk.org; sta...@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after
> freeing
> 
> Hi Pablo,
> 
> On 4/27/2018 5:06 PM, De Lara Guarch, Pablo wrote:
> > Hi Akhil,
> >
> >> -Original Message-
> >> From: Akhil Goyal [mailto:akhil.go...@nxp.com]
> >> Sent: Friday, April 27, 2018 9:47 AM
> >> To: De Lara Guarch, Pablo ; Zhang, Roy Fan
> >> 
> >> Cc: dev@dpdk.org; sta...@dpdk.org
> >> Subject: Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer
> after
> >> freeing
> >>
> >> Hi Pablo,
> >>
> >> On 4/26/2018 8:39 PM, Pablo de Lara wrote:
> >>> When freeing memory, pointers should be set to NULL, to avoid memory
> >>> corruption/segmentation faults.
> >>
> >> Shouldn't this be handled in the rte_free itself. A lot of other driver are
> also not
> >> setting null after rte_free.
> >> This would require change at a lot of places if this is not handled in
> rte_free.
> >>
> >
> > The glibc function "free" works the same way. Users are responsible for
> > setting to NULL these pointers (because sometimes, it is not necessary to do
> such thing).
> Yes it is correct but rte_free is custom free API in DPDK which can be
> modified or we can have a safer API rte_free_safe which can set the
> pointer to null.
> >
> > Anyway, in case we still wanted to change it, we would need to pass a
> pointer
> > to a pointer in rte_free, which would imply an API breakage.


Actually there is an alternative solution, by creating a macro like so;

#define rte_free(x) do {
 rte_free_(x); /* call the real implementation, now with _ */
 x = NULL;
} while (0)

This is not an ABI break if symbol versioning is used for rte_free().

It is an API change however - not that the calling code has to change,
but rather that the effect of rte_free() changes transparently.

I'm not sure what the correct thing to do is in this case - just pointing
out that this is another possible solution.


> I think if the community agrees, we can add this change may be in next
> releases.

+1 to discuss this with the community, regardless of the implementation :)


> > Thanks,
> > Pablo
> >
> >> Thanks,
> >> Akhil

[dpdk-dev] [PATCH v2] doc: update eal option usage for FreeBSD

[Reshma Pattan]

EAL option -m is supported in FreeBSD,
so move it under supported heading from non
supported heading.

Signed-off-by: Reshma Pattan 
---
v2: remove recommendation of socket-mem option.
---
 doc/guides/freebsd_gsg/build_sample_apps.rst | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/doc/guides/freebsd_gsg/build_sample_apps.rst 
b/doc/guides/freebsd_gsg/build_sample_apps.rst
index 90f050232..a085b6187 100644
--- a/doc/guides/freebsd_gsg/build_sample_apps.rst
+++ b/doc/guides/freebsd_gsg/build_sample_apps.rst
@@ -128,6 +128,9 @@ The EAL options for FreeBSD are as follows:
 *   ``--proc-type``:
 The type of process instance.
 
+*   ``-m MB``:
+Memory to allocate from hugepages, regardless of processor socket.
+
 Other options, specific to Linux and are not supported under FreeBSD are as 
follows:
 
 *   ``socket-mem``:
@@ -142,10 +145,6 @@ Other options, specific to Linux and are not supported 
under FreeBSD are as foll
 *   ``--file-prefix``:
 The prefix text used for hugepage filenames.
 
-*   ``-m MB``:
-Memory to allocate from hugepages, regardless of processor socket.
-It is recommended that ``--socket-mem`` be used instead of this option.
-
 The ``-c`` or ``-l`` option is mandatory; the others are optional.
 
 Copy the DPDK application binary to your target, then run the application
-- 
2.14.3

Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after freeing

[Bruce Richardson]

On Fri, Apr 27, 2018 at 12:37:08PM +, Van Haaren, Harry wrote:
> > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Akhil Goyal
> > Sent: Friday, April 27, 2018 12:59 PM
> > To: De Lara Guarch, Pablo ; Akhil Goyal
> > ; Zhang, Roy Fan 
> > Cc: dev@dpdk.org; sta...@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer after
> > freeing
> > 
> > Hi Pablo,
> > 
> > On 4/27/2018 5:06 PM, De Lara Guarch, Pablo wrote:
> > > Hi Akhil,
> > >
> > >> -Original Message-
> > >> From: Akhil Goyal [mailto:akhil.go...@nxp.com]
> > >> Sent: Friday, April 27, 2018 9:47 AM
> > >> To: De Lara Guarch, Pablo ; Zhang, Roy 
> > >> Fan
> > >> 
> > >> Cc: dev@dpdk.org; sta...@dpdk.org
> > >> Subject: Re: [dpdk-dev] [PATCH 1/2] crypto/scheduler: set null pointer
> > after
> > >> freeing
> > >>
> > >> Hi Pablo,
> > >>
> > >> On 4/26/2018 8:39 PM, Pablo de Lara wrote:
> > >>> When freeing memory, pointers should be set to NULL, to avoid memory
> > >>> corruption/segmentation faults.
> > >>
> > >> Shouldn't this be handled in the rte_free itself. A lot of other driver 
> > >> are
> > also not
> > >> setting null after rte_free.
> > >> This would require change at a lot of places if this is not handled in
> > rte_free.
> > >>
> > >
> > > The glibc function "free" works the same way. Users are responsible for
> > > setting to NULL these pointers (because sometimes, it is not necessary to 
> > > do
> > such thing).
> > Yes it is correct but rte_free is custom free API in DPDK which can be
> > modified or we can have a safer API rte_free_safe which can set the
> > pointer to null.
> > >
> > > Anyway, in case we still wanted to change it, we would need to pass a
> > pointer
> > > to a pointer in rte_free, which would imply an API breakage.
> 
> 
> Actually there is an alternative solution, by creating a macro like so;
> 
> #define rte_free(x) do {
>  rte_free_(x); /* call the real implementation, now with _ */
>  x = NULL;
> } while (0)
> 
> This is not an ABI break if symbol versioning is used for rte_free().
> 
> It is an API change however - not that the calling code has to change,
> but rather that the effect of rte_free() changes transparently.
> 
> I'm not sure what the correct thing to do is in this case - just pointing
> out that this is another possible solution.
> 
> 
> > I think if the community agrees, we can add this change may be in next
> > releases.
> 
> +1 to discuss this with the community, regardless of the implementation :)
> 
> 
I really don't think this change is necessary. I think having rte_free
behave as libc free is fine.

However, if we want to add a new API called rte_free_and_null(void **x),
I could be ok with that, though I'd be somewhat dubious of its necessity.
Static analysis tools should be able to pick up use-after-free errors,
though we may need to provide metadata to the tools in some form indicating
that rte_free is a free-ing function.

/Bruce

Re: [dpdk-dev] [PATCH 1/1] doc: update doc and release notes for szedata2 driver

[Ferruh Yigit]

On 4/27/2018 9:57 AM, Matej Vido wrote:
> New version of the packages with dependencies for the szedata2
> driver is needed due to the new API of the libsze2 library which
> is used in the driver.
> The documentation and the release notes are updated to contain
> the information about the required versions.
> 
> Signed-off-by: Matej Vido 
> Acked-by: Jan Remes 

Applied to dpdk-next-net/master, thanks.

(Thanks for the update)

> @@ -54,6 +54,15 @@ separately:
>  Information about getting the dependencies can be found `here
>  `_.
>  
> +Versions of the packages
> +~~~

Giving warning because of missing char [1], fixed while applying.

[1]
dpdk/doc/guides/nics/szedata2.rst:58: WARNING: Title underline too short.

Versions of the packages
~~~

Re: [dpdk-dev] [PATCH v2] doc: update eal option usage for FreeBSD

[Burakov, Anatoly]

On 27-Apr-18 1:59 PM, Reshma Pattan wrote:

EAL option -m is supported in FreeBSD,
so move it under supported heading from non
supported heading.

Signed-off-by: Reshma Pattan 
---
v2: remove recommendation of socket-mem option.
---


Reviewed-by: Anatoly Burakov 

--
Thanks,
Anatoly

[dpdk-dev] [PATCH] app/test: enhance sanity script

[Reshma Pattan]

1. Sanity Script is enhanced with additional test cases
being added to autotest_data.py
2. Fixed in autotest_test_funcs.py to handle test cases
which returns "Skipped" as result.
  The issue was skipped test cases got timed out,
causing delay in sanity script execution.
3. Enhanced support for FreeBSD to add the respective memory limit,
as FreeBSD doesn't support socket-mem, file-prefix options.

Signed-off-by: Jananee Parthasarathy 
---
 test/test/autotest_data.py   | 471 ++-
 test/test/autotest_runner.py |   9 +-
 test/test/autotest_test_funcs.py |   6 +-
 3 files changed, 474 insertions(+), 12 deletions(-)
 mode change 100644 => 100755 test/test/autotest_data.py
 mode change 100644 => 100755 test/test/autotest_test_funcs.py

diff --git a/test/test/autotest_data.py b/test/test/autotest_data.py
old mode 100644
new mode 100755
index aacfe0a..2c6828d
--- a/test/test/autotest_data.py
+++ b/test/test/autotest_data.py
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
 
 # Test data for autotests
 
@@ -70,6 +70,31 @@ def per_sockets(num):
 "Func":default_autotest,
 "Report":  None,
 },
+{
+"Name":"Eventdev selftest octeontx",
+"Command": "eventdev_selftest_octeontx",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Event ring autotest",
+"Command": "event_ring_autotest",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Table autotest",
+"Command": "table_autotest",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Flow classify autotest",
+"Command": "flow_classify_autotest",
+"Func":default_autotest,
+"Report":  None,
+},
+
 ]
 },
 {
@@ -125,6 +150,50 @@ def per_sockets(num):
 "Func":default_autotest,
 "Report":  None,
 },
+{
+"Name":"Dump struct sizes",
+"Command": "dump_struct_sizes",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Dump mempool",
+"Command": "dump_mempool",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Dump malloc stats",
+"Command": "dump_malloc_stats",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Dump devargs",
+"Command": "dump_devargs",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Dump log types",
+"Command": "dump_log_types",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Dump_ring",
+"Command": "dump_ring",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Quit",
+"Command": "quit",
+"Func":default_autotest,
+"Report":  None,
+},
+
+
 ],
 },
 {
@@ -168,6 +237,26 @@ def per_sockets(num):
 "Func":default_autotest,
 "Report":  None,
 },
+{
+"Name":"Set rxtx mode",
+"Command": "set_rxtx_mode",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Set rxtx anchor",
+"Command": "set_rxtx_anchor",
+"Func":default_autotest,
+"Report":  None,
+},
+{
+"Name":"Set rxtx sc",
+"Command": "set_rxtx_sc",
+"Func":default_autotest,
+"Report":  None,
+},
+
+
 ]
 },
 {
@@ -176,12 +265,6 @@ def per_sockets(num):
 "Tests":
 [
 {
-"Name":"PCI autotest",
-"Command": "pci_autotest",
-"Func":default_autotest,
-"Report":  None,
-},
-{
 "Name":"Malloc autotest",
 "Command": "malloc_autotest",
 "Func":default_autotest,
@@ -211,6 +294,37 @@ def per_sockets(num):
 "

[dpdk-dev] [PATCH v6 00/14] Implement compression API

[Pablo de Lara]

With the vast amounts of data being transported around networks
and stored in storage systems, reducing data size is becoming
ever more important.

There are both software libraries and hardware devices available
that provide compression, but no common API.
Such an API is proposed in this patchset, which supports
the following features:

- Deflate Algorithm (https://tools.ietf.org/html/rfc1951)
- LZS algorithm (https://tools.ietf.org/html/rfc2395)
- Static and Dynamic Huffman encoding.
- Compression levels
- Checksum generation
- Asynchronous burst API
- private_xform - a place for PMDs to hold private data derived from
  a xform and used by stateless operations.
- stream - a place for PMDs to hold private data derived from
  a xform and also maintain state and history data. For
  stateful flows.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 

Changes in v6:
 - Add programmer's guide section for library
 - Change macro for no hash
 - Fix compilation in patch 4
 - Remove next pointer in xform
 - Add queue pair release in device closure

Changes in v5:
 - Make rte_compressdev_is_valid_dev, rte_compressdev_pmd_get_dev
   functions internal
 - Add rte_comp.h to API doc
 - Remove unused rte_comp_op_get_user_data,
   rte_comp_op_get_user_data_size functions
 - Fix comment
 - Move private data size arg out of PMD init params structure
 - Split out of space error status into two
 - Make rte_comp_op_alloc, rte_comp_op_bulk_alloc, rte_comp_free,
   rte_compressdev_enqueue_burst and rte_compressdev_dequeue_burst
   functions not inline.
 - Move rte_compressdev and rte_compressdev_data structures out of
   rte_compressdev.h, making them internal to the API
 - Remove unneeded driver registration (used in cryptodev, for multi PMD
   sessions)
 - Remove unneeded includes
 - rte_comp_op_bulk_alloc returns now 0 on success, instead of the
   number of allocated operations
 - Move internals macros to rte_compressdev_internals.h, to avoid
   exposing them unnecessarily

Changes in v4:
 - Fix build (missing ";")
 - Change order in common_base, so compression is placed after security

Changes in v3:
 - Remove rte_comp_op_ctod helper functions
 - Remove param_range_check macro
 - Rename from phys_addr to iova_addr
 - Remove rte_comp_algo_strings
 - Rename rte_compressdev_pmd_is_valid_dev to rte_compressdev_is_valid_dev
 - Remove feature flags from compressdev
 - Support hash operations
 - Add shareable priv xform in feature flags, instead of returnin it
   on xform creation
 - Allow max number of queue pairs to be 0, meaning that there is no
   limit.
 - Add invalid configuration checks
 - Add capability helper functions

Changes in v2:
 - Add stream APIs
 - Remove session
 - Add SHAREABLE / NON_SHAREABLE private_xform types
 - Add algo enum 'UNSPECIFIED' to fix warning in capabilities
 - Change one remaining log to use dynamic logging.
 - Add rte_cache_aligned keyword to op
 - Rename enums with better names _ALGO, __CHECKSUM, _HUFFMAN_
 - Use const keyword when passing xform
 - Remove qp_count fn from dev_ops as never used
 - Remove max_nb_queue-pairs from compressdev_init_param as never used
 - Clarify device configure and start sequence
 - Replace OUT_OF_SPACE with OUT_OF_SPACE_RECOVERABLE and TERMINATED
   and clarified usage. 
 - Add stream and private_xform sizes to device config for use in
   mempool creation
 - Add capability helper fn
 - Use Base2 log value for window size on xforms
 - Add Meson build
 - Update MAINTAINERS
 - Update Map file
 - Change order in doxy file
 - Update Release note 

Ashish Gupta (1):
  doc: add compressdev library guide

Fiona Trahe (12):
  compressdev: add basic device management
  compressdev: add queue pair management
  compressdev: add compression specific data
  compressdev: add enqueue/dequeue functions
  compressdev: add operation management
  compressdev: support stateless operations
  compressdev: support stateful operations
  compressdev: add device feature flags
  compressdev: add compression service feature flags
  compressdev: add device stats
  compressdev: add device capabilities
  compressdev: get device id from name

Shally Verma (1):
  compressdev: support hash operations

 MAINTAINERS|   8 +
 config/common_base |   6 +
 config/rte_config.h|   3 +
 doc/api/doxy-api-index.md  |   2 +
 doc/api/doxy-api.conf  |   1 +
 doc/guides/prog_guide/compressdev_lib.rst  | 623 +
 doc/guides/prog_guide/img/stateful-op.svg  | 116 +++
 doc/guides/prog_guide/img/stateless-op-shared.svg  | 124 
 doc/guides/prog_guide/img/stateless-op.svg | 140 
 doc/guides/prog_guide/index.rst|   1 +
 doc/guides/rel_notes/release_18_05.rst |   6 +
 lib/Makefile   |   3 +
 

[dpdk-dev] [PATCH] devtools: add git log checks for vDPA MSS and UDP

[Ferruh Yigit]

Signed-off-by: Ferruh Yigit 
---
 devtools/check-git-log.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/devtools/check-git-log.sh b/devtools/check-git-log.sh
index c601f6ae9..2dfa3a1cd 100755
--- a/devtools/check-git-log.sh
+++ b/devtools/check-git-log.sh
@@ -129,6 +129,7 @@ bad=$(echo "$headlines" | grep -E --color=always \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
+   -e ':.*\' \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
@@ -138,7 +139,9 @@ bad=$(echo "$headlines" | grep -E --color=always \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
+   -e ':.*\' \
-e ':.*\<[Vv]lan\>' \
+   -e ':.*\' \
-e ':.*\' \
| sed 's,^,\t,')
 [ -z "$bad" ] || printf "Wrong headline lowercase:\n$bad\n"
-- 
2.14.3

[dpdk-dev] [PATCH v6 02/14] compressdev: add queue pair management

[Pablo de Lara]

From: Fiona Trahe 

Add functions to manage device queue pairs.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 178 -
 lib/librte_compressdev/rte_compressdev.h   |  45 ++
 lib/librte_compressdev/rte_compressdev_internal.h  |   5 +
 lib/librte_compressdev/rte_compressdev_pmd.h   |  47 ++
 lib/librte_compressdev/rte_compressdev_version.map |   2 +
 5 files changed, 276 insertions(+), 1 deletion(-)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 751517c3d..6667528b5 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -227,10 +227,136 @@ rte_compressdev_pmd_release_device(struct 
rte_compressdev *compressdev)
return 0;
 }
 
+uint16_t __rte_experimental
+rte_compressdev_queue_pair_count(uint8_t dev_id)
+{
+   struct rte_compressdev *dev;
+
+   dev = &rte_comp_devices[dev_id];
+   return dev->data->nb_queue_pairs;
+}
+
+static int
+rte_compressdev_queue_pairs_config(struct rte_compressdev *dev,
+   uint16_t nb_qpairs, int socket_id)
+{
+   struct rte_compressdev_info dev_info;
+   void **qp;
+   unsigned int i;
+
+   if ((dev == NULL) || (nb_qpairs < 1)) {
+   COMPRESSDEV_LOG(ERR, "invalid param: dev %p, nb_queues %u",
+   dev, nb_qpairs);
+   return -EINVAL;
+   }
+
+   COMPRESSDEV_LOG(DEBUG, "Setup %d queues pairs on device %u",
+   nb_qpairs, dev->data->dev_id);
+
+   memset(&dev_info, 0, sizeof(struct rte_compressdev_info));
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_infos_get, -ENOTSUP);
+   (*dev->dev_ops->dev_infos_get)(dev, &dev_info);
+
+   if ((dev_info.max_nb_queue_pairs != 0) &&
+   (nb_qpairs > dev_info.max_nb_queue_pairs)) {
+   COMPRESSDEV_LOG(ERR, "Invalid num queue_pairs (%u) for dev %u",
+   nb_qpairs, dev->data->dev_id);
+   return -EINVAL;
+   }
+
+   if (dev->data->queue_pairs == NULL) { /* first time configuration */
+   dev->data->queue_pairs = rte_zmalloc_socket(
+   "compressdev->queue_pairs",
+   sizeof(dev->data->queue_pairs[0]) * nb_qpairs,
+   RTE_CACHE_LINE_SIZE, socket_id);
+
+   if (dev->data->queue_pairs == NULL) {
+   dev->data->nb_queue_pairs = 0;
+   COMPRESSDEV_LOG(ERR,
+   "failed to get memory for qp meta data, nb_queues %u",
+   nb_qpairs);
+   return -(ENOMEM);
+   }
+   } else { /* re-configure */
+   int ret;
+   uint16_t old_nb_queues = dev->data->nb_queue_pairs;
+
+   qp = dev->data->queue_pairs;
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_release,
+   -ENOTSUP);
+
+   for (i = nb_qpairs; i < old_nb_queues; i++) {
+   ret = (*dev->dev_ops->queue_pair_release)(dev, i);
+   if (ret < 0)
+   return ret;
+   }
+
+   qp = rte_realloc(qp, sizeof(qp[0]) * nb_qpairs,
+   RTE_CACHE_LINE_SIZE);
+   if (qp == NULL) {
+   COMPRESSDEV_LOG(ERR,
+   "failed to realloc qp meta data, nb_queues %u",
+   nb_qpairs);
+   return -(ENOMEM);
+   }
+
+   if (nb_qpairs > old_nb_queues) {
+   uint16_t new_qs = nb_qpairs - old_nb_queues;
+
+   memset(qp + old_nb_queues, 0,
+   sizeof(qp[0]) * new_qs);
+   }
+
+   dev->data->queue_pairs = qp;
+
+   }
+   dev->data->nb_queue_pairs = nb_qpairs;
+   return 0;
+}
+
+static int
+rte_compressdev_queue_pairs_release(struct rte_compressdev *dev)
+{
+   uint16_t num_qps, i;
+   int ret;
+
+   if (dev == NULL) {
+   COMPRESSDEV_LOG(ERR, "invalid param: dev %p", dev);
+   return -EINVAL;
+   }
+
+   num_qps = dev->data->nb_queue_pairs;
+
+   if (num_qps == 0)
+   return 0;
+
+   COMPRESSDEV_LOG(DEBUG, "Free %d queues pairs on device %u",
+   dev->data->nb_queue_pairs, dev->data->dev_id);
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->queue_pair_release,
+   -ENOTSUP);
+
+   for (i = 0; i < num_qps; i++) {
+   ret = (*dev->dev_ops->queue_pair_release)(dev, i);
+   if (ret < 0)
+  

[dpdk-dev] [PATCH v6 01/14] compressdev: add basic device management

[Pablo de Lara]

From: Fiona Trahe 

Add basic functions to manage compress devices,
including driver and device allocation, and the basic
interface with compressdev PMDs.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 MAINTAINERS|   7 +
 config/common_base |   6 +
 config/rte_config.h|   3 +
 doc/api/doxy-api-index.md  |   1 +
 doc/api/doxy-api.conf  |   1 +
 doc/guides/rel_notes/release_18_05.rst |   6 +
 lib/Makefile   |   3 +
 lib/librte_compressdev/Makefile|  30 ++
 lib/librte_compressdev/meson.build |  10 +
 lib/librte_compressdev/rte_compressdev.c   | 378 +
 lib/librte_compressdev/rte_compressdev.h   | 169 +
 lib/librte_compressdev/rte_compressdev_internal.h  |  64 
 lib/librte_compressdev/rte_compressdev_pmd.c   | 160 +
 lib/librte_compressdev/rte_compressdev_pmd.h   | 236 +
 lib/librte_compressdev/rte_compressdev_version.map |  24 ++
 lib/meson.build|   2 +-
 mk/rte.app.mk  |   1 +
 17 files changed, 1100 insertions(+), 1 deletion(-)
 create mode 100644 lib/librte_compressdev/Makefile
 create mode 100644 lib/librte_compressdev/meson.build
 create mode 100644 lib/librte_compressdev/rte_compressdev.c
 create mode 100644 lib/librte_compressdev/rte_compressdev.h
 create mode 100644 lib/librte_compressdev/rte_compressdev_internal.h
 create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.c
 create mode 100644 lib/librte_compressdev/rte_compressdev_pmd.h
 create mode 100644 lib/librte_compressdev/rte_compressdev_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index b59fa0531..c1370258f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -363,6 +363,13 @@ F: drivers/raw/skeleton_rawdev/
 F: test/test/test_rawdev.c
 F: doc/guides/prog_guide/rawdev.rst
 
+Compression API - EXPERIMENTAL
+M: Fiona Trahe 
+M: Pablo de Lara 
+M: Ashish Gupta 
+T: git://dpdk.org/next/dpdk-next-crypto
+F: lib/librte_compressdev/
+
 
 Memory Pool Drivers
 ---
diff --git a/config/common_base b/config/common_base
index f24417cb0..23cf00011 100644
--- a/config/common_base
+++ b/config/common_base
@@ -568,6 +568,12 @@ CONFIG_RTE_LIBRTE_PMD_MRVL_CRYPTO_DEBUG=n
 #
 CONFIG_RTE_LIBRTE_SECURITY=y
 
+#
+# Compile generic compression device library
+#
+CONFIG_RTE_LIBRTE_COMPRESSDEV=y
+CONFIG_RTE_COMPRESS_MAX_DEVS=64
+
 #
 # Compile generic event device library
 #
diff --git a/config/rte_config.h b/config/rte_config.h
index b1c0b3920..5db654534 100644
--- a/config/rte_config.h
+++ b/config/rte_config.h
@@ -56,6 +56,9 @@
 #define RTE_CRYPTO_MAX_DEVS 64
 #define RTE_CRYPTODEV_NAME_LEN 64
 
+/* compressdev defines */
+#define RTE_COMPRESS_MAX_DEVS 64
+
 /* eventdev defines */
 #define RTE_EVENT_MAX_DEVS 16
 #define RTE_EVENT_MAX_QUEUES_PER_DEV 64
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index 26ce7b44b..a3dfc0899 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -18,6 +18,7 @@ The public API headers are grouped by topics:
   [bbdev]  (@ref rte_bbdev.h),
   [cryptodev]  (@ref rte_cryptodev.h),
   [security]   (@ref rte_security.h),
+  [compressdev](@ref rte_compressdev.h),
   [eventdev]   (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_timer_adapter](@ref rte_event_timer_adapter.h),
diff --git a/doc/api/doxy-api.conf b/doc/api/doxy-api.conf
index ad8bdcf61..3a159eaed 100644
--- a/doc/api/doxy-api.conf
+++ b/doc/api/doxy-api.conf
@@ -45,6 +45,7 @@ INPUT   = doc/api/doxy-api-index.md \
   lib/librte_cfgfile \
   lib/librte_cmdline \
   lib/librte_compat \
+  lib/librte_compressdev \
   lib/librte_cryptodev \
   lib/librte_distributor \
   lib/librte_efd \
diff --git a/doc/guides/rel_notes/release_18_05.rst 
b/doc/guides/rel_notes/release_18_05.rst
index f93141b57..6290c759e 100644
--- a/doc/guides/rel_notes/release_18_05.rst
+++ b/doc/guides/rel_notes/release_18_05.rst
@@ -149,6 +149,11 @@ New Features
   stats/xstats on shared memory from secondary process, and also pdump packets 
on
   those virtual devices.
 
+* **Added Compressdev Library, a generic compression service library.**
+
+  The compressdev library provides an API for offload of compression and
+  decompression operations to hardware or software accelerator devices.
+
 
 API Changes
 ---
@@ -353,6 +358,7 @@ The libraries prepended with a plus sign were incremented 
in this version.
  librte_

[dpdk-dev] [PATCH v6 04/14] compressdev: add enqueue/dequeue functions

[Pablo de Lara]

From: Fiona Trahe 

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   |  22 +
 lib/librte_compressdev/rte_compressdev.h   | 107 +
 lib/librte_compressdev/rte_compressdev_internal.h  |  49 ++
 lib/librte_compressdev/rte_compressdev_version.map |   2 +
 4 files changed, 180 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 6667528b5..af0616980 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -512,6 +512,28 @@ rte_compressdev_queue_pair_setup(uint8_t dev_id, uint16_t 
queue_pair_id,
max_inflight_ops, socket_id);
 }
 
+uint16_t __rte_experimental
+rte_compressdev_dequeue_burst(uint8_t dev_id, uint16_t qp_id,
+   struct rte_comp_op **ops, uint16_t nb_ops)
+{
+   struct rte_compressdev *dev = &rte_compressdevs[dev_id];
+
+   nb_ops = (*dev->dequeue_burst)
+   (dev->data->queue_pairs[qp_id], ops, nb_ops);
+
+   return nb_ops;
+}
+
+uint16_t __rte_experimental
+rte_compressdev_enqueue_burst(uint8_t dev_id, uint16_t qp_id,
+   struct rte_comp_op **ops, uint16_t nb_ops)
+{
+   struct rte_compressdev *dev = &rte_compressdevs[dev_id];
+
+   return (*dev->enqueue_burst)(
+   dev->data->queue_pairs[qp_id], ops, nb_ops);
+}
+
 void __rte_experimental
 rte_compressdev_info_get(uint8_t dev_id, struct rte_compressdev_info *dev_info)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index 81710bbea..9660e5af6 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -19,6 +19,8 @@ extern "C" {
 
 #include 
 
+#include "rte_comp.h"
+
 /**  comp device information */
 struct rte_compressdev_info {
const char *driver_name;/**< Driver name. */
@@ -207,6 +209,111 @@ rte_compressdev_queue_pair_count(uint8_t dev_id);
 void __rte_experimental
 rte_compressdev_info_get(uint8_t dev_id, struct rte_compressdev_info 
*dev_info);
 
+/**
+ *
+ * Dequeue a burst of processed compression operations from a queue on the comp
+ * device. The dequeued operation are stored in *rte_comp_op* structures
+ * whose pointers are supplied in the *ops* array.
+ *
+ * The rte_compressdev_dequeue_burst() function returns the number of ops
+ * actually dequeued, which is the number of *rte_comp_op* data structures
+ * effectively supplied into the *ops* array.
+ *
+ * A return value equal to *nb_ops* indicates that the queue contained
+ * at least *nb_ops* operations, and this is likely to signify that other
+ * processed operations remain in the devices output queue. Applications
+ * implementing a "retrieve as many processed operations as possible" policy
+ * can check this specific case and keep invoking the
+ * rte_compressdev_dequeue_burst() function until a value less than
+ * *nb_ops* is returned.
+ *
+ * The rte_compressdev_dequeue_burst() function does not provide any error
+ * notification to avoid the corresponding overhead.
+ *
+ * @note: operation ordering is not maintained within the queue pair.
+ *
+ * @note: In case op status = OUT_OF_SPACE_TERMINATED, op.consumed=0 and the
+ * op must be resubmitted with the same input data and a larger output buffer.
+ * op.produced is usually 0, but in decompression cases a PMD may return > 0
+ * and the application may find it useful to inspect that data.
+ * This status is only returned on STATELESS ops.
+ *
+ * @note: In case op status = OUT_OF_SPACE_RECOVERABLE, op.produced can be used
+ * and next op in stream should continue on from op.consumed+1 with a fresh
+ * output buffer.
+ * Consumed=0, produced=0 is an unusual but allowed case. There may be useful
+ * state/history stored in the PMD, even though no output was produced yet.
+ *
+ *
+ * @param dev_id
+ *   Compress device identifier
+ * @param qp_id
+ *   The index of the queue pair from which to retrieve
+ *   processed operations. The value must be in the range
+ *   [0, nb_queue_pair - 1] previously supplied to
+ *   rte_compressdev_configure()
+ * @param ops
+ *   The address of an array of pointers to
+ *   *rte_comp_op* structures that must be
+ *   large enough to store *nb_ops* pointers in it
+ * @param nb_ops
+ *   The maximum number of operations to dequeue
+ * @return
+ *   - The number of operations actually dequeued, which is the number
+ *   of pointers to *rte_comp_op* structures effectively supplied to the
+ *   *ops* array.
+ */
+uint16_t __rte_experimental
+rte_compressdev_dequeue_burst(uint8_t dev_id, uint16_t qp_id,
+   struct rte_comp_op **ops, uint16_t nb_ops);
+
+/**
+ * Enqueue a burst of operations for processing on a compression device.
+ *
+ * The rte_compressdev_enqueue_burst() function is invoked t

[dpdk-dev] [PATCH v6 03/14] compressdev: add compression specific data

[Pablo de Lara]

From: Fiona Trahe 

Added structures and enums specific to compression,
including the compression operation structure and the
different supported algorithms, checksums and compression
levels.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 doc/api/doxy-api-index.md  |   1 +
 lib/librte_compressdev/Makefile|   1 +
 lib/librte_compressdev/meson.build |   3 +-
 lib/librte_compressdev/rte_comp.h  | 316 +
 4 files changed, 320 insertions(+), 1 deletion(-)
 create mode 100644 lib/librte_compressdev/rte_comp.h

diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md
index a3dfc0899..f53bd2431 100644
--- a/doc/api/doxy-api-index.md
+++ b/doc/api/doxy-api-index.md
@@ -19,6 +19,7 @@ The public API headers are grouped by topics:
   [cryptodev]  (@ref rte_cryptodev.h),
   [security]   (@ref rte_security.h),
   [compressdev](@ref rte_compressdev.h),
+  [compress]   (@ref rte_comp.h),
   [eventdev]   (@ref rte_eventdev.h),
   [event_eth_rx_adapter]   (@ref rte_event_eth_rx_adapter.h),
   [event_timer_adapter](@ref rte_event_timer_adapter.h),
diff --git a/lib/librte_compressdev/Makefile b/lib/librte_compressdev/Makefile
index b84769417..4dae6ca42 100644
--- a/lib/librte_compressdev/Makefile
+++ b/lib/librte_compressdev/Makefile
@@ -19,6 +19,7 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_kvargs
 SRCS-y += rte_compressdev.c rte_compressdev_pmd.c
 
 # export include files
+SYMLINK-y-include += rte_comp.h
 SYMLINK-y-include += rte_compressdev.h
 # export include files (for PMDs)
 SYMLINK-y-include += rte_compressdev_pmd.h
diff --git a/lib/librte_compressdev/meson.build 
b/lib/librte_compressdev/meson.build
index 895e959d3..2115961c2 100644
--- a/lib/librte_compressdev/meson.build
+++ b/lib/librte_compressdev/meson.build
@@ -6,5 +6,6 @@ sources = files('rte_compressdev.c',
'rte_compressdev_pmd.c')
 headers = files('rte_compressdev.h',
'rte_compressdev_pmd.h',
-   'rte_compressdev_internal.h')
+   'rte_compressdev_internal.h',
+   'rte_comp.h')
 deps += ['kvargs', 'mbuf']
diff --git a/lib/librte_compressdev/rte_comp.h 
b/lib/librte_compressdev/rte_comp.h
new file mode 100644
index 0..9f35c37c5
--- /dev/null
+++ b/lib/librte_compressdev/rte_comp.h
@@ -0,0 +1,316 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#ifndef _RTE_COMP_H_
+#define _RTE_COMP_H_
+
+/**
+ * @file rte_comp.h
+ *
+ * RTE definitions for Data Compression Service
+ *
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include 
+#include 
+
+/** Status of comp operation */
+enum rte_comp_op_status {
+   RTE_COMP_OP_STATUS_SUCCESS = 0,
+   /**< Operation completed successfully */
+   RTE_COMP_OP_STATUS_NOT_PROCESSED,
+   /**< Operation has not yet been processed by the device */
+   RTE_COMP_OP_STATUS_INVALID_ARGS,
+   /**< Operation failed due to invalid arguments in request */
+   RTE_COMP_OP_STATUS_ERROR,
+   /**< Error handling operation */
+   RTE_COMP_OP_STATUS_INVALID_STATE,
+   /**< Operation is invoked in invalid state */
+   RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED,
+   /**< Output buffer ran out of space before operation completed.
+* Error case. Application must resubmit all data with a larger
+* output buffer.
+*/
+   RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE,
+   /**< Output buffer ran out of space before operation completed, but this
+* is not an error case. Output data up to op.produced can be used and
+* next op in the stream should continue on from op.consumed+1.
+*/
+};
+
+/** Compression Algorithms */
+enum rte_comp_algorithm {
+   RTE_COMP_ALGO_UNSPECIFIED = 0,
+   /** No Compression algorithm */
+   RTE_COMP_ALGO_NULL,
+   /**< No compression.
+* Pass-through, data is copied unchanged from source buffer to
+* destination buffer.
+*/
+   RTE_COMP_ALGO_DEFLATE,
+   /**< DEFLATE compression algorithm
+* https://tools.ietf.org/html/rfc1951
+*/
+   RTE_COMP_ALGO_LZS,
+   /**< LZS compression algorithm
+* https://tools.ietf.org/html/rfc2395
+*/
+   RTE_COMP_ALGO_LIST_END
+};
+
+/**< Compression Level.
+ * The number is interpreted by each PMD differently. However, lower numbers
+ * give fastest compression, at the expense of compression ratio while
+ * higher numbers may give better compression ratios but are likely slower.
+ */
+#defineRTE_COMP_LEVEL_PMD_DEFAULT  (-1)
+/** Use PMD Default */
+#defineRTE_COMP_LEVEL_NONE (0)
+/** Output uncompressed blocks if supported by the specified algorithm */
+#define RTE_COMP_LEVEL_MIN (1)
+/** Use minimum compression level supported by the PMD */
+#define RTE_COMP_LEVEL_MAX (9)
+/** Use maximum com

[dpdk-dev] [PATCH v6 05/14] compressdev: add operation management

[Pablo de Lara]

From: Fiona Trahe 

Added functions to allocate and free compression operations.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/Makefile|   2 +-
 lib/librte_compressdev/meson.build |   3 +-
 lib/librte_compressdev/rte_comp.c  | 176 +
 lib/librte_compressdev/rte_comp.h  |  64 
 lib/librte_compressdev/rte_compressdev_internal.h  |   1 -
 lib/librte_compressdev/rte_compressdev_version.map |   4 +
 6 files changed, 247 insertions(+), 3 deletions(-)
 create mode 100644 lib/librte_compressdev/rte_comp.c

diff --git a/lib/librte_compressdev/Makefile b/lib/librte_compressdev/Makefile
index 4dae6ca42..7ef89e613 100644
--- a/lib/librte_compressdev/Makefile
+++ b/lib/librte_compressdev/Makefile
@@ -16,7 +16,7 @@ CFLAGS += -DALLOW_EXPERIMENTAL_API
 LDLIBS += -lrte_eal -lrte_mempool -lrte_kvargs
 
 # library source files
-SRCS-y += rte_compressdev.c rte_compressdev_pmd.c
+SRCS-y += rte_compressdev.c rte_compressdev_pmd.c rte_comp.c
 
 # export include files
 SYMLINK-y-include += rte_comp.h
diff --git a/lib/librte_compressdev/meson.build 
b/lib/librte_compressdev/meson.build
index 2115961c2..5416571c9 100644
--- a/lib/librte_compressdev/meson.build
+++ b/lib/librte_compressdev/meson.build
@@ -3,7 +3,8 @@
 
 allow_experimental_apis = true
 sources = files('rte_compressdev.c',
-   'rte_compressdev_pmd.c')
+   'rte_compressdev_pmd.c',
+   'rte_comp.c')
 headers = files('rte_compressdev.h',
'rte_compressdev_pmd.h',
'rte_compressdev_internal.h',
diff --git a/lib/librte_compressdev/rte_comp.c 
b/lib/librte_compressdev/rte_comp.c
new file mode 100644
index 0..eb99e99d2
--- /dev/null
+++ b/lib/librte_compressdev/rte_comp.c
@@ -0,0 +1,176 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2017-2018 Intel Corporation
+ */
+
+#include "rte_comp.h"
+#include "rte_compressdev.h"
+#include "rte_compressdev_internal.h"
+
+/**
+ * Reset the fields of an operation to their default values.
+ *
+ * @note The private data associated with the operation is not zeroed.
+ *
+ * @param op
+ *   The operation to be reset
+ */
+static inline void
+rte_comp_op_reset(struct rte_comp_op *op)
+{
+   struct rte_mempool *tmp_mp = op->mempool;
+   rte_iova_t tmp_iova_addr = op->iova_addr;
+
+   memset(op, 0, sizeof(struct rte_comp_op));
+   op->status = RTE_COMP_OP_STATUS_NOT_PROCESSED;
+   op->iova_addr = tmp_iova_addr;
+   op->mempool = tmp_mp;
+}
+
+/**
+ * Private data structure belonging to an operation pool.
+ */
+struct rte_comp_op_pool_private {
+   uint16_t user_size;
+   /**< Size of private user data with each operation. */
+};
+
+/**
+ * Bulk allocate raw element from mempool and return as comp operations
+ *
+ * @param mempool
+ *   Compress operation mempool
+ * @param ops
+ *   Array to place allocated operations
+ * @param nb_ops
+ *   Number of operations to allocate
+ * @return
+ *   - 0: Success
+ *   - -ENOENT: Not enough entries in the mempool; no ops are retrieved.
+ */
+static inline int
+rte_comp_op_raw_bulk_alloc(struct rte_mempool *mempool,
+   struct rte_comp_op **ops, uint16_t nb_ops)
+{
+   if (rte_mempool_get_bulk(mempool, (void **)ops, nb_ops) == 0)
+   return nb_ops;
+
+   return 0;
+}
+
+/** Initialise rte_comp_op mempool element */
+static void
+rte_comp_op_init(struct rte_mempool *mempool,
+   __rte_unused void *opaque_arg,
+   void *_op_data,
+   __rte_unused unsigned int i)
+{
+   struct rte_comp_op *op = _op_data;
+
+   memset(_op_data, 0, mempool->elt_size);
+
+   op->status = RTE_COMP_OP_STATUS_NOT_PROCESSED;
+   op->iova_addr = rte_mem_virt2iova(_op_data);
+   op->mempool = mempool;
+}
+
+struct rte_mempool * __rte_experimental
+rte_comp_op_pool_create(const char *name,
+   unsigned int nb_elts, unsigned int cache_size,
+   uint16_t user_size, int socket_id)
+{
+   struct rte_comp_op_pool_private *priv;
+
+   unsigned int elt_size = sizeof(struct rte_comp_op) + user_size;
+
+   /* lookup mempool in case already allocated */
+   struct rte_mempool *mp = rte_mempool_lookup(name);
+
+   if (mp != NULL) {
+   priv = (struct rte_comp_op_pool_private *)
+   rte_mempool_get_priv(mp);
+
+   if (mp->elt_size != elt_size ||
+   mp->cache_size < cache_size ||
+   mp->size < nb_elts ||
+   priv->user_size <  user_size) {
+   mp = NULL;
+   COMPRESSDEV_LOG(ERR,
+   "Mempool %s already exists but with incompatible parameters",
+   name);
+   return NULL;
+   }
+   re

[dpdk-dev] [PATCH v6 06/14] compressdev: support stateless operations

[Pablo de Lara]

From: Fiona Trahe 

Added private transform data (priv_xform) in compression
operation, which will contain the private data from each
PMD to support stateless operations.
Also, added functions to create/free this data.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 48 ++
 lib/librte_compressdev/rte_compressdev.h   | 48 ++
 lib/librte_compressdev/rte_compressdev_pmd.h   | 40 ++
 lib/librte_compressdev/rte_compressdev_version.map |  2 +
 4 files changed, 138 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index af0616980..0af1b4927 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -554,6 +554,54 @@ rte_compressdev_info_get(uint8_t dev_id, struct 
rte_compressdev_info *dev_info)
dev_info->driver_name = dev->device->driver->name;
 }
 
+int __rte_experimental
+rte_compressdev_private_xform_create(uint8_t dev_id,
+   const struct rte_comp_xform *xform,
+   void **priv_xform)
+{
+   struct rte_compressdev *dev;
+   int ret;
+
+   dev = rte_compressdev_get_dev(dev_id);
+
+   if (xform == NULL || priv_xform == NULL || dev == NULL)
+   return -EINVAL;
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->private_xform_create, -ENOTSUP);
+   ret = (*dev->dev_ops->private_xform_create)(dev, xform, priv_xform);
+   if (ret < 0) {
+   COMPRESSDEV_LOG(ERR,
+   "dev_id %d failed to create private_xform: err=%d",
+   dev_id, ret);
+   return ret;
+   };
+
+   return 0;
+}
+
+int __rte_experimental
+rte_compressdev_private_xform_free(uint8_t dev_id, void *priv_xform)
+{
+   struct rte_compressdev *dev;
+   int ret;
+
+   dev = rte_compressdev_get_dev(dev_id);
+
+   if (dev == NULL || priv_xform == NULL)
+   return -EINVAL;
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->private_xform_free, -ENOTSUP);
+   ret = dev->dev_ops->private_xform_free(dev, priv_xform);
+   if (ret < 0) {
+   COMPRESSDEV_LOG(ERR,
+   "dev_id %d failed to free private xform: err=%d",
+   dev_id, ret);
+   return ret;
+   };
+
+   return 0;
+}
+
 const char * __rte_experimental
 rte_compressdev_name_get(uint8_t dev_id)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index 9660e5af6..705e9fe79 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -89,6 +89,8 @@ struct rte_compressdev_config {
/**< Socket on which to allocate resources */
uint16_t nb_queue_pairs;
/**< Total number of queue pairs to configure on a device */
+   uint16_t max_nb_priv_xforms;
+   /**< Max number of private_xforms which will be created on the device */
 };
 
 /**
@@ -314,6 +316,52 @@ uint16_t __rte_experimental
 rte_compressdev_enqueue_burst(uint8_t dev_id, uint16_t qp_id,
struct rte_comp_op **ops, uint16_t nb_ops);
 
+/**
+ * This should alloc a private_xform from the device's mempool and initialise
+ * it. The application should call this API when setting up for stateless
+ * processing on a device. If it returns non-shareable, then the appl cannot
+ * share this handle with multiple in-flight ops and should call this API again
+ * to get a separate handle for every in-flight op.
+ * The handle returned is only valid for use with ops of op_type STATELESS.
+ *
+ * @param dev_id
+ *   Compress device identifier
+ * @param xform
+ *   xform data
+ * @param private_xform
+ *   Pointer to where PMD's private_xform handle should be stored
+ *
+ * @return
+ *  - if successful returns 0
+ *and valid private_xform handle
+ *  - <0 in error cases
+ *  - Returns -EINVAL if input parameters are invalid.
+ *  - Returns -ENOTSUP if comp device does not support the comp transform.
+ *  - Returns -ENOMEM if the private_xform could not be allocated.
+ */
+int __rte_experimental
+rte_compressdev_private_xform_create(uint8_t dev_id,
+   const struct rte_comp_xform *xform,
+   void **private_xform);
+
+/**
+ * This should clear the private_xform and return it to the device's mempool.
+ *
+ * @param dev_id
+ *   Compress device identifier
+ *
+ * @param private_xform
+ *   PMD's private_xform data
+ *
+ * @return
+ *  - 0 if successful
+ *  - <0 in error cases
+ *  - Returns -EINVAL if input parameters are invalid.
+ *  - Returns -EBUSY if can't free private_xform due to inflight operations
+ */
+int __rte_experimental
+rte_compressdev_private_xform_free(uint8_t dev_id, void *private_xform);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_compressdev/rte_compres

[dpdk-dev] [PATCH v6 07/14] compressdev: support stateful operations

[Pablo de Lara]

From: Fiona Trahe 

Added stream data (stream) in compression operation,
which will contain the private data from each PMD
to support stateful operations.
Also, added functions to create/free this data.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 49 +
 lib/librte_compressdev/rte_compressdev.h   | 50 ++
 lib/librte_compressdev/rte_compressdev_pmd.h   | 41 ++
 lib/librte_compressdev/rte_compressdev_version.map |  2 +
 4 files changed, 142 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 0af1b4927..0ffc43bb9 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -602,6 +602,55 @@ rte_compressdev_private_xform_free(uint8_t dev_id, void 
*priv_xform)
return 0;
 }
 
+int __rte_experimental
+rte_compressdev_stream_create(uint8_t dev_id,
+   const struct rte_comp_xform *xform,
+   void **stream)
+{
+   struct rte_compressdev *dev;
+   int ret;
+
+   dev = rte_compressdev_get_dev(dev_id);
+
+   if (xform == NULL || dev == NULL || stream == NULL)
+   return -EINVAL;
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stream_create, -ENOTSUP);
+   ret = (*dev->dev_ops->stream_create)(dev, xform, stream);
+   if (ret < 0) {
+   COMPRESSDEV_LOG(ERR,
+   "dev_id %d failed to create stream: err=%d",
+   dev_id, ret);
+   return ret;
+   };
+
+   return 0;
+}
+
+
+int __rte_experimental
+rte_compressdev_stream_free(uint8_t dev_id, void *stream)
+{
+   struct rte_compressdev *dev;
+   int ret;
+
+   dev = rte_compressdev_get_dev(dev_id);
+
+   if (dev == NULL || stream == NULL)
+   return -EINVAL;
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stream_free, -ENOTSUP);
+   ret = dev->dev_ops->stream_free(dev, stream);
+   if (ret < 0) {
+   COMPRESSDEV_LOG(ERR,
+   "dev_id %d failed to free stream: err=%d",
+   dev_id, ret);
+   return ret;
+   };
+
+   return 0;
+}
+
 const char * __rte_experimental
 rte_compressdev_name_get(uint8_t dev_id)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index 705e9fe79..f18c7ccf6 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -91,6 +91,8 @@ struct rte_compressdev_config {
/**< Total number of queue pairs to configure on a device */
uint16_t max_nb_priv_xforms;
/**< Max number of private_xforms which will be created on the device */
+   uint16_t max_nb_streams;
+   /**< Max number of streams which will be created on the device */
 };
 
 /**
@@ -316,6 +318,54 @@ uint16_t __rte_experimental
 rte_compressdev_enqueue_burst(uint8_t dev_id, uint16_t qp_id,
struct rte_comp_op **ops, uint16_t nb_ops);
 
+/**
+ * This should alloc a stream from the device's mempool and initialise it.
+ * The application should call this API when setting up for the stateful
+ * processing of a set of data on a device. The API can be called multiple
+ * times to set up a stream for each data set. The handle returned is only for
+ * use with ops of op_type STATEFUL and must be passed to the PMD
+ * with every op in the data stream
+ *
+ * @param dev_id
+ *   Compress device identifier
+ * @param xform
+ *   xform data
+ * @param stream
+ *   Pointer to where PMD's private stream handle should be stored
+ *
+ * @return
+ *  - 0 if successful and valid stream handle
+ *  - <0 in error cases
+ *  - Returns -EINVAL if input parameters are invalid.
+ *  - Returns -ENOTSUP if comp device does not support STATEFUL operations.
+ *  - Returns -ENOTSUP if comp device does not support the comp transform.
+ *  - Returns -ENOMEM if the private stream could not be allocated.
+ *
+ */
+int __rte_experimental
+rte_compressdev_stream_create(uint8_t dev_id,
+   const struct rte_comp_xform *xform,
+   void **stream);
+
+/**
+ * This should clear the stream and return it to the device's mempool.
+ *
+ * @param dev_id
+ *   Compress device identifier
+ *
+ * @param stream
+ *   PMD's private stream data
+ *
+ * @return
+ *  - 0 if successful
+ *  - <0 in error cases
+ *  - Returns -EINVAL if input parameters are invalid.
+ *  - Returns -ENOTSUP if comp device does not support STATEFUL operations.
+ *  - Returns -EBUSY if can't free stream as there are inflight operations
+ */
+int __rte_experimental
+rte_compressdev_stream_free(uint8_t dev_id, void *stream);
+
 /**
  * This should alloc a private_xform from the device's mempool and initialise
  * it. The application should call this API when setting up for 

[dpdk-dev] [PATCH v6 08/14] compressdev: support hash operations

[Pablo de Lara]

From: Shally Verma 

- Added hash algo enumeration and params in xform and rte_comp_op
- Updated compress/decompress xform to input hash algorithm
- Updated struct rte_comp_op to input hash buffer

User in capability query will know about support hashes via
device info comp_feature_flag. If supported, application can initialize
desired algorithm enumeration in xform structure and pass valid hash
buffer during enqueue_burst().

Signed-off-by: Shally Verma 
Signed-off-by: Sunila Sahu 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_comp.h | 32 
 1 file changed, 32 insertions(+)

diff --git a/lib/librte_compressdev/rte_comp.h 
b/lib/librte_compressdev/rte_comp.h
index 996b17871..6df1779af 100644
--- a/lib/librte_compressdev/rte_comp.h
+++ b/lib/librte_compressdev/rte_comp.h
@@ -63,6 +63,17 @@ enum rte_comp_algorithm {
RTE_COMP_ALGO_LIST_END
 };
 
+/** Compression Hash Algorithms */
+enum rte_comp_hash_algorithm {
+   RTE_COMP_HASH_ALGO_NONE = 0,
+   /**< No hash */
+   RTE_COMP_HASH_ALGO_SHA1,
+   /**< SHA1 hash algorithm */
+   RTE_COMP_HASH_ALGO_SHA2_256,
+   /**< SHA256 hash algorithm of SHA2 family */
+   RTE_COMP_HASH_ALGO_LIST_END
+};
+
 /**< Compression Level.
  * The number is interpreted by each PMD differently. However, lower numbers
  * give fastest compression, at the expense of compression ratio while
@@ -172,6 +183,10 @@ struct rte_comp_compress_xform {
 */
enum rte_comp_checksum_type chksum;
/**< Type of checksum to generate on the uncompressed data */
+   enum rte_comp_hash_algorithm hash_algo;
+   /**< Hash algorithm to be used with compress operation. Hash is always
+* done on plaintext.
+*/
 };
 
 /**
@@ -187,6 +202,10 @@ struct rte_comp_decompress_xform {
 * compressed data. If window size can't be supported by the PMD then
 * setup of stream or private_xform should fail.
 */
+   enum rte_comp_hash_algorithm hash_algo;
+   /**< Hash algorithm to be used with decompress operation. Hash is always
+* done on plaintext.
+*/
 };
 
 /**
@@ -272,6 +291,19 @@ struct rte_comp_op {
 * decompress direction.
 */
} dst;
+   struct {
+   uint8_t *digest;
+   /**< Output buffer to store hash output, if enabled in xform.
+* Buffer would contain valid value only after an op with
+* flush flag = RTE_COMP_FLUSH_FULL/FLUSH_FINAL is processed
+* successfully.
+*
+* Length of buffer should be contiguous and large enough to
+* accommodate digest produced by specific hash algo.
+*/
+   rte_iova_t iova_addr;
+   /**< IO address of the buffer */
+   } hash;
enum rte_comp_flush_flag flush_flag;
/**< Defines flush characteristics for the output data.
 * Only applicable in compress direction
-- 
2.14.3

[dpdk-dev] [PATCH v6 09/14] compressdev: add device feature flags

[Pablo de Lara]

From: Fiona Trahe 

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 21 ++
 lib/librte_compressdev/rte_compressdev.h   | 33 ++
 lib/librte_compressdev/rte_compressdev_version.map |  1 +
 3 files changed, 55 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 0ffc43bb9..03f1ab8a7 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -31,6 +31,27 @@ static struct rte_compressdev_global compressdev_globals = {
 
 struct rte_compressdev_global *rte_compressdev_globals = &compressdev_globals;
 
+const char * __rte_experimental
+rte_compressdev_get_feature_name(uint64_t flag)
+{
+   switch (flag) {
+   case RTE_COMPDEV_FF_HW_ACCELERATED:
+   return "HW_ACCELERATED";
+   case RTE_COMPDEV_FF_CPU_SSE:
+   return "CPU_SSE";
+   case RTE_COMPDEV_FF_CPU_AVX:
+   return "CPU_AVX";
+   case RTE_COMPDEV_FF_CPU_AVX2:
+   return "CPU_AVX2";
+   case RTE_COMPDEV_FF_CPU_AVX512:
+   return "CPU_AVX512";
+   case RTE_COMPDEV_FF_CPU_NEON:
+   return "CPU_NEON";
+   default:
+   return NULL;
+   }
+}
+
 static struct rte_compressdev *
 rte_compressdev_get_dev(uint8_t dev_id)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index f18c7ccf6..30cf64c6e 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -21,9 +21,42 @@ extern "C" {
 
 #include "rte_comp.h"
 
+/**
+ * compression device supported feature flags
+ *
+ * @note New features flags should be added to the end of the list
+ *
+ * Keep these flags synchronised with rte_compressdev_get_feature_name()
+ */
+#defineRTE_COMPDEV_FF_HW_ACCELERATED   (1ULL << 0)
+/**< Operations are off-loaded to an external hardware accelerator */
+#defineRTE_COMPDEV_FF_CPU_SSE  (1ULL << 1)
+/**< Utilises CPU SIMD SSE instructions */
+#defineRTE_COMPDEV_FF_CPU_AVX  (1ULL << 2)
+/**< Utilises CPU SIMD AVX instructions */
+#defineRTE_COMPDEV_FF_CPU_AVX2 (1ULL << 3)
+/**< Utilises CPU SIMD AVX2 instructions */
+#defineRTE_COMPDEV_FF_CPU_AVX512   (1ULL << 4)
+/**< Utilises CPU SIMD AVX512 instructions */
+#defineRTE_COMPDEV_FF_CPU_NEON (1ULL << 5)
+/**< Utilises CPU NEON instructions */
+
+/**
+ * Get the name of a compress device feature flag.
+ *
+ * @param flag
+ *   The mask describing the flag
+ *
+ * @return
+ *   The name of this flag, or NULL if it's not a valid feature flag.
+ */
+const char * __rte_experimental
+rte_compressdev_get_feature_name(uint64_t flag);
+
 /**  comp device information */
 struct rte_compressdev_info {
const char *driver_name;/**< Driver name. */
+   uint64_t feature_flags; /**< Feature flags */
uint16_t max_nb_queue_pairs;
/**< Maximum number of queues pairs supported by device.
 * (If 0, there is no limit in maximum number of queue pairs)
diff --git a/lib/librte_compressdev/rte_compressdev_version.map 
b/lib/librte_compressdev/rte_compressdev_version.map
index ca4d9d3d8..f9223bcfe 100644
--- a/lib/librte_compressdev/rte_compressdev_version.map
+++ b/lib/librte_compressdev/rte_compressdev_version.map
@@ -9,6 +9,7 @@ EXPERIMENTAL {
rte_compressdev_dequeue_burst;
rte_compressdev_devices_get;
rte_compressdev_enqueue_burst;
+   rte_compressdev_get_feature_name;
rte_compressdev_info_get;
rte_compressdev_name_get;
rte_compressdev_pmd_allocate;
-- 
2.14.3

[dpdk-dev] [PATCH v6 10/14] compressdev: add compression service feature flags

[Pablo de Lara]

From: Fiona Trahe 

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_comp.c  | 31 +++
 lib/librte_compressdev/rte_comp.h  | 46 ++
 lib/librte_compressdev/rte_compressdev_version.map |  1 +
 3 files changed, 78 insertions(+)

diff --git a/lib/librte_compressdev/rte_comp.c 
b/lib/librte_compressdev/rte_comp.c
index eb99e99d2..d596ba872 100644
--- a/lib/librte_compressdev/rte_comp.c
+++ b/lib/librte_compressdev/rte_comp.c
@@ -6,6 +6,37 @@
 #include "rte_compressdev.h"
 #include "rte_compressdev_internal.h"
 
+const char * __rte_experimental
+rte_comp_get_feature_name(uint64_t flag)
+{
+   switch (flag) {
+   case RTE_COMP_FF_STATEFUL_COMPRESSION:
+   return "STATEFUL_COMPRESSION";
+   case RTE_COMP_FF_STATEFUL_DECOMPRESSION:
+   return "STATEFUL_DECOMPRESSION";
+   case RTE_COMP_FF_MBUF_SCATTER_GATHER:
+   return "MBUF_SCATTER_GATHER";
+   case RTE_COMP_FF_MULTI_PKT_CHECKSUM:
+   return "MULTI_PKT_CHECKSUM";
+   case RTE_COMP_FF_ADLER32_CHECKSUM:
+   return "ADLER32_CHECKSUM";
+   case RTE_COMP_FF_CRC32_CHECKSUM:
+   return "CRC32_CHECKSUM";
+   case RTE_COMP_FF_CRC32_ADLER32_CHECKSUM:
+   return "CRC32_ADLER32_CHECKSUM";
+   case RTE_COMP_FF_NONCOMPRESSED_BLOCKS:
+   return "NONCOMPRESSED_BLOCKS";
+   case RTE_COMP_FF_SHA1_HASH:
+   return "SHA1_HASH";
+   case RTE_COMP_FF_SHA2_SHA256_HASH:
+   return "SHA2_SHA256_HASH";
+   case RTE_COMP_FF_SHAREABLE_PRIV_XFORM:
+   return "SHAREABLE_PRIV_XFORM";
+   default:
+   return NULL;
+   }
+}
+
 /**
  * Reset the fields of an operation to their default values.
  *
diff --git a/lib/librte_compressdev/rte_comp.h 
b/lib/librte_compressdev/rte_comp.h
index 6df1779af..4c802195f 100644
--- a/lib/librte_compressdev/rte_comp.h
+++ b/lib/librte_compressdev/rte_comp.h
@@ -19,6 +19,40 @@ extern "C" {
 #include 
 #include 
 
+/**
+ * compression service feature flags
+ *
+ * @note New features flags should be added to the end of the list
+ *
+ * Keep these flags synchronised with rte_comp_get_feature_name()
+ */
+#define RTE_COMP_FF_STATEFUL_COMPRESSION   (1ULL << 0)
+/**< Stateful compression is supported */
+#define RTE_COMP_FF_STATEFUL_DECOMPRESSION (1ULL << 1)
+/**< Stateful decompression is supported */
+#defineRTE_COMP_FF_MBUF_SCATTER_GATHER (1ULL << 2)
+/**< Scatter-gather mbufs are supported */
+#define RTE_COMP_FF_ADLER32_CHECKSUM   (1ULL << 3)
+/**< Adler-32 Checksum is supported */
+#define RTE_COMP_FF_CRC32_CHECKSUM (1ULL << 4)
+/**< CRC32 Checksum is supported */
+#define RTE_COMP_FF_CRC32_ADLER32_CHECKSUM (1ULL << 5)
+/**< Adler-32/CRC32 Checksum is supported */
+#define RTE_COMP_FF_MULTI_PKT_CHECKSUM (1ULL << 6)
+/**< Generation of checksum across multiple stateless packets is supported */
+#define RTE_COMP_FF_SHA1_HASH  (1ULL << 7)
+/**< SHA1 Hash is supported */
+#define RTE_COMP_FF_SHA2_SHA256_HASH   (1ULL << 8)
+/**< SHA256 Hash of SHA2 family is supported */
+#define RTE_COMP_FF_NONCOMPRESSED_BLOCKS   (1ULL << 9)
+/**< Creation of non-compressed blocks using RTE_COMP_LEVEL_NONE is supported 
*/
+#define RTE_COMP_FF_SHAREABLE_PRIV_XFORM   (1ULL << 10)
+/**< Private xforms created by the PMD can be shared
+ * across multiple stateless operations. If not set, then app needs
+ * to create as many priv_xforms as it expects to have stateless
+ * operations in-flight.
+ */
+
 /** Status of comp operation */
 enum rte_comp_op_status {
RTE_COMP_OP_STATUS_SUCCESS = 0,
@@ -405,6 +439,18 @@ rte_comp_op_bulk_alloc(struct rte_mempool *mempool,
 void __rte_experimental
 rte_comp_op_free(struct rte_comp_op *op);
 
+/**
+ * Get the name of a compress service feature flag
+ *
+ * @param flag
+ *   The mask describing the flag
+ *
+ * @return
+ *   The name of this flag, or NULL if it's not a valid feature flag.
+ */
+const char * __rte_experimental
+rte_comp_get_feature_name(uint64_t flag);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_compressdev/rte_compressdev_version.map 
b/lib/librte_compressdev/rte_compressdev_version.map
index f9223bcfe..5ed3080b4 100644
--- a/lib/librte_compressdev/rte_compressdev_version.map
+++ b/lib/librte_compressdev/rte_compressdev_version.map
@@ -28,6 +28,7 @@ EXPERIMENTAL {
rte_compressdev_stop;
rte_compressdev_stream_create;
rte_compressdev_stream_free;
+   rte_comp_get_feature_name;
rte_comp_op_alloc;
rte_comp_op_bulk_alloc;
rte_comp_op_free;
-- 
2.14.3

[dpdk-dev] [PATCH v6 11/14] compressdev: add device stats

[Pablo de Lara]

From: Fiona Trahe 

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 40 ++
 lib/librte_compressdev/rte_compressdev.h   | 39 +
 lib/librte_compressdev/rte_compressdev_pmd.h   | 26 ++
 lib/librte_compressdev/rte_compressdev_version.map |  2 ++
 4 files changed, 107 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 03f1ab8a7..855385e10 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -555,6 +555,46 @@ rte_compressdev_enqueue_burst(uint8_t dev_id, uint16_t 
qp_id,
dev->data->queue_pairs[qp_id], ops, nb_ops);
 }
 
+int __rte_experimental
+rte_compressdev_stats_get(uint8_t dev_id, struct rte_compressdev_stats *stats)
+{
+   struct rte_compressdev *dev;
+
+   if (!rte_compressdev_is_valid_dev(dev_id)) {
+   COMPRESSDEV_LOG(ERR, "Invalid dev_id=%d", dev_id);
+   return -ENODEV;
+   }
+
+   if (stats == NULL) {
+   COMPRESSDEV_LOG(ERR, "Invalid stats ptr");
+   return -EINVAL;
+   }
+
+   dev = &rte_comp_devices[dev_id];
+   memset(stats, 0, sizeof(*stats));
+
+   RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->stats_get, -ENOTSUP);
+   (*dev->dev_ops->stats_get)(dev, stats);
+   return 0;
+}
+
+void __rte_experimental
+rte_compressdev_stats_reset(uint8_t dev_id)
+{
+   struct rte_compressdev *dev;
+
+   if (!rte_compressdev_is_valid_dev(dev_id)) {
+   COMPRESSDEV_LOG(ERR, "Invalid dev_id=%" PRIu8, dev_id);
+   return;
+   }
+
+   dev = &rte_comp_devices[dev_id];
+
+   RTE_FUNC_PTR_OR_RET(*dev->dev_ops->stats_reset);
+   (*dev->dev_ops->stats_reset)(dev);
+}
+
+
 void __rte_experimental
 rte_compressdev_info_get(uint8_t dev_id, struct rte_compressdev_info *dev_info)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index 30cf64c6e..5ad926976 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -63,6 +63,19 @@ struct rte_compressdev_info {
 */
 };
 
+/** comp device statistics */
+struct rte_compressdev_stats {
+   uint64_t enqueued_count;
+   /**< Count of all operations enqueued */
+   uint64_t dequeued_count;
+   /**< Count of all operations dequeued */
+
+   uint64_t enqueue_err_count;
+   /**< Total error count on operations enqueued */
+   uint64_t dequeue_err_count;
+   /**< Total error count on operations dequeued */
+};
+
 /**
  * Get the compress device name given a device identifier.
  *
@@ -229,6 +242,32 @@ rte_compressdev_queue_pair_setup(uint8_t dev_id, uint16_t 
queue_pair_id,
 uint16_t __rte_experimental
 rte_compressdev_queue_pair_count(uint8_t dev_id);
 
+
+/**
+ * Retrieve the general I/O statistics of a device.
+ *
+ * @param dev_id
+ *   The identifier of the device
+ * @param stats
+ *   A pointer to a structure of type
+ *   *rte_compressdev_stats* to be filled with the
+ *   values of device counters
+ * @return
+ *   - Zero if successful.
+ *   - Non-zero otherwise.
+ */
+int __rte_experimental
+rte_compressdev_stats_get(uint8_t dev_id, struct rte_compressdev_stats *stats);
+
+/**
+ * Reset the general I/O statistics of a device.
+ *
+ * @param dev_id
+ *   The identifier of the device.
+ */
+void __rte_experimental
+rte_compressdev_stats_reset(uint8_t dev_id);
+
 /**
  * Retrieve the contextual information of a device.
  *
diff --git a/lib/librte_compressdev/rte_compressdev_pmd.h 
b/lib/librte_compressdev/rte_compressdev_pmd.h
index c08a3ee63..38e9ea02b 100644
--- a/lib/librte_compressdev/rte_compressdev_pmd.h
+++ b/lib/librte_compressdev/rte_compressdev_pmd.h
@@ -116,6 +116,27 @@ typedef void (*compressdev_stop_t)(struct rte_compressdev 
*dev);
 typedef int (*compressdev_close_t)(struct rte_compressdev *dev);
 
 
+/**
+ * Function used to get statistics of a device.
+ *
+ * @param dev
+ *   Compress device
+ * @param stats
+ *   Compress device stats to populate
+ */
+typedef void (*compressdev_stats_get_t)(struct rte_compressdev *dev,
+   struct rte_compressdev_stats *stats);
+
+
+/**
+ * Function used to reset statistics of a device.
+ *
+ * @param dev
+ *   Compress device
+ */
+typedef void (*compressdev_stats_reset_t)(struct rte_compressdev *dev);
+
+
 /**
  * Function used to get specific information of a device.
  *
@@ -247,6 +268,11 @@ struct rte_compressdev_ops {
 
compressdev_info_get_t dev_infos_get;   /**< Get device info. */
 
+   compressdev_stats_get_t stats_get;
+   /**< Get device statistics. */
+   compressdev_stats_reset_t stats_reset;
+   /**< Reset device statistics. */
+
compressdev_queue_pair_setup_t queue_pair_se

[dpdk-dev] [PATCH v6 12/14] compressdev: add device capabilities

[Pablo de Lara]

From: Fiona Trahe 

Added structure which each PMD will fill out,
providing the capabilities of each driver
(containing mainly which compression services
it supports).

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 23 ++
 lib/librte_compressdev/rte_compressdev.h   | 35 ++
 lib/librte_compressdev/rte_compressdev_version.map |  1 +
 3 files changed, 59 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 855385e10..2d7127ce2 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -31,6 +31,29 @@ static struct rte_compressdev_global compressdev_globals = {
 
 struct rte_compressdev_global *rte_compressdev_globals = &compressdev_globals;
 
+const struct rte_compressdev_capabilities * __rte_experimental
+rte_compressdev_capability_get(uint8_t dev_id,
+   enum rte_comp_algorithm algo)
+{
+   const struct rte_compressdev_capabilities *capability;
+   struct rte_compressdev_info dev_info;
+   int i = 0;
+
+   if (dev_id >= compressdev_globals.nb_devs) {
+   COMPRESSDEV_LOG(ERR, "Invalid dev_id=%d", dev_id);
+   return NULL;
+   }
+   rte_compressdev_info_get(dev_id, &dev_info);
+
+   while ((capability = &dev_info.capabilities[i++])->algo !=
+   RTE_COMP_ALGO_UNSPECIFIED){
+   if (capability->algo == algo)
+   return capability;
+   }
+
+   return NULL;
+}
+
 const char * __rte_experimental
 rte_compressdev_get_feature_name(uint64_t flag)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index 5ad926976..d52103e15 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -21,6 +21,39 @@ extern "C" {
 
 #include "rte_comp.h"
 
+/**
+ * Parameter log base 2 range description.
+ * Final value will be 2^value.
+ */
+struct rte_param_log2_range {
+   uint8_t min;/**< Minimum log2 value */
+   uint8_t max;/**< Maximum log2 value */
+   uint8_t increment;
+   /**< If a range of sizes are supported,
+* this parameter is used to indicate
+* increments in base 2 log byte value
+* that are supported between the minimum and maximum
+*/
+};
+
+/** Structure used to capture a capability of a comp device */
+struct rte_compressdev_capabilities {
+   enum rte_comp_algorithm algo;
+   /* Compression algorithm */
+   uint64_t comp_feature_flags;
+   /**< Bitmask of flags for compression service features */
+   struct rte_param_log2_range window_size;
+   /**< Window size range in base two log byte values */
+};
+
+/** Macro used at end of comp PMD list */
+#define RTE_COMP_END_OF_CAPABILITIES_LIST() \
+   { RTE_COMP_ALGO_UNSPECIFIED }
+
+const struct rte_compressdev_capabilities * __rte_experimental
+rte_compressdev_capability_get(uint8_t dev_id,
+   enum rte_comp_algorithm algo);
+
 /**
  * compression device supported feature flags
  *
@@ -57,6 +90,8 @@ rte_compressdev_get_feature_name(uint64_t flag);
 struct rte_compressdev_info {
const char *driver_name;/**< Driver name. */
uint64_t feature_flags; /**< Feature flags */
+   const struct rte_compressdev_capabilities *capabilities;
+   /**< Array of devices supported capabilities */
uint16_t max_nb_queue_pairs;
/**< Maximum number of queues pairs supported by device.
 * (If 0, there is no limit in maximum number of queue pairs)
diff --git a/lib/librte_compressdev/rte_compressdev_version.map 
b/lib/librte_compressdev/rte_compressdev_version.map
index 8e4004631..6c553341a 100644
--- a/lib/librte_compressdev/rte_compressdev_version.map
+++ b/lib/librte_compressdev/rte_compressdev_version.map
@@ -3,6 +3,7 @@ EXPERIMENTAL {
 
rte_compressdev_callback_register;
rte_compressdev_callback_unregister;
+   rte_compressdev_capability_get;
rte_compressdev_close;
rte_compressdev_configure;
rte_compressdev_count;
-- 
2.14.3

[dpdk-dev] [PATCH v6 13/14] compressdev: get device id from name

[Pablo de Lara]

From: Fiona Trahe 

Added API to retrieve the device id provided the device name.

Signed-off-by: Fiona Trahe 
Signed-off-by: Pablo de Lara 
Signed-off-by: Shally Verma 
Signed-off-by: Ashish Gupta 
---
 lib/librte_compressdev/rte_compressdev.c   | 18 ++
 lib/librte_compressdev/rte_compressdev.h   | 13 +
 lib/librte_compressdev/rte_compressdev_version.map |  1 +
 3 files changed, 32 insertions(+)

diff --git a/lib/librte_compressdev/rte_compressdev.c 
b/lib/librte_compressdev/rte_compressdev.c
index 2d7127ce2..6a38917dd 100644
--- a/lib/librte_compressdev/rte_compressdev.c
+++ b/lib/librte_compressdev/rte_compressdev.c
@@ -117,6 +117,24 @@ rte_compressdev_is_valid_dev(uint8_t dev_id)
 }
 
 
+int __rte_experimental
+rte_compressdev_get_dev_id(const char *name)
+{
+   unsigned int i;
+
+   if (name == NULL)
+   return -1;
+
+   for (i = 0; i < rte_compressdev_globals->nb_devs; i++)
+   if ((strcmp(rte_compressdev_globals->devs[i].data->name, name)
+   == 0) &&
+   (rte_compressdev_globals->devs[i].attached ==
+   RTE_COMPRESSDEV_ATTACHED))
+   return i;
+
+   return -1;
+}
+
 uint8_t __rte_experimental
 rte_compressdev_count(void)
 {
diff --git a/lib/librte_compressdev/rte_compressdev.h 
b/lib/librte_compressdev/rte_compressdev.h
index d52103e15..c86035f14 100644
--- a/lib/librte_compressdev/rte_compressdev.h
+++ b/lib/librte_compressdev/rte_compressdev.h
@@ -111,6 +111,19 @@ struct rte_compressdev_stats {
/**< Total error count on operations dequeued */
 };
 
+
+/**
+ * Get the device identifier for the named compress device.
+ *
+ * @param name
+ *   Device name to select the device structure
+ * @return
+ *   - Returns compress device identifier on success.
+ *   - Return -1 on failure to find named compress device.
+ */
+int __rte_experimental
+rte_compressdev_get_dev_id(const char *name);
+
 /**
  * Get the compress device name given a device identifier.
  *
diff --git a/lib/librte_compressdev/rte_compressdev_version.map 
b/lib/librte_compressdev/rte_compressdev_version.map
index 6c553341a..e5cbc569e 100644
--- a/lib/librte_compressdev/rte_compressdev_version.map
+++ b/lib/librte_compressdev/rte_compressdev_version.map
@@ -10,6 +10,7 @@ EXPERIMENTAL {
rte_compressdev_dequeue_burst;
rte_compressdev_devices_get;
rte_compressdev_enqueue_burst;
+   rte_compressdev_get_dev_id;
rte_compressdev_get_feature_name;
rte_compressdev_info_get;
rte_compressdev_name_get;
-- 
2.14.3

[dpdk-dev] [PATCH v6 14/14] doc: add compressdev library guide

[Pablo de Lara]

From: Ashish Gupta 

Add section in programmer’s guide for Compressdev library.

Signed-off-by: Shally Verma 
Signed-off-by: Sunila Sahu 
Signed-off-by: Ashish Gupta 
---
 MAINTAINERS   |   1 +
 doc/guides/prog_guide/compressdev_lib.rst | 623 ++
 doc/guides/prog_guide/img/stateful-op.svg | 116 
 doc/guides/prog_guide/img/stateless-op-shared.svg | 124 +
 doc/guides/prog_guide/img/stateless-op.svg| 140 +
 doc/guides/prog_guide/index.rst   |   1 +
 6 files changed, 1005 insertions(+)
 create mode 100644 doc/guides/prog_guide/compressdev_lib.rst
 create mode 100644 doc/guides/prog_guide/img/stateful-op.svg
 create mode 100644 doc/guides/prog_guide/img/stateless-op-shared.svg
 create mode 100644 doc/guides/prog_guide/img/stateless-op.svg

diff --git a/MAINTAINERS b/MAINTAINERS
index c1370258f..8de296866 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -369,6 +369,7 @@ M: Pablo de Lara 
 M: Ashish Gupta 
 T: git://dpdk.org/next/dpdk-next-crypto
 F: lib/librte_compressdev/
+F: doc/guides/prog_guide/compressdev_lib.rst
 
 
 Memory Pool Drivers
diff --git a/doc/guides/prog_guide/compressdev_lib.rst 
b/doc/guides/prog_guide/compressdev_lib.rst
new file mode 100644
index 0..87e264906
--- /dev/null
+++ b/doc/guides/prog_guide/compressdev_lib.rst
@@ -0,0 +1,623 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+Copyright(c) 2017-2018 Cavium Networks.
+
+Compression Device Library
+===
+
+The compression framework provides a generic set of APIs to perform 
compression services
+as well as to query and configure compression devices both physical(hardware) 
and virtual(software)
+to perform those services. The framework currently only supports lossless 
compression schemes:
+Deflate and LZS.
+
+Device Management
+-
+
+Device Creation
+~~~
+
+Physical compression devices are discovered during the bus probe of the EAL 
function
+which is executed at DPDK initialization, based on their unique device 
identifier.
+For eg. PCI devices can be identified using PCI BDF (bus/bridge, device, 
function).
+Specific physical compression devices, like other physical devices in DPDK can 
be
+white-listed or black-listed using the EAL command line options.
+
+Virtual devices can be created by two mechanisms, either using the EAL command
+line options or from within the application using an EAL API directly.
+
+From the command line using the --vdev EAL option
+
+.. code-block:: console
+
+   --vdev  ',socket_id=0'
+
+.. Note::
+
+   * If DPDK application requires multiple software compression PMD devices 
then required
+ number of ``--vdev`` with appropriate libraries are to be added.
+
+   * An Application with multiple compression device instances exposed by the 
same PMD must
+ specify a unique name for each device.
+
+   Example: ``--vdev  'pmd0' --vdev  'pmd1'``
+
+Or, by using the rte_vdev_init API within the application code.
+
+.. code-block:: c
+
+   rte_vdev_init("","socket_id=0")
+
+All virtual compression devices support the following initialization 
parameters:
+
+* ``socket_id`` - socket on which to allocate the device resources on.
+
+Device Identification
+~
+
+Each device, whether virtual or physical is uniquely designated by two
+identifiers:
+
+- A unique device index used to designate the compression device in all 
functions
+  exported by the compressdev API.
+
+- A device name used to designate the compression device in console messages, 
for
+  administration or debugging purposes.
+
+Device Configuration
+
+
+The configuration of each compression device includes the following operations:
+
+- Allocation of resources, including hardware resources if a physical device.
+- Resetting the device into a well-known default state.
+- Initialization of statistics counters.
+
+The ``rte_compressdev_configure`` API is used to configure a compression 
device.
+
+The ``rte_compressdev_config`` structure is used to pass the configuration
+parameters.
+
+See *DPDK API Reference* for details.
+
+Configuration of Queue Pairs
+
+
+Each compression device queue pair is individually configured through the
+``rte_compressdev_queue_pair_setup`` API.
+
+The ``max_inflight_ops`` is used to pass maximum number of
+rte_comp_op that could be present in a queue at-a-time.
+PMD then can allocate resources accordingly on a specified socket.
+
+See *DPDK API Reference* for details.
+
+Logical Cores, Memory and Queues Pair Relationships
+~~~
+
+Library supports NUMA similarly as described in Cryptodev library section.
+
+A queue pair cannot be shared and should be exclusively used by a single 
processing
+context for enqueuing operations or dequeuing operations on the same 
compression device
+since sharing would require global locks and hinder performance. It i

Re: [dpdk-dev] [PATCH] app/test: enhance sanity script

[Burakov, Anatoly]

On 27-Apr-18 2:20 PM, Reshma Pattan wrote:

1. Sanity Script is enhanced with additional test cases
being added to autotest_data.py
2. Fixed in autotest_test_funcs.py to handle test cases
which returns "Skipped" as result.
   The issue was skipped test cases got timed out,
causing delay in sanity script execution.
3. Enhanced support for FreeBSD to add the respective memory limit,
as FreeBSD doesn't support socket-mem, file-prefix options.

Signed-off-by: Jananee Parthasarathy 
---


I don't think number 3 on that list is needed. FreeBSD doesn't support 
file-prefixes, so it's not possible to run multiple unit tests in 
parallel in the first place, so no need for grouping and no need to 
limit allocated memory.


On a more general note, IMO this whole thing needs to be redone and 
simplified. I wanted to fix all this as part of Memory Hotplug patches, 
but didn't get to it yet.


(not to mention that this infrastructure itself is a lot like IVSHMEM, 
in that it was built to solve a specific problem, consequences be damned...)


The reasoning behind splitting tests into groups was largely the fact 
that we had to preallocate memory, while also running tests in parallel. 
This is no longer needed for two reasons.


First, most (if not all) tests now clean up properly after themselves 
(when this was written, it wasn't possible to un-reserve memzones, rings 
etc. - now it is), so we can run them one by one without restarting the 
test application.


Second, memory is now allocated on the fly, so there is no more reason 
to split autotests into groups at all. Instead, we can just group all 
tests into "performance" (to be run one by one) and "non performance" 
(can be run concurrently via different prefixes), and run them with a 
pool of workers.


For FreeBSD, since there's no dynamic memory allocation and no support 
for file prefixes, there needs to be only one worker, running all tests 
one by one. For Linux, you can add all unit tests into a queue, and use 
a pool of workers dequeuing and doing tests in parallel under different 
prefixes. No '-m' or '--socket-mem' switches needed.


--
Thanks,
Anatoly

[dpdk-dev] [PATCH v2] devtools: add git log checks for acronyms

[Ferruh Yigit]

checks added for vDPA, MSS, UDP, EEPROM

Signed-off-by: Ferruh Yigit 
---
v2:
* Add more checks: eeprom
---
 devtools/check-git-log.sh | 4 
 1 file changed, 4 insertions(+)

diff --git a/devtools/check-git-log.sh b/devtools/check-git-log.sh
index c601f6ae9..c8c149685 100755
--- a/devtools/check-git-log.sh
+++ b/devtools/check-git-log.sh
@@ -123,12 +123,14 @@ bad=$(echo "$headlines" | grep -E --color=always \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
+   -e ':.*\' \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
+   -e ':.*\' \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
@@ -138,7 +140,9 @@ bad=$(echo "$headlines" | grep -E --color=always \
-e ':.*\' \
-e ':.*\' \
-e ':.*\' \
+   -e ':.*\' \
-e ':.*\<[Vv]lan\>' \
+   -e ':.*\' \
-e ':.*\' \
| sed 's,^,\t,')
 [ -z "$bad" ] || printf "Wrong headline lowercase:\n$bad\n"
-- 
2.14.3

[dpdk-dev] [PATCH v2 1/3] build: fix kernel compile on cross-build

[Bruce Richardson]

When cross-compiling, if no kernel_dir was specified, then the kernel
modules were still being compiled for the build machine. Fix this by
only building modules on cross-compile when we have a kernel_dir value
set. Print out a message indicating why we are skipping kernel
compilation, and in case that the headers for kernel compile are not
found, print a warning instead of erroring out.

Fixes: a52f4574f798 ("igb_uio: build with meson")

Signed-off-by: Bruce Richardson 
---
 kernel/linux/igb_uio/meson.build |  6 --
 kernel/linux/meson.build | 40 +---
 meson.build  |  6 +-
 3 files changed, 42 insertions(+), 10 deletions(-)

diff --git a/kernel/linux/igb_uio/meson.build b/kernel/linux/igb_uio/meson.build
index 356f4ab32..71ed2e7a8 100644
--- a/kernel/linux/igb_uio/meson.build
+++ b/kernel/linux/igb_uio/meson.build
@@ -1,12 +1,6 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-kernel_dir = get_option('kernel_dir')
-if kernel_dir == ''
-   kernel_version = run_command('uname', '-r').stdout().strip()
-   kernel_dir = '/lib/modules/' + kernel_version + '/build'
-endif
-
 mkfile = custom_target('igb_uio_makefile',
output: 'Makefile',
command: ['touch', '@OUTPUT@'])
diff --git a/kernel/linux/meson.build b/kernel/linux/meson.build
index a5b88f0c6..a924c7b6f 100644
--- a/kernel/linux/meson.build
+++ b/kernel/linux/meson.build
@@ -3,6 +3,40 @@
 
 subdirs = ['igb_uio']
 
-foreach d:subdirs
-   subdir(d)
-endforeach
+WARN_CROSS_COMPILE='Need "kernel_dir" option for kmod compilation when 
cross-compiling'
+WARN_NO_HEADERS='Cannot compile kernel modules as requested - are kernel 
headers installed?'
+
+# if we are cross-compiling we need kernel_dir specified
+# NOTE: warning() function only available from version 0.44 onwards
+if get_option('kernel_dir') == '' and meson.is_cross_build()
+   if meson.version().version_compare('>=0.44')
+   warning(WARN_CROSS_COMPILE)
+   else
+   message('WARNING: ' + WARN_CROSS_COMPILE)
+   endif
+else
+
+   kernel_dir = get_option('kernel_dir')
+   if kernel_dir == ''
+   # use default path for native builds
+   kernel_version = run_command('uname', '-r').stdout().strip()
+   kernel_dir = '/lib/modules/' + kernel_version + '/build'
+   endif
+
+   # test running make in kernel directory, using "make kernelversion"
+   make_returncode = run_command('make', '-sC', kernel_dir,
+   'kernelversion').returncode()
+   if make_returncode != 0
+   if meson.version().version_compare('>=0.44')
+   warning(WARN_NO_HEADERS)
+   else
+   message('WARNING: ' + WARN_NO_HEADERS)
+   endif
+   else # returncode == 0
+
+# DO ACTUAL MODULE BUILDING
+   foreach d:subdirs
+   subdir(d)
+   endforeach
+   endif
+endif
diff --git a/meson.build b/meson.build
index cc16595cb..52a8746c5 100644
--- a/meson.build
+++ b/meson.build
@@ -26,7 +26,6 @@ subdir('config')
 
 # build libs and drivers
 subdir('lib')
-subdir('kernel')
 subdir('buildtools')
 subdir('drivers')
 
@@ -40,6 +39,11 @@ if get_option('examples') != ''
subdir('examples')
 endif
 
+# build kernel modules if enabled
+if get_option('enable_kmods')
+   subdir('kernel')
+endif
+
 # write the build config
 build_cfg = 'rte_build_config.h'
 configure_file(output: build_cfg,
-- 
2.14.3

[dpdk-dev] [PATCH v2 0/3] small improvements to meson build

[Bruce Richardson]

A few small improvements and fixes for the meson build system, including
improved behaviour for building kmods - especially on cross-compile, a fix
for our libbsd detect on CentOS 7, and a small change to ensure compatibility
with future meson versions.

Bruce Richardson (3):
  build: fix kernel compile on cross-build
  build: fix check for libbsd in meson
  build: ensure compatibility with future meson versions

 config/meson.build   |  3 ++-
 kernel/linux/igb_uio/meson.build |  6 --
 kernel/linux/meson.build | 40 +---
 lib/meson.build  |  2 +-
 meson.build  |  6 +-
 5 files changed, 45 insertions(+), 12 deletions(-)

-- 
2.14.3

[dpdk-dev] [PATCH v2 3/3] build: ensure compatibility with future meson versions

[Bruce Richardson]

Meson 0.46 fixed a bug where "extract_all_objects" would not recursively
extract objects not compiled from source for a target. To keep backward
compatibility, a "recursive" keyword-arg was added to make this optional.
The value is "false" by default for now, but will change to "true" in
future, so we hard-code it to "false" in our code to ensure future
compatiblity.

Signed-off-by: Bruce Richardson 
---
 lib/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/meson.build b/lib/meson.build
index 73d6f25c7..ae445153b 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -96,7 +96,7 @@ foreach l:libraries
 
# then use pre-build objects to build shared lib
sources = []
-   objs += static_lib.extract_all_objects()
+   objs += static_lib.extract_all_objects(recursive: false)
version_map = '@0@/@1@/rte_@2@_version.map'.format(
meson.current_source_dir(), dir_name, 
name)
shared_lib = shared_library(libname,
-- 
2.14.3

[dpdk-dev] [PATCH v2 2/3] build: fix check for libbsd in meson

[Bruce Richardson]

When we check for libbsd we were just checking for the library itself.
However, on some distros, e.g. CentOS, libbsd will be found even though
it's not usable by DPDK as it's missing the string header. Therefore we
should check for both the library and the header file.

Fixes: 5364de644a4b ("eal: support strlcpy function")

Signed-off-by: Bruce Richarson 
---
 config/meson.build | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/config/meson.build b/config/meson.build
index 5815b5602..2179c689c 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -43,7 +43,8 @@ if numa_dep.found() and cc.has_header('numaif.h')
 endif
 
 # check for strlcpy
-if host_machine.system() == 'linux' and cc.find_library('bsd', required: 
false).found()
+if host_machine.system() == 'linux' and cc.find_library('bsd',
+   required: false).found() and cc.has_header('bsd/string.h')
dpdk_conf.set('RTE_USE_LIBBSD', 1)
add_project_link_arguments('-lbsd', language: 'c')
dpdk_extra_ldflags += '-lbsd'
-- 
2.14.3

Re: [dpdk-dev] [PATCH v2 2/3] build: fix check for libbsd in meson

[Bruce Richardson]

On Fri, Apr 27, 2018 at 02:49:27PM +0100, Bruce Richardson wrote:
> When we check for libbsd we were just checking for the library itself.
> However, on some distros, e.g. CentOS, libbsd will be found even though
> it's not usable by DPDK as it's missing the string header. Therefore we
> should check for both the library and the header file.
> 
> Fixes: 5364de644a4b ("eal: support strlcpy function")
> 

Sorry, forgot to add:

Reported-by: Vladimir Medvedkin 

> Signed-off-by: Bruce Richarson 
> ---
>  config/meson.build | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/config/meson.build b/config/meson.build
> index 5815b5602..2179c689c 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -43,7 +43,8 @@ if numa_dep.found() and cc.has_header('numaif.h')
>  endif
>  
>  # check for strlcpy
> -if host_machine.system() == 'linux' and cc.find_library('bsd', required: 
> false).found()
> +if host_machine.system() == 'linux' and cc.find_library('bsd',
> + required: false).found() and cc.has_header('bsd/string.h')
>   dpdk_conf.set('RTE_USE_LIBBSD', 1)
>   add_project_link_arguments('-lbsd', language: 'c')
>   dpdk_extra_ldflags += '-lbsd'
> -- 
> 2.14.3
> 

Re: [dpdk-dev] [PATCH] app/test: enhance sanity script

[Burakov, Anatoly]

On 27-Apr-18 2:20 PM, Reshma Pattan wrote:

1. Sanity Script is enhanced with additional test cases
being added to autotest_data.py
2. Fixed in autotest_test_funcs.py to handle test cases
which returns "Skipped" as result.
   The issue was skipped test cases got timed out,
causing delay in sanity script execution.
3. Enhanced support for FreeBSD to add the respective memory limit,
as FreeBSD doesn't support socket-mem, file-prefix options.

Signed-off-by: Jananee Parthasarathy 
---


<...>


@@ -1,5 +1,5 @@
  # SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2010-2014 Intel Corporation
+# Copyright(c) 2010-2018 Intel Corporation
  
  # The main logic behind running autotests in parallel
  
@@ -201,6 +201,13 @@ def __init__(self, cmdline, target, blacklist, whitelist):

  # set up cmdline string
  def __get_cmdline(self, test):
  cmdline = self.cmdline
+os_name = platform.system()
+
+# append the respective memory limitation on FREEBSD
+# as socket-mem, file-prefix is not supported
+if os_name == "FreeBSD" :
+cmdline += " -m " + str(sum(map(int, test["Memory"].split(","
+return cmdline


A better fix would've been to completely avoid calling __get_cmdline() 
for FreeBSD. Since file prefixes are not supported, tests cannot be run 
in parallel, so there is no need to limit allocated memory in the first 
place, so neither -m nor --socket-mem switches need to be added either. 
And without those, there's no need to modify the test cmdline at all.


--
Thanks,
Anatoly

[dpdk-dev] [PATCH] vhost/crypto: fix missing head correction

[Fan Zhang]

Fixes: 473387aaae4a ("vhost/crypto: move to safe GPA translation API")

This patch fixes the missing head descriptor correction for
indirect descriptors.

Signed-off-by: Fan Zhang 
---
 lib/librte_vhost/vhost_crypto.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index c38eb3bb5..81fab7e94 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -919,6 +919,7 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
if (unlikely(!desc || dlen != head->len))
return -1;
desc_idx = 0;
+   head = desc;
} else {
desc = head;
}
-- 
2.13.6

[dpdk-dev] [PATCH] vhost/crypto: fix condition check

[Fan Zhang]

Fixes: 3bb595ecd682 ("vhost/crypto: add request handler")

This patch fix final condition check while moving virtqueue
descriptors.

Signed-off-by: Fan Zhang 
---
 lib/librte_vhost/vhost_crypto.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index c38eb3bb5..4126c3bc1 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -506,7 +506,7 @@ move_desc(struct vring_desc *head, struct vring_desc 
**cur_desc,
left -= desc->len;
}
 
-   if (unlikely(left < 0)) {
+   if (unlikely(left > 0)) {
VC_LOG_ERR("Incorrect virtio descriptor");
return -1;
}
@@ -553,7 +553,7 @@ copy_data(void *dst_data, struct vring_desc *head, struct 
rte_vhost_memory *mem,
left -= to_copy;
}
 
-   if (unlikely(left < 0)) {
+   if (unlikely(left > 0)) {
VC_LOG_ERR("Incorrect virtio descriptor");
return -1;
}
-- 
2.13.6

[dpdk-dev] [PATCH v3 0/5] Initial compressdev unit tests

[Pablo de Lara]

Added initial tests for Compressdev library.
The tests are performed compressing a test buffer (or multiple test buffers)
with compressdev or Zlib, and decompressing it/them with the other library
(if compression is done with compressdev, decompression is done with Zlib,
and viceversa).

Tests added so far are based on the deflate algorithm,
including:
- Fixed huffman on single buffer
- Dynamic huffman on single buffer
- Multi compression level test on single buffer
- Multi buffer
- Multi xform using the same buffer

Due to a dependency on Zlib, the test is not enabled by default.
Once the library is installed, the configuration option
CONFIG_RTE_COMPRESSDEV_TEST must be set to Y.
However, if building with Meson, the test will be built automatically,
if Zlib is installed.

The test requires a compressdev PMD to be initialized,
when running the test app. For example:

./build/app/test --vdev="compress_X"

RTE>>compressdev_autotest

This patchset depends on the Compressdev API patchset:
http://dpdk.org/ml/archives/dev/2018-April/099580.html
("[PATCH v6 00/14] Implement compression API")

Changes in v3:
- Remove next pointer in xform setting
- Remove unneeded DIV_CEIL macro
- Add rte_compressdev_close() call after finishing test cases

Changes in v2:
- Add meson build
- Add invalid configuration tests
- Use new Compressdev API:
  * Substitute session with priv xform
  * Check if priv xform is shareable and create one per operation if not

Pablo de Lara (5):
  test/compress: add initial unit tests
  test/compress: add multi op test
  test/compress: add multi level test
  test/compress: add multi xform test
  test/compress: add invalid configuration tests

 config/common_base   |5 +
 test/test/Makefile   |9 +
 test/test/meson.build|8 +
 test/test/test_compressdev.c | 1092 ++
 test/test/test_compressdev_test_buffer.h |  295 
 5 files changed, 1409 insertions(+)
 create mode 100644 test/test/test_compressdev.c
 create mode 100644 test/test/test_compressdev_test_buffer.h

-- 
2.14.3

[dpdk-dev] [PATCH v3 1/5] test/compress: add initial unit tests

[Pablo de Lara]

This commit introduces the initial tests for compressdev,
performing basic compression and decompression operations
of sample test buffers, using the Zlib library in one direction
and compressdev in another direction, to make sure that
the library is compatible with Zlib.

Due to the use of Zlib API, the test is disabled by default,
to avoid adding a new dependency on DPDK.

Signed-off-by: Pablo de Lara 
Signed-off-by: Ashish Gupta 
Signed-off-by: Shally Verma 
---
 config/common_base   |   5 +
 test/test/Makefile   |   9 +
 test/test/meson.build|   8 +
 test/test/test_compressdev.c | 725 +++
 test/test/test_compressdev_test_buffer.h | 295 +
 5 files changed, 1042 insertions(+)
 create mode 100644 test/test/test_compressdev.c
 create mode 100644 test/test/test_compressdev_test_buffer.h

diff --git a/config/common_base b/config/common_base
index 23cf00011..fdec213cc 100644
--- a/config/common_base
+++ b/config/common_base
@@ -574,6 +574,11 @@ CONFIG_RTE_LIBRTE_SECURITY=y
 CONFIG_RTE_LIBRTE_COMPRESSDEV=y
 CONFIG_RTE_COMPRESS_MAX_DEVS=64
 
+#
+# Compile compressdev unit test
+#
+CONFIG_RTE_COMPRESSDEV_TEST=n
+
 #
 # Compile generic event device library
 #
diff --git a/test/test/Makefile b/test/test/Makefile
index 2630ab484..77d6369d6 100644
--- a/test/test/Makefile
+++ b/test/test/Makefile
@@ -180,6 +180,10 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring_perf.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_blockcipher.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev.c
 
+ifeq ($(CONFIG_RTE_COMPRESSDEV_TEST),y)
+SRCS-$(CONFIG_RTE_LIBRTE_COMPRESSDEV) += test_compressdev.c
+endif
+
 ifeq ($(CONFIG_RTE_LIBRTE_EVENTDEV),y)
 SRCS-y += test_eventdev.c
 SRCS-y += test_event_ring.c
@@ -201,6 +205,11 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS += -D_GNU_SOURCE
 
 LDLIBS += -lm
+ifeq ($(CONFIG_RTE_COMPRESSDEV_TEST),y)
+ifeq ($(CONFIG_RTE_LIBRTE_COMPRESSDEV),y)
+LDLIBS += -lz
+endif
+endif
 
 # Disable VTA for memcpy test
 ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
diff --git a/test/test/meson.build b/test/test/meson.build
index ad0a65080..d71cd3c83 100644
--- a/test/test/meson.build
+++ b/test/test/meson.build
@@ -234,6 +234,14 @@ if dpdk_conf.has('RTE_LIBRTE_KNI')
 endif
 
 test_dep_objs = []
+compress_test_dep = dependency('zlib', required: false)
+if compress_test_dep.found()
+   test_dep_objs += compress_test_dep
+   test_sources += 'test_compressdev.c'
+   test_deps += 'compressdev'
+   test_names += 'compressdev_autotest'
+endif
+
 foreach d:test_deps
def_lib = get_option('default_library')
test_dep_objs += get_variable(def_lib + '_rte_' + d)
diff --git a/test/test/test_compressdev.c b/test/test/test_compressdev.c
new file mode 100644
index 0..a1c4f1027
--- /dev/null
+++ b/test/test/test_compressdev.c
@@ -0,0 +1,725 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2018 Intel Corporation
+ */
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "test_compressdev_test_buffer.h"
+#include "test.h"
+
+#define DEFAULT_WINDOW_SIZE 15
+#define DEFAULT_MEM_LEVEL 8
+#define MAX_DEQD_RETRIES 10
+#define DEQUEUE_WAIT_TIME 1
+
+/*
+ * 30% extra size for compressed data compared to original data,
+ * in case data size cannot be reduced and it is actually bigger
+ * due to the compress block headers
+ */
+#define COMPRESS_BUF_SIZE_RATIO 1.3
+#define NUM_MBUFS 16
+#define NUM_OPS 16
+#define NUM_MAX_XFORMS 1
+#define NUM_MAX_INFLIGHT_OPS 128
+#define CACHE_SIZE 0
+
+const char *
+huffman_type_strings[] = {
+   [RTE_COMP_HUFFMAN_DEFAULT]  = "PMD default",
+   [RTE_COMP_HUFFMAN_FIXED]= "Fixed",
+   [RTE_COMP_HUFFMAN_DYNAMIC]  = "Dynamic"
+};
+
+enum zlib_direction {
+   ZLIB_NONE,
+   ZLIB_COMPRESS,
+   ZLIB_DECOMPRESS,
+   ZLIB_ALL
+};
+
+struct comp_testsuite_params {
+   struct rte_mempool *mbuf_pool;
+   struct rte_mempool *op_pool;
+   struct rte_comp_xform def_comp_xform;
+   struct rte_comp_xform def_decomp_xform;
+};
+
+static struct comp_testsuite_params testsuite_params = { 0 };
+
+static void
+testsuite_teardown(void)
+{
+   struct comp_testsuite_params *ts_params = &testsuite_params;
+
+   rte_mempool_free(ts_params->mbuf_pool);
+   rte_mempool_free(ts_params->op_pool);
+}
+
+static int
+testsuite_setup(void)
+{
+   struct comp_testsuite_params *ts_params = &testsuite_params;
+   unsigned int i;
+
+   if (rte_compressdev_count() == 0) {
+   RTE_LOG(ERR, USER1, "Need at least one compress device\n");
+   return -EINVAL;
+   }
+
+   uint32_t max_buf_size = 0;
+   for (i = 0; i < RTE_DIM(compress_test_bufs); i++)
+   max_buf_size = RTE_MAX(max_buf_size,
+   strlen(compress_test_bufs[i]) + 1);
+
+   max_buf_size *= COMPRESS_BUF_SIZE_RATI

[dpdk-dev] [PATCH v3 2/5] test/compress: add multi op test

[Pablo de Lara]

Add test that checks if multiple operations with
different buffers can be handled on a single enqueue call.

Signed-off-by: Pablo de Lara 
---
 test/test/test_compressdev.c | 476 +--
 1 file changed, 319 insertions(+), 157 deletions(-)

diff --git a/test/test/test_compressdev.c b/test/test/test_compressdev.c
index a1c4f1027..8e67faec2 100644
--- a/test/test/test_compressdev.c
+++ b/test/test/test_compressdev.c
@@ -45,6 +45,10 @@ enum zlib_direction {
ZLIB_ALL
 };
 
+struct priv_op_data {
+   uint16_t orig_idx;
+};
+
 struct comp_testsuite_params {
struct rte_mempool *mbuf_pool;
struct rte_mempool *op_pool;
@@ -97,7 +101,8 @@ testsuite_setup(void)
}
 
ts_params->op_pool = rte_comp_op_pool_create("op_pool", NUM_OPS,
-   0, 0, rte_socket_id());
+   0, sizeof(struct priv_op_data),
+   rte_socket_id());
if (ts_params->op_pool == NULL) {
RTE_LOG(ERR, USER1, "Operation pool could not be created\n");
goto exit;
@@ -337,7 +342,9 @@ decompress_zlib(struct rte_comp_op *op,
  * Compresses and decompresses buffer with compressdev API and Zlib API
  */
 static int
-test_deflate_comp_decomp(const char *test_buffer,
+test_deflate_comp_decomp(const char * const test_bufs[],
+   unsigned int num_bufs,
+   uint16_t buf_idx[],
struct rte_comp_xform *compress_xform,
struct rte_comp_xform *decompress_xform,
enum rte_comp_op_type state,
@@ -346,95 +353,146 @@ test_deflate_comp_decomp(const char *test_buffer,
struct comp_testsuite_params *ts_params = &testsuite_params;
int ret_status = -1;
int ret;
-   struct rte_mbuf *comp_buf = NULL;
-   struct rte_mbuf *uncomp_buf = NULL;
-   struct rte_comp_op *op = NULL;
-   struct rte_comp_op *op_processed = NULL;
-   void *priv_xform = NULL;
-   uint16_t num_deqd;
+   struct rte_mbuf *uncomp_bufs[num_bufs];
+   struct rte_mbuf *comp_bufs[num_bufs];
+   struct rte_comp_op *ops[num_bufs];
+   struct rte_comp_op *ops_processed[num_bufs];
+   void *priv_xforms[num_bufs];
+   uint16_t num_enqd, num_deqd, num_total_deqd;
+   uint16_t num_priv_xforms = 0;
unsigned int deqd_retries = 0;
+   struct priv_op_data *priv_data;
char *data_ptr;
-
-   /* Prepare the source mbuf with the data */
-   uncomp_buf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
-   if (uncomp_buf == NULL) {
+   unsigned int i;
+   const struct rte_compressdev_capabilities *capa =
+   rte_compressdev_capability_get(0, RTE_COMP_ALGO_DEFLATE);
+
+   /* Initialize all arrays to NULL */
+   memset(uncomp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs);
+   memset(comp_bufs, 0, sizeof(struct rte_mbuf *) * num_bufs);
+   memset(ops, 0, sizeof(struct rte_comp_op *) * num_bufs);
+   memset(ops_processed, 0, sizeof(struct rte_comp_op *) * num_bufs);
+   memset(priv_xforms, 0, sizeof(void *) * num_bufs);
+
+   /* Prepare the source mbufs with the data */
+   ret = rte_pktmbuf_alloc_bulk(ts_params->mbuf_pool, uncomp_bufs, 
num_bufs);
+   if (ret < 0) {
RTE_LOG(ERR, USER1,
-   "Source mbuf could not be allocated "
+   "Source mbufs could not be allocated "
"from the mempool\n");
goto exit;
}
 
-   data_ptr = rte_pktmbuf_append(uncomp_buf, strlen(test_buffer) + 1);
-   snprintf(data_ptr, strlen(test_buffer) + 1, "%s", test_buffer);
+   for (i = 0; i < num_bufs; i++) {
+   data_ptr = rte_pktmbuf_append(uncomp_bufs[i],
+   strlen(test_bufs[i]) + 1);
+   snprintf(data_ptr, strlen(test_bufs[i]) + 1, "%s",
+   test_bufs[i]);
+   }
 
-   /* Prepare the destination mbuf */
-   comp_buf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
-   if (comp_buf == NULL) {
+   /* Prepare the destination mbufs */
+   ret = rte_pktmbuf_alloc_bulk(ts_params->mbuf_pool, comp_bufs, num_bufs);
+   if (ret < 0) {
RTE_LOG(ERR, USER1,
-   "Destination mbuf could not be allocated "
+   "Destination mbufs could not be allocated "
"from the mempool\n");
goto exit;
}
 
-   rte_pktmbuf_append(comp_buf,
-   strlen(test_buffer) * COMPRESS_BUF_SIZE_RATIO);
+   for (i = 0; i < num_bufs; i++)
+   rte_pktmbuf_append(comp_bufs[i],
+   strlen(test_bufs[i]) * COMPRESS_BUF_SIZE_RATIO);
 
/* Build the compression operations */
-   op = rte_comp_op_alloc(ts_params->op_pool);
-   if (op == NULL) {
+   ret = rte_comp_op_bulk_alloc(

[dpdk-dev] [PATCH v3 4/5] test/compress: add multi xform test

[Pablo de Lara]

Add test that checks if multiple xforms can be
handled on a single enqueue call.

Signed-off-by: Pablo de Lara 
---
 test/test/test_compressdev.c | 257 ---
 1 file changed, 191 insertions(+), 66 deletions(-)

diff --git a/test/test/test_compressdev.c b/test/test/test_compressdev.c
index bb026d74f..0253d12ea 100644
--- a/test/test/test_compressdev.c
+++ b/test/test/test_compressdev.c
@@ -27,7 +27,7 @@
 #define COMPRESS_BUF_SIZE_RATIO 1.3
 #define NUM_MBUFS 16
 #define NUM_OPS 16
-#define NUM_MAX_XFORMS 1
+#define NUM_MAX_XFORMS 16
 #define NUM_MAX_INFLIGHT_OPS 128
 #define CACHE_SIZE 0
 
@@ -52,8 +52,8 @@ struct priv_op_data {
 struct comp_testsuite_params {
struct rte_mempool *mbuf_pool;
struct rte_mempool *op_pool;
-   struct rte_comp_xform def_comp_xform;
-   struct rte_comp_xform def_decomp_xform;
+   struct rte_comp_xform *def_comp_xform;
+   struct rte_comp_xform *def_decomp_xform;
 };
 
 static struct comp_testsuite_params testsuite_params = { 0 };
@@ -65,6 +65,8 @@ testsuite_teardown(void)
 
rte_mempool_free(ts_params->mbuf_pool);
rte_mempool_free(ts_params->op_pool);
+   rte_free(ts_params->def_comp_xform);
+   rte_free(ts_params->def_decomp_xform);
 }
 
 static int
@@ -108,19 +110,24 @@ testsuite_setup(void)
goto exit;
}
 
+   ts_params->def_comp_xform =
+   rte_malloc(NULL, sizeof(struct rte_comp_xform), 0);
+   ts_params->def_decomp_xform =
+   rte_malloc(NULL, sizeof(struct rte_comp_xform), 0);
+
/* Initializes default values for compress/decompress xforms */
-   ts_params->def_comp_xform.type = RTE_COMP_COMPRESS;
-   ts_params->def_comp_xform.compress.algo = RTE_COMP_ALGO_DEFLATE,
-   ts_params->def_comp_xform.compress.deflate.huffman =
+   ts_params->def_comp_xform->type = RTE_COMP_COMPRESS;
+   ts_params->def_comp_xform->compress.algo = RTE_COMP_ALGO_DEFLATE,
+   ts_params->def_comp_xform->compress.deflate.huffman =
RTE_COMP_HUFFMAN_DEFAULT;
-   ts_params->def_comp_xform.compress.level = RTE_COMP_LEVEL_PMD_DEFAULT;
-   ts_params->def_comp_xform.compress.chksum = RTE_COMP_CHECKSUM_NONE;
-   ts_params->def_comp_xform.compress.window_size = DEFAULT_WINDOW_SIZE;
+   ts_params->def_comp_xform->compress.level = RTE_COMP_LEVEL_PMD_DEFAULT;
+   ts_params->def_comp_xform->compress.chksum = RTE_COMP_CHECKSUM_NONE;
+   ts_params->def_comp_xform->compress.window_size = DEFAULT_WINDOW_SIZE;
 
-   ts_params->def_decomp_xform.type = RTE_COMP_DECOMPRESS;
-   ts_params->def_decomp_xform.decompress.algo = RTE_COMP_ALGO_DEFLATE,
-   ts_params->def_decomp_xform.decompress.chksum = RTE_COMP_CHECKSUM_NONE;
-   ts_params->def_decomp_xform.decompress.window_size = 
DEFAULT_WINDOW_SIZE;
+   ts_params->def_decomp_xform->type = RTE_COMP_DECOMPRESS;
+   ts_params->def_decomp_xform->decompress.algo = RTE_COMP_ALGO_DEFLATE,
+   ts_params->def_decomp_xform->decompress.chksum = RTE_COMP_CHECKSUM_NONE;
+   ts_params->def_decomp_xform->decompress.window_size = 
DEFAULT_WINDOW_SIZE;
 
return TEST_SUCCESS;
 
@@ -345,8 +352,9 @@ static int
 test_deflate_comp_decomp(const char * const test_bufs[],
unsigned int num_bufs,
uint16_t buf_idx[],
-   struct rte_comp_xform *compress_xform,
-   struct rte_comp_xform *decompress_xform,
+   struct rte_comp_xform *compress_xforms[],
+   struct rte_comp_xform *decompress_xforms[],
+   unsigned int num_xforms,
enum rte_comp_op_type state,
enum zlib_direction zlib_dir)
 {
@@ -441,8 +449,9 @@ test_deflate_comp_decomp(const char * const test_bufs[],
/* Compress data (either with Zlib API or compressdev API */
if (zlib_dir == ZLIB_COMPRESS || zlib_dir == ZLIB_ALL) {
for (i = 0; i < num_bufs; i++) {
-   ret = compress_zlib(ops[i],
-   (const struct rte_comp_xform *)compress_xform,
+   const struct rte_comp_xform *compress_xform =
+   compress_xforms[i % num_xforms];
+   ret = compress_zlib(ops[i], compress_xform,
DEFAULT_MEM_LEVEL);
if (ret < 0)
goto exit;
@@ -450,11 +459,11 @@ test_deflate_comp_decomp(const char * const test_bufs[],
ops_processed[i] = ops[i];
}
} else {
-   if (capa->comp_feature_flags & 
RTE_COMP_FF_SHAREABLE_PRIV_XFORM) {
-   /* Create single compress private xform data */
+   /* Create compress private xform data */
+   for (i = 0; i < num_xforms; i++) {
ret = rte_compressdev_private_

[dpdk-dev] [PATCH v3 5/5] test/compress: add invalid configuration tests

[Pablo de Lara]

Add tests that check if device configuration
is not successful when providing invalid parameters.

Signed-off-by: Pablo de Lara 
---
 test/test/test_compressdev.c | 49 +++-
 1 file changed, 48 insertions(+), 1 deletion(-)

diff --git a/test/test/test_compressdev.c b/test/test/test_compressdev.c
index 0253d12ea..bb683b998 100644
--- a/test/test/test_compressdev.c
+++ b/test/test/test_compressdev.c
@@ -175,6 +175,51 @@ generic_ut_teardown(void)
RTE_LOG(ERR, USER1, "Device could not be closed\n");
 }
 
+static int
+test_compressdev_invalid_configuration(void)
+{
+   struct rte_compressdev_config invalid_config;
+   struct rte_compressdev_config valid_config = {
+   .socket_id = rte_socket_id(),
+   .nb_queue_pairs = 1,
+   .max_nb_priv_xforms = NUM_MAX_XFORMS,
+   .max_nb_streams = 0
+   };
+   struct rte_compressdev_info dev_info;
+
+   /* Invalid configuration with 0 queue pairs */
+   memcpy(&invalid_config, &valid_config,
+   sizeof(struct rte_compressdev_config));
+   invalid_config.nb_queue_pairs = 0;
+
+   TEST_ASSERT_FAIL(rte_compressdev_configure(0, &invalid_config),
+   "Device configuration was successful "
+   "with no queue pairs (invalid)\n");
+
+   /*
+* Invalid configuration with too many queue pairs
+* (if there is an actual maximum number of queue pairs)
+*/
+   rte_compressdev_info_get(0, &dev_info);
+   if (dev_info.max_nb_queue_pairs != 0) {
+   memcpy(&invalid_config, &valid_config,
+   sizeof(struct rte_compressdev_config));
+   invalid_config.nb_queue_pairs = dev_info.max_nb_queue_pairs + 1;
+
+   TEST_ASSERT_FAIL(rte_compressdev_configure(0, &invalid_config),
+   "Device configuration was successful "
+   "with too many queue pairs (invalid)\n");
+   }
+
+   /* Invalid queue pair setup, with no number of queue pairs set */
+   TEST_ASSERT_FAIL(rte_compressdev_queue_pair_setup(0, 0,
+   NUM_MAX_INFLIGHT_OPS, rte_socket_id()),
+   "Queue pair setup was successful "
+   "with no queue pairs set (invalid)\n");
+
+   return TEST_SUCCESS;
+}
+
 static int
 compare_buffers(const char *buffer1, uint32_t buffer1_len,
const char *buffer2, uint32_t buffer2_len)
@@ -690,7 +735,7 @@ test_deflate_comp_decomp(const char * const test_bufs[],
 
/* Attach non shareable private xform data to ops */
for (i = 0; i < num_bufs; i++) {
-   priv_data = (struct priv_op_data *) (ops[i] + 
1);
+   priv_data = (struct priv_op_data *)(ops[i] + 1);
uint16_t xform_idx = priv_data->orig_idx;
ops[i]->private_xform = priv_xforms[xform_idx];
}
@@ -1022,6 +1067,8 @@ static struct unit_test_suite compressdev_testsuite  = {
.setup = testsuite_setup,
.teardown = testsuite_teardown,
.unit_test_cases = {
+   TEST_CASE_ST(NULL, NULL,
+   test_compressdev_invalid_configuration),
TEST_CASE_ST(generic_ut_setup, generic_ut_teardown,
test_compressdev_deflate_stateless_fixed),
TEST_CASE_ST(generic_ut_setup, generic_ut_teardown,
-- 
2.14.3

[dpdk-dev] [PATCH v3 3/5] test/compress: add multi level test

[Pablo de Lara]

Add test that checks if all compression levels
are supported and compress a buffer correctly.

Signed-off-by: Pablo de Lara 
---
 test/test/test_compressdev.c | 33 +
 1 file changed, 33 insertions(+)

diff --git a/test/test/test_compressdev.c b/test/test/test_compressdev.c
index 8e67faec2..bb026d74f 100644
--- a/test/test/test_compressdev.c
+++ b/test/test/test_compressdev.c
@@ -863,6 +863,37 @@ test_compressdev_deflate_stateless_multi_op(void)
return TEST_SUCCESS;
 }
 
+
+static int
+test_compressdev_deflate_stateless_multi_level(void)
+{
+   struct comp_testsuite_params *ts_params = &testsuite_params;
+   const char *test_buffer;
+   unsigned int level;
+   uint16_t i;
+   struct rte_comp_xform compress_xform;
+
+   memcpy(&compress_xform, &ts_params->def_comp_xform,
+   sizeof(struct rte_comp_xform));
+
+   for (i = 0; i < RTE_DIM(compress_test_bufs); i++) {
+   test_buffer = compress_test_bufs[i];
+   for (level = RTE_COMP_LEVEL_MIN; level <= RTE_COMP_LEVEL_MAX;
+   level++) {
+   compress_xform.compress.level = level;
+   /* Compress with compressdev, decompress with Zlib */
+   if (test_deflate_comp_decomp(&test_buffer, 1,
+   &i,
+   &compress_xform,
+   &ts_params->def_decomp_xform,
+   RTE_COMP_OP_STATELESS,
+   ZLIB_DECOMPRESS) < 0)
+   return TEST_FAILED;
+   }
+   }
+
+   return TEST_SUCCESS;
+}
 static struct unit_test_suite compressdev_testsuite  = {
.suite_name = "compressdev unit test suite",
.setup = testsuite_setup,
@@ -874,6 +905,8 @@ static struct unit_test_suite compressdev_testsuite  = {
test_compressdev_deflate_stateless_dynamic),
TEST_CASE_ST(generic_ut_setup, generic_ut_teardown,
test_compressdev_deflate_stateless_multi_op),
+   TEST_CASE_ST(generic_ut_setup, generic_ut_teardown,
+   test_compressdev_deflate_stateless_multi_level),
TEST_CASES_END() /**< NULL terminate unit test array */
}
 };
-- 
2.14.3

[dpdk-dev] [PATCH v2 1/2] bus/pci: fix find device implementation

[Gaetan Rivet]

If start is set, and a device before it matches the data
passed for comparison, then this first device is returned.

This induces potentially infinite loops.

Fixes: c7fe1eea8a74 ("bus: simplify finding starting point")
Cc: sta...@dpdk.org

Signed-off-by: Gaetan Rivet 
---
 drivers/bus/pci/pci_common.c  | 21 -
 drivers/bus/pci/rte_bus_pci.h |  3 +++
 2 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index 6bed0bc9d..7215aaec3 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -455,17 +455,20 @@ static struct rte_device *
 pci_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
const void *data)
 {
-   struct rte_pci_device *dev;
+   const struct rte_pci_device *pstart;
+   struct rte_pci_device *pdev;
 
-   FOREACH_DEVICE_ON_PCIBUS(dev) {
-   if (start && &dev->device == start) {
-   start = NULL; /* starting point found */
-   continue;
-   }
-   if (cmp(&dev->device, data) == 0)
-   return &dev->device;
+   if (start != NULL) {
+   pstart = RTE_DEV_TO_PCI_CONST(start);
+   pdev = TAILQ_NEXT(pstart, next);
+   } else {
+   pdev = TAILQ_FIRST(&rte_pci_bus.device_list);
+   }
+   while (pdev != NULL) {
+   if (cmp(&pdev->device, data) == 0)
+   return &pdev->device;
+   pdev = TAILQ_NEXT(pdev, next);
}
-
return NULL;
 }
 
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 357afb912..458e6d076 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -74,6 +74,9 @@ struct rte_pci_device {
  */
 #define RTE_DEV_TO_PCI(ptr) container_of(ptr, struct rte_pci_device, device)
 
+#define RTE_DEV_TO_PCI_CONST(ptr) \
+   container_of(ptr, const struct rte_pci_device, device)
+
 #define RTE_ETH_DEV_TO_PCI(eth_dev)RTE_DEV_TO_PCI((eth_dev)->device)
 
 /** Any PCI device identifier (vendor, device, ...) */
-- 
2.11.0

[dpdk-dev] [PATCH v2 2/2] bus/vdev: fix find device implementation

[Gaetan Rivet]

If start is set and a device before it matches the data,
this device is returned.

This induces potentially infinite loops.

Fixes: c7fe1eea8a74 ("bus: simplify finding starting point")
Cc: sta...@dpdk.org

Signed-off-by: Gaetan Rivet 
---
 drivers/bus/vdev/rte_bus_vdev.h |  3 +++
 drivers/bus/vdev/vdev.c | 14 +-
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/bus/vdev/rte_bus_vdev.h b/drivers/bus/vdev/rte_bus_vdev.h
index 646d6c090..f9b5eb596 100644
--- a/drivers/bus/vdev/rte_bus_vdev.h
+++ b/drivers/bus/vdev/rte_bus_vdev.h
@@ -25,6 +25,9 @@ struct rte_vdev_device {
 #define RTE_DEV_TO_VDEV(ptr) \
container_of(ptr, struct rte_vdev_device, device)
 
+#define RTE_DEV_TO_VDEV_CONST(ptr) \
+   container_of(ptr, const struct rte_vdev_device, device)
+
 static inline const char *
 rte_vdev_device_name(const struct rte_vdev_device *dev)
 {
diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c
index 84fa51f98..9f576eb80 100644
--- a/drivers/bus/vdev/vdev.c
+++ b/drivers/bus/vdev/vdev.c
@@ -495,16 +495,20 @@ static struct rte_device *
 vdev_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 const void *data)
 {
+   const struct rte_vdev_device *vstart;
struct rte_vdev_device *dev;
 
rte_spinlock_lock(&vdev_device_list_lock);
-   TAILQ_FOREACH(dev, &vdev_device_list, next) {
-   if (start && &dev->device == start) {
-   start = NULL;
-   continue;
-   }
+   if (start != NULL) {
+   vstart = RTE_DEV_TO_VDEV_CONST(start);
+   dev = TAILQ_NEXT(vstart, next);
+   } else {
+   dev = TAILQ_FIRST(&vdev_device_list);
+   }
+   while (dev != NULL) {
if (cmp(&dev->device, data) == 0)
break;
+   dev = TAILQ_NEXT(dev, next);
}
rte_spinlock_unlock(&vdev_device_list_lock);
 
-- 
2.11.0

[dpdk-dev] [PATCH v2 0/2] Bus find_device fixes

[Gaetan Rivet]

These patches are extracted from the device querying series [1]
for earlier integration.

[1]: https://dpdk.org/ml/archives/dev/2018-March/092891.html

v2:

  * Rebase on master

Gaetan Rivet (2):
  bus/pci: fix find device implementation
  bus/vdev: fix find device implementation

 drivers/bus/pci/pci_common.c| 21 -
 drivers/bus/pci/rte_bus_pci.h   |  3 +++
 drivers/bus/vdev/rte_bus_vdev.h |  3 +++
 drivers/bus/vdev/vdev.c | 14 +-
 4 files changed, 27 insertions(+), 14 deletions(-)

-- 
2.11.0

Re: [dpdk-dev] [PATCH] vhost/crypto: fix missing head correction

[Maxime Coquelin]

Hi Fan,

On 04/27/2018 03:52 PM, Fan Zhang wrote:

Fixes: 473387aaae4a ("vhost/crypto: move to safe GPA translation API")

This patch fixes the missing head descriptor correction for
indirect descriptors.

Signed-off-by: Fan Zhang 
---
  lib/librte_vhost/vhost_crypto.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c
index c38eb3bb5..81fab7e94 100644
--- a/lib/librte_vhost/vhost_crypto.c
+++ b/lib/librte_vhost/vhost_crypto.c
@@ -919,6 +919,7 @@ vhost_crypto_process_one_req(struct vhost_crypto *vcrypto,
if (unlikely(!desc || dlen != head->len))
return -1;
desc_idx = 0;
+   head = desc;


Thanks for fixing this, and sorry for having introduced the regression.

BTW, my patch is half-complete. Indeed, it ensures no out-of-bound
accesses will be done, but it does not handle legit buffers that are
contiguous in guest physical address space, but discontiguous in the
host VA space. Such case can for example happen when having buffers
larger than the huge page size.

I have implemented the handling of such buffers in virtio-net.c, for 
example:


fb3815cc6 vhost: handle virtually non-contiguous buffers in Rx-mrg
6727f5a73 vhost: handle virtually non-contiguous buffers in Rx
91b7b4080 vhost: handle virtually non-contiguous buffers in Tx
d0c24508e vhost: add support for non-contiguous indirect descs tables

Can you try to work on such handling for next -rc release?


For this patch:
Reviewed-by: Maxime Coquelin 

Thanks,
Maxime


} else {
desc = head;
}

[dpdk-dev] [PATCH] net/vhost: Initialise vid to -1

[Ciara Loftus]

rte_eth_vhost_get_vid_from_port_id returns a value of 0 if called before
the first call to the new_device callback. A vid value >=0 suggests the
device is active which is not the case in this instance. Initialise vid
to a negative value to prevent this.

Signed-off-by: Ciara Loftus 
---
 drivers/net/vhost/rte_eth_vhost.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/vhost/rte_eth_vhost.c 
b/drivers/net/vhost/rte_eth_vhost.c
index 99a7727..f47950c 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1051,6 +1051,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t 
rx_queue_id,
return -ENOMEM;
}
 
+   vq->vid = -1;
vq->mb_pool = mb_pool;
vq->virtqueue_id = rx_queue_id * VIRTIO_QNUM + VIRTIO_TXQ;
dev->data->rx_queues[rx_queue_id] = vq;
-- 
2.7.5

Re: [dpdk-dev] [PATCH] vhost/crypto: fix condition check

[Maxime Coquelin]

On 04/27/2018 04:06 PM, Fan Zhang wrote:

Fixes: 3bb595ecd682 ("vhost/crypto: add request handler")

This patch fix final condition check while moving virtqueue
descriptors.

Signed-off-by: Fan Zhang 
---
  lib/librte_vhost/vhost_crypto.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)



Reviewed-by: Maxime Coquelin 

Thanks,
Maxime

Re: [dpdk-dev] [PATCH] vhost/crypto: fix missing head correction

[Zhang, Roy Fan]

Hi Maxime,

No problem. I will work on it.

Regards,
Fan

> -Original Message-
> From: Maxime Coquelin [mailto:maxime.coque...@redhat.com]
> Sent: Friday, April 27, 2018 3:19 PM
> To: Zhang, Roy Fan ; dev@dpdk.org
> Cc: Yigit, Ferruh 
> Subject: Re: [PATCH] vhost/crypto: fix missing head correction
> 
> Hi Fan,
> 
> On 04/27/2018 03:52 PM, Fan Zhang wrote:
> > Fixes: 473387aaae4a ("vhost/crypto: move to safe GPA translation API")
> >
> > This patch fixes the missing head descriptor correction for indirect
> > descriptors.
> >
> > Signed-off-by: Fan Zhang 
> > ---
> >   lib/librte_vhost/vhost_crypto.c | 1 +
> >   1 file changed, 1 insertion(+)
> >
> > diff --git a/lib/librte_vhost/vhost_crypto.c
> > b/lib/librte_vhost/vhost_crypto.c index c38eb3bb5..81fab7e94 100644
> > --- a/lib/librte_vhost/vhost_crypto.c
> > +++ b/lib/librte_vhost/vhost_crypto.c
> > @@ -919,6 +919,7 @@ vhost_crypto_process_one_req(struct
> vhost_crypto *vcrypto,
> > if (unlikely(!desc || dlen != head->len))
> > return -1;
> > desc_idx = 0;
> > +   head = desc;
> 
> Thanks for fixing this, and sorry for having introduced the regression.
> 
> BTW, my patch is half-complete. Indeed, it ensures no out-of-bound
> accesses will be done, but it does not handle legit buffers that are 
> contiguous
> in guest physical address space, but discontiguous in the host VA space. Such
> case can for example happen when having buffers larger than the huge page
> size.
> 
> I have implemented the handling of such buffers in virtio-net.c, for
> example:
> 
> fb3815cc6 vhost: handle virtually non-contiguous buffers in Rx-mrg
> 6727f5a73 vhost: handle virtually non-contiguous buffers in Rx
> 91b7b4080 vhost: handle virtually non-contiguous buffers in Tx d0c24508e
> vhost: add support for non-contiguous indirect descs tables
> 
> Can you try to work on such handling for next -rc release?
> 
> 
> For this patch:
> Reviewed-by: Maxime Coquelin 
> 
> Thanks,
> Maxime
> 
> > } else {
> > desc = head;
> > }
> >

Re: [dpdk-dev] [PATCH v9 00/10] eal: replace calls to rte_panic and refrain from new instances

[Thomas Monjalon]

26/04/2018 08:20, Arnon Warshavsky:
> The purpose of this patch series is to cleanup the library code
> from paths that end up aborting the process,
> and move to checking error values, in order to allow the running process
> perform an orderly teardown or other mitigation of the event.
> 
> This patch modifies the majority of rte_panic calls
> under lib and drivers, and replaces them with a log message
> and an error return code according to context,
> that can be propagated up the call stack.
> 
> - Focus was given to the dpdk initialization path
> - Some of the panic calls within drivers were left in place where
>   the call is from within an interrupt or calls that are
>   on the data path,where there is no simple applicative
>   route to propagate the error to temination.
>   These should be handled by the driver maintainers..
> - local void functions with no api were changed to retrun a value
>   where needed
> - No change took place in example and test files
> - No change took place for debug assertions calling panic
> - A new function was added to devtools/checkpatches.sh
>   in order to prevent new additions of calls to rte_panic
>   under lib and drivers.
> 
> Keep calm and don't panic

What is the status of this patchset?
It seems not ready for RC1 (today).
I don't want to push it in RC2 because it changes too many things.
So it's today or wait for 18.08. What do you think?

The last patch for check tooling can be separated and pushed at anytime.

Re: [dpdk-dev] [PATCH v9 00/10] eal: replace calls to rte_panic and refrain from new instances

[Thomas Monjalon]

26/04/2018 08:20, Arnon Warshavsky:
> v8:
> - Seperate the 2 drivers salad back to distinct bond and dpaa patches
> - Add missing file descriptor closing when returnning an error
> - Remove half baked thread patch to be handled in the next version
> - Remove duplicate function call after rebase

You forgot the ack from Stephen on the series

Re: [dpdk-dev] [PATCH] Revert "vhost: avoid concurrency when logging dirty pages"

[Ferruh Yigit]

On 4/20/2018 9:39 AM, Maxime Coquelin wrote:
> This reverts commit 394313fff39d0f994325c47f7eab39daf5dc9e11.
> 
> While the patch did solve concurrency issue, it induces more
> pages copies as some clean pages are marked as dirty for
> performance reasons. Moreover, as there is no more contention
> doing the logging, the rate of packets than can be processed is
> higher, leading to even more pages to be dirtied.
> 
> It has been reported that with more than one queue pair, and
> with a relatively low packet rate (1Mpps), the live migration
> never converges until the flow is stopped.
> 
> While a better solution is found, it is better to reset to the
> old behaviour, i.e. using atomic operation for dirty pages
> logging.
> 
> Signed-off-by: Maxime Coquelin 

Applied to dpdk-next-net/master, thanks.

Re: [dpdk-dev] [PATCH v2 0/2] Bus find_device fixes

[Thomas Monjalon]

> Gaetan Rivet (2):
>   bus/pci: fix find device implementation
>   bus/vdev: fix find device implementation

Applied, thanks

Re: [dpdk-dev] [PATCH] net/vhost: Initialise vid to -1

[Maxime Coquelin]

On 04/27/2018 04:19 PM, Ciara Loftus wrote:

rte_eth_vhost_get_vid_from_port_id returns a value of 0 if called before
the first call to the new_device callback. A vid value >=0 suggests the
device is active which is not the case in this instance. Initialise vid
to a negative value to prevent this.

Signed-off-by: Ciara Loftus 
---
  drivers/net/vhost/rte_eth_vhost.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/drivers/net/vhost/rte_eth_vhost.c 
b/drivers/net/vhost/rte_eth_vhost.c
index 99a7727..f47950c 100644
--- a/drivers/net/vhost/rte_eth_vhost.c
+++ b/drivers/net/vhost/rte_eth_vhost.c
@@ -1051,6 +1051,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t 
rx_queue_id,
return -ENOMEM;
}
  
+	vq->vid = -1;

vq->mb_pool = mb_pool;
vq->virtqueue_id = rx_queue_id * VIRTIO_QNUM + VIRTIO_TXQ;
dev->data->rx_queues[rx_queue_id] = vq;



Reviewed-by: Maxime Coquelin 

Thanks,
Maxime

Re: [dpdk-dev] [PATCH v2 0/3] small improvements to meson build

[Luca Boccassi]

On Fri, 2018-04-27 at 14:49 +0100, Bruce Richardson wrote:
> A few small improvements and fixes for the meson build system,
> including
> improved behaviour for building kmods - especially on cross-compile,
> a fix
> for our libbsd detect on CentOS 7, and a small change to ensure
> compatibility
> with future meson versions.
> 
> Bruce Richardson (3):
>   build: fix kernel compile on cross-build
>   build: fix check for libbsd in meson
>   build: ensure compatibility with future meson versions
> 
>  config/meson.build   |  3 ++-
>  kernel/linux/igb_uio/meson.build |  6 --
>  kernel/linux/meson.build | 40
> +---
>  lib/meson.build  |  2 +-
>  meson.build  |  6 +-
>  5 files changed, 45 insertions(+), 12 deletions(-)

Series-acked-by: Luca Boccassi 

LGTM!

-- 
Kind regards,
Luca Boccassi

Re: [dpdk-dev] [PATCH v7 4/4] ethdev: add shared counter support to rte_flow

[Ferruh Yigit]

On 4/27/2018 1:25 PM, Ferruh Yigit wrote:
> On 4/26/2018 7:55 PM, Thomas Monjalon wrote:
>> 26/04/2018 19:29, Declan Doherty:
>>> Add rte_flow_action_count action data structure to enable shared
>>> counters across multiple flows on a single port or across multiple
>>> flows on multiple ports within the same switch domain. Also this enables
>>> multiple count actions to be specified in a single flow action.
>>>
>>> This patch also modifies the existing rte_flow_query API to take the
>>> rte_flow_action structure as an input parameter instead of the
>>> rte_flow_action_type enumeration to allow querying a specific action
>>> from a flow rule when multiple actions of the same type are specified.
>>>
>>> This patch also contains updates for the bonding, failsafe and mlx5 PMDs
>>> and testpmd application which are affected by this API change.
>>
>> The API changes must be notified in the release notes (there is a section
>> for API changes).
> 
> Hi Declan,
> 
> If you can send the update as a separate patch, I can squash it in next-net.

Thanks Declan, following addition squashed into related commit:

* ethdev: change flow APIs regarding count action:
  * ``rte_flow_create()`` API count action now requires the ``struct
rte_flow_action_count``.
  * ``rte_flow_query()`` API parameter changed from action type to action 
structure.

Re: [dpdk-dev] [PATCH] vhost: fix vDPA set features

[Ferruh Yigit]

On 4/26/2018 9:07 AM, Maxime Coquelin wrote:
> 
> 
> On 04/25/2018 04:18 AM, Xiao Wang wrote:
>> We should call set_features callback after setting features in virtio_net
>> structure, otherwise vDPA driver cannot get the right features.
>>
>> Fixes: 07718b4f87aa ("vhost: adapt library for selective datapath")
>>
>> Signed-off-by: Xiao Wang
>> ---
>>   lib/librte_vhost/vhost_user.c | 10 +-
>>   1 file changed, 5 insertions(+), 5 deletions(-)
> 

Acked-by: Zhihong Wang 

> Reviewed-by: Maxime Coquelin 

Applied to dpdk-next-net/master, thanks.

Re: [dpdk-dev] [PATCH v3 1/9] mem: use strlcpy instead of snprintf

[Bruce Richardson]

On Wed, Apr 25, 2018 at 10:56:39AM +0100, Anatoly Burakov wrote:
> Signed-off-by: Anatoly Burakov 
> ---
> 
Acked-by: Bruce Richardson 

Re: [dpdk-dev] [PATCH v3 2/9] mem: fix resource leak

[Bruce Richardson]

On Wed, Apr 25, 2018 at 10:56:40AM +0100, Anatoly Burakov wrote:
> Coverity issue: 272601
> 
> Fixes: 66cc45e293ed ("mem: replace memseg with memseg lists")
> Cc: anatoly.bura...@intel.com
> 
> Signed-off-by: Anatoly Burakov 
> ---
Acked-by: Bruce Richardson 

Re: [dpdk-dev] [PATCH v3 3/9] mem: fix potential double close

[Bruce Richardson]

On Wed, Apr 25, 2018 at 10:56:41AM +0100, Anatoly Burakov wrote:
> We were closing descriptor before checking if mapping has
> failed, but if it did, we did a second close afterwards. Fix
> it by moving closing descriptor to after we check if mmap has
> succeeded.
> 
> Coverity issue: 272560
> 
> Fixes: 2a04139f66b4 ("eal: add single file segments option")
> Cc: anatoly.bura...@intel.com
> 
> Signed-off-by: Anatoly Burakov 
> ---
Is a better fix not to assign fd to -1 after closing and then checking that
in the error leg?

[dpdk-dev] [PATCH] lib: clear experimental version tag in linker scripts

[Ferruh Yigit]

Remove version tag from experimental block in linker version scripts
(.map files).

That label is not used by linker and information only. It is useful
for version blocks but not useful for experimental block but confusing.
Removing those labels.

Signed-off-by: Ferruh Yigit 
---
Cc: Neil Horman 
Cc: Thomas Monjalon 
---
 drivers/net/dpaa/rte_pmd_dpaa_version.map  | 4 +---
 lib/librte_cryptodev/rte_cryptodev_version.map | 2 +-
 lib/librte_eal/rte_eal_version.map | 2 +-
 lib/librte_ethdev/rte_ethdev_version.map   | 2 +-
 lib/librte_eventdev/rte_eventdev_version.map   | 2 +-
 lib/librte_mbuf/rte_mbuf_version.map   | 3 +--
 lib/librte_mempool/rte_mempool_version.map | 3 +--
 lib/librte_meter/rte_meter_version.map | 2 +-
 lib/librte_net/rte_net_version.map | 2 +-
 lib/librte_pipeline/rte_pipeline_version.map   | 2 +-
 lib/librte_vhost/rte_vhost_version.map | 2 +-
 11 files changed, 11 insertions(+), 15 deletions(-)

diff --git a/drivers/net/dpaa/rte_pmd_dpaa_version.map 
b/drivers/net/dpaa/rte_pmd_dpaa_version.map
index 3b937b10f..c7ad4030a 100644
--- a/drivers/net/dpaa/rte_pmd_dpaa_version.map
+++ b/drivers/net/dpaa/rte_pmd_dpaa_version.map
@@ -9,6 +9,4 @@ EXPERIMENTAL {
dpaa_eth_eventq_attach;
dpaa_eth_eventq_detach;
rte_pmd_dpaa_set_tx_loopback;
-
-   local: *;
-} DPDK_17.11;
+};
diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map 
b/lib/librte_cryptodev/rte_cryptodev_version.map
index 560e46411..da6905ee6 100644
--- a/lib/librte_cryptodev/rte_cryptodev_version.map
+++ b/lib/librte_cryptodev/rte_cryptodev_version.map
@@ -91,4 +91,4 @@ EXPERIMENTAL {
 
rte_cryptodev_sym_session_get_private_data;
rte_cryptodev_sym_session_set_private_data;
-} DPDK_17.11;
+};
diff --git a/lib/librte_eal/rte_eal_version.map 
b/lib/librte_eal/rte_eal_version.map
index 859723974..19ec04438 100644
--- a/lib/librte_eal/rte_eal_version.map
+++ b/lib/librte_eal/rte_eal_version.map
@@ -311,4 +311,4 @@ EXPERIMENTAL {
rte_vfio_container_group_bind;
rte_vfio_container_group_unbind;
 
-} DPDK_18.02;
+};
diff --git a/lib/librte_ethdev/rte_ethdev_version.map 
b/lib/librte_ethdev/rte_ethdev_version.map
index 9c9394c58..0569bfe62 100644
--- a/lib/librte_ethdev/rte_ethdev_version.map
+++ b/lib/librte_ethdev/rte_ethdev_version.map
@@ -243,4 +243,4 @@ EXPERIMENTAL {
rte_mtr_stats_read;
rte_mtr_stats_update;
 
-} DPDK_18.02;
+};
diff --git a/lib/librte_eventdev/rte_eventdev_version.map 
b/lib/librte_eventdev/rte_eventdev_version.map
index 3ee28f759..3b9180f76 100644
--- a/lib/librte_eventdev/rte_eventdev_version.map
+++ b/lib/librte_eventdev/rte_eventdev_version.map
@@ -97,4 +97,4 @@ EXPERIMENTAL {
rte_event_timer_arm_burst;
rte_event_timer_arm_tmo_tick_burst;
rte_event_timer_cancel_burst;
-} DPDK_18.05;
+};
diff --git a/lib/librte_mbuf/rte_mbuf_version.map 
b/lib/librte_mbuf/rte_mbuf_version.map
index 2e056d994..1bb9538db 100644
--- a/lib/librte_mbuf/rte_mbuf_version.map
+++ b/lib/librte_mbuf/rte_mbuf_version.map
@@ -44,5 +44,4 @@ EXPERIMENTAL {
rte_mbuf_set_user_mempool_ops;
rte_mbuf_user_mempool_ops;
rte_pktmbuf_pool_create_by_ops;
-
-} DPDK_16.11;
+};
diff --git a/lib/librte_mempool/rte_mempool_version.map 
b/lib/librte_mempool/rte_mempool_version.map
index 1c406b5b0..7091b954b 100644
--- a/lib/librte_mempool/rte_mempool_version.map
+++ b/lib/librte_mempool/rte_mempool_version.map
@@ -63,5 +63,4 @@ EXPERIMENTAL {
global:
 
rte_mempool_ops_get_info;
-
-} DPDK_18.05;
+};
diff --git a/lib/librte_meter/rte_meter_version.map 
b/lib/librte_meter/rte_meter_version.map
index 391b80c82..2873a723d 100644
--- a/lib/librte_meter/rte_meter_version.map
+++ b/lib/librte_meter/rte_meter_version.map
@@ -17,4 +17,4 @@ EXPERIMENTAL {
rte_meter_srtcm_profile_config;
rte_meter_trtcm_profile_config;
 
-} DPDK_2.0;
+};
diff --git a/lib/librte_net/rte_net_version.map 
b/lib/librte_net/rte_net_version.map
index 8bc57d51c..26c06e7c7 100644
--- a/lib/librte_net/rte_net_version.map
+++ b/lib/librte_net/rte_net_version.map
@@ -18,4 +18,4 @@ EXPERIMENTAL {
 
rte_net_make_rarp_packet;
rte_net_skip_ip6_ext;
-} DPDK_17.05;
+};
diff --git a/lib/librte_pipeline/rte_pipeline_version.map 
b/lib/librte_pipeline/rte_pipeline_version.map
index 53d31a9cb..129ae5330 100644
--- a/lib/librte_pipeline/rte_pipeline_version.map
+++ b/lib/librte_pipeline/rte_pipeline_version.map
@@ -73,4 +73,4 @@ EXPERIMENTAL {
rte_table_action_time_read;
rte_table_action_ttl_read;
 
-} DPDK_16.04;
+};
diff --git a/lib/librte_vhost/rte_vhost_version.map 
b/lib/librte_vhost/rte_vhost_version.map
index 8243bcabf..da220dd02 100644
--- a/lib/librte_vhost/rte_vhost_version.map
+++ b/lib/librte_vhost/rte_vhost_version.map
@@ -82,4 +82,4 @@ EXPERIMENTAL {
rte_vhost_crypto_finalize_requests;
rte_vhost_crypto_set_zero_copy;
rte_vhost_va_fro

Re: [dpdk-dev] [PATCH v3 4/9] mem: fix potential resource leak

[Bruce Richardson]

On Wed, Apr 25, 2018 at 10:56:42AM +0100, Anatoly Burakov wrote:
> We close fd if we managed to find it in the list of allocated
> segment lists (which should always be the case under normal
> conditions), but if we didn't, the fd was leaking. Close it if
> we couldn't find it in the segment list. This is not an issue
> as if the segment is zero length, we're getting rid of it
> anyway, so there's no harm in not storing the fd anywhere.
> 
> Coverity issue: 272568
> 

This coverity issue indicates two resource leaks, while I think this patch
only closes one of them.

/Bruce

> Fixes: 2a04139f66b4 ("eal: add single file segments option")
> Cc: anatoly.bura...@intel.com
> 
> Signed-off-by: Anatoly Burakov 
> ---
>  lib/librte_eal/linuxapp/eal/eal_memalloc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/lib/librte_eal/linuxapp/eal/eal_memalloc.c 
> b/lib/librte_eal/linuxapp/eal/eal_memalloc.c
> index 9156f8b..fab5a98 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_memalloc.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_memalloc.c
> @@ -569,6 +569,8 @@ free_seg(struct rte_memseg *ms, struct hugepage_info *hi,
>   if (te != NULL && te->fd >= 0) {
>   close(te->fd);
>   te->fd = -1;
> + } else {
> + close(fd);
>   }
>   unlink(path);
>   }
> -- 
> 2.7.4

[dpdk-dev] [PATCH 3/3] vhost: fix typo in fdset_event_dispatch comment

[xiangxia . m . yue]

From: Tonghao Zhang 

Signed-off-by: Tonghao Zhang 
---
 lib/librte_vhost/fd_man.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_vhost/fd_man.c b/lib/librte_vhost/fd_man.c
index b24c27d..38347ab 100644
--- a/lib/librte_vhost/fd_man.c
+++ b/lib/librte_vhost/fd_man.c
@@ -293,7 +293,7 @@
 * because the fd is closed in the cb,
 * the old fd val could be reused by when creates new
 * listen fd in another thread, we couldn't call
-* fd_set_del.
+* fdset_del.
 */
if (remove1 || remove2) {
pfdentry->fd = -1;
-- 
1.8.3.1

[dpdk-dev] [PATCH 1/3] vhost: fix deadlock due to vhostuser socket and fdset

[xiangxia . m . yue]

From: Tonghao Zhang 

When qemu close the unix socket fd of the vhostuser as a
server, and then immediately delete the vhostuser port on
openvswitch. There will be a deadlock.

A thread (fdset event thread):   B thread:
1. fdset_event_dispatch  rte_vhost_driver_unregister
2. set the fd busy to 1. lock vsocket->conn_mutex
3. vhost_user_read_cbfdset_del waits busy changed to 0.
4. vhost peer closed, remove the
   conn from vsocket->conn_list:
   lock vsocket->conn_mutex

5. set the fd busy to 0

Fixes: 65388b43 ("vhost: fix fd leaks for vhost-user server mode")
Cc: sta...@dpdk.org
Cc: Yuanhan Liu 
Signed-off-by: Tonghao Zhang 
---
 lib/librte_vhost/fd_man.c | 32 
 lib/librte_vhost/fd_man.h |  1 +
 lib/librte_vhost/socket.c | 13 -
 3 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/lib/librte_vhost/fd_man.c b/lib/librte_vhost/fd_man.c
index 8590ee5..b24c27d 100644
--- a/lib/librte_vhost/fd_man.c
+++ b/lib/librte_vhost/fd_man.c
@@ -174,6 +174,38 @@
return dat;
 }
 
+/**
+ *  Unregister the fd from the fdset.
+ *
+ *  If parameters are invalid, return directly -2.
+ *  And check whether fd is busy, if yes, return -1.
+ *  Otherwise, try to delete the fd from fdset and
+ *  return true.
+ */
+int
+fdset_try_del(struct fdset *pfdset, int fd)
+{
+   int i;
+
+   if (pfdset == NULL || fd == -1)
+   return -2;
+
+   pthread_mutex_lock(&pfdset->fd_mutex);
+   i = fdset_find_fd(pfdset, fd);
+   if (i != -1 && pfdset->fd[i].busy) {
+   pthread_mutex_unlock(&pfdset->fd_mutex);
+   return -1;
+   }
+
+   if (i != -1) {
+   pfdset->fd[i].fd = -1;
+   pfdset->fd[i].rcb = pfdset->fd[i].wcb = NULL;
+   pfdset->fd[i].dat = NULL;
+   }
+
+   pthread_mutex_unlock(&pfdset->fd_mutex);
+   return 0;
+}
 
 /**
  * This functions runs in infinite blocking loop until there is no fd in
diff --git a/lib/librte_vhost/fd_man.h b/lib/librte_vhost/fd_man.h
index 76a42fb..3331bcd 100644
--- a/lib/librte_vhost/fd_man.h
+++ b/lib/librte_vhost/fd_man.h
@@ -44,6 +44,7 @@ int fdset_add(struct fdset *pfdset, int fd,
fd_cb rcb, fd_cb wcb, void *dat);
 
 void *fdset_del(struct fdset *pfdset, int fd);
+int fdset_try_del(struct fdset *pfdset, int fd);
 
 void *fdset_event_dispatch(void *arg);
 
diff --git a/lib/librte_vhost/socket.c b/lib/librte_vhost/socket.c
index 4a561ad..822db41 100644
--- a/lib/librte_vhost/socket.c
+++ b/lib/librte_vhost/socket.c
@@ -922,13 +922,24 @@ struct vhost_user_reconnect_list {
vhost_user_remove_reconnect(vsocket);
}
 
+again:
pthread_mutex_lock(&vsocket->conn_mutex);
for (conn = TAILQ_FIRST(&vsocket->conn_list);
 conn != NULL;
 conn = next) {
next = TAILQ_NEXT(conn, next);
 
-   fdset_del(&vhost_user.fdset, conn->connfd);
+   /*
+* If r/wcb is executing, release the
+* conn_mutex lock, and try again since
+* the r/wcb may use the conn_mutex lock.
+*/
+   if (fdset_try_del(&vhost_user.fdset,
+ conn->connfd) == -1) {
+   
pthread_mutex_unlock(&vsocket->conn_mutex);
+   goto again;
+   }
+
RTE_LOG(INFO, VHOST_CONFIG,
"free connfd = %d for device '%s'\n",
conn->connfd, path);
-- 
1.8.3.1

[dpdk-dev] [PATCH 2/3] vhost: fix crash and fd leak due to vhostuser destroyed

[xiangxia . m . yue]

From: Tonghao Zhang 

when rte_vhost_driver_unregister detstroy the vsocket, we
should set it to NULL after freeing it, because in client mode,
the conn may be added to reconnect thread while vsocket is
destroyed. In one case, if qemu create vhostuser port as a
server with the same unix path, the reconnect thread will
reconnect to it while vsocket is destroyed.

To fix this:
1. set vsocket to NULL after free it.
2. remove the reconnection from reconnection thread in suitable
   position.

Cc: sta...@dpdk.org
Signed-off-by: Tonghao Zhang 
---
 lib/librte_vhost/socket.c | 41 -
 1 file changed, 28 insertions(+), 13 deletions(-)

diff --git a/lib/librte_vhost/socket.c b/lib/librte_vhost/socket.c
index 822db41..d5a6ac8 100644
--- a/lib/librte_vhost/socket.c
+++ b/lib/librte_vhost/socket.c
@@ -199,6 +199,9 @@ struct vhost_user {
struct vhost_user_connection *conn;
int ret;
 
+   if (vsocket == NULL)
+   return;
+
conn = malloc(sizeof(*conn));
if (conn == NULL) {
close(fd);
@@ -778,6 +781,20 @@ struct vhost_user_reconnect_list {
return ret;
 }
 
+static void
+vhost_user_socket_mem_free(struct vhost_user_socket *vsocket)
+{
+   if (vsocket && vsocket->path) {
+   free(vsocket->path);
+   vsocket->path = NULL;
+   }
+
+   if (vsocket) {
+   free(vsocket);
+   vsocket = NULL;
+   }
+}
+
 /*
  * Register a new vhost-user socket; here we could act as server
  * (the default case), or client (when RTE_VHOST_USER_CLIENT) flag
@@ -808,7 +825,7 @@ struct vhost_user_reconnect_list {
if (vsocket->path == NULL) {
RTE_LOG(ERR, VHOST_CONFIG,
"error: failed to copy socket path string\n");
-   free(vsocket);
+   vhost_user_socket_mem_free(vsocket);
goto out;
}
TAILQ_INIT(&vsocket->conn_list);
@@ -866,8 +883,7 @@ struct vhost_user_reconnect_list {
"error: failed to destroy connection mutex\n");
}
 out_free:
-   free(vsocket->path);
-   free(vsocket);
+   vhost_user_socket_mem_free(vsocket);
 out:
pthread_mutex_unlock(&vhost_user.mutex);
 
@@ -914,14 +930,6 @@ struct vhost_user_reconnect_list {
struct vhost_user_socket *vsocket = vhost_user.vsockets[i];
 
if (!strcmp(vsocket->path, path)) {
-   if (vsocket->is_server) {
-   fdset_del(&vhost_user.fdset, 
vsocket->socket_fd);
-   close(vsocket->socket_fd);
-   unlink(path);
-   } else if (vsocket->reconnect) {
-   vhost_user_remove_reconnect(vsocket);
-   }
-
 again:
pthread_mutex_lock(&vsocket->conn_mutex);
for (conn = TAILQ_FIRST(&vsocket->conn_list);
@@ -950,9 +958,16 @@ struct vhost_user_reconnect_list {
}
pthread_mutex_unlock(&vsocket->conn_mutex);
 
+   if (vsocket->is_server) {
+   fdset_del(&vhost_user.fdset, 
vsocket->socket_fd);
+   close(vsocket->socket_fd);
+   unlink(path);
+   } else if (vsocket->reconnect) {
+   vhost_user_remove_reconnect(vsocket);
+   }
+
pthread_mutex_destroy(&vsocket->conn_mutex);
-   free(vsocket->path);
-   free(vsocket);
+   vhost_user_socket_mem_free(vsocket);
 
count = --vhost_user.vsocket_cnt;
vhost_user.vsockets[i] = vhost_user.vsockets[count];
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH v3 5/9] mem: fix potential resource leak

[Bruce Richardson]

On Wed, Apr 25, 2018 at 10:56:43AM +0100, Anatoly Burakov wrote:
> Normally, tailq entry should have a valid fd by the time we attempt
> to map the segment. However, in case it doesn't, we're leaking fd,
> so fix it.
> 
> Coverity issue: 272570
> 
> Fixes: 2a04139f66b4 ("eal: add single file segments option")
> Cc: anatoly.bura...@intel.com
> 
> Signed-off-by: Anatoly Burakov 
> ---
>  lib/librte_eal/linuxapp/eal/eal_memalloc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/lib/librte_eal/linuxapp/eal/eal_memalloc.c 
> b/lib/librte_eal/linuxapp/eal/eal_memalloc.c
> index fab5a98..b02e3a5 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_memalloc.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_memalloc.c
> @@ -524,6 +524,8 @@ alloc_seg(struct rte_memseg *ms, void *addr, int 
> socket_id,
>   if (te != NULL && te->fd >= 0) {
>   close(te->fd);
>   te->fd = -1;

Is "fd" still not being leaked here, since we won't hit the else case and
then jump to the end of the function where it goes out of scope?

> + } else {
> + close(fd);
>   }
>   /* ignore errors, can't make it any worse */
>   unlink(path);
> -- 
> 2.7.4

Re: [dpdk-dev] [PATCH v3 6/9] mem: fix comparing pointer to value

[Bruce Richardson]

On Wed, Apr 25, 2018 at 10:56:44AM +0100, Anatoly Burakov wrote:
> Previous code had an old rebase leftover from the time when
> oldpolicy was an actual int, instead of a pointer. Fix it to
> do comparison with dereferencing the pointer.
> 
> Coverity issue: 272589
> 
> Fixes: 582bed1e1d1d ("mem: support mapping hugepages at runtime")
> Cc: anatoly.bura...@intel.com
> 
> Signed-off-by: Anatoly Burakov 
> ---
Acked-by: Bruce Richardson