On 5/11/2017 12:51 PM, Gowrishankar wrote:
> From: Gowrishankar Muthukrishnan <gowrishanka...@linux.vnet.ibm.com>
>
> In kni_allocate_mbufs(), we attempt to add max_burst (32) count of mbuf
> always into alloc_q, which is excessively leading too many rte_pktmbuf_
> free() when alloc_q is contending at high packet rate (for eg 10Gig data).
> In a situation when alloc_q fifo can only accommodate very few (or zero)
> mbuf, create only what needed and add in fifo.
I remember I have tried similar, also tried allocating amount of
nb_packets read from kernel, both produced worse performance.
Can you please share your before/after performance numbers?
kni_allocate_mbufs() called within rte_kni_rx_burst() if any packet
received from kernel. If there is a heavy traffic, kernel will always
consume the alloc_q before this function called and this function will
fill it back. So there shouldn't be much cases that alloc_q fifo already
full.
Perhaps this can happen if application burst Rx from kernel in a number
less than 32, but fifo filled with fixed 32mbufs, is this your case?
Can you measure number of times rte_pktmbuf_free() called because of
alloc_q is full?
>
> With this patch, we could stop random network stall in KNI at higher packet
> rate (eg 1G or 10G data between vEth0 and PMD) sufficiently exhausting
> alloc_q on above condition. I tested i40e PMD for this purpose in ppc64le.
If stall happens from NIC to kernel, this is kernel receive path, and
alloc_q is in kernel transmit path.
>
> Changes:
> v2 - alloc_q free count calculation corrected.
> line wrap fixed for commit message.
>
> Signed-off-by: Gowrishankar Muthukrishnan <gowrishanka...@linux.vnet.ibm.com>
> ---
> lib/librte_kni/rte_kni.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c
> index c3f9208..9c5d485 100644
> --- a/lib/librte_kni/rte_kni.c
> +++ b/lib/librte_kni/rte_kni.c
> @@ -624,6 +624,7 @@ struct rte_kni *
> int i, ret;
> struct rte_mbuf *pkts[MAX_MBUF_BURST_NUM];
> void *phys[MAX_MBUF_BURST_NUM];
> + int allocq_free;
>
> RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pool) !=
> offsetof(struct rte_kni_mbuf, pool));
> @@ -646,7 +647,9 @@ struct rte_kni *
> return;
> }
>
> - for (i = 0; i < MAX_MBUF_BURST_NUM; i++) {
> + allocq_free = (kni->alloc_q->read - kni->alloc_q->write - 1) \
> + & (MAX_MBUF_BURST_NUM - 1);
> + for (i = 0; i < allocq_free; i++) {
> pkts[i] = rte_pktmbuf_alloc(kni->pktmbuf_pool);
> if (unlikely(pkts[i] == NULL)) {
> /* Out of memory */
>
