On 10/2/19 1:16 AM, David Miller wrote:
From: Ka-Cheong Poon <ka-cheong.p...@oracle.com>
Date: Mon, 30 Sep 2019 02:08:00 -0700
Currently, RDS calls ib_dma_alloc_coherent() to allocate a large piece
of contiguous DMA coherent memory to store struct rds_header for
sending/receiving packets. The memory allocated is then partitioned
into struct rds_header. This is not necessary and can be costly at
times when memory is fragmented. Instead, RDS should use the DMA
memory pool interface to handle this.
Suggested-by: Håkon Bugge <haakon.bu...@oracle.com>
Signed-off-by: Ka-Cheong Poon <ka-cheong.p...@oracle.com>
This is trading a one-time overhead for extra levels of dereferencing
on every single descriptor access in the fast paths.
I do not agree with this tradeoff, please implement this more
reasonably.
The problem with the existing way of pre-allocation is
that when there are a lot of RDS connections, the call to
ib_dma_alloc_coherent() can fail because there are not
enough contiguous memory pages available. It is causing
problems in production systems.
And the i_{recv|send|_hdrs_dma array dereferencing is done
at send/receive ring initialization and refill. It is not
done at every access of the header.
Thanks.
--
K. Poon
ka-cheong.p...@oracle.com
