This commit introduces rte_dma_wmb() and rte_dma_rmb(), in order to guarantee the ordering of coherent shared memory between the CPU and a DMA capable device.
Signed-off-by: Yongseok Koh <ys...@mellanox.com> --- lib/librte_eal/common/include/generic/rte_atomic.h | 52 ++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/lib/librte_eal/common/include/generic/rte_atomic.h b/lib/librte_eal/common/include/generic/rte_atomic.h index 3ba7245a3..1ffa51e31 100644 --- a/lib/librte_eal/common/include/generic/rte_atomic.h +++ b/lib/librte_eal/common/include/generic/rte_atomic.h @@ -98,6 +98,58 @@ static inline void rte_io_wmb(void); */ static inline void rte_io_rmb(void); +/** + * Write memory barrier for coherent memory between lcore and IO device + * + * Guarantees that the STORE operations on coherent memory that + * precede the rte_dma_wmb() call are visible to I/O device before the + * STORE operations that follow it. + * + * DMA memory barrier is a lightweight version of I/O device barriers + * which are system-wide data synchronization barriers. This is for + * only coherent memory domain between lcore and IO device but it is + * same as the I/O device barriers in most of architectures. However, + * some architecture provides even lighter barriers which are + * somewhere in between I/O device barriers and SMP barriers. For + * example, in case of ARMv8, data memory barrier can have different + * shareability domains - inner-shareable and outer-shareable. And + * inner-shareable data memory barrier fits for SMP barriers and + * outer-shareable one for DMA barriers, which acts on coherent + * memory. + * + * In most cases, I/O device barriers are safer but if operations are + * on coherent memory instead of incoherent MMIO region of a device, + * then DMA barriers can be used and this could bring performance gain + * depending on architectures. + */ +static inline void rte_dma_wmb(void); + +/** + * Read memory barrier for coherent memory between lcore and IO device + * + * Guarantees that the LOAD operations on coherent memory updated by + * IO device that precede the rte_dma_rmb() call are visible to CPU + * before the LOAD operations that follow it. + * + * DMA memory barrier is a lightweight version of I/O device barriers + * which are system-wide data synchronization barriers. This is for + * only coherent memory domain between lcore and IO device but it is + * same as the I/O device barriers in most of architectures. However, + * some architecture provides even lighter barriers which are + * somewhere in between I/O device barriers and SMP barriers. For + * example, in case of ARMv8, data memory barrier can have different + * shareability domains - inner-shareable and outer-shareable. And + * inner-shareable data memory barrier fits for SMP barriers and + * outer-shareable one for DMA barriers, which acts on coherent + * memory. + * + * In most cases, I/O device barriers are safer but if operations are + * on coherent memory instead of incoherent MMIO region of a device, + * then DMA barriers can be used and this could bring performance gain + * depending on architectures. + */ +static inline void rte_dma_rmb(void); + #endif /* __DOXYGEN__ */ /** -- 2.11.0
