Hi, Looking at the code, it looks like mlx5 keeps a MR cache per TX queue (each MR registers a rte_mempool).
Once a TX queue is created, mlx5 scans existing mempools and pre-registers a MR for each mempool it meets (using rte_mempool_walk). For each MR registration that exceeds the TX queue cache, it removes the first entry from the cache and deregisters that MR (in txq_mp2mr_reg). Now on TX burst, if the driver sees a mbuf from an unknown mempool, it registers the mempool on the fly and again *deregister* the first MR in the cache. My question is, what guarantees that no inflight send operations posted on the TX queue when we deregister and remove a MR from the cache? AFAICT, it is the driver responsibility to guarantee to never deregister a memory region that has inflight send operations posted, otherwise the send operation *will* complete with a local protection error. Is that taken care of? Another question, why is the MR cache maintained per TX queue and not per device? If the application starts N TX queues then a single mempool will be registered N times instead of just once. Having lots of MR instances will pollute the device ICMC pretty badly. Am I missing something? Thanks, Sagi.
