Hi Olivier, On Tue, Nov 14, 2017 at 10:58:02AM +0100, Olivier Gournet wrote: > Hi, > > I can't get TX on secondary process to works with the lastest dpdk, it was > running fine with dpdk-16.11. > RX/TX is ok on primary processs, and I'm not interested in RX on secondary > process. Each process has its > owns TX queues. > > I upgraded everything to: > linux 4.14 > dpdk 17.11-rc4 > rdma-core from yesterday git-master > mlx-fw 12.21.1000 > > On seconday process, TX queue gets full and is never emptied. > It seems like bf_reg is correctly re-mapped from secondary process; here's > dmesg from primary process: > > [...] > [85361.895778] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x6, pfn > 0x00000000000f8020 > [85361.895781] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at > 0x7ffff7faa000, PA 0x00000000f8020000 > [85361.895784] mlx5_1:uar_mmap:1722:(pid 9533): uar idx 0x7, pfn > 0x00000000000f8021 > [85361.895787] mlx5_1:uar_mmap:1735:(pid 9533): mapped best effort WC at > 0x7ffff7fa9000, PA 0x00000000f8021000 > > Then from secondary process: > > [...] > [85408.038295] mlx5_1:mlx5_ib_mmap:1778:(pid 9551): mapped internal timer at > 0x7ffff7f7f000, PA 0xf8001000 > [85408.040229] mlx5_1:uar_mmap:1722:(pid 9551): uar idx 0x6, pfn > 0x00000000000f8020 > [85408.040233] mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at > 0x7ffff7faa000, PA 0x00000000f8020000 > [85408.040239] mlx5_1:uar_mmap:1722:(pid 9551): uar idx 0x7, pfn > 0x00000000000f8021 > [85408.040241] mlx5_1:uar_mmap:1735:(pid 9551): mapped best effort WC at > 0x7ffff7fa9000, PA 0x00000000f8021000 > > But then from mlx5_rxtx.h:mlx5_tx_dbrec_cond_wmb() it doesn't seems like > writing to txq->bf_reg has any effect. > I know it isn't supposed to be read, but when reading from primary process is > always give *dst==0xe5ccdabae5ccdaba, > whereas from secondary process *dst is zero or the last written value. > > I don't really have any other clues, and don't know where to search. Can > anybody give me some hints ?
Since Yongseok patch [1] the UAR pages are no more configured though the Write Combining of the PCI but in your logs it seems it is still using the cache. Do you have the "MLX5_SHUT_UP_BF" set to 1 in your environment? If not does adding it solves your issue? Regards, [1] http://dpdk.org/browse/dpdk/commit/?id=fb870be5a879c6617fecabf47873ae2b576e6e69 -- Nélio Laranjeiro 6WIND
