> -----Original Message----- > From: Mike Kravetz [mailto:mike.krav...@oracle.com] > Sent: Saturday, August 22, 2020 5:53 AM > To: Song Bao Hua (Barry Song) <song.bao....@hisilicon.com>; h...@lst.de; > m.szyprow...@samsung.com; robin.mur...@arm.com; w...@kernel.org; > ganapatrao.kulka...@cavium.com; catalin.mari...@arm.com; > a...@linux-foundation.org > Cc: io...@lists.linux-foundation.org; linux-arm-ker...@lists.infradead.org; > linux-kernel@vger.kernel.org; Zengtao (B) <prime.z...@hisilicon.com>; > huangdaode <huangda...@huawei.com>; Linuxarm <linux...@huawei.com> > Subject: Re: [PATCH v7 0/3] make dma_alloc_coherent NUMA-aware by > per-NUMA CMA > > Hi Barry, > Sorry for jumping in so late. > > On 8/21/20 4:33 AM, Barry Song wrote: > > > > with per-numa CMA, smmu will get memory from local numa node to save > command > > queues and page tables. that means dma_unmap latency will be shrunk > much. > > Since per-node CMA areas for hugetlb was introduced, I have been thinking > about the limited number of CMA areas. In most configurations, I believe > it is limited to 7. And, IIRC it is not something that can be changed at > runtime, you need to reconfig and rebuild to increase the number. In contrast > some configs have NODES_SHIFT set to 10. I wasn't too worried because of > the limited hugetlb use case. However, this series is adding another user > of per-node CMA areas. > > With more users, should try to sync up number of CMA areas and number of > nodes? Or, perhaps I am worrying about nothing?
Hi Mike, The current limitation is 8. If the server has 4 nodes and we enable both pernuma CMA and hugetlb, the last node will fail to get one cma area as the default global cma area will take 1 of 8. So users need to change menuconfig. If the server has 8 nodes, we enable one of pernuma cma and hugetlb, one node will fail to get cma. We may set the default number of CMA areas as 8+MAX_NODES(if hugetlb enabled) + MAX_NODES(if pernuma cma enabled) if we don't expect users to change config, but right now hugetlb has not an option in Kconfig to enable or disable like pernuma cma has DMA_PERNUMA_CMA. > -- > Mike Kravetz Thanks Barry
