On Tue, Nov 6, 2018 at 10:31 AM Burakov, Anatoly <anatoly.bura...@intel.com> wrote:
> On 06-Nov-18 9:32 AM, Alejandro Lucero wrote: > > > > > > On Mon, Nov 5, 2018 at 4:35 PM Burakov, Anatoly > > <anatoly.bura...@intel.com <mailto:anatoly.bura...@intel.com>> wrote: > > > > On 05-Nov-18 3:33 PM, Alejandro Lucero wrote: > > > > > > > > > On Mon, Nov 5, 2018 at 3:12 PM Burakov, Anatoly > > > <anatoly.bura...@intel.com <mailto:anatoly.bura...@intel.com> > > <mailto:anatoly.bura...@intel.com > > <mailto:anatoly.bura...@intel.com>>> wrote: > > > > > > On 05-Nov-18 10:13 AM, Alejandro Lucero wrote: > > > > On Mon, Nov 5, 2018 at 10:01 AM Li, WenjieX A > > > <wenjiex.a...@intel.com <mailto:wenjiex.a...@intel.com> > > <mailto:wenjiex.a...@intel.com <mailto:wenjiex.a...@intel.com>>> > > > > wrote: > > > > > > > >> 1. With GCC32, testpmd could not startup without > > '--iova-mode pa'. > > > >> ./i686-native-linuxapp-gcc/app/testpmd -c f -n 4 -- -i > > > >> The output is: > > > >> EAL: Detected 16 lcore(s) > > > >> EAL: Detected 1 NUMA nodes > > > >> EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > > > >> EAL: Some devices want iova as va but pa will be used > > because.. > > > EAL: few > > > >> device bound to UIO > > > >> EAL: No free hugepages reported in hugepages-1048576kB > > > >> EAL: Probing VFIO support... > > > >> EAL: VFIO support initialized > > > >> EAL: wrong dma mask size 48 (Max: 31) > > > >> EAL: alloc_pages_on_heap(): couldn't allocate memory due > > to IOVA > > > exceeding > > > >> limits of current DMA mask > > > >> error allocating rte services array > > > >> EAL: FATAL: rte_service_init() failed > > > >> EAL: rte_service_init() failed > > > >> PANIC in main(): > > > >> Cannot init EAL > > > >> 5: [./i686-native-linuxapp-gcc/app/testpmd(+0x95fda) > > [0x56606fda]] > > > >> 4: [/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf6) > > > [0xf74d1276]] > > > >> 3: [./i686-native-linuxapp-gcc/app/testpmd(main+0xf21) > > [0x565fcee1]] > > > >> 2: > [./i686-native-linuxapp-gcc/app/testpmd(__rte_panic+0x3d) > > > [0x565edc68]] > > > >> 1: > > [./i686-native-linuxapp-gcc/app/testpmd(rte_dump_stack+0x33) > > > >> [0x5675f333]] > > > >> Aborted > > > >> > > > >> 2. With '--iova-mode pa', testpmd could startup. > > > >> 3. With GCC64, there is no such issue. > > > >> Thanks! > > > >> > > > >> > > > > Does 32 bits support require IOMMU? It would be a > surprise. If > > > there is no > > > > IOMMU hardware, no dma mask should be there at all. > > > > > > IOMMU is supported on 32-bits, however limited the address > > space might > > > be. Maybe limit IOMMU width to RTE_MIN(31, value) bits for > > > everything on > > > 32-bit? > > > > > > > > > If IOMMU is supported in 32 bits, then the DMA mask check should > > not be > > > happening. AFAIK, the IOMMU hardware addressing limitations is a > > problem > > > only in 64 bits systems. The worst situation I have head of is 39 > > bits > > > for virtualized IOMMU with QEMU. > > > > > > I would prefer not to invoke rte_mem_set_dma_mask for 32 bits > > system for > > > the Intel IOMMU case. The only other dma mask client is the NFP > > PMD and > > > we do not support 32 bits systems. > > > > > > > I don't think not invoking DMA mask check is the right choice here. > In > > practice it may be, but i'd rather the behavior to be "correct", if > at > > all possible :) It is theoretically possible to have an IOMMU with an > > addressing limitation of, say, 30 bits (even though they don't exist > in > > reality), so therefore our code should handle it, should it encounter > > one, and it should also handle the "proper" ones correctly (as in, > > treat > > them as 32-bit-limited instead of 39- or 48-bit-limited). > > > > > > Fine. > > > > The problem is the current sanity check about the dma mask width, what > > is 31 for 32 bits systems. > > Should we just leave a single max dma width to 63? This covers the > > possibility of 32 bit systems integrating an IOMMU designed for 64 > > bits. I really doubt this is a real possibility in x86, although I can > > see it more likely in embedded systems where this sort of hardware > > components integration happens. > > Actually (and after a quick chat with Ferruh), is this even needed? IOVA > addresses are independent from VA width, IOVA can happily be bigger than > 32-bits if i understand things correctly. All of our IOVA addresses are > always 64-bit throughout DPDK. I don't think this check is even valid. > > Although iova_t is 64 bits, there should not be a IOVA higher than 32 bits, although there could be exceptions like PAE extensions (I'm old enough for remembering that option :-( ). Anyway, the original idea of dma mask sanity check is 32 bits systems was assuming there should not be a dma mask above 32 bits, but I'm happy with removing that sanity check for 32 bits systems. So, do you agree to just leave the sanity check for a max width of 63 bits? > > > > > > > > -- > > > Thanks, > > > Anatoly > > > > > > > > > -- > > Thanks, > > Anatoly > > > > > -- > Thanks, > Anatoly >
