Hi Dariousz, On Mon, Oct 29, 2018 at 4:08 PM Dariusz Stojaczyk <darek.stojac...@gmail.com> wrote:
> On Fri, Oct 5, 2018 at 2:47 PM Alejandro Lucero > <alejandro.luc...@netronome.com> wrote: > > > > Linux kernel uses a really high address as starting address for > > serving mmaps calls. If there exist addressing limitations and > > IOVA mode is VA, this starting address is likely too high for > > those devices. However, it is possible to use a lower address in > > the process virtual address space as with 64 bits there is a lot > > of available space. > > > > This patch adds an address hint as starting address for 64 bits > > systems and increments the hint for next invocations. If the mmap > > call does not use the hint address, repeat the mmap call using > > the hint address incremented by page size. > > > > Signed-off-by: Alejandro Lucero <alejandro.luc...@netronome.com> > > Reviewed-by: Anatoly Burakov <anatoly.bura...@intel.com> > > --- > > lib/librte_eal/common/eal_common_memory.c | 34 > ++++++++++++++++++++++++++++++- > > 1 file changed, 33 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > > index c482f0d..853c44c 100644 > > --- a/lib/librte_eal/common/eal_common_memory.c > > +++ b/lib/librte_eal/common/eal_common_memory.c > > @@ -37,6 +37,23 @@ > > static void *next_baseaddr; > > static uint64_t system_page_sz; > > > > +#ifdef RTE_ARCH_64 > > +/* > > + * Linux kernel uses a really high address as starting address for > serving > > + * mmaps calls. If there exists addressing limitations and IOVA mode is > VA, > > + * this starting address is likely too high for those devices. However, > it > > + * is possible to use a lower address in the process virtual address > space > > + * as with 64 bits there is a lot of available space. > > + * > > + * Current known limitations are 39 or 40 bits. Setting the starting > address > > + * at 4GB implies there are 508GB or 1020GB for mapping the available > > + * hugepages. This is likely enough for most systems, although a device > with > > + * addressing limitations should call rte_eal_check_dma_mask for > ensuring all > > + * memory is within supported range. > > + */ > > +static uint64_t baseaddr = 0x100000000; > > +#endif > > This breaks running with ASAN unless a custom --base-virtaddr option > is specified. The default base-virtaddr introduced by this patch falls > into an area that's already reserved by ASAN. > > See here: > https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm > The only available address space starts at 0x10007fff8000, which > unfortunately doesn't fit in 39 bits. > > Right now the very first eal_get_virtual_area() in EAL initialization > is used with 4KB pagesize, meaning that DPDK will try to mmap at each > 4KB-aligned offset all the way from 0x100000000 to 0x10007fff8000, > which takes quite a long, long time. > > I'm not sure about the solution to this problem, but I verify that > starting DPDK 18.11-rc1 with `--base-virtaddr 0x200000000000` works > just fine under ASAN. > > Do we have documentation about using Address Sanitizer? I understand the goal but, which is the cost? Do you have numbers about the impact on performance? Solving this is not trivial. I would say someone interested in this but using a hardware with addressing limitations needs to choose. Could it be possible to modify the virtual addresses used by default? I guess the shadow regions can be higher that the default ones. > D. > > > > > <snip> >
