Marc Rocas wrote On 09/27/05 21:27,:
> I spoke to one of our I/O guys to see whether there is a prescribed way
> to allocate physical memory in the least significant 4 gig for DMA from
> userland. Solaris doesn't provide an existing way. The philosophy is
> that users shouldn't have to care and that the I/O buffer will be copied
> to/from the least significant physical memory inside the kernel if the
> device can only reach that far to DMA. I think that you may be able to
> write a driver that can allocate physical memory in the desired range
> and allow access to it with mmap(2) or ioctl(2).
>
>
> We already have such a driver but have not found a way to force the use
> of 2M pages! Is there a new DDI interface to request large page size?
I'm not sure, but can try to find out.
What does the driver use to allocate the memory (eg. ddi_dma_mem_alloc(9F))?
We actually got the driver from the local Sun office and I just checked it and it uses page_create() which is both private and deprecated in Solaris 10. It would be easy to use ddi_dma_mem_alloc() if we can coerce it to allocate large pages. To be fair, we got the driver back in 1997 and discontinued its use once ISM became available in Solaris 8u3.
> By writing our own device driver, do you mean one to export 4GB PA range
> to user-land or our own segment driver?
I mean whether your driver can allocate the DMA buffer in the right
place and provide acccess to it through mmap(2) or ioctl(2). It sounds
like you have a driver like that already, but want the buffer to be on
large pages in the user application.
Is that right?
That's correct. We've measured 3-5% improvement with 4MB pages on SPARC and simply wanted to carry over to amd64. We have not carried out the measurements on amd64 to compare 4KB vs. 2MB pages since the ratio is the same as in SPARC. Also, off the top of my head, I can't remember whether Opteron is on par or worse as far as its TLB is concerned as compare to SPARC.
--Marc
_______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
