Re: iommu dma mapping alignment requirements

2007-12-20 Thread Benjamin Herrenschmidt
BTW. I need to know urgently what HW is broken by this Ben. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml

Re: iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Benjamin Herrenschmidt wrote: Sounds good. Thanks! Note, that these smaller sub-host-page-sized mappings might pollute the address space causing full aligned host-page-size maps to become scarce... Maybe there's a clever way to keep those in their own segment of the address space? We al

Re: iommu dma mapping alignment requirements

2007-12-20 Thread Benjamin Herrenschmidt
> Sounds good. Thanks! > > Note, that these smaller sub-host-page-sized mappings might pollute the > address space causing full aligned host-page-size maps to become > scarce... Maybe there's a clever way to keep those in their own segment > of the address space? We already have a large vs.

Re: iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Benjamin Herrenschmidt wrote: On Thu, 2007-12-20 at 15:02 -0600, Steve Wise wrote: Benjamin Herrenschmidt wrote: Adding A few more people to the discussion. You may well be right and we would have to provide the same alignment, though that sucks a bit as one of the reason we switched to 4K for

Re: iommu dma mapping alignment requirements

2007-12-20 Thread Benjamin Herrenschmidt
On Thu, 2007-12-20 at 15:02 -0600, Steve Wise wrote: > Benjamin Herrenschmidt wrote: > > Adding A few more people to the discussion. You may well be right and we > > would have to provide the same alignment, though that sucks a bit as one > > of the reason we switched to 4K for the IOMMU is that t

Re: [ofa-general] iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Benjamin Herrenschmidt wrote: On Thu, 2007-12-20 at 13:29 -0600, Steve Wise wrote: Or based on the alignment of vaddr actually... The later wouldn't be realistic. What I think might be necessay, though it would definitely cause us problems with running out of iommu space (which is the reason

Re: iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Benjamin Herrenschmidt wrote: Adding A few more people to the discussion. You may well be right and we would have to provide the same alignment, though that sucks a bit as one of the reason we switched to 4K for the IOMMU is that the iommu space available on pSeries is very small and we were runn

Re: [ofa-general] iommu dma mapping alignment requirements

2007-12-20 Thread Benjamin Herrenschmidt
On Thu, 2007-12-20 at 13:29 -0600, Steve Wise wrote: > Or based on the alignment of vaddr actually... The later wouldn't be realistic. What I think might be necessay, though it would definitely cause us problems with running out of iommu space (which is the reason we did the switch down to 4K),

Re: iommu dma mapping alignment requirements

2007-12-20 Thread Benjamin Herrenschmidt
Adding A few more people to the discussion. You may well be right and we would have to provide the same alignment, though that sucks a bit as one of the reason we switched to 4K for the IOMMU is that the iommu space available on pSeries is very small and we were running out of it with 64K pages and

Re: [ofa-general] iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Steve Wise wrote: Roland Dreier wrote: > It appears that my problem boils down to a single host page of memory > that is mapped for dma, and the dma address returned by dma_map_sg() > is _not_ 64KB aligned. Here is an example: > My first question is: Is there an assumption or requirement i

Re: [ofa-general] iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Roland Dreier wrote: > It appears that my problem boils down to a single host page of memory > that is mapped for dma, and the dma address returned by dma_map_sg() > is _not_ 64KB aligned. Here is an example: > My first question is: Is there an assumption or requirement in linux > that dma

Re: [ofa-general] iommu dma mapping alignment requirements

2007-12-20 Thread Roland Dreier
> It appears that my problem boils down to a single host page of memory > that is mapped for dma, and the dma address returned by dma_map_sg() > is _not_ 64KB aligned. Here is an example: > My first question is: Is there an assumption or requirement in linux > that dma_addressess should have

iommu dma mapping alignment requirements

2007-12-20 Thread Steve Wise
Hey Roland (and any iommu/ppc/dma experts out there): I'm debugging a data corruption issue that happens on PPC64 systems running rdma on kernels where the iommu page size is 4KB yet the host page size is 64KB. This "feature" was added to the PPC64 code recently, and is in kernel.org from 2.6

Re: [ofa-general] iommu dma mapping alignment requirements

2007-12-20 Thread Tom Tucker
On Thu, 2007-12-20 at 11:14 -0600, Steve Wise wrote: > Hey Roland (and any iommu/ppc/dma experts out there): > > I'm debugging a data corruption issue that happens on PPC64 systems > running rdma on kernels where the iommu page size is 4KB yet the host > page size is 64KB. This "feature" was a