On Tue, May 20, 2014 at 10:43:18PM +0000, Scott Long wrote: > Author: scottl > Date: Tue May 20 22:43:17 2014 > New Revision: 266481 > URL: http://svnweb.freebsd.org/changeset/base/266481 > > Log: > Old PCIe implementations cannot allow a DMA transfer to cross a 4GB > boundary. This was addressed several years ago by creating a parent > tag hierarchy for the root buses that set the boundary restriction > for appropriate buses and allowed child deviced to inherit it. > Somewhere along the way, this restriction was turned into a case for > marking the tag as a candidate for needing bounce buffers, instead > of just splitting the segment along the boundary line. This flag > also causes all maps associated with this tag to be non-NULL, which > in turn causes bus_dmamap_sync() to take the slow path of function > pointer indirection to discover that there's no bouncing work to > do. The end result is a lot of pages set aside in bounce pools > that will never be used, and a slow path for data buffers in nearly > every DMA-capable PCIe device. For example, our workload at Netflix > was spending nearly 1% of all CPU time going through this slow path. > > Fix this problem by being more selective about when to set the > COULD_BOUNCE flag. Only set it when the boundary restriction > exists and the consumer cannot do more than a single DMA segment > at once. This fixes the case of dynamic buffers (mbufs, bio's) > but doesn't address static buffers allocated from bus_dmamem_alloc(). > That case will be addressed in the future. > > For those interested, this was discovered thanks to Dtrace Flame > Graphs. > > Discussed with: jhb, kib > Obtained from: Netflix, Inc. > MFC after: 3 days > > Modified: > head/sys/x86/x86/busdma_bounce.c > > Modified: head/sys/x86/x86/busdma_bounce.c > ============================================================================== > --- head/sys/x86/x86/busdma_bounce.c Tue May 20 22:11:52 2014 > (r266480) > +++ head/sys/x86/x86/busdma_bounce.c Tue May 20 22:43:17 2014 > (r266481) > @@ -172,12 +172,35 @@ bounce_bus_dma_tag_create(bus_dma_tag_t > newtag->map_count = 0; > newtag->segments = NULL; > > + /* > + * Bouncing might be needed if there's a filter. > + * XXX Filters are likely broken as there's no way to > + * guarantee that bounce pages will also satisfy the > + * filter requirement. > + */ > if (parent != NULL && ((newtag->common.filter != NULL) || > ((parent->common.flags & BUS_DMA_COULD_BOUNCE) != 0))) > newtag->common.flags |= BUS_DMA_COULD_BOUNCE; > > - if (newtag->common.lowaddr < ptoa((vm_paddr_t)Maxmem) || > - newtag->common.alignment > 1) > + /* > + * Bouncing might be needed if there's an upper memory > + * restriction. > + */ > + if (newtag->common.lowaddr < ptoa((vm_paddr_t)Maxmem)) > + newtag->common.flags |= BUS_DMA_COULD_BOUNCE; > + > + /* > + * Bouncing might be needed if there's an alignment > + * restriction that can't be satisfied by breaking up > + * the segment. > + * XXX Need to consider non-natural alignment. > + * XXX Static allocations that tie to bus_dmamem_alloc() > + * will likely pass this test and be penalized with > + * the COULD_BOUNCE flag. Should probably have > + * bus_dmamem_alloc() clear this flag. > + */ > + if ((newtag->common.nsegments <= 1) && > + (newtag->common.alignment > 1)) > newtag->common.flags |= BUS_DMA_COULD_BOUNCE; > > if (((newtag->common.flags & BUS_DMA_COULD_BOUNCE) != 0) && You changed the handling of the alignment, which is probably not correct. The problematic parameter, if any, is boundary.
pgpMOO_E4_Tix.pgp
Description: PGP signature
