-----Original Message----- > Date: Wed, 10 May 2017 12:31:48 +0100 > From: Declan Doherty <declan.dohe...@intel.com> > To: Jerin Jacob <jerin.ja...@caviumnetworks.com> > CC: Thomas Monjalon <tho...@monjalon.net>, Radu Nicolau > <radu.nico...@intel.com>, dev@dpdk.org > Subject: Re: [dpdk-dev] [RFC][PATCH 2/5] pci: allow shared device instances. > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 > Thunderbird/45.8.0 > > On 10/05/2017 12:08 PM, Jerin Jacob wrote: > > -----Original Message----- > > > Date: Wed, 10 May 2017 11:52:45 +0100 > > > From: Declan Doherty <declan.dohe...@intel.com> > > > To: Thomas Monjalon <tho...@monjalon.net>, Radu Nicolau > > > <radu.nico...@intel.com> > > > CC: dev@dpdk.org > > > Subject: Re: [dpdk-dev] [RFC][PATCH 2/5] pci: allow shared device > > > instances. > > > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 > > > Thunderbird/45.8.0 > > > > > > Hey Thomas, I've been working on this with Radu, so see my take below > > > > > > On 10/05/2017 11:28 AM, Thomas Monjalon wrote: > > > > 10/05/2017 12:11, Radu Nicolau: > > > > > Hi > > > > > > > > > > > > > > > On 5/10/2017 10:09 AM, Thomas Monjalon wrote: > > > > > > Hi, > > > > > > > > > > > > 09/05/2017 16:57, Radu Nicolau: > > > > > > > Updated PCI initialization code to allow devices to be shared > > > > > > > across multiple PMDs. > > > > > > > > > > > > > > Signed-off-by: Radu Nicolau <radu.nico...@intel.com> > > > > > > I was waiting the day when we have a device shared > > > > > > by two different interfaces. > > > > > > Note that some Mellanox and Chelsio devices already instantiate > > > > > > two ethdev ports per PCI device. > > > > > > > > > > > > Please explain your idea behind this "shared" flag. > > > > > > What is your exact need? > > > > > > > > > > Currently for each pci device a look-up into a list of PMDs is > > > > > performed, and when a match is found the system moves to the next > > > > > device. Having this flag will allow a PMD to inform the system that > > > > > there may be more matches, more PMDs that can be used for this > > > > > particular device. > > > > > There is a difference when comparing to the devices you mentioned > > > > > above, > > > > > in this case the PMDs are totally different types, one network and one > > > > > cryptodev PMD for each IXGBE network card. > > > > > > > > Yes I know it is a lack in DPDK. > > > > Linux introduced MultiFunction Device in 2005: > > > > > > > > http://events.linuxfoundation.org/sites/events/files/slides/belloni-mfd-regmap-syscon_0.pdf > > > > > > > > > > So at the most basic level the intention is to allow more than one device > > > of > > > different types, in our case a net PMD and a crypto PMD, to be > > > instantiated > > > on a single PCI bar, in essence to share the bar. I'm not familiar with > > > the > > > approaches taken in the Mellanox and Chelsio devices but I assume they are > > > handled with the driver probe/create functions independently from the EAL > > > infrastructure? > > > > > > For the initial proto-typing of this RFC we only implemented the > > > multi-device creation but I envisage that there will be a requirement for > > > sharing state between drivers, or at a minimum implementing locking around > > > shared resources, registers etc. And I would like to see this done in a > > > generic fashion that can me leverage by any driver and not have each > > > driver > > > having to solve this independently. > > > > Cavium's next generation PCI based NW devices has similar scheme where we > > need to share the same BAR with multiple DPDK subsystems(ethdev, > > eventdev etc) unlike current generation(OcteonTX). > > > > Have you done investigation into how you would like to support this, and are > you trending to any particular approach. The rte_bus approach as you outline > below does sound like it would suit this multi-function device.
Not much investigation has been done as its for next generation. It is no PCIe multi function device. There will be a lot shared functions between these shared DPDK devices and there should be place holder for this in code.I thought driver/bus/foo may a option. In additional to this, If we expose new function pointer based interfaces in bus for the shared device register access and other shared resource alloc/free between these two DPDK devices, it can be centralized to one place(driver/bus/foo) and generalized. Just 2c. We haven't done any prototype. > > > I think, Another possible way to handle this in generic way is to: > > Register a new rte_bus for the shared PCI access which sits on top PCIe bus. > > With new bus's scan and probe scheme, it can probe the two devices. > > > > > > Yes, this would work and I think it makes a lot of sense in the case where > you have logically independent hardware functional blocks on a shared bus. > In our particular case, we only have a single physical device, which we are > presenting as 2 logical devices purely to improve the sw model through DPDK > existing infrastructure. We may also need to implement some shared context > for protecting access to shared resources such as register and to > synchronized exposure of capabilities. In the case of the IXBGE family of > devices they can support MACsec or IPsec functionality but not both at the > same time, so some mechanism of passing this state between the net and > crypto PMDs will be required. I guess it should be possible to do this > through the bus model as well but we'll need to have another look, although > my initial feeling is they are slightly different problems.
