[dpdk-dev] pci device access from dpdk secondary process with igb_uio
Hi All, I am trying to use pci vf device with secondary process in a multiple process mode and finding pci_dev->mem_resource[0].addr to be NULL. This happens when the pci device is attached to igb_uio.ko but with vfio_pci it works fine. Looking at the pci device initialization part for secondary process, when device is presented through igb_uio, (in pci_uio_map_secondary), it doesn't seems to update pci_dev->mem_resource[X].addr. For vfio-pci, it updates the addresses in pci_vfio_map_resource_secondary. With the following patch, i am able to use the device from secondary process but would like to know if I am missing something. diff --git a/drivers/bus/pci/pci_common_uio.c b/drivers/bus/pci/pci_common_uio.c index 7ea73db..f4dca9d 100644 --- a/drivers/bus/pci/pci_common_uio.c +++ b/drivers/bus/pci/pci_common_uio.c @@ -70,6 +70,7 @@ } return -1; } + dev->mem_resource[i].addr = mapaddr; } return 0; }
Re: [dpdk-dev] pci device access from dpdk secondary process with igb_uio
On Thu, 26 Dec 2019 at 16:58, vijay mohan wrote: > > Hi All, > I am trying to use pci vf device with secondary process in a multiple process > mode and finding pci_dev->mem_resource[0].addr to be NULL. This happens when > the pci device is attached to igb_uio.ko but with vfio_pci it works fine. > > Looking at the pci device initialization part for secondary process, when > device is presented through igb_uio, (in pci_uio_map_secondary), it doesn't > seems to update pci_dev->mem_resource[X].addr. For vfio-pci, it updates the > addresses in pci_vfio_map_resource_secondary. With the following patch, i am > able to use the device from secondary process but would like to know if I am > missing something. > > diff --git a/drivers/bus/pci/pci_common_uio.c > b/drivers/bus/pci/pci_common_uio.c > index 7ea73db..f4dca9d 100644 > --- a/drivers/bus/pci/pci_common_uio.c > +++ b/drivers/bus/pci/pci_common_uio.c > @@ -70,6 +70,7 @@ > } > return -1; > } > + dev->mem_resource[i].addr = mapaddr; > } > return 0; > } Hi Ferruh Yigit/John Griffin, I am trying to use QAT crypto VF device attached to igb_uio.ko with secondary process and has observed following errors while setting up the queues due to pci_dev->mem_resource[0].addr check from qat_qp_setup. mem_resource[0].addr happens to be NULL when the device is presented to dpdk through igb_uio but works fine with vfio_pci.ko. Could you please let me know if the fix mentioned above is sufficient for the device to work with secondary process. CRYPTODEV: Initialisation parameters - name: BDF_qat_sym,socket id: 0, max queue pairs: 0 qat_sym_dev_create(): Created QAT SYM device BDF_qat_sym as cryptodev instance 0 qat_comp_dev_create(): Creating QAT COMP device BDF_qat_comp qat_comp_dev_create(): Created QAT COMP device BDF_qat_comp as compressdev instance 0 Dev config Setting up queues qat_qp_setup(): Setup qp 0 on qat pci device 0 gen 2 qat_qp_setup(): Could not find VF config space (UIO driver attached?). Crypto device queue pair init failed Thanks, Vijay
Re: [dpdk-dev] [PATCH] bus/pci: pcidev access from secondary process
Hi Thomas, As long as pmd or application doesn't access dev->mem_resource[i].addr, secondary process works with igb_uio. But for pci devices like crypto_qat, pmd checks for dev->mem_resource[i].addr and bails out if the address is null. Root cause is, not initializing the mapped address in rte_pcidev when device is exposed through igb_uio. Thanks, Vijay On Mon, 30 Mar 2020 at 18:54, Thomas Monjalon wrote: > > 06/03/2020 13:28, Ferruh Yigit: > > On 3/6/2020 12:02 AM, Vijaya Mohan Guvva wrote: > > > For pci devices presented through igb_uio, pcidev->mem_resource[] is > > > not populated when the device is initialized for secondary process. > > > > > > Initialize pcidev->mem_resource[] with pci-bar mapped addresses. > > > > > > Signed-off-by: Vijaya Mohan Guvva > > > > Reviewed-by: Ferruh Yigit > > Do you mean secondary process never worked with igb_uio? > Or is there a specific case? > What is the root cause? > Should it be backported to stable? > >
Re: [dpdk-dev] pci device access from dpdk secondary process with igb_uio
Hi Fiona, Thanks for verifying the fix. I will send this patch out for review. Thanks, Vijay On Tue, 25 Feb 2020 at 07:09, Trahe, Fiona wrote: > > Hi Vijay, > > Thanks for debugging. We've recreated your issue and validated that this > fixes it. > Do you want to go ahead and send up a patch or will I do that? > > Fiona > > > > -Original Message- > > From: Trahe, Fiona > > Sent: Wednesday, January 22, 2020 5:19 PM > > To: vijay mohan ; dev@dpdk.org; Yigit, Ferruh > > > > Cc: Trahe, Fiona > > Subject: RE: [dpdk-dev] pci device access from dpdk secondary process with > > igb_uio > > > > Hi Vijay, > > > > Sorry for the delay, we missed this as It arrived while on Christmas > > holidays. > > We'll investigate, though it may take some time as this looks like a pci > > lib issue > > and up to recently we have only been using QAT in primary processes so > > don't have > > expertise yet in this area. > > > > If you could provide some background to your use-case for qat in > > primary/secondary, that may help. > > > > Fiona > > > > > -Original Message- > > > From: dev On Behalf Of vijay mohan > > > Sent: Thursday, January 2, 2020 6:58 PM > > > To: dev@dpdk.org; Yigit, Ferruh ; Griffin, John > > > > > > Subject: Re: [dpdk-dev] pci device access from dpdk secondary process > > > with igb_uio > > > > > > On Thu, 26 Dec 2019 at 16:58, vijay mohan wrote: > > > > > > > > Hi All, > > > > I am trying to use pci vf device with secondary process in a multiple > > > > process mode and finding > > > pci_dev->mem_resource[0].addr to be NULL. This happens when the pci > > > device is attached to > > > igb_uio.ko but with vfio_pci it works fine. > > > > > > > > Looking at the pci device initialization part for secondary process, > > > > when device is presented > > through > > > igb_uio, (in pci_uio_map_secondary), it doesn't seems to update > > > pci_dev->mem_resource[X].addr. > > > For vfio-pci, it updates the addresses in > > > pci_vfio_map_resource_secondary. With the following > > patch, i > > > am able to use the device from secondary process but would like to know > > > if I am missing something. > > > > > > > > diff --git a/drivers/bus/pci/pci_common_uio.c > > > > b/drivers/bus/pci/pci_common_uio.c > > > > index 7ea73db..f4dca9d 100644 > > > > --- a/drivers/bus/pci/pci_common_uio.c > > > > +++ b/drivers/bus/pci/pci_common_uio.c > > > > @@ -70,6 +70,7 @@ > > > > } > > > > return -1; > > > > } > > > > + dev->mem_resource[i].addr = mapaddr; > > > > } > > > > return 0; > > > > } > > > > > > Hi Ferruh Yigit/John Griffin, > > > I am trying to use QAT crypto VF device attached to igb_uio.ko with > > > secondary process and has observed following errors while setting up > > > the queues due to pci_dev->mem_resource[0].addr check from > > > qat_qp_setup. mem_resource[0].addr happens to be NULL when the device > > > is presented to dpdk through igb_uio but works fine with vfio_pci.ko. > > > Could you please let me know if the fix mentioned above is sufficient > > > for the device to work with secondary process. > > > > > > CRYPTODEV: Initialisation parameters - name: BDF_qat_sym,socket id: 0, > > > max queue pairs: 0 > > > qat_sym_dev_create(): Created QAT SYM device BDF_qat_sym as cryptodev > > > instance 0 > > > qat_comp_dev_create(): Creating QAT COMP device BDF_qat_comp > > > qat_comp_dev_create(): Created QAT COMP device BDF_qat_comp as > > > compressdev instance 0 > > > Dev config > > > Setting up queues > > > qat_qp_setup(): Setup qp 0 on qat pci device 0 gen 2 > > > qat_qp_setup(): Could not find VF config space (UIO driver attached?). > > > Crypto device queue pair init failed > > > > > > Thanks, > > > Vijay
