Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-28 Thread Jakub Kicinski
On Thu, 28 Jan 2021 16:03:02 -0800 Saeed Mahameed wrote: > On Tue, 2021-01-26 at 17:34 -0800, Jakub Kicinski wrote: > > On Fri, 22 Jan 2021 11:36:44 -0800 Saeed Mahameed wrote: > > > This series form Parav was the theme of this mlx5 release cycle, > > > we've been waiting anxiously for the auxbus

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-28 Thread Saeed Mahameed
On Tue, 2021-01-26 at 17:34 -0800, Jakub Kicinski wrote: > On Fri, 22 Jan 2021 11:36:44 -0800 Saeed Mahameed wrote: > > This series form Parav was the theme of this mlx5 release cycle, > > we've been waiting anxiously for the auxbus infrastructure to make > > it into > > the kernel, and now as the

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-26 Thread Jakub Kicinski
On Fri, 22 Jan 2021 11:36:44 -0800 Saeed Mahameed wrote: > This series form Parav was the theme of this mlx5 release cycle, > we've been waiting anxiously for the auxbus infrastructure to make it into > the kernel, and now as the auxbus is in and all the stars are aligned, I > can finally submit th

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 5:22 AM Jason Gunthorpe wrote: > SRIOV and SF's require a simple linear lookup to learn the "function" > because the BAR space is required to be linear. Isn't this still true even for NumVF's > 256? Wouldn't there still be a contiguous VF BAR space? Don't the routing IDs

RE: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Parav Pandit
Hi Edwin, > From: Edwin Peer > Sent: Monday, January 25, 2021 2:17 AM > > On Fri, Jan 22, 2021 at 11:37 AM Saeed Mahameed > wrote: > > > For more detailed information about subfunctions please see detailed tag > > log below. > > Apologies for the tardy question out of left field, but I've bee

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 10:57:14AM +, Parav Pandit wrote: > Hi Edwin, > > > From: Edwin Peer > > Sent: Monday, January 25, 2021 2:17 AM > > > > On Fri, Jan 22, 2021 at 11:37 AM Saeed Mahameed > > wrote: > > > > > For more detailed information about subfunctions please see detailed tag > >

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 10:35 AM Edwin Peer wrote: > > Several weeks back, Jason already answered this VF scaling question from > > you at discussion [1]. > > > > [1] https://lore.kernel.org/netdev/20201216023928.gg552...@nvidia.com/ Regarding these costs: > A lot of the trappings that PCI-SIG

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 01:23:04PM -0800, Edwin Peer wrote: > On Mon, Jan 25, 2021 at 12:41 PM Jason Gunthorpe wrote: > > > > That's an implementation decision. Nothing mandates that the state has > > > to physically exist in the same structure, only that reads and writes > > > are appropriately

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 12:41 PM Jason Gunthorpe wrote: > > That's an implementation decision. Nothing mandates that the state has > > to physically exist in the same structure, only that reads and writes > > are appropriately responded to. > > Yes, PCI does mandate this, you can't store the data

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 11:23:56AM -0800, Edwin Peer wrote: > On Mon, Jan 25, 2021 at 5:22 AM Jason Gunthorpe wrote: > > > SRIOV and SF's require a simple linear lookup to learn the "function" > > because the BAR space is required to be linear. > > Isn't this still true even for NumVF's > 256? W

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 12:22:13PM -0800, Edwin Peer wrote: > On Mon, Jan 25, 2021 at 11:59 AM Jason Gunthorpe wrote: > > > Every writable data mandated by the PCI spec requires very expensive > > on-die SRAM to store it. > > That's an implementation decision. Nothing mandates that the state has

RE: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Parav Pandit
> From: Edwin Peer > Sent: Tuesday, January 26, 2021 1:36 AM > > On Mon, Jan 25, 2021 at 11:49 AM Jason Gunthorpe > wrote: > > > I've never seen someone implement a NumVF > 256 by co-opting the bus > > number. > > Usually the VF offset already places the VF routing IDs into a > different bus

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Michael Chan
On Mon, Jan 25, 2021 at 12:09 PM Edwin Peer wrote: > > On Mon, Jan 25, 2021 at 11:49 AM Jason Gunthorpe wrote: > > > I've never seen someone implement a NumVF > 256 by co-opting the bus > > number. > > Usually the VF offset already places the VF routing IDs into a > different bus number range fro

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 11:59 AM Jason Gunthorpe wrote: > Every writable data mandated by the PCI spec requires very expensive > on-die SRAM to store it. That's an implementation decision. Nothing mandates that the state has to physically exist in the same structure, only that reads and writes a

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 11:49 AM Jason Gunthorpe wrote: > I've never seen someone implement a NumVF > 256 by co-opting the bus > number. Usually the VF offset already places the VF routing IDs into a different bus number range from the PF. That much at least works today. > Can Linux even assign

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Jason Gunthorpe
On Mon, Jan 25, 2021 at 11:34:49AM -0800, Edwin Peer wrote: > What do these amount to in practice? Presumably config space is backed > by normal memory controlled by firmware. Do VF's need to expose ECAM? > Also, don't MSI tables come out of the BAR budget? Is the required BAR > space necessarily

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-25 Thread Edwin Peer
On Mon, Jan 25, 2021 at 2:57 AM Parav Pandit wrote: > > Apologies for the tardy question out of left field, but I've been > > thinking about this some more. If I recall, the primary motivation for > > this was a means to effectively address more VFs? But, why can't the > > device simply expose mo

Re: [pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-24 Thread Edwin Peer
On Fri, Jan 22, 2021 at 11:37 AM Saeed Mahameed wrote: > This series form Parav was the theme of this mlx5 release cycle, > we've been waiting anxiously for the auxbus infrastructure to make it into > the kernel, and now as the auxbus is in and all the stars are aligned, I > can finally submit th

[pull request][net-next V10 00/14] Add mlx5 subfunction support

2021-01-22 Thread Saeed Mahameed
From: Saeed Mahameed Hi Dave, Jakub, Jason, This series form Parav was the theme of this mlx5 release cycle, we've been waiting anxiously for the auxbus infrastructure to make it into the kernel, and now as the auxbus is in and all the stars are aligned, I can finally submit this patchset of the