Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-08-09 Thread Alex Williamson
On Thu, 9 Aug 2018 14:21:29 +1000 Alexey Kardashevskiy wrote: > On 08/08/2018 18:39, Alexey Kardashevskiy wrote: > > > > > > On 02/08/2018 02:16, Alex Williamson wrote: > >> On Wed, 1 Aug 2018 18:37:35 +1000 > >> Alexey Kardashevskiy wrote: > >> > >>> On 01/08/2018 00:29, Alex Williamson w

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-08-08 Thread Alexey Kardashevskiy
On 08/08/2018 18:39, Alexey Kardashevskiy wrote: > > > On 02/08/2018 02:16, Alex Williamson wrote: >> On Wed, 1 Aug 2018 18:37:35 +1000 >> Alexey Kardashevskiy wrote: >> >>> On 01/08/2018 00:29, Alex Williamson wrote: On Tue, 31 Jul 2018 14:03:35 +1000 Alexey Kardashevskiy wrote: >

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-08-08 Thread Alexey Kardashevskiy
On 02/08/2018 02:16, Alex Williamson wrote: > On Wed, 1 Aug 2018 18:37:35 +1000 > Alexey Kardashevskiy wrote: > >> On 01/08/2018 00:29, Alex Williamson wrote: >>> On Tue, 31 Jul 2018 14:03:35 +1000 >>> Alexey Kardashevskiy wrote: >>> On 31/07/2018 02:29, Alex Williamson wrote: >

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-08-01 Thread Alex Williamson
On Wed, 1 Aug 2018 18:37:35 +1000 Alexey Kardashevskiy wrote: > On 01/08/2018 00:29, Alex Williamson wrote: > > On Tue, 31 Jul 2018 14:03:35 +1000 > > Alexey Kardashevskiy wrote: > > > >> On 31/07/2018 02:29, Alex Williamson wrote: > >>> On Mon, 30 Jul 2018 18:58:49 +1000 > >>> Alexey Karda

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-08-01 Thread Alexey Kardashevskiy
On 01/08/2018 00:29, Alex Williamson wrote: > On Tue, 31 Jul 2018 14:03:35 +1000 > Alexey Kardashevskiy wrote: > >> On 31/07/2018 02:29, Alex Williamson wrote: >>> On Mon, 30 Jul 2018 18:58:49 +1000 >>> Alexey Kardashevskiy wrote: After some local discussions, it was pointed out that for

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-31 Thread Alex Williamson
On Tue, 31 Jul 2018 14:03:35 +1000 Alexey Kardashevskiy wrote: > On 31/07/2018 02:29, Alex Williamson wrote: > > On Mon, 30 Jul 2018 18:58:49 +1000 > > Alexey Kardashevskiy wrote: > >> After some local discussions, it was pointed out that force disabling > >> nvlinks won't bring us much as for a

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-30 Thread Alexey Kardashevskiy
On 31/07/2018 02:29, Alex Williamson wrote: > On Mon, 30 Jul 2018 18:58:49 +1000 > Alexey Kardashevskiy wrote: > >> On 11/07/2018 19:26, Alexey Kardashevskiy wrote: >>> On Tue, 10 Jul 2018 16:37:15 -0600 >>> Alex Williamson wrote: >>> On Tue, 10 Jul 2018 14:10:20 +1000 Alexey Kar

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-30 Thread Alex Williamson
On Mon, 30 Jul 2018 18:58:49 +1000 Alexey Kardashevskiy wrote: > On 11/07/2018 19:26, Alexey Kardashevskiy wrote: > > On Tue, 10 Jul 2018 16:37:15 -0600 > > Alex Williamson wrote: > > > >> On Tue, 10 Jul 2018 14:10:20 +1000 > >> Alexey Kardashevskiy wrote: > >> > >>> On Thu, 7 Jun 2018 23:

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-30 Thread Alexey Kardashevskiy
On 11/07/2018 19:26, Alexey Kardashevskiy wrote: > On Tue, 10 Jul 2018 16:37:15 -0600 > Alex Williamson wrote: > >> On Tue, 10 Jul 2018 14:10:20 +1000 >> Alexey Kardashevskiy wrote: >> >>> On Thu, 7 Jun 2018 23:03:23 -0600 >>> Alex Williamson wrote: >>> On Fri, 8 Jun 2018 14:14:23 +1

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-11 Thread Alexey Kardashevskiy
On Tue, 10 Jul 2018 16:37:15 -0600 Alex Williamson wrote: > On Tue, 10 Jul 2018 14:10:20 +1000 > Alexey Kardashevskiy wrote: > > > On Thu, 7 Jun 2018 23:03:23 -0600 > > Alex Williamson wrote: > > > > > On Fri, 8 Jun 2018 14:14:23 +1000 > > > Alexey Kardashevskiy wrote: > > > > > > > O

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-10 Thread Alex Williamson
On Tue, 10 Jul 2018 14:10:20 +1000 Alexey Kardashevskiy wrote: > On Thu, 7 Jun 2018 23:03:23 -0600 > Alex Williamson wrote: > > > On Fri, 8 Jun 2018 14:14:23 +1000 > > Alexey Kardashevskiy wrote: > > > > > On 8/6/18 1:44 pm, Alex Williamson wrote: > > > > On Fri, 8 Jun 2018 13:08:54 +10

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-07-09 Thread Alexey Kardashevskiy
On Thu, 7 Jun 2018 23:03:23 -0600 Alex Williamson wrote: > On Fri, 8 Jun 2018 14:14:23 +1000 > Alexey Kardashevskiy wrote: > > > On 8/6/18 1:44 pm, Alex Williamson wrote: > > > On Fri, 8 Jun 2018 13:08:54 +1000 > > > Alexey Kardashevskiy wrote: > > > > > >> On 8/6/18 8:15 am, Alex Willi

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alex Williamson
On Fri, 8 Jun 2018 14:14:23 +1000 Alexey Kardashevskiy wrote: > On 8/6/18 1:44 pm, Alex Williamson wrote: > > On Fri, 8 Jun 2018 13:08:54 +1000 > > Alexey Kardashevskiy wrote: > > > >> On 8/6/18 8:15 am, Alex Williamson wrote: > >>> On Fri, 08 Jun 2018 07:54:02 +1000 > >>> Benjamin Herrensc

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alexey Kardashevskiy
On 8/6/18 1:44 pm, Alex Williamson wrote: > On Fri, 8 Jun 2018 13:08:54 +1000 > Alexey Kardashevskiy wrote: > >> On 8/6/18 8:15 am, Alex Williamson wrote: >>> On Fri, 08 Jun 2018 07:54:02 +1000 >>> Benjamin Herrenschmidt wrote: >>> On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alex Williamson
On Fri, 8 Jun 2018 13:08:54 +1000 Alexey Kardashevskiy wrote: > On 8/6/18 8:15 am, Alex Williamson wrote: > > On Fri, 08 Jun 2018 07:54:02 +1000 > > Benjamin Herrenschmidt wrote: > > > >> On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote: > >>> > >>> Can we back up and discuss whethe

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alexey Kardashevskiy
On 8/6/18 8:15 am, Alex Williamson wrote: > On Fri, 08 Jun 2018 07:54:02 +1000 > Benjamin Herrenschmidt wrote: > >> On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote: >>> >>> Can we back up and discuss whether the IOMMU grouping of NVLink >>> connected devices makes sense? AIUI we have a

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alex Williamson
On Fri, 08 Jun 2018 10:58:54 +1000 Benjamin Herrenschmidt wrote: > On Thu, 2018-06-07 at 18:34 -0600, Alex Williamson wrote: > > > We *can* allow individual GPUs to be passed through, either if somebody > > > designs a system without cross links, or if the user is ok with the > > > security risk

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Benjamin Herrenschmidt
On Thu, 2018-06-07 at 18:34 -0600, Alex Williamson wrote: > > We *can* allow individual GPUs to be passed through, either if somebody > > designs a system without cross links, or if the user is ok with the > > security risk as the guest driver will not enable them if it doesn't > > "find" both side

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alex Williamson
On Fri, 08 Jun 2018 09:20:30 +1000 Benjamin Herrenschmidt wrote: > On Thu, 2018-06-07 at 16:15 -0600, Alex Williamson wrote: > > On Fri, 08 Jun 2018 07:54:02 +1000 > > Benjamin Herrenschmidt wrote: > > > > > On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote: > > > > > > > > Can we b

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Benjamin Herrenschmidt
On Thu, 2018-06-07 at 16:15 -0600, Alex Williamson wrote: > On Fri, 08 Jun 2018 07:54:02 +1000 > Benjamin Herrenschmidt wrote: > > > On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote: > > > > > > Can we back up and discuss whether the IOMMU grouping of NVLink > > > connected devices makes

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alex Williamson
On Fri, 08 Jun 2018 07:54:02 +1000 Benjamin Herrenschmidt wrote: > On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote: > > > > Can we back up and discuss whether the IOMMU grouping of NVLink > > connected devices makes sense? AIUI we have a PCI view of these > > devices and from that pers

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Benjamin Herrenschmidt
On Thu, 2018-06-07 at 11:04 -0600, Alex Williamson wrote: > > Can we back up and discuss whether the IOMMU grouping of NVLink > connected devices makes sense? AIUI we have a PCI view of these > devices and from that perspective they're isolated. That's the view of > the device used to generate t

Re: [RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alex Williamson
On Thu, 7 Jun 2018 18:44:15 +1000 Alexey Kardashevskiy wrote: > Here is an rfc of some patches adding psaa-through support > for NVIDIA V100 GPU found in some POWER9 boxes. > > The example P9 system has 6 GPUs, each accompanied with 2 bridges > representing the hardware links (aka NVLink2): >

[RFC PATCH kernel 0/5] powerpc/P9/vfio: Pass through NVIDIA Tesla V100

2018-06-07 Thread Alexey Kardashevskiy
Here is an rfc of some patches adding psaa-through support for NVIDIA V100 GPU found in some POWER9 boxes. The example P9 system has 6 GPUs, each accompanied with 2 bridges representing the hardware links (aka NVLink2): 4 0004:04:00.0 3D: NVIDIA Corporation GV100GL [Tesla V100 SXM2] (rev a1) 5