Re: PCI pass-through problem for SN570 NVME SSD

2022-07-08 Thread G.R.
On Fri, Jul 8, 2022 at 10:28 AM G.R. wrote: > > On Fri, Jul 8, 2022 at 12:38 AM Jan Beulich wrote: > > > > > I built both 4.14.3 debug version and 4.16.1 release version for > > > testing purposes. > > > Unfortunately they gave me absolutely zero information, since both of > > > them are not able

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-08 Thread G.R.
On Fri, Jul 8, 2022 at 10:28 AM G.R. wrote: > > On Fri, Jul 8, 2022 at 12:38 AM Jan Beulich wrote: > > > But the 'xl pci-assignable-remove' will lead to xl segmentation fault... > > >> [ 655.041442] xl[975]: segfault at 0 ip 7f2cccdaf71f sp > > >> 7ffd73a3d4d0 error 4 in libxenlight.so.

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-07 Thread G.R.
On Fri, Jul 8, 2022 at 12:38 AM Jan Beulich wrote: > > On 07.07.2022 17:24, G.R. wrote: > > On Wed, Jul 6, 2022 at 2:33 PM Jan Beulich wrote: > >> > >> On 06.07.2022 08:25, G.R. wrote: > >>> On Tue, Jul 5, 2022 at 7:59 PM Jan Beulich wrote: > Nothing useful in there. Yet independent of that

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-07 Thread Jan Beulich
On 07.07.2022 17:24, G.R. wrote: > On Wed, Jul 6, 2022 at 2:33 PM Jan Beulich wrote: >> >> On 06.07.2022 08:25, G.R. wrote: >>> On Tue, Jul 5, 2022 at 7:59 PM Jan Beulich wrote: Nothing useful in there. Yet independent of that I guess we need to separate the issues you're seeing. Otherw

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-07 Thread Jan Beulich
On 07.07.2022 17:36, G.R. wrote: > On Thu, Jul 7, 2022 at 11:24 PM G.R. wrote: >> >> On Wed, Jul 6, 2022 at 2:33 PM Jan Beulich wrote: >>> Should I expect a debug build of XEN hypervisor to give better diagnose messages, without the debug patch that Roger mentioned? >>> >>> Well, "expec

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-07 Thread G.R.
On Thu, Jul 7, 2022 at 11:24 PM G.R. wrote: > > On Wed, Jul 6, 2022 at 2:33 PM Jan Beulich wrote: > > > > > Should I expect a debug build of XEN hypervisor to give better > > > diagnose messages, without the debug patch that Roger mentioned? > > > > Well, "expect" is perhaps too much to say, but

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-07 Thread G.R.
On Wed, Jul 6, 2022 at 2:33 PM Jan Beulich wrote: > > On 06.07.2022 08:25, G.R. wrote: > > On Tue, Jul 5, 2022 at 7:59 PM Jan Beulich wrote: > >> Nothing useful in there. Yet independent of that I guess we need to > >> separate the issues you're seeing. Otherwise it'll be impossible to > >> know

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-05 Thread Jan Beulich
On 06.07.2022 08:25, G.R. wrote: > On Tue, Jul 5, 2022 at 7:59 PM Jan Beulich wrote: >> Nothing useful in there. Yet independent of that I guess we need to >> separate the issues you're seeing. Otherwise it'll be impossible to >> know what piece of data belongs where. > Yep, I think I'm seeing sev

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-05 Thread G.R.
On Tue, Jul 5, 2022 at 7:59 PM Jan Beulich wrote: > Nothing useful in there. Yet independent of that I guess we need to > separate the issues you're seeing. Otherwise it'll be impossible to > know what piece of data belongs where. Yep, I think I'm seeing several different issues here: 1. The FLR r

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-05 Thread Jason Andryuk
On Mon, Jul 4, 2022 at 11:57 AM Roger Pau Monné wrote: > > On Mon, Jul 04, 2022 at 11:44:14PM +0800, G.R. wrote: > > On Mon, Jul 4, 2022 at 11:33 PM Roger Pau Monné > > wrote: > > > > > > Right, so hvmloader attempts to place a BAR from 05:00.0 and a BAR > > > from 00:17.0 into the same page, wh

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-05 Thread Jan Beulich
On 05.07.2022 13:31, G.R. wrote: > On Tue, Jul 5, 2022 at 5:04 PM Jan Beulich wrote: >> >> On 04.07.2022 18:31, G.R. wrote: >>> On Tue, Jul 5, 2022 at 12:21 AM Roger Pau Monné >>> wrote: > I retried with the following: > pci=['05:00.0,permissive=1,msitranslate=1'] > Those extra optio

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-05 Thread G.R.
On Tue, Jul 5, 2022 at 5:04 PM Jan Beulich wrote: > > On 04.07.2022 18:31, G.R. wrote: > > On Tue, Jul 5, 2022 at 12:21 AM Roger Pau Monné > > wrote: > >>> I retried with the following: > >>> pci=['05:00.0,permissive=1,msitranslate=1'] > >>> Those extra options suppressed some error logging, but

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-05 Thread Jan Beulich
On 04.07.2022 18:31, G.R. wrote: > On Tue, Jul 5, 2022 at 12:21 AM Roger Pau Monné wrote: >> >> On Mon, Jul 04, 2022 at 11:37:13PM +0800, G.R. wrote: >>> On Mon, Jul 4, 2022 at 11:15 PM G.R. >>> wrote: On Mon, Jul 4, 2022 at 10:51 PM G.R. wrote: > > On Mon, Jul 4, 2022 a

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Tue, Jul 5, 2022 at 12:21 AM Roger Pau Monné wrote: > > On Mon, Jul 04, 2022 at 11:37:13PM +0800, G.R. wrote: > > On Mon, Jul 4, 2022 at 11:15 PM G.R. > > wrote: > > > > > > On Mon, Jul 4, 2022 at 10:51 PM G.R. > > > wrote: > > > > > > > > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné >

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Jan Beulich
On 04.07.2022 18:05, Roger Pau Monné wrote: > On Mon, Jul 04, 2022 at 11:37:13PM +0800, G.R. wrote: >> On Mon, Jul 4, 2022 at 11:15 PM G.R. >> wrote: >>> >>> On Mon, Jul 4, 2022 at 10:51 PM G.R. >>> wrote: On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné wrote: > Can you past

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Jan Beulich
On 04.07.2022 17:33, Roger Pau Monné wrote: > On Mon, Jul 04, 2022 at 10:51:53PM +0800, G.R. wrote: >> On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné wrote: 05:00.0 Non-Volatile memory controller: Sandisk Corp Device 501a (prog-if 02 [NVM Express]) Subsystem: Sandisk Cor

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Roger Pau Monné
On Mon, Jul 04, 2022 at 11:37:13PM +0800, G.R. wrote: > On Mon, Jul 4, 2022 at 11:15 PM G.R. wrote: > > > > On Mon, Jul 4, 2022 at 10:51 PM G.R. > > wrote: > > > > > > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné > > > wrote: > > > > Can you paste the lspci -vvv output for any other device y

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Roger Pau Monné
On Mon, Jul 04, 2022 at 11:44:14PM +0800, G.R. wrote: > On Mon, Jul 4, 2022 at 11:33 PM Roger Pau Monné wrote: > > > > Right, so hvmloader attempts to place a BAR from 05:00.0 and a BAR > > from 00:17.0 into the same page, which is not that good behavior. It > > might be sensible to attempt to sh

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Mon, Jul 4, 2022 at 11:33 PM Roger Pau Monné wrote: > > Right, so hvmloader attempts to place a BAR from 05:00.0 and a BAR > from 00:17.0 into the same page, which is not that good behavior. It > might be sensible to attempt to share the page if both BARs belong to > the same device, but not i

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Mon, Jul 4, 2022 at 11:15 PM G.R. wrote: > > On Mon, Jul 4, 2022 at 10:51 PM G.R. wrote: > > > > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné wrote: > > > Can you paste the lspci -vvv output for any other device you are also > > > passing through to this guest? > > > > > As reminded by this

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Roger Pau Monné
On Mon, Jul 04, 2022 at 10:51:53PM +0800, G.R. wrote: > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné wrote: > > > > > > 05:00.0 Non-Volatile memory controller: Sandisk Corp Device 501a (prog-if > > > 02 [NVM Express]) > > > Subsystem: Sandisk Corp Device 501a > > > Control: I/O+ Mem

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Mon, Jul 4, 2022 at 10:51 PM G.R. wrote: > > On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné wrote: > > > > > > 05:00.0 Non-Volatile memory controller: Sandisk Corp Device 501a (prog-if > > > 02 [NVM Express]) > > > Subsystem: Sandisk Corp Device 501a > > > Control: I/O+ Mem+ BusMa

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Mon, Jul 4, 2022 at 9:09 PM Roger Pau Monné wrote: > > > > 05:00.0 Non-Volatile memory controller: Sandisk Corp Device 501a (prog-if > > 02 [NVM Express]) > > Subsystem: Sandisk Corp Device 501a > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > > Stepping-

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Roger Pau Monné
On Mon, Jul 04, 2022 at 07:34:47PM +0800, G.R. wrote: > On Mon, Jul 4, 2022 at 5:53 PM Roger Pau Monné wrote: > > > > On Sun, Jul 03, 2022 at 01:43:11AM +0800, G.R. wrote: > > > Hi everybody, > > > > > > I run into problems passing through a SN570 NVME SSD to a HVM guest. > > > So far I have no id

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Mon, Jul 4, 2022 at 7:34 PM G.R. wrote: > > On Mon, Jul 4, 2022 at 5:53 PM Roger Pau Monné wrote: > > > > Would also be helpful if you could get the RMRR regions from that > > box. Booting with `iommu=verbose` on the Xen command line should print > > those. > Coming in my next reply... See at

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread G.R.
On Mon, Jul 4, 2022 at 5:53 PM Roger Pau Monné wrote: > > On Sun, Jul 03, 2022 at 01:43:11AM +0800, G.R. wrote: > > Hi everybody, > > > > I run into problems passing through a SN570 NVME SSD to a HVM guest. > > So far I have no idea if the problem is with this specific SSD or with > > the CPU + mo

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Jan Beulich
On 04.07.2022 11:50, Roger Pau Monné wrote: > On Sun, Jul 03, 2022 at 01:43:11AM +0800, G.R. wrote: >> Hi everybody, >> >> I run into problems passing through a SN570 NVME SSD to a HVM guest. >> So far I have no idea if the problem is with this specific SSD or with >> the CPU + motherboard combinat

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Jan Beulich
(Note: please don't cross-post; removing xen-users@.) On 04.07.2022 08:37, G.R. wrote: > Update some findings with extra triage effort... > Detailed log could be found in the attachments. > 1. Confirm stock Debian 11.2 kernel (5.10) shares the same syndrome.. > 2. With loglvl=all, it reveals why t

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-04 Thread Roger Pau Monné
On Sun, Jul 03, 2022 at 01:43:11AM +0800, G.R. wrote: > Hi everybody, > > I run into problems passing through a SN570 NVME SSD to a HVM guest. > So far I have no idea if the problem is with this specific SSD or with > the CPU + motherboard combination or the SW stack. > Looking for some suggestion

Re: PCI pass-through problem for SN570 NVME SSD

2022-07-03 Thread G.R.
Update some findings with extra triage effort... Detailed log could be found in the attachments. 1. Confirm stock Debian 11.2 kernel (5.10) shares the same syndrome.. 2. With loglvl=all, it reveals why the mapping failure happens, looks like it comes from some duplicated mapping.. (XEN) memory_map:

PCI pass-through problem for SN570 NVME SSD

2022-07-02 Thread G.R.
Hi everybody, I run into problems passing through a SN570 NVME SSD to a HVM guest. So far I have no idea if the problem is with this specific SSD or with the CPU + motherboard combination or the SW stack. Looking for some suggestions on troubleshooting. List of build info: CPU+motherboard: E-2146