Hi Jan, On Wed, Jul 21, 2021 at 04:49:26PM +0200, Jan Beulich wrote: > On 21.07.2021 16:19, Andy Smith wrote: > > I understand that below 4GiB memory use of swiotlb is disabled so > > all the time previously this was not used, and now is. Perhaps the > > bug is in there? > > > > I was told that the only simple way on a Xen dom0 to disable use of > > swiotlb would be to set the memory below 4GiB again, so I might try > > that. > > I have no idea where you take this 4GiB aspect from. What the kernel > considers "below 4GiB" in its view of the world may be at a much > higher address in system address space. And the mere amount of > memory doesn't matter here at all.
Ah, I was taking that from: https://elixir.bootlin.com/linux/v5.10.40/source/arch/x86/kernel/pci-swiotlb.c#L41 …which I had found while looking around how to disable use of swiotlb. But I think I'm probably confused - should I only be looking at arch/x86/xen/pci-swiotlb-xen.c in the case of a PV domain like dom0? I have not been able to reproduce the problem by giving a test system with identical hardware more RAM and getting fio in a guest to do random reads with a blocksize between 4kiB and 4MiB. Perhaps it is highly workload dependent then. In some ways it's a pity that I do not get call traces for the later occurrences as then I could see if it's always the same 62.xvda-0 process (and thus same guest) triggering it. It's happened three more times since my previous email, but these have been up to 46 hours apart. These were all reads, so MD just satisfied the read from the other device without kicking the nvme device out. Hmm, I have the sector offset in the MD device so maybe I can convert that into a logical volume to know if a particular guest is provoking it… If anyone happens to have any suggestions as to what kind of IO might provoke this at all so I could maybe get fio to reproduce it, please do let me know. Thanks, Andy