Hi, > On 5 Jun 2020, at 03:29, CodeWiz2280 <codewiz2...@gmail.com> wrote: > > On Thu, Jun 4, 2020 at 2:24 PM Julien Grall <jul...@xen.org> wrote: >> >> Hi, >> >> On 04/06/2020 13:07, CodeWiz2280 wrote: >>> On Thu, Jun 4, 2020 at 6:16 AM Julien Grall <jul...@xen.org> wrote: >>>> >>>> Hi, >>>> >>>> On 04/06/2020 10:08, Bertrand Marquis wrote: >>>>> I would have thought that linux would have need some memory, even small >>>>> in the 32bit space in order to boot. >>>> >>>> Yes it needs some, but then they are switching to use the high memory >>>> alias after the MMU has been switch on. >>>> >>>> From my understanding, the only difference is the page-tables will >>>> point to the high memory alias address rather than the low memory one. >>>> Linux will still be located at the same place but now accessed from the >>>> high memory alias rather than the low one. >>>> >>>> Note that AFAICT the secondary CPUs will still be brought-up using the >>>> low memory alias. >>>> >>>>> I could understand that some memory in the low address space needs to be >>>>> reserved by Linux as DMA area for peripherals not supporting 36-bit >>>>> addresses, but the whole low memory sounds like a big restriction. >>>> Many platforms have devices only supporting 32-bit DMA, but none of them >>>> require such aliasing. So this doesn't look to be the issue here. >>>> >>>> TBH, this code is only used by Keystone and switching address space is >>>> expensive (you have to turn off the MMU, updates page-tables, flush the >>>> cache...). I find hard to believe a developper would have come up with >>>> this complexity if it were possible to always use the low memory address >>>> range. It is even harder to believe Linux community would have accepted it. >>>> >>>>> >>>>> Would it be possible to have a bit more information on the “problem with >>>>> peripherals” here ? >>>> >>>> I am curious as well, so I looked in more depth :). Going through the >>>> Linux history, one of the commit message [1] suggests they are switching >>>> to a coherent address space. The datasheet [2] (page 75) also confirm >>>> that the low region is not IO coherent. >>>> >>>> So I think you would not be able to do DMA without flush the cache which >>>> can be pretty expensive. For a PoC, it might be possible to force Linux >>>> flushing the area before and after each DMA request. This should be >>>> possible by marking the devices as not coherent. >>>> >>>> Although, I am not entirely sure if there is any fallout. >>>> >>>> @Dave, do you think it is possible for you to have a try? I can provide >>>> the patch for Linux to disable DMA coherency if possible. >>> I attempted to do that, where I removed the "dma-coherent" flags from >>> the device tree. There are likely other issues, but the most glaring >>> problem that I ran into is that the ethernet does not work. Eth0 >>> shows up in ifconfig but there is no activity on it after a small >>> handful of message exchanges, whereas booting without Xen it seems to >>> work fine even if left in 32-bit mode (with the dma-coherent >>> disabled). I don't know what implications behind the scenes there are >>> trying to stay in the lower 0x8000_0000 alias range either though. >> >> Thank you for the answer. As wrote, Linux is working fine in 32-bit mode >> when dma-coherent is left in 32-bit mode. So this suggest a different >> issue on the platform. >> >> Given that you receive an handful of packet and then nothing, this would >> lead to maybe an interrupt problem. Can you check whether the number of >> interrupts increments the same way on baremetal and on Xen? >> >> Dumping /proc/interrupts should be sufficient. >> > I am able to ping the board from itself, do you think it could still > be an interrupt issue? It just cannot seem to ping out to a different > host (or ping from > my pc). Unfortunately, the interrupts for the netcp Ethernet driver > on this board don't show up in the cat /proc/interrupts output under > the non-Xen kernel or > Xen loaded kernel from what I can tell. I'm not sure how I would confirm > that.
Could you check the content of /proc/interrupts ? I did raise an issue several years ago on the keystone 2 related to interrupts and virtualization (no with Xen but the context should still be right): https://e2e.ti.com/support/processors/f/791/t/462126?Keystone-2-no-interrupts-received-out-of-80-and-92- There might be something to check in regards to level vs front interrupts for forwarded interrupts. Regards Bertrand > >>> I >>> would rather run it as intended by switching to the upper >>> 0x8_0000_0000 alias region. >> >> I agree this would be ideal :). >> >> Cheers, >> >> -- >> Julien Grall
