On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell <c...@cs.ualberta.ca> wrote:
> Hi,
>
> I'm trying to use a 64-bit BAR for my shared memory device. In simply
> changing the memory type in pci_register_bar() to
> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
> that BAR (and my driver crashes in pci_ioremap).
>
> from lspci:
>
> 00:04.0 RAM memory: Qumranet, Inc. Device 1110
> Subsystem: Qumranet, Inc. Device 1100
> Flags: fast devsel, IRQ 10
> Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
> Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
> Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
> Capabilities: <access denied>
> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
>
> with DEBUG_MEMREG, I see
>
> kvm_unregister_memory_area:666 Unregistering memory region
> c20000000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
> IVSHMEM: addr = 3221225472 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
> kvm_unregister_memory_area:666 Unregistering memory region
> c200c0000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
> IVSHMEM: addr = 0 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
> kvm_unregister_memory_area:666 Unregistering memory region
> c20000000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
> IVSHMEM: addr = 0 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
> kvm_unregister_memory_area:666 Unregistering memory region
> ffffffff00000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
> IVSHMEM: addr = 0 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>
> (the IVSHMEM lines are my debug statements)
>
> address sizes : 40 bits physical, 48 bits virtual (guest)
> address sizes : 38 bits physical, 48 bits virtual (host)
>
Hi, I happened to run into this problem again when trying to use a
64-bit BAR. I did a bit more digging and the test that is failing is
called from arch/x86/mm/ioremap.c in the guest and here it is.
static inline int phys_addr_valid(resource_size_t addr)
{
#ifdef CONFIG_PHYS_ADDR_T_64BIT
return !(addr >> boot_cpu_data.x86_phys_bits);
#else
return 1;
#endif
}
the value of addr (in this case the 48-bit virtual address
c20000000000) is shifted to the right shift by
boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address
size), so a non-zero value is returned which causes the test to fail
and generates the "invalid physical address" error in the guest.
Any help is appreciated as to whether this is a Qemu or guest kernel issue.
Thanks,
Cam
