>> Ok. I'd failed to register that the bare metal code was doing this though >> with hindsight I guess that is obvious. Though without more info or >> a bare metal example being given its also possible the BIOS was doing >> enumeration etc (like CXL does for all < 2.0 devices) and hence was >> building SRAT with the necessary memory ranges in place - even if the >> driver then does the hot add dance later. > > Ankit, maybe you can share some relavent ACPI dumps from the physical > hardware and explain how this compares?
Yeah, we are pretty much emulating the baremetal behavior here (I should have been clearer). The PXM domains 8-15 are the "device associated" NUMA nodes on the host. Here are the kernel logs from the baremetal system related to these NUMA nodes. [ 0.000000] ACPI: SRAT: Node 1 PXM 8 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 2 PXM 9 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 3 PXM 10 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 4 PXM 11 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 5 PXM 12 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 6 PXM 13 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 7 PXM 14 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 8 PXM 15 [mem 0x00000000-0xffffffffffffffff] hotplug ... [ 0.000000] Initmem setup node 2 as memoryless [ 0.000000] Initmem setup node 3 as memoryless [ 0.000000] Initmem setup node 4 as memoryless [ 0.000000] Initmem setup node 5 as memoryless [ 0.000000] Initmem setup node 6 as memoryless [ 0.000000] Initmem setup node 7 as memoryless [ 0.000000] Initmem setup node 8 as memoryless [ 0.000000] Initmem setup node 9 as memoryless On the VM, it looks like the following with the PXM 2-9 associated with the device. [ 0.000000] ACPI: SRAT: Node 2 PXM 2 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 3 PXM 3 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 4 PXM 4 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 5 PXM 5 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 6 PXM 6 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 7 PXM 7 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 8 PXM 8 [mem 0x00000000-0xffffffffffffffff] hotplug [ 0.000000] ACPI: SRAT: Node 9 PXM 9 [mem 0x00000000-0xffffffffffffffff] hotplug
