Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-29 Thread Dario Faggioli
On Tue, 2015-07-28 at 17:11 +0200, Juergen Gross wrote: > On 07/28/2015 06:29 AM, Juergen Gross wrote: > > On 07/27/2015 04:09 PM, Dario Faggioli wrote: > >> On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: > >>> On 07/24/2015 05:58 PM, Dario Faggioli wrote: > >> > So, just to check if

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-29 Thread Dario Faggioli
On Wed, 2015-07-29 at 08:04 +0200, Juergen Gross wrote: > On 07/28/2015 06:17 PM, Dario Faggioli wrote: > >> On 07/28/2015 06:29 AM, Juergen Gross wrote: > > > >>> I'll make some performance tests on a big machine (4 sockets, 60 cores, > >>> 120 threads) regarding topology information: > >>> > > I'

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Juergen Gross
On 07/28/2015 06:17 PM, Dario Faggioli wrote: On Tue, 2015-07-28 at 17:11 +0200, Juergen Gross wrote: On 07/28/2015 06:29 AM, Juergen Gross wrote: I'll make some performance tests on a big machine (4 sockets, 60 cores, 120 threads) regarding topology information: - bare metal - "random" topo

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Dario Faggioli
On Tue, 2015-07-28 at 18:17 +0200, Dario Faggioli wrote: > So, my test box looks like this: > cpu_topology : > cpu:coresocket node > 0: 010 > 1: 010 > 2: 110 > 3: 110 > 4:

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Dario Faggioli
On Tue, 2015-07-28 at 17:11 +0200, Juergen Gross wrote: > On 07/28/2015 06:29 AM, Juergen Gross wrote: > > I'll make some performance tests on a big machine (4 sockets, 60 cores, > > 120 threads) regarding topology information: > > > > - bare metal > > - "random" topology (like today) > > - "simpl

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Dario Faggioli
On Tue, 2015-07-28 at 11:05 +0100, Wei Liu wrote: > On Fri, Jul 24, 2015 at 06:05:59PM +0200, Dario Faggioli wrote: > > BTW, I've also been grepping, and I'm not seeing XENMEM_get_vnumainfo > > being called anywhere either... Well, no wonder, we're seeing vNUMA > > setup issues! If I did check for

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Juergen Gross
On 07/28/2015 06:29 AM, Juergen Gross wrote: On 07/27/2015 04:09 PM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, just to check if I'm understanding is correct: you'd like to add an abstraction layer, in Linux

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Wei Liu
On Fri, Jul 24, 2015 at 06:05:59PM +0200, Dario Faggioli wrote: > On Fri, 2015-07-24 at 17:14 +0200, Juergen Gross wrote: > > On 07/24/2015 04:44 PM, Dario Faggioli wrote: > > > > Ok. And I already have a question (as I lost track of things a bit). > > > What you just said about ACPI tables is cer

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-28 Thread Andrew Cooper
On 28/07/15 04:52, Juergen Gross wrote: > On 07/28/2015 01:19 AM, Andrew Cooper wrote: >> On 27/07/2015 18:42, Dario Faggioli wrote: >>> On Mon, 2015-07-27 at 17:33 +0100, Andrew Cooper wrote: On 27/07/15 17:31, David Vrabel wrote: > >> Yeah, indeed. >> That's the downside of Juerg

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 04:09 PM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, just to check if I'm understanding is correct: you'd like to add an abstraction layer, in Linux, like in generic (or, perhaps, scheduling) co

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/28/2015 01:19 AM, Andrew Cooper wrote: On 27/07/2015 18:42, Dario Faggioli wrote: On Mon, 2015-07-27 at 17:33 +0100, Andrew Cooper wrote: On 27/07/15 17:31, David Vrabel wrote: Yeah, indeed. That's the downside of Juergen's "Linux scheduler approach". But the issue is there, even witho

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Andrew Cooper
On 27/07/2015 18:42, Dario Faggioli wrote: > On Mon, 2015-07-27 at 17:33 +0100, Andrew Cooper wrote: >> On 27/07/15 17:31, > David Vrabel wrote: >>> Yeah, indeed. That's the downside of Juergen's "Linux scheduler approach". But the issue is there, even without taking vNUMA into acco

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Konrad Rzeszutek Wilk
. snip.. > So, it looks to me that: > 1) any application using CPUID for either licensing or > placement/performance optimization will get (potentially) random > results; Right, that is a bug that Andrew outlined in this leveling document I believe. We just pluck the cpuid results on wha

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Mon, 2015-07-27 at 17:33 +0100, Andrew Cooper wrote: > On 27/07/15 17:31, David Vrabel wrote: > > > >> Yeah, indeed. That's the downside of Juergen's "Linux scheduler > >> approach". But the issue is there, even without taking vNUMA into > >> account, and I think something like that would really

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Andrew Cooper
On 27/07/15 17:31, David Vrabel wrote: > >>> 2. For HVM guests, use the existing hardware interfaces to present NUMA >>> topology. i.e., CPUID, ACPI tables etc. This will work for both kernel >>> and userspace and both will see the same topology. >>> >>> This also has the advantage that any hyper

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread David Vrabel
On 27/07/15 17:02, Dario Faggioli wrote: > On Mon, 2015-07-27 at 16:13 +0100, David Vrabel wrote: >> On 16/07/15 11:32, Dario Faggioli wrote: >>> >>> Anyway, is there anything we can do to fix or workaround things? >> >> This thread has gotten a bit long... >> > Yep, indeed... :-( > >> For Linux I

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Mon, 2015-07-27 at 16:13 +0100, David Vrabel wrote: > On 16/07/15 11:32, Dario Faggioli wrote: > > > > Anyway, is there anything we can do to fix or workaround things? > > This thread has gotten a bit long... > Yep, indeed... :-( > For Linux I would like to see: > > 1. No support for NUMA i

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread David Vrabel
On 16/07/15 11:32, Dario Faggioli wrote: > > Anyway, is there anything we can do to fix or workaround things? This thread has gotten a bit long... For Linux I would like to see: 1. No support for NUMA in PV guests -- if you want new MM features in a guest use HVM. 2. For HVM guests, use the ex

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 04:51 PM, Boris Ostrovsky wrote: On 07/27/2015 10:43 AM, Juergen Gross wrote: On 07/27/2015 04:34 PM, Boris Ostrovsky wrote: On 07/27/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, j

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Mon, 2015-07-27 at 10:34 -0400, Boris Ostrovsky wrote: > On 07/27/2015 10:09 AM, Dario Faggioli wrote: > > Of course, it's not that my opinion on where should be in Linux counts > > that much! :-D Nevertheless, I wanted to make it clear that, while > > skeptic at the beginning, I now think th

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Boris Ostrovsky
On 07/27/2015 10:43 AM, Juergen Gross wrote: On 07/27/2015 04:34 PM, Boris Ostrovsky wrote: On 07/27/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, just to check if I'm understanding is correct: you

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 04:34 PM, Boris Ostrovsky wrote: On 07/27/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, just to check if I'm understanding is correct: you'd like to add an abstraction layer, in Linux

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 04:34 PM, Boris Ostrovsky wrote: On 07/27/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, just to check if I'm understanding is correct: you'd like to add an abstraction layer, in Linux

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Boris Ostrovsky
On 07/27/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: On 07/24/2015 05:58 PM, Dario Faggioli wrote: So, just to check if I'm understanding is correct: you'd like to add an abstraction layer, in Linux, like in generic (or, perhaps, scheduling) code

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Fri, 2015-07-24 at 18:10 +0200, Juergen Gross wrote: > On 07/24/2015 05:58 PM, Dario Faggioli wrote: > > So, just to check if I'm understanding is correct: you'd like to add an > > abstraction layer, in Linux, like in generic (or, perhaps, scheduling) > > code, to hide the direct interaction wi

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Mon, 2015-07-27 at 12:11 +0100, George Dunlap wrote: > 1. Userspace applications are in the habit of reading CPUID to determine > the topology of the system they're running on > I'd add this item here: 1b. Linux kernel uses CPUID to configure some bits of its scheduler. The result of that

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 03:23 PM, Dario Faggioli wrote: On Mon, 2015-07-27 at 14:01 +0200, Juergen Gross wrote: On 07/27/2015 01:11 PM, George Dunlap wrote: Or alternately, if the user wants to give up on the "consolidation" aspect of virtualization, they can pin vcpus to pcpus and then pass in the act

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Fri, 2015-07-24 at 13:11 -0400, Boris Ostrovsky wrote: > On 07/24/2015 12:48 PM, Juergen Gross wrote: > > On 07/24/2015 06:40 PM, Boris Ostrovsky wrote: > >> On 07/24/2015 12:10 PM, Juergen Gross wrote: > >>> > >>> If we can fiddle with the masks on boot, we could do it in a running > >>> system

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Mon, 2015-07-27 at 14:01 +0200, Juergen Gross wrote: > On 07/27/2015 01:11 PM, George Dunlap wrote: > > Or alternately, if the user wants to give up on the "consolidation" > > aspect of virtualization, they can pin vcpus to pcpus and then pass in > > the actual host topology (hyperthreads and a

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Dario Faggioli
On Mon, 2015-07-27 at 11:49 +0100, Andrew Cooper wrote: > On 27/07/15 11:41, George Dunlap wrote: > > Can you expand a little on this? I'm having trouble figuring out > > exactly what user-space applications are reading and how they're using > > it -- and, how they work currently in virtual envir

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Tim Deegan
At 14:01 +0200 on 27 Jul (1438005701), Juergen Gross wrote: > There would be another solution, of course: > > Support hyperthreads in the Xen scheduler via gang scheduling. While > this is not a simple solution, it is a fair one. Hyperthreads on one > core can influence each other rather much. Wit

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 01:11 PM, George Dunlap wrote: On 07/27/2015 11:54 AM, Juergen Gross wrote: On 07/27/2015 12:43 PM, George Dunlap wrote: On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross wrote: On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: On 07/24/2015 12:39 PM, Juergen Gross wrote: I don'

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 12:54 PM, Andrew Cooper wrote: On 27/07/15 11:43, George Dunlap wrote: On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross wrote: On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: On 07/24/2015 12:39 PM, Juergen Gross wrote: I don't say mangling cpuids can't solve the scheduling prob

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread George Dunlap
On 07/27/2015 11:54 AM, Juergen Gross wrote: > On 07/27/2015 12:43 PM, George Dunlap wrote: >> On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross wrote: >>> On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: On 07/24/2015 12:39 PM, Juergen Gross wrote: > > > > I don't say manglin

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Andrew Cooper
On 27/07/15 11:43, George Dunlap wrote: > On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross wrote: >> On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: >>> On 07/24/2015 12:39 PM, Juergen Gross wrote: I don't say mangling cpuids can't solve the scheduling problem. It surely can. But

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Juergen Gross
On 07/27/2015 12:43 PM, George Dunlap wrote: On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross wrote: On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: On 07/24/2015 12:39 PM, Juergen Gross wrote: I don't say mangling cpuids can't solve the scheduling problem. It surely can. But it can't solve

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread Andrew Cooper
On 27/07/15 11:41, George Dunlap wrote: > On Fri, Jul 24, 2015 at 5:09 PM, Konrad Rzeszutek Wilk > wrote: >> On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: >>> On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: On 07/24/2015 05:14 PM, Juergen Gross wrote: > On 07/24/

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread George Dunlap
On Mon, Jul 27, 2015 at 5:35 AM, Juergen Gross wrote: > On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: >> >> On 07/24/2015 12:39 PM, Juergen Gross wrote: >>> >>> >>> >>> I don't say mangling cpuids can't solve the scheduling problem. It >>> surely can. But it can't solve the scheduling problem wit

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-27 Thread George Dunlap
On Fri, Jul 24, 2015 at 5:09 PM, Konrad Rzeszutek Wilk wrote: > On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: >> On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: >> > On 07/24/2015 05:14 PM, Juergen Gross wrote: >> > > On 07/24/2015 04:44 PM, Dario Faggioli wrote: >> >> >

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-26 Thread Juergen Gross
On 07/24/2015 06:44 PM, Boris Ostrovsky wrote: On 07/24/2015 12:39 PM, Juergen Gross wrote: I don't say mangling cpuids can't solve the scheduling problem. It surely can. But it can't solve the scheduling problem without hiding information like number of sockets or cores which might be require

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-26 Thread Juergen Gross
On 07/24/2015 06:40 PM, Boris Ostrovsky wrote: On 07/24/2015 12:10 PM, Juergen Gross wrote: If we can fiddle with the masks on boot, we could do it in a running system, too. Another advantage with not relying on cpuid. :-) I am trying to catch up with this thread so I may have missed it, but

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Elena Ufimtseva
On Fri, Jul 24, 2015 at 04:44:36PM +0200, Dario Faggioli wrote: > On Fri, 2015-07-24 at 12:28 +0200, Juergen Gross wrote: > > On 07/23/2015 04:07 PM, Dario Faggioli wrote: > > > > FWIW, I was thinking that the kernel were a better place, as Juergen is > > > saying, while now I'm more convinced tha

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Boris Ostrovsky
On 07/24/2015 12:48 PM, Juergen Gross wrote: On 07/24/2015 06:40 PM, Boris Ostrovsky wrote: On 07/24/2015 12:10 PM, Juergen Gross wrote: If we can fiddle with the masks on boot, we could do it in a running system, too. Another advantage with not relying on cpuid. :-) I am trying to catch up

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/24/2015 06:40 PM, Boris Ostrovsky wrote: On 07/24/2015 12:10 PM, Juergen Gross wrote: If we can fiddle with the masks on boot, we could do it in a running system, too. Another advantage with not relying on cpuid. :-) I am trying to catch up with this thread so I may have missed it, but

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Boris Ostrovsky
On 07/24/2015 12:39 PM, Juergen Gross wrote: I don't say mangling cpuids can't solve the scheduling problem. It surely can. But it can't solve the scheduling problem without hiding information like number of sockets or cores which might be required for license purposes. If we don't care, fine.

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Boris Ostrovsky
On 07/24/2015 12:10 PM, Juergen Gross wrote: If we can fiddle with the masks on boot, we could do it in a running system, too. Another advantage with not relying on cpuid. :-) I am trying to catch up with this thread so I may have missed it, but I still don't understand why we don't want to

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/24/2015 06:29 PM, Konrad Rzeszutek Wilk wrote: On Fri, Jul 24, 2015 at 06:18:56PM +0200, Juergen Gross wrote: On 07/24/2015 06:09 PM, Konrad Rzeszutek Wilk wrote: On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: On 0

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Konrad Rzeszutek Wilk
On Fri, Jul 24, 2015 at 06:18:56PM +0200, Juergen Gross wrote: > On 07/24/2015 06:09 PM, Konrad Rzeszutek Wilk wrote: > >On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: > >>On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: > >>>On 07/24/2015 05:14 PM, Juergen Gross wrote: > >>

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/24/2015 06:09 PM, Konrad Rzeszutek Wilk wrote: On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: On 07/24/2015 05:14 PM, Juergen Gross wrote: On 07/24/2015 04:44 PM, Dario Faggioli wrote: In fact, I think that it is

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Dario Faggioli
On Fri, 2015-07-24 at 12:09 -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: > > So, just to check if I'm understanding is correct: you'd like to add an > > abstraction layer, in Linux, like in generic (or, perhaps, scheduling) > > code, to hide

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/24/2015 05:58 PM, Dario Faggioli wrote: On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: On 07/24/2015 05:14 PM, Juergen Gross wrote: On 07/24/2015 04:44 PM, Dario Faggioli wrote: In fact, I think that it is the topology, i.e., what comes from MSRs, that needs to adapt, and foll

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Konrad Rzeszutek Wilk
On Fri, Jul 24, 2015 at 05:58:29PM +0200, Dario Faggioli wrote: > On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: > > On 07/24/2015 05:14 PM, Juergen Gross wrote: > > > On 07/24/2015 04:44 PM, Dario Faggioli wrote: > > > >> In fact, I think that it is the topology, i.e., what comes from MS

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Dario Faggioli
On Fri, 2015-07-24 at 17:14 +0200, Juergen Gross wrote: > On 07/24/2015 04:44 PM, Dario Faggioli wrote: > > Ok. And I already have a question (as I lost track of things a bit). > > What you just said about ACPI tables is certainly true for baremetal and > > HVM guests, but for PV? At the time I wa

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Boris Ostrovsky
On 07/23/2015 03:25 AM, Jan Beulich wrote: On 22.07.15 at 20:10, wrote: I don't think this is currently doable with what we have for CPUID support in xl syntax. I am pretty sure we need to at least be able to specify all leaf 4's indexes. And we can't. BTW, irrespective of this particular prob

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Dario Faggioli
On Fri, 2015-07-24 at 17:24 +0200, Juergen Gross wrote: > On 07/24/2015 05:14 PM, Juergen Gross wrote: > > On 07/24/2015 04:44 PM, Dario Faggioli wrote: > >> In fact, I think that it is the topology, i.e., what comes from MSRs, > >> that needs to adapt, and follow vNUMA, as much as possible. Do we

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/24/2015 05:14 PM, Juergen Gross wrote: On 07/24/2015 04:44 PM, Dario Faggioli wrote: On Fri, 2015-07-24 at 12:28 +0200, Juergen Gross wrote: On 07/23/2015 04:07 PM, Dario Faggioli wrote: FWIW, I was thinking that the kernel were a better place, as Juergen is saying, while now I'm more

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/24/2015 04:44 PM, Dario Faggioli wrote: On Fri, 2015-07-24 at 12:28 +0200, Juergen Gross wrote: On 07/23/2015 04:07 PM, Dario Faggioli wrote: FWIW, I was thinking that the kernel were a better place, as Juergen is saying, while now I'm more convinced that tools would be more appropriate

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Dario Faggioli
On Fri, 2015-07-24 at 12:28 +0200, Juergen Gross wrote: > On 07/23/2015 04:07 PM, Dario Faggioli wrote: > > FWIW, I was thinking that the kernel were a better place, as Juergen is > > saying, while now I'm more convinced that tools would be more > > appropriate, as Boris is saying. > > I've colle

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-24 Thread Juergen Gross
On 07/23/2015 04:07 PM, Dario Faggioli wrote: On Thu, 2015-07-23 at 06:43 +0200, Juergen Gross wrote: On 07/22/2015 04:44 PM, Boris Ostrovsky wrote: On 07/22/2015 10:09 AM, Juergen Gross wrote: I think we have 2 possible solutions: 1. Try to handle this all in the hypervisor via CPUID mangl

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-23 Thread Juergen Gross
On 07/23/2015 04:07 PM, Dario Faggioli wrote: On Thu, 2015-07-23 at 06:43 +0200, Juergen Gross wrote: On 07/22/2015 04:44 PM, Boris Ostrovsky wrote: On 07/22/2015 10:09 AM, Juergen Gross wrote: I think we have 2 possible solutions: 1. Try to handle this all in the hypervisor via CPUID mangl

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-23 Thread Dario Faggioli
On Thu, 2015-07-23 at 06:43 +0200, Juergen Gross wrote: > On 07/22/2015 04:44 PM, Boris Ostrovsky wrote: > > On 07/22/2015 10:09 AM, Juergen Gross wrote: > I think we have 2 possible solutions: > > 1. Try to handle this all in the hypervisor via CPUID mangling. > > 2. Add

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-23 Thread Dario Faggioli
On Wed, 2015-07-22 at 14:10 -0400, Boris Ostrovsky wrote: > On 07/22/2015 11:49 AM, Dario Faggioli wrote: > > In fact, of course there are other issues (like the ones you're > > mentioning, caused by this), but it's only with vNUMA that I see 2 out > > of 4 vcpus completely lost! :-/ > > My guess

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-23 Thread Andrew Cooper
On 23/07/15 05:43, Juergen Gross wrote: > On 07/22/2015 04:44 PM, Boris Ostrovsky wrote: >> On 07/22/2015 10:09 AM, Juergen Gross wrote: >>> On 07/22/2015 03:58 PM, Boris Ostrovsky wrote: On 07/22/2015 09:50 AM, Juergen Gross wrote: > On 07/22/2015 03:36 PM, Dario Faggioli wrote: >> On

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-23 Thread Jan Beulich
>>> On 23.07.15 at 06:43, wrote: > Hmm, I didn't think of user processes. Are you aware of cases where they > are to be considered? Why wouldn't a sophisticated user mode program attempt to adjust certain memory objects' sizes based on cache size? Jan __

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-23 Thread Jan Beulich
>>> On 22.07.15 at 20:10, wrote: > I don't think this is currently doable with what we have for CPUID > support in xl syntax. I am pretty sure we need to at least be able to > specify all leaf 4's indexes. And we can't. > > BTW, irrespective of this particular problem, adding support for indexe

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Juergen Gross
On 07/22/2015 04:44 PM, Boris Ostrovsky wrote: On 07/22/2015 10:09 AM, Juergen Gross wrote: On 07/22/2015 03:58 PM, Boris Ostrovsky wrote: On 07/22/2015 09:50 AM, Juergen Gross wrote: On 07/22/2015 03:36 PM, Dario Faggioli wrote: On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote: On 0

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Boris Ostrovsky
On 07/22/2015 11:49 AM, Dario Faggioli wrote: On Wed, 2015-07-22 at 11:32 -0400, Boris Ostrovsky wrote: On 07/22/2015 10:50 AM, Dario Faggioli wrote: Yep. Exacty. As Boris says, this is a generic scheduling issue, although it's tru that it's only (as far as I can tell) with vNUMA that it bite u

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Dario Faggioli
On Wed, 2015-07-22 at 11:32 -0400, Boris Ostrovsky wrote: > On 07/22/2015 10:50 AM, Dario Faggioli wrote: > > Yep. Exacty. As Boris says, this is a generic scheduling issue, although > > it's tru that it's only (as far as I can tell) with vNUMA that it bite > > us so hard... > > I am not sure tha

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Boris Ostrovsky
On 07/22/2015 10:50 AM, Dario Faggioli wrote: On Wed, 2015-07-22 at 16:09 +0200, Juergen Gross wrote: On 07/22/2015 03:58 PM, Boris Ostrovsky wrote: What if I configure a guest to follow HW topology? I.e. I pin VCPUs to appropriate cores/threads? With elfnote I am stuck with disabled topology.

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Dario Faggioli
On Wed, 2015-07-22 at 16:09 +0200, Juergen Gross wrote: > On 07/22/2015 03:58 PM, Boris Ostrovsky wrote: > > What if I configure a guest to follow HW topology? I.e. I pin VCPUs to > > appropriate cores/threads? With elfnote I am stuck with disabled topology. > > Add an option to do exactly that:

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Boris Ostrovsky
On 07/22/2015 10:09 AM, Juergen Gross wrote: On 07/22/2015 03:58 PM, Boris Ostrovsky wrote: On 07/22/2015 09:50 AM, Juergen Gross wrote: On 07/22/2015 03:36 PM, Dario Faggioli wrote: On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote: On 07/20/2015 10:43 AM, Boris Ostrovsky wrote: On 0

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Juergen Gross
On 07/22/2015 03:58 PM, Boris Ostrovsky wrote: On 07/22/2015 09:50 AM, Juergen Gross wrote: On 07/22/2015 03:36 PM, Dario Faggioli wrote: On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote: On 07/20/2015 10:43 AM, Boris Ostrovsky wrote: On 07/20/2015 10:09 AM, Dario Faggioli wrote: I

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Boris Ostrovsky
On 07/22/2015 09:50 AM, Juergen Gross wrote: On 07/22/2015 03:36 PM, Dario Faggioli wrote: On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote: On 07/20/2015 10:43 AM, Boris Ostrovsky wrote: On 07/20/2015 10:09 AM, Dario Faggioli wrote: I'll need to see how LLC IDs are calculated, prob

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Juergen Gross
On 07/22/2015 03:36 PM, Dario Faggioli wrote: On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote: On 07/20/2015 10:43 AM, Boris Ostrovsky wrote: On 07/20/2015 10:09 AM, Dario Faggioli wrote: I'll need to see how LLC IDs are calculated, probably also from some CPUID bits. No, can't d

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-22 Thread Dario Faggioli
On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote: > On 07/20/2015 10:43 AM, Boris Ostrovsky wrote: > > On 07/20/2015 10:09 AM, Dario Faggioli wrote: > > I'll need to see how LLC IDs are calculated, probably also from some > > CPUID bits. > > > No, can't do this: LLC is calculated from C

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-21 Thread Boris Ostrovsky
On 07/20/2015 10:43 AM, Boris Ostrovsky wrote: On 07/20/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-17 at 14:17 -0400, Boris Ostrovsky wrote: On 07/17/2015 03:27 AM, Dario Faggioli wrote: In the meanwhile, what should we do? Document this? How? "don't use vNUMA with PV guest in SMT en

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-20 Thread Boris Ostrovsky
On 07/20/2015 10:09 AM, Dario Faggioli wrote: On Fri, 2015-07-17 at 14:17 -0400, Boris Ostrovsky wrote: On 07/17/2015 03:27 AM, Dario Faggioli wrote: In the meanwhile, what should we do? Document this? How? "don't use vNUMA with PV guest in SMT enabled systems" seems a bit harsh... Is there a w

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-20 Thread Dario Faggioli
On Fri, 2015-07-17 at 14:17 -0400, Boris Ostrovsky wrote: > On 07/17/2015 03:27 AM, Dario Faggioli wrote: > > In the meanwhile, what should we do? Document this? How? "don't use > > vNUMA with PV guest in SMT enabled systems" seems a bit harsh... Is > > there a workaround we can put in place/sugge

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-17 Thread Boris Ostrovsky
On 07/17/2015 03:27 AM, Dario Faggioli wrote: On Fri, 2015-07-17 at 07:09 +0100, Jan Beulich wrote: On 16.07.15 at 18:59, wrote: And in general (both for PV and HVM) --- is there any reason to expose CPU topology at all? I can see it being useful if VCPUs are pinned but if they are not then it

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-17 Thread Andrew Cooper
On 16/07/15 17:59, Boris Ostrovsky wrote: > On 07/16/2015 12:39 PM, Andrew Cooper wrote: >> On 16/07/15 17:29, Jan Beulich wrote: >> On 16.07.15 at 17:50, wrote: Can't we set leaf 1's EBX[32:16] to 1? > > (I obviously fat-fingered this --- I meant EBX[23:16]) > >>> I don't think we should

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-17 Thread Wei Liu
On Fri, Jul 17, 2015 at 09:27:55AM +0200, Dario Faggioli wrote: > On Fri, 2015-07-17 at 07:09 +0100, Jan Beulich wrote: > > >>> On 16.07.15 at 18:59, wrote: > > > And in general (both for PV and HVM) --- is there any reason to expose > > > CPU topology at all? I can see it being useful if VCPUs a

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-17 Thread Jan Beulich
>>> On 17.07.15 at 09:27, wrote: > In the meanwhile, what should we do? Document this? How? "don't use > vNUMA with PV guest in SMT enabled systems" seems a bit harsh... Is > there a workaround we can put in place/suggest? Use SLE / openSUSE kernels ;-) ? Jan __

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-17 Thread Dario Faggioli
On Fri, 2015-07-17 at 07:09 +0100, Jan Beulich wrote: > >>> On 16.07.15 at 18:59, wrote: > > And in general (both for PV and HVM) --- is there any reason to expose > > CPU topology at all? I can see it being useful if VCPUs are pinned but > > if they are not then it can make performance worse. >

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Jan Beulich
>>> On 16.07.15 at 18:59, wrote: > And in general (both for PV and HVM) --- is there any reason to expose > CPU topology at all? I can see it being useful if VCPUs are pinned but > if they are not then it can make performance worse. Indeed - that's what our kernels have been doing for years, an

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Boris Ostrovsky
On 07/16/2015 12:39 PM, Andrew Cooper wrote: On 16/07/15 17:29, Jan Beulich wrote: On 16.07.15 at 17:50, wrote: Can't we set leaf 1's EBX[32:16] to 1? (I obviously fat-fingered this --- I meant EBX[23:16]) I don't think we should partially overwrite the relevant parts of CPUID output - eit

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Andrew Cooper
On 16/07/15 17:29, Jan Beulich wrote: On 16.07.15 at 17:50, wrote: >> Can't we set leaf 1's EBX[32:16] to 1? > I don't think we should partially overwrite the relevant parts of > CPUID output - either all or nothing (so that things at least > remain consistent). Also, there are no masking/ov

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Jan Beulich
>>> On 16.07.15 at 17:50, wrote: > Can't we set leaf 1's EBX[32:16] to 1? I don't think we should partially overwrite the relevant parts of CPUID output - either all or nothing (so that things at least remain consistent). Jan ___ Xen-devel mailing li

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Boris Ostrovsky
On 07/16/2015 11:45 AM, Andrew Cooper wrote: On 16/07/15 16:25, Wei Liu wrote: On Thu, Jul 16, 2015 at 11:56:50AM +0100, Andrew Cooper wrote: On 16/07/15 11:47, Jan Beulich wrote: On 16.07.15 at 12:32, wrote: root@test:~# numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 node 0 si

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Andrew Cooper
On 16/07/15 16:25, Wei Liu wrote: > On Thu, Jul 16, 2015 at 11:56:50AM +0100, Andrew Cooper wrote: >> On 16/07/15 11:47, Jan Beulich wrote: >> On 16.07.15 at 12:32, wrote: root@test:~# numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 node 0 size: 475 MB nod

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Wei Liu
On Thu, Jul 16, 2015 at 12:32:42PM +0200, Dario Faggioli wrote: > Hey, > > This started on IRC, but it's actually appropriate to have the > conversation here. > > I just discovered an issue with vNUMA, when PV guests are used. In fact, > creating a 4 vCPUs PV guest, and making up things so that a

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Wei Liu
On Thu, Jul 16, 2015 at 11:56:50AM +0100, Andrew Cooper wrote: > On 16/07/15 11:47, Jan Beulich wrote: > On 16.07.15 at 12:32, wrote: > >> root@test:~# numactl --hardware > >> available: 2 nodes (0-1) > >> node 0 cpus: 0 1 > >> node 0 size: 475 MB > >> node 0 free: 382 MB > >> node 1 cpus: 2

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Andrew Cooper
On 16/07/15 11:47, Jan Beulich wrote: On 16.07.15 at 12:32, wrote: >> root@test:~# numactl --hardware >> available: 2 nodes (0-1) >> node 0 cpus: 0 1 >> node 0 size: 475 MB >> node 0 free: 382 MB >> node 1 cpus: 2 3 >> node 1 size: 495 MB >> node 1 free: 475 MB >> node distances: >> node 0

Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Jan Beulich
>>> On 16.07.15 at 12:32, wrote: > root@test:~# numactl --hardware > available: 2 nodes (0-1) > node 0 cpus: 0 1 > node 0 size: 475 MB > node 0 free: 382 MB > node 1 cpus: 2 3 > node 1 size: 495 MB > node 1 free: 475 MB > node distances: > node 0 1 > 0: 10 10 > 1: 20 10 > > root@tes

[Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

2015-07-16 Thread Dario Faggioli
Hey, This started on IRC, but it's actually appropriate to have the conversation here. I just discovered an issue with vNUMA, when PV guests are used. In fact, creating a 4 vCPUs PV guest, and making up things so that all the 4 vCPUs should be busy, I see this: root@Zhaman:~# xl vcpu-list test N