Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-23 Thread Edgar E. Iglesias
On Wed, Feb 22, 2017 at 09:22:25AM -0800, Stefano Stabellini wrote: > On Wed, 22 Feb 2017, Edgar E. Iglesias wrote: > > On Tue, Feb 21, 2017 at 07:20:29PM +, Julien Grall wrote: > > > > > > > > > On 21/02/2017 18:30, Stefano Stabellini wrote: > > > >On Tue, 21 Feb 2017, Julien Grall wrote: >

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-22 Thread Stefano Stabellini
On Wed, 22 Feb 2017, Edgar E. Iglesias wrote: > On Tue, Feb 21, 2017 at 07:20:29PM +, Julien Grall wrote: > > > > > > On 21/02/2017 18:30, Stefano Stabellini wrote: > > >On Tue, 21 Feb 2017, Julien Grall wrote: > > >>Hi Stefano, > > >> > > >>On 21/02/17 17:49, Stefano Stabellini wrote: > > >>

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-22 Thread Dario Faggioli
On Tue, 2017-02-21 at 18:17 +, George Dunlap wrote: > On 21/02/17 17:49, Stefano Stabellini wrote: > > I don't know the inner working of the scheduler, but does it always > > send > > an interrupt to other pcpu to schedule something? > > Letting a guest call WFI is as safe as letting a guest >

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Edgar E. Iglesias
On Tue, Feb 21, 2017 at 07:20:29PM +, Julien Grall wrote: > > > On 21/02/2017 18:30, Stefano Stabellini wrote: > >On Tue, 21 Feb 2017, Julien Grall wrote: > >>Hi Stefano, > >> > >>On 21/02/17 17:49, Stefano Stabellini wrote: > >>>On Tue, 21 Feb 2017, Dario Faggioli wrote: > On Tue, 2017-0

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
On 21/02/2017 18:30, Stefano Stabellini wrote: On Tue, 21 Feb 2017, Julien Grall wrote: Hi Stefano, On 21/02/17 17:49, Stefano Stabellini wrote: On Tue, 21 Feb 2017, Dario Faggioli wrote: On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote: Oh, actually, if --which I only now realize may

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Stefano Stabellini
On Tue, 21 Feb 2017, Julien Grall wrote: > Hi Stefano, > > On 21/02/17 17:49, Stefano Stabellini wrote: > > On Tue, 21 Feb 2017, Dario Faggioli wrote: > > > On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote: > > > Oh, actually, if --which I only now realize may be what you are > > > referring

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi Stefano, On 21/02/17 18:03, Stefano Stabellini wrote: On Tue, 21 Feb 2017, Julien Grall wrote: Hi George, On 21/02/17 13:46, George Dunlap wrote: I think our options look like: Thank you for the summary of the options! A. Don't trap guest WFI at all -- allow it to 'halt' in moderate-

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread George Dunlap
On 21/02/17 17:49, Stefano Stabellini wrote: > On Tue, 21 Feb 2017, Dario Faggioli wrote: >> On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote: >>> >>> A. Don't trap guest WFI at all -- allow it to 'halt' in >>> moderate-power-but-ready-for-interrupt mode. >>> >>> [..] >>> >>> A is safe becau

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Stefano Stabellini
On Tue, 21 Feb 2017, Julien Grall wrote: > Hi George, > > On 21/02/17 13:46, George Dunlap wrote: > > I think our options look like: > > Thank you for the summary of the options! > > > > > A. Don't trap guest WFI at all -- allow it to 'halt' in > > moderate-power-but-ready-for-interrupt mode.

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi Stefano, On 21/02/17 17:49, Stefano Stabellini wrote: On Tue, 21 Feb 2017, Dario Faggioli wrote: On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote: Oh, actually, if --which I only now realize may be what you are referring to, since you're talking about "guest burning its credits"-- you

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Stefano Stabellini
On Tue, 21 Feb 2017, Dario Faggioli wrote: > On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote: > > > > A.  Don't trap guest WFI at all -- allow it to 'halt' in > > moderate-power-but-ready-for-interrupt mode. > > > > [..] > > > > A is safe because the scheduler should already have set a time

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Stefano Stabellini
On Tue, 21 Feb 2017, Dario Faggioli wrote: > On Tue, 2017-02-21 at 12:30 +, Julien Grall wrote: > > On 21/02/2017 09:09, Dario Faggioli wrote: > > > Well, TBH, we still are not entirely sure who the culprit is for > > > high > > > latency. There are spikes in Credit2, and I'm investigating that

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread George Dunlap
On 21/02/17 15:14, Julien Grall wrote: > Hi George, > > On 21/02/17 13:46, George Dunlap wrote: >> I think our options look like: > > Thank you for the summary of the options! > >> >> A. Don't trap guest WFI at all -- allow it to 'halt' in >> moderate-power-but-ready-for-interrupt mode. >> >> B

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Dario Faggioli
On Tue, 2017-02-21 at 12:30 +, Julien Grall wrote: > On 21/02/2017 09:09, Dario Faggioli wrote: > > Well, TBH, we still are not entirely sure who the culprit is for > > high > > latency. There are spikes in Credit2, and I'm investigating that. > > But > > apart from them? I think we need other

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi George, On 21/02/17 13:46, George Dunlap wrote: I think our options look like: Thank you for the summary of the options! A. Don't trap guest WFI at all -- allow it to 'halt' in moderate-power-but-ready-for-interrupt mode. B. Trap guest WFI and block normally. C. Trap guest WFI and pol

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Dario Faggioli
On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote: > > A.  Don't trap guest WFI at all -- allow it to 'halt' in > moderate-power-but-ready-for-interrupt mode. > > [..] > > A is safe because the scheduler should already have set a timer to > break > out of it if necessary.  The only potential

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread George Dunlap
On 21/02/17 12:30, Julien Grall wrote: > Hi Dario, > > On 21/02/2017 09:09, Dario Faggioli wrote: >> On Tue, 2017-02-21 at 07:59 +, Julien Grall wrote: >>> On 20/02/2017 22:53, Dario Faggioli wrote: For instance, as you say, executing a WFI from a guest directly on hardware, only mak

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi Dario, On 21/02/17 09:24, Dario Faggioli wrote: On Tue, 2017-02-21 at 08:10 +, Julien Grall wrote: On 21/02/2017 00:38, Stefano Stabellini wrote: On Mon, 20 Feb 2017, Dario Faggioli wrote: Mmm... ok, yes, in that case, it may make sense and work, from a, let's say, purely functional pe

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi Dario, On 21/02/2017 09:09, Dario Faggioli wrote: On Tue, 2017-02-21 at 07:59 +, Julien Grall wrote: On 20/02/2017 22:53, Dario Faggioli wrote: For instance, as you say, executing a WFI from a guest directly on hardware, only makes sense if we have 1:1 static pinning. Which means it can

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Dario Faggioli
On Tue, 2017-02-21 at 08:10 +, Julien Grall wrote: > On 21/02/2017 00:38, Stefano Stabellini wrote: > > On Mon, 20 Feb 2017, Dario Faggioli wrote: > > > Mmm... ok, yes, in that case, it may make sense and work, from a, > > > let's > > > say, purely functional perspective. But still I struggle t

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Dario Faggioli
On Tue, 2017-02-21 at 07:59 +, Julien Grall wrote: > On 20/02/2017 22:53, Dario Faggioli wrote: > > For instance, as you say, executing a WFI from a guest directly on > > hardware, only makes sense if we have 1:1 static pinning. Which > > means > > it can't just be done by default, or with a bo

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi Stefano, On 21/02/2017 00:38, Stefano Stabellini wrote: On Mon, 20 Feb 2017, Dario Faggioli wrote: On Mon, 2017-02-20 at 19:38 +, Julien Grall wrote: On 20/02/17 19:20, Dario Faggioli wrote: E.g., if vCPU x of domain A wants to go idle with a WFI/WFE, but the host is overbooked and cur

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-21 Thread Julien Grall
Hi Dario, On 20/02/2017 22:53, Dario Faggioli wrote: On Mon, 2017-02-20 at 19:38 +, Julien Grall wrote: On 20/02/17 19:20, Dario Faggioli wrote: E.g., if vCPU x of domain A wants to go idle with a WFI/WFE, but the host is overbooked and currently really busy, Xen wants to run some other vC

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Stefano Stabellini
On Mon, 20 Feb 2017, Dario Faggioli wrote: > On Mon, 2017-02-20 at 19:38 +, Julien Grall wrote: > > On 20/02/17 19:20, Dario Faggioli wrote: > > > E.g., if vCPU x of domain A wants to go idle with a WFI/WFE, but > > > the > > > host is overbooked and currently really busy, Xen wants to run some

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Dario Faggioli
On Mon, 2017-02-20 at 19:38 +, Julien Grall wrote: > On 20/02/17 19:20, Dario Faggioli wrote: > > E.g., if vCPU x of domain A wants to go idle with a WFI/WFE, but > > the > > host is overbooked and currently really busy, Xen wants to run some > > other vCPU (of either the same of another domain

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Julien Grall
Hi Dario, On 20/02/17 19:20, Dario Faggioli wrote: On Mon, 2017-02-20 at 18:53 +, Julien Grall wrote: On 20/02/17 18:47, Stefano Stabellini wrote: This is a good question, I have already answered: I think it would break the scheduler. Dario confirmed it in his reply (1487382463.6732.146.ca

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Dario Faggioli
On Mon, 2017-02-20 at 18:53 +, Julien Grall wrote: > On 20/02/17 18:47, Stefano Stabellini wrote: > > This is a good question, I have already answered: I think it would > > break > > the scheduler. Dario confirmed it in his reply > > (1487382463.6732.146.ca...@citrix.com). > > I don't think it

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Julien Grall
Hi Stefano, On 20/02/17 18:47, Stefano Stabellini wrote: On Sun, 19 Feb 2017, Julien Grall wrote: I don't think this is acceptable even to get a better interrupt latency. Some workload will care about interrupt latency and power. I think a better approach would be to check whether the schedule

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Stefano Stabellini
On Mon, 20 Feb 2017, George Dunlap wrote: > On 20/02/17 18:43, Stefano Stabellini wrote: > > On Mon, 20 Feb 2017, Dario Faggioli wrote: > >> On Sun, 2017-02-19 at 21:34 +, Julien Grall wrote: > >>> Hi Stefano, > >>> > >>> I have CCed another ARM person who has more knowledge than me on > >>> s

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Stefano Stabellini
On Sun, 19 Feb 2017, Julien Grall wrote: > Hi Stefano, > > I have CCed another ARM person who has more knowledge than me on > scheduling/power. > > On 02/17/2017 10:50 PM, Stefano Stabellini wrote: > > CC'ing xen-devel, I forgot on the original patch > > > > On Fri, 17 Feb 2017, Julien Grall wro

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread George Dunlap
On 20/02/17 18:43, Stefano Stabellini wrote: > On Mon, 20 Feb 2017, Dario Faggioli wrote: >> On Sun, 2017-02-19 at 21:34 +, Julien Grall wrote: >>> Hi Stefano, >>> >>> I have CCed another ARM person who has more knowledge than me on >>> scheduling/power. >>> >> Ah, when I saw this, I thought y

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Stefano Stabellini
On Mon, 20 Feb 2017, Dario Faggioli wrote: > On Sun, 2017-02-19 at 21:34 +, Julien Grall wrote: > > Hi Stefano, > > > > I have CCed another ARM person who has more knowledge than me on  > > scheduling/power. > > > Ah, when I saw this, I thought you were Cc-ing my friend Juri, which > also work

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Dario Faggioli
On Sun, 2017-02-19 at 21:34 +, Julien Grall wrote: > Hi Stefano, > > I have CCed another ARM person who has more knowledge than me on  > scheduling/power. > Ah, when I saw this, I thought you were Cc-ing my friend Juri, which also works there, and is doing that stuff. :-) > > In both cases th

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread Dario Faggioli
On Sun, 2017-02-19 at 21:27 +, Julien Grall wrote: > Hi Dario, > Hi, > On 02/18/2017 01:47 AM, Dario Faggioli wrote: > >  - vcpu A yields, and there are no runnable but not running vcpus > >    around. In this case, A gets to run again. Full stop. > > Which turn to be the busy looping I was

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-20 Thread George Dunlap
On 19/02/17 21:27, Julien Grall wrote: > Hi Dario, > > On 02/18/2017 01:47 AM, Dario Faggioli wrote: >> On Fri, 2017-02-17 at 14:50 -0800, Stefano Stabellini wrote: >>> On Fri, 17 Feb 2017, Julien Grall wrote: Please explain in which context this will be beneficial. My gut feeling is >>>

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-19 Thread Julien Grall
Hi Stefano, I have CCed another ARM person who has more knowledge than me on scheduling/power. On 02/17/2017 10:50 PM, Stefano Stabellini wrote: CC'ing xen-devel, I forgot on the original patch On Fri, 17 Feb 2017, Julien Grall wrote: Hi Stefano, On 02/16/2017 11:04 PM, Stefano Stabellini

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-19 Thread Julien Grall
Hi Dario, On 02/18/2017 01:47 AM, Dario Faggioli wrote: On Fri, 2017-02-17 at 14:50 -0800, Stefano Stabellini wrote: On Fri, 17 Feb 2017, Julien Grall wrote: Please explain in which context this will be beneficial. My gut feeling is only will make performance worst if a multiple vCPU of the sa

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-17 Thread Dario Faggioli
On Fri, 2017-02-17 at 14:50 -0800, Stefano Stabellini wrote: > On Fri, 17 Feb 2017, Julien Grall wrote: > > Please explain in which context this will be beneficial. My gut > > feeling is > > only will make performance worst if a multiple vCPU of the same > > guest is > > running on vCPU > > I am n

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

2017-02-17 Thread Stefano Stabellini
CC'ing xen-devel, I forgot on the original patch On Fri, 17 Feb 2017, Julien Grall wrote: > Hi Stefano, > > On 02/16/2017 11:04 PM, Stefano Stabellini wrote: > > Introduce new Xen command line parameter called "vwfi", which stands for > > virtual wfi. The default is "sleep": on guest wfi, Xen cal