Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-20 Thread Dario Faggioli
On Wed, 2017-07-19 at 12:21 +0100, Julien Grall wrote: > On 17/07/17 12:28, George Dunlap wrote: > > Just checking -- you do mean its own core, as opposed to its own > > socket? > >  (Or NUMA node?) > > I don't know much about the scheduler, so I might say something > stupid  > here :). Below the

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-20 Thread Dario Faggioli
On Mon, 2017-07-17 at 12:28 +0100, George Dunlap wrote: > Most schedulers have one runqueue per logical cpu.  Credit2 has the > option of having one runqueue per logical cpu, one per core (i.e., > hyperthreads share a runqueue), one runqueue per socket (i.e., all > cores > on the same socket share

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-20 Thread Dario Faggioli
On Mon, 2017-07-17 at 10:25 +0100, George Dunlap wrote: > On 07/12/2017 07:14 AM, Dario Faggioli wrote: > > > > That being said, I personally have never liked rate-limiting, it > > always > > looked to me like the wrong solution. > > In fact, I *think* the only reason it may have been introduced

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-19 Thread Julien Grall
Hi George, On 17/07/17 12:28, George Dunlap wrote: On 07/17/2017 11:04 AM, Julien Grall wrote: Hi, On 17/07/17 10:25, George Dunlap wrote: On 07/12/2017 07:14 AM, Dario Faggioli wrote: On Fri, 2017-07-07 at 14:12 -0700, Stefano Stabellini wrote: On Fri, 7 Jul 2017, Volodymyr Babchuk wrote:

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-17 Thread George Dunlap
On 07/17/2017 11:04 AM, Julien Grall wrote: > Hi, > > On 17/07/17 10:25, George Dunlap wrote: >> On 07/12/2017 07:14 AM, Dario Faggioli wrote: >>> On Fri, 2017-07-07 at 14:12 -0700, Stefano Stabellini wrote: On Fri, 7 Jul 2017, Volodymyr Babchuk wrote: >>> >> Since you are using Credi

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-17 Thread Julien Grall
Hi, On 17/07/17 10:25, George Dunlap wrote: On 07/12/2017 07:14 AM, Dario Faggioli wrote: On Fri, 2017-07-07 at 14:12 -0700, Stefano Stabellini wrote: On Fri, 7 Jul 2017, Volodymyr Babchuk wrote: Since you are using Credit, can you try to disable context switch rate limiting? Yep. You are

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-17 Thread George Dunlap
On 07/12/2017 07:14 AM, Dario Faggioli wrote: > On Fri, 2017-07-07 at 14:12 -0700, Stefano Stabellini wrote: >> On Fri, 7 Jul 2017, Volodymyr Babchuk wrote: > Since you are using Credit, can you try to disable context switch rate limiting? >>> >>> Yep. You are right. In the envir

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-11 Thread Dario Faggioli
On Fri, 2017-07-07 at 14:12 -0700, Stefano Stabellini wrote: > On Fri, 7 Jul 2017, Volodymyr Babchuk wrote: > > > > > > > Since you are using Credit, can you try to disable context switch > > > rate > > > limiting? > > > > Yep. You are right. In the environment described above (Case 2) I > > now >

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-08 Thread Dario Faggioli
On Fri, 2017-07-07 at 10:03 -0700, Volodymyr Babchuk wrote: > On 7 July 2017 at 09:41, Dario Faggioli > wrote: > > > > Also, are you sure (e.g., because of how the Linux driver is done) > > that > > this always happen on one vCPU? > > No, I can't guarantee that. Linux driver is single threaded,

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-07 Thread Stefano Stabellini
On Fri, 7 Jul 2017, Volodymyr Babchuk wrote: > >> I run test in DomU: > >> real 113.08 > >> user 0.00 > >> sys 113.04 > >> > > Ok, so there's contention for pCPUs. Dom0's vCPUs are CPU hogs, while, > > if my assumption above is correct, the "SMC vCPU" of the DomU is I/O > > bound, in the sense that

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-07 Thread Volodymyr Babchuk
Hi again, On 7 July 2017 at 09:41, Dario Faggioli wrote: > On Fri, 2017-07-07 at 18:02 +0300, Volodymyr Babchuk wrote: >> Hello Dario, >> > Hi! > >> On 20 June 2017 at 13:11, Dario Faggioli >> wrote: >> > On Mon, 2017-06-19 at 11:36 -0700, Volodymyr Babchuk wrote: >> > > >> > > Thanks. Actually,

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-07 Thread Dario Faggioli
On Fri, 2017-07-07 at 18:02 +0300, Volodymyr Babchuk wrote: > Hello Dario, > Hi! > On 20 June 2017 at 13:11, Dario Faggioli > wrote: > > On Mon, 2017-06-19 at 11:36 -0700, Volodymyr Babchuk wrote: > > > > > > Thanks. Actually, we discussed this topic internally today. Main > > > concern today i

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-07-07 Thread Volodymyr Babchuk
Hello Dario, On 20 June 2017 at 13:11, Dario Faggioli wrote: > On Mon, 2017-06-19 at 11:36 -0700, Volodymyr Babchuk wrote: >> On 19 June 2017 at 10:54, Stefano Stabellini >> wrote: >> > True. However, Volodymyr took the time to demonstrate the >> > performance of >> > EL0 apps vs. stubdoms with

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-21 Thread Julien Grall
On 20/06/17 17:23, Volodymyr Babchuk wrote: Hi Julien, Hi Volodymyr, On 20 June 2017 at 03:45, Julien Grall wrote: On 19 June 2017 at 10:54, Stefano Stabellini wrote: But given the conversation so far, it seems likely that that is mainly due to the fact that context switching on ARM h

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-20 Thread Volodymyr Babchuk
Hi Julien, On 20 June 2017 at 03:45, Julien Grall wrote: >> On 19 June 2017 at 10:54, Stefano Stabellini >> wrote: >> But given the conversation so far, it seems likely that that is mainly due to the fact that context switching on ARM has not been optimized. >>> >>> >>> True. However,

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-20 Thread Julien Grall
On 06/19/2017 07:36 PM, Volodymyr Babchuk wrote: Hi Stefano, Hi, On 19 June 2017 at 10:54, Stefano Stabellini wrote: But given the conversation so far, it seems likely that that is mainly due to the fact that context switching on ARM has not been optimized. True. However, Volodymyr too

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-20 Thread George Dunlap
On 20/06/17 11:00, Dario Faggioli wrote: > On Mon, 2017-06-19 at 11:26 -0700, Volodymyr Babchuk wrote: >> On 19 June 2017 at 02:37, George Dunlap >> wrote: >>> If you want this "EL0 app" thing to be able to provide extra >>> security >>> over just running the code inside of Xen, then the code must

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-20 Thread Dario Faggioli
On Mon, 2017-06-19 at 11:36 -0700, Volodymyr Babchuk wrote: > On 19 June 2017 at 10:54, Stefano Stabellini > wrote: > > True. However, Volodymyr took the time to demonstrate the > > performance of > > EL0 apps vs. stubdoms with a PoC, which is much more than most Xen > > contributors do. Nodoby pr

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-20 Thread Dario Faggioli
On Mon, 2017-06-19 at 11:26 -0700, Volodymyr Babchuk wrote: > On 19 June 2017 at 02:37, George Dunlap > wrote: > > If you want this "EL0 app" thing to be able to provide extra > > security > > over just running the code inside of Xen, then the code must not be > > able > > to DoS the host by spinn

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-19 Thread Volodymyr Babchuk
Hi Stefano, On 19 June 2017 at 10:54, Stefano Stabellini wrote: >> But given the conversation so far, it seems likely that that is mainly >> due to the fact that context switching on ARM has not been optimized. > > True. However, Volodymyr took the time to demonstrate the performance of > EL0 ap

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-19 Thread Volodymyr Babchuk
Hi George, On 19 June 2017 at 02:37, George Dunlap wrote: > There is no way out: if the stubdom needs events, then we'll have to > expose and context switch the vGIC. If it doesn't, then we can skip the > vGIC. However, we would have a similar problem with EL0 apps: I am > assumin

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-19 Thread Stefano Stabellini
On Mon, 19 Jun 2017, George Dunlap wrote: > On 17/06/17 01:14, Volodymyr Babchuk wrote: > > Hello George, > > > > On 31 May 2017 at 20:02, George Dunlap wrote: > There is no way out: if the stubdom needs events, then we'll have to > expose and context switch the vGIC. If it doesn't, the

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-19 Thread George Dunlap
On 17/06/17 01:14, Volodymyr Babchuk wrote: > Hello George, > > On 31 May 2017 at 20:02, George Dunlap wrote: There is no way out: if the stubdom needs events, then we'll have to expose and context switch the vGIC. If it doesn't, then we can skip the vGIC. However, we would have a

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-16 Thread Volodymyr Babchuk
Hello Juilen, >> The polling can be minimized if you block the vCPU when there are >> nothing to do. It would get unblock when you have to schedule him >> because of a request. > Thinking a bit more about this. So far, we rely on the domain to use the > vGIC interrupt controller which require the c

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-16 Thread Volodymyr Babchuk
Hello George, On 31 May 2017 at 20:02, George Dunlap wrote: >>> There is no way out: if the stubdom needs events, then we'll have to >>> expose and context switch the vGIC. If it doesn't, then we can skip the >>> vGIC. However, we would have a similar problem with EL0 apps: I am >>> assuming that

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread Stefano Stabellini
On Thu, 1 Jun 2017, George Dunlap wrote: > > On May 31, 2017, at 6:45 PM, Stefano Stabellini > > wrote: > > > > On Wed, 31 May 2017, George Dunlap wrote: > >> On 30/05/17 18:29, Stefano Stabellini wrote: > >>> On Fri, 26 May 2017, Volodymyr Babchuk wrote: > >>> The other issue with stubdoms

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread George Dunlap
On 01/06/17 13:40, Dario Faggioli wrote: > On Thu, 2017-06-01 at 12:52 +0200, George Dunlap wrote: >>> On May 31, 2017, at 6:45 PM, Stefano Stabellini >> .org> wrote: >>> >>> I don't think we should provide that. If the user wants a stable >>> interface, she can use domains. I suggested that the co

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread Dario Faggioli
On Thu, 2017-06-01 at 12:52 +0200, George Dunlap wrote: > > On May 31, 2017, at 6:45 PM, Stefano Stabellini > .org> wrote: > > > > I don't think we should provide that. If the user wants a stable > > interface, she can use domains. I suggested that the code for the > > EL0 > > app should come out

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread George Dunlap
> On Jun 1, 2017, at 11:52 AM, George Dunlap wrote: > > >> On May 31, 2017, at 6:45 PM, Stefano Stabellini >> wrote: >> >> On Wed, 31 May 2017, George Dunlap wrote: >>> On 30/05/17 18:29, Stefano Stabellini wrote: On Fri, 26 May 2017, Volodymyr Babchuk wrote: The other issue wi

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread George Dunlap
> On May 31, 2017, at 6:45 PM, Stefano Stabellini > wrote: > > On Wed, 31 May 2017, George Dunlap wrote: >> On 30/05/17 18:29, Stefano Stabellini wrote: >>> On Fri, 26 May 2017, Volodymyr Babchuk wrote: >>> The other issue with stubdoms is context switch times. Volodymyr showed >>> that

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread Julien Grall
Hi Stefano, On 31/05/17 18:45, Stefano Stabellini wrote: On Wed, 31 May 2017, George Dunlap wrote: On 30/05/17 18:29, Stefano Stabellini wrote: On Fri, 26 May 2017, Volodymyr Babchuk wrote: The other issue with stubdoms is context switch times. Volodymyr showed that minios has much higher con

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-06-01 Thread Julien Grall
Hi, On 30/05/17 18:33, Julien Grall wrote: On 30/05/17 18:29, Stefano Stabellini wrote: On Fri, 26 May 2017, Volodymyr Babchuk wrote: The other issue with stubdoms is context switch times. Volodymyr showed that minios has much higher context switch times compared to EL0 apps. It is probably

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-31 Thread Stefano Stabellini
On Wed, 31 May 2017, George Dunlap wrote: > On 30/05/17 18:29, Stefano Stabellini wrote: > > On Fri, 26 May 2017, Volodymyr Babchuk wrote: > > The other issue with stubdoms is context switch times. Volodymyr showed > > that minios has much higher context switch times compared to EL0 apps. >

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-31 Thread George Dunlap
On 26/05/17 20:28, Volodymyr Babchuk wrote: >> There is no way out: if the stubdom needs events, then we'll have to >> expose and context switch the vGIC. If it doesn't, then we can skip the >> vGIC. However, we would have a similar problem with EL0 apps: I am >> assuming that EL0 apps don't need t

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-31 Thread Volodymyr Babchuk
Hi Dario, >> > > On other hand, EL0 app (as I see them) does not need such events. >> > > Basically, you just call function `handle_mmio()` right in the >> > > app. >> > > So, apps can live without interrupts and they still be able to >> > > handle >> > > request. >> > >> > That's true. >> >> Well

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-31 Thread Dario Faggioli
On Wed, 2017-05-31 at 10:09 +0100, George Dunlap wrote: > On 30/05/17 18:29, Stefano Stabellini wrote: > > On Fri, 26 May 2017, Volodymyr Babchuk wrote: > > > On other hand, EL0 app (as I see them) does not need such events. > > > Basically, you just call function `handle_mmio()` right in the > > >

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-31 Thread George Dunlap
On 30/05/17 18:29, Stefano Stabellini wrote: > On Fri, 26 May 2017, Volodymyr Babchuk wrote: > The other issue with stubdoms is context switch times. Volodymyr showed > that minios has much higher context switch times compared to EL0 apps. > It is probably due to GIC context switch, tha

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-30 Thread Julien Grall
On 30/05/17 18:29, Stefano Stabellini wrote: On Fri, 26 May 2017, Volodymyr Babchuk wrote: The other issue with stubdoms is context switch times. Volodymyr showed that minios has much higher context switch times compared to EL0 apps. It is probably due to GIC context switch, that is skipped fo

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-30 Thread Stefano Stabellini
On Fri, 26 May 2017, Volodymyr Babchuk wrote: > >> > The other issue with stubdoms is context switch times. Volodymyr showed > >> > that minios has much higher context switch times compared to EL0 apps. > >> > It is probably due to GIC context switch, that is skipped for EL0 apps. > >> > Maybe we c

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-26 Thread Dario Faggioli
On Fri, 2017-05-26 at 13:09 -0700, Volodymyr Babchuk wrote: > Hello Dario, > Hi, > > Feel free to ask anything. :-) > > I'm so unfamiliar, so even don't know what to ask :) But thank you. > Surely I'll have questions. > Sure. As soon as you have one, go ahead with it. > > The null scheduler is

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-26 Thread Volodymyr Babchuk
Hello Dario, >> I'm not very familiar with XEN schedulers. > Feel free to ask anything. :-) I'm so unfamiliar, so even don't know what to ask :) But thank you. Surely I'll have questions. >> Looks like null scheduler >> is good for hard RT, but isn't fine for a generic consumer system. >> > The nu

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-26 Thread Volodymyr Babchuk
On 26 May 2017 at 12:43, Volodymyr Babchuk wrote: > Hi Dario, > Oops, sorry, George. There was two emails in a row: yours one and Dario's one. And I overlooked to whom I'm answering. -- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babc...@gmail.com __

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-26 Thread Volodymyr Babchuk
Hi Dario, >>> Explanation of what EL0 apps are. What should be their interface with >>> Xen? Could the interface be the regular hypercall interface? In that >>> case, what's the benefit compared to stubdoms? >> I imagined this as separate syscall interface (with finer policy >> rules). But this ca

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-26 Thread Volodymyr Babchuk
Hello Stefano, >> > The problem with stubdoms is latency and scheduling. It is not >> > deterministic. We could easily improve the null scheduler to introduce >> > some sort of non-preemptive scheduling of stubdoms on the same pcpus of >> > the guest vcpus. It would still require manually pinning

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-23 Thread George Dunlap
On 19/05/17 20:45, Volodymyr Babchuk wrote: > Hi Stefano, > > On 18 May 2017 at 22:00, Stefano Stabellini wrote: > >> Description of the problem: need for a place to run emulators and >> mediators outside of Xen, with low latency. >> >> Explanation of what EL0 apps are. What should be their inte

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-23 Thread Dario Faggioli
On Fri, 2017-05-19 at 22:45 +0300, Volodymyr Babchuk wrote: > On 18 May 2017 at 22:00, Stefano Stabellini > wrote: > > ACTIONS: > > Improve the null scheduler to enable decent stubdoms scheduling on > > latency sensitive systems. > > I'm not very familiar with XEN schedulers. > Feel free to ask

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-22 Thread Stefano Stabellini
On Fri, 19 May 2017, Volodymyr Babchuk wrote: > On 18 May 2017 at 22:00, Stefano Stabellini wrote: > > > Description of the problem: need for a place to run emulators and > > mediators outside of Xen, with low latency. > > > > Explanation of what EL0 apps are. What should be their interface with

Re: [Xen-devel] Notes on stubdoms and latency on ARM

2017-05-19 Thread Volodymyr Babchuk
Hi Stefano, On 18 May 2017 at 22:00, Stefano Stabellini wrote: > Description of the problem: need for a place to run emulators and > mediators outside of Xen, with low latency. > > Explanation of what EL0 apps are. What should be their interface with > Xen? Could the interface be the regular hyp

[Xen-devel] Notes on stubdoms and latency on ARM

2017-05-18 Thread Stefano Stabellini
Hi all, Julien, Dario, George and I had a quick meeting to discuss stubdom scheduling. These are my notes. Description of the problem: need for a place to run emulators and mediators outside of Xen, with low latency. Explanation of what EL0 apps are. What should be their interface with Xen? Cou