Re: [Xen-devel] [PATCH] x86/xen: Add "xen_timer_slop" command line option
Hi all, On Sat, 23 Mar 2019 03:58:16 +0100 Dario Faggioli wrote: > On Fri, 2019-03-22 at 18:10 -0400, Boris Ostrovsky wrote: > > On 3/22/19 2:29 PM, thibo...@gmail.com wrote: > > > From: Ryan Thibodeaux > > > > > > The original timer slop value has not changed since the > > > introduction > > > of the Xen-aware Linux kernel code. This commit provides users an > > > opportunity to tune timer performance given the refinements to > > > hardware and the Xen event channel processing. It also mirrors > > > a feature in the Xen hypervisor - the "timer_slop" Xen command > > > line option. > > > > Is there any data that shows effects of using this new parameter? > > > Yes, I've done some research and experiments on this. I did it > together with a friend, which I'm Cc-ing, as I'm not sure we're > ready/capable to share the results, yet (Luca?). I think we can easily share the experimental data (cyclictest output and plots). Moreover, we can share the scripts and tools for running the experiments (so, everyone can easily reproduce the numbers by simply typing "make" and waiting for some time :) I'll try to package the results and the scripts/tools this evening, and I'll send them. Luca > > What I think I can anticipate is that having such a high value for > timer slop in the kernel, for the Xen clockevent device is (together > with the also quite high default value of timer_slop in Xen itself) > responsible for really high vcpu wakeup latencies. > > Lowering those two values, reduces such latencies dramatically. > > Regards, > Dario ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] x86/xen: Add "xen_timer_slop" command line option
Hi all, On Sat, 23 Mar 2019 11:41:51 +0100 luca abeni wrote: [...] > > > Is there any data that shows effects of using this new parameter? > > > > > Yes, I've done some research and experiments on this. I did it > > together with a friend, which I'm Cc-ing, as I'm not sure we're > > ready/capable to share the results, yet (Luca?). > > I think we can easily share the experimental data (cyclictest output > and plots). > > Moreover, we can share the scripts and tools for running the > experiments (so, everyone can easily reproduce the numbers by simply > typing "make" and waiting for some time :) > > > I'll try to package the results and the scripts/tools this evening, > and I'll send them. Sorry for the delay. I put some quick results here: http://retis.santannapisa.it/luca/XenTimers/ (there also is a link to the scripts to be used for reproducing the results). The latencies have been measured by running cyclictest in the guest (see the scripts for details). The picture shows the latencies measured with an unpatched guest kernel and with a guest kernel having TIMER_SLOP set to 1000 (arbitrary small value :). All the experiments have been performed booting the hypervisor with a small timer_slop (the hypervisor's one) value. So, they show that decreasing the hypervisor's timer_slop is not enough to measure low latencies with cyclictest. Luca ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] x86/xen: Add "xen_timer_slop" command line option
Hi, On Mon, 25 Mar 2019 09:43:20 -0400 Boris Ostrovsky wrote: [...] > > http://retis.santannapisa.it/luca/XenTimers/ > > (there also is a link to the scripts to be used for reproducing the > > results). The latencies have been measured by running cyclictest in > > the guest (see the scripts for details). > > > > The picture shows the latencies measured with an unpatched guest > > kernel and with a guest kernel having TIMER_SLOP set to 1000 > > (arbitrary small value :). > > All the experiments have been performed booting the hypervisor with > > a small timer_slop (the hypervisor's one) value. So, they show that > > decreasing the hypervisor's timer_slop is not enough to measure low > > latencies with cyclictest. > > > > I have a couple of questions: > * Does it make sense to make this a tunable for other clockevent > devices as well? > * This patch adjusts min value. Could max value (ever) need a similar > adjustment? Sorry, I do not know much about clockevent devices, so I have no answers to these questions... What I can say is that when I repeated the cyclictest experiments on VMs using a different clockevent device (lapic) I did not measure large latencies. So, I guess the "lapic" clockevent device already defaults to a smaller min value (not sure about other clockevent devices, I do not know how to test them). Luca ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH] x86/xen: Add "xen_timer_slop" command line option
Hi all, On Tue, 26 Mar 2019 10:13:32 +0100 Dario Faggioli wrote: > On Mon, 2019-03-25 at 09:43 -0400, Boris Ostrovsky wrote: > > On 3/25/19 8:05 AM, luca abeni wrote: > > > > > > The picture shows the latencies measured with an unpatched guest > > > kernel > > > and with a guest kernel having TIMER_SLOP set to 1000 (arbitrary > > > small > > > value :). > > > All the experiments have been performed booting the hypervisor > > > with a > > > small timer_slop (the hypervisor's one) value. So, they show that > > > decreasing the hypervisor's timer_slop is not enough to measure > > > low latencies with cyclictest. > > > > I have a couple of questions: > > * Does it make sense to make this a tunable for other clockevent > > devices > > as well? > > > So, AFAIUI, the thing is as follows. In clockevents_program_event(), > we keep the delta between now and the next timer event within > dev->max_delta_ns and dev->min_delta_ns: > > delta = min(delta, (int64_t) dev->max_delta_ns); > delta = max(delta, (int64_t) dev->min_delta_ns); > > For Xen (well, for the Xen clock) we have: > > .max_delta_ns = 0x, > .min_delta_ns = TIMER_SLOP, > > which means a guest can't ask for a timer to fire earlier than 100us [...] I know this is not fully related with the current discussion, but in these days I had a look at the code again, and... The comment for TIMER_SLOP in arch/x86/xen/time.c says: /* Xen may fire a timer up to this many ns early */ Isn't the comment wrong? shouldn't it be "...many ns late" instead of "early"? Thanks, Luca ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
