Hsuan Hsu has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/30919 )
Change subject: cpu-kvm,arch-arm: Improve KvmCPU tick event scheduling
......................................................................
cpu-kvm,arch-arm: Improve KvmCPU tick event scheduling
The memory-mapped timer emulated by gem5 is driven by the underlying
gem5 tick, which means that we must align the tick with the host time
to make the timer interrupt fire at a nearly native rate.
In each KVM execution round, the number of ticks incremented is
directly calculated from the number of instructions executed. However,
when a guest CPU switches to idle state, KVM seems to stay in kernel-
space until the POSIX timer set up in user-space raises an expiration
signal, instead of trapping to user-space immediately; and somehow the
instruction count is just too low to match the elapsed host time. This
makes the gem5 tick increment very slowly when the guest is idle and
drastically slow down workloads being sensitive to the guest time which
is driven by timer interrupt.
Before switching to KVM to execute the guest code, gem5 programs the
POSIX timer to expire according to the remaining ticks before the next
event in the event queue. Based on this, we can come up with the
following solution: If KVM returns to user-space due to POSIX timer
expiration, it must be time to process the next gem5 event, so we just
fast-forward the tick (by scheduling the next CPU tick event) to that
event directly without calculating from the instruction count.
There is one more related issue needed to be solved. The KVM exit
reason, KVM_EXIT_INTR, was treated as the case where the KVM execution
was disturbed by POSIX timer expiration. However, there exists a case
where the exit reason is KVM_EXIT_INTR but the POSIX timer has not
expired. Its cause is still unknown, but it can be observed via the
"old_value" argument returned by timer_settime() when disarming the
POSIX timer. In addition, it seems to happen often when a guest CPU is
not in idle state. When this happens, the above tick event scheduling
incorrectly treats KVM_EXIT_INTR as POSIX timer expiration and fast-
forwards the tick to process the next event too early. This makes the
guest feel external events come too fast, and will sometimes cause
trouble. One example is the VSYNC interrupt from HDLCD. The guest seems
to get stuck in VSYNC handling if the KVM CPU is not given enough time
between each VSYNC interrupt to complete a service. (Honestly I did not
dig in to see how the guest handled the VSYNC interrupt and how the
above situation became trouble. I just observed from the debug trace of
GIC & HDLCD & timer, and made this conclusion.) This change also uses
a workaround to detect POSIX timer expiration correctly to make the
guest work with HDLCD.
JIRA: https://gem5.atlassian.net/browse/GEM5-663
Change-Id: I6159238a36fc18c0c881d177a742d8a7745a23ca
Signed-off-by: Hsuan Hsu <[email protected]>
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/30919
Reviewed-by: Andreas Sandberg <[email protected]>
Maintainer: Andreas Sandberg <[email protected]>
Tested-by: kokoro <[email protected]>
---
M src/cpu/kvm/base.cc
M src/cpu/kvm/timer.cc
M src/cpu/kvm/timer.hh
3 files changed, 24 insertions(+), 5 deletions(-)
Approvals:
Andreas Sandberg: Looks good to me, approved; Looks good to me, approved
kokoro: Regressions pass
diff --git a/src/cpu/kvm/base.cc b/src/cpu/kvm/base.cc
index 18aead8..0afab1e 100644
--- a/src/cpu/kvm/base.cc
+++ b/src/cpu/kvm/base.cc
@@ -691,8 +691,13 @@
}
// Schedule a new tick if we are still running
- if (_status != Idle && _status != RunningMMIOPending)
- schedule(tickEvent, clockEdge(ticksToCycles(delay)));
+ if (_status != Idle && _status != RunningMMIOPending) {
+ if (_kvmRun->exit_reason == KVM_EXIT_INTR && runTimer->expired())
+ schedule(tickEvent, clockEdge(ticksToCycles(
+ curEventQueue()->nextTick() - curTick() + 1)));
+ else
+ schedule(tickEvent, clockEdge(ticksToCycles(delay)));
+ }
}
Tick
diff --git a/src/cpu/kvm/timer.cc b/src/cpu/kvm/timer.cc
index e7b185f..538fd56 100644
--- a/src/cpu/kvm/timer.cc
+++ b/src/cpu/kvm/timer.cc
@@ -122,10 +122,19 @@
struct itimerspec ts;
memset(&ts, 0, sizeof(ts));
- DPRINTF(KvmTimer, "Disarming POSIX timer\n");
-
- if (timer_settime(timer, 0, &ts, NULL) == -1)
+ if (timer_settime(timer, 0, &ts, &prevTimerSpec) == -1)
panic("PosixKvmTimer: Failed to disarm timer\n");
+
+ DPRINTF(KvmTimer, "Disarmed POSIX timer: %is%ins left\n",
+ prevTimerSpec.it_value.tv_sec,
+ prevTimerSpec.it_value.tv_nsec);
+}
+
+bool
+PosixKvmTimer::expired()
+{
+ return (prevTimerSpec.it_value.tv_nsec == 0 &&
+ prevTimerSpec.it_value.tv_sec == 0);
}
Tick
diff --git a/src/cpu/kvm/timer.hh b/src/cpu/kvm/timer.hh
index 32222e5..376ba7c 100644
--- a/src/cpu/kvm/timer.hh
+++ b/src/cpu/kvm/timer.hh
@@ -93,6 +93,9 @@
* signals upon timeout.
*/
virtual void disarm() = 0;
+ virtual bool expired() {
+ return true;
+ }
/**
* Determine the resolution of the timer in ticks. This method is
@@ -193,6 +196,7 @@
void arm(Tick ticks);
void disarm();
+ bool expired() override;
protected:
Tick calcResolution();
@@ -200,6 +204,7 @@
private:
clockid_t clockID;
timer_t timer;
+ struct itimerspec prevTimerSpec;
};
/**
--
To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/30919
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I6159238a36fc18c0c881d177a742d8a7745a23ca
Gerrit-Change-Number: 30919
Gerrit-PatchSet: 8
Gerrit-Owner: Hsuan Hsu <[email protected]>
Gerrit-Reviewer: Andreas Sandberg <[email protected]>
Gerrit-Reviewer: Giacomo Travaglini <[email protected]>
Gerrit-Reviewer: Hsuan Hsu <[email protected]>
Gerrit-Reviewer: kokoro <[email protected]>
Gerrit-MessageType: merged
_______________________________________________
gem5-dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
