> Date: Wed, 10 Jun 2020 23:11:32 +0000 > From: Andrew Doran <[email protected]> > > On Tue, Jun 09, 2020 at 05:16:27PM +0000, Taylor R Campbell wrote: > > > At the same time, I wonder whether we should _also_: > > > > 1. Modify the tsc timecounter so that it uses a global atomic to > > ensure that there is a global view of time as counted by the tsc. > > A long time ago we used to do something quite like this for x86 but it > worked poorly which is somewhat understandable because it's really difficult > to get something like that right. (Which reminds me someone posted a patch > for kern_cctr.c for alpha a couple of years ago.)
It's not hard to get global monotonicity right. We do that for Xen these days, in arch/xen/xen/xen_clock.c. The Alpha patch was probably this thread: https://mail-index.netbsd.org/port-alpha/2017/11/01/msg000873.html What's trickier is synchronizing per-CPU timecounters so that they all give a reasonably consistent view of absolute wall clock time -- and so it's not just one CPU that leads while the others play catchup every time they try to read the clock. (In other words, adding atomic catchup logic certainly does not obviate the need to synchronize per-CPU timecounters!) But neither synchronization nor global monotonicity is always necessary -- e.g., for rusage we really only need a local view of time since we're only measuring relative time durations spent on the current CPU anyway. > > This is what the timecounter(9) API per se expects of timecounters, > > and right now tsc (along with various other per-CPU cycle counters) > > fails to guarantee that. > > Howso, do you see a bug? I think it's okay. The TSC is only used for the > timecounter where it's known that it's insensitive to core speed variations > and is driven by PLL related to the bus clock. Fortunately that means most > x86 systems, excepting a window of some years from roughly around the time > of the Pentium 4 onwards. If tc_get_timecount goes backward by a little, e.g. because you queried it on cpu0 the first time and on cpu1 the second time, kern_tc.c will interpret that to mean that it has instead jumped forward by a lot -- nothing in the timecounter abstraction copes with a timecounter that goes backwards at all. (There's also an issue where the `monotonic' clock goes backwards sometimes, as reported by sched_pstats. I'm not sure anyone has tracked down where that's coming from -- it seems unlikely to be related to cross-CPU tsc synchronization because lwp rtime should generally be computed from differences between samples on a single CPU at a time, but I don't know.)
