ntpviz - Don't plot a line during data absence
In the interest of keeping longer technical discussions on the devel list, we have this: Gary, Eric, and I had a discussion on #ntpsec about what I referred to as "interpolation" performed by ntpviz when there is a hole in data collection. Gary pointed out that it's really just gnuplot drawing lines between data points. I suppose gnuplot can't "know" the difference between a real interval between points and one where something has broken. That's probably it. I expect you'd get a gap there in an impulse or dot plot. Might mean gemiller should tweak the plot mode. jasonium: not exactly nptviz filling the blank spots, that is gnuplot making lines between points. There was some more discussion about adding runtime options to create different kinds of graphs. Then: hmm, some people use up to poll 1024, I could preprocess the plot file so that any gap longer than 1024 seconds does not get a connecting line. Sound good? Or would people prefer to see just dots first? I like that compromise. and: jasonium: I just pushed a change to ntpviz. If no data for over 1024 seconds, then tell gnuplot not to connect the lines. and even later: jasonium: further testing shows a 10% slowdown with my recent hack. So tell me if it is doing what you want before I try to regain some of the lost time. I have some ideas, but they will require some contortions to the data set schema. It was taking me 39 seconds to run a weeks logs on a RasPi3, after the patch is is 43 second I like the change. I've attached a sample image (I hope that's kosher on the list), where you see peer "chilipepper" appear and disappear. In the earlier version of ntpviz, gnuplot would have drawn straight lines between "real" data points. Whether or not it is worth a 10% slowdown is certainly debatable. -- Jason Azze ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Jason Azze : > Whether or not it is worth a 10% slowdown is certainly debatable. Let's step back a bit. Why is it important for ntpviz to be fast? I can see why it's important for ntpviz to use as few processor clocks as possible, in order to avoid causing artifacts in the data. Is that the same issue? -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Jason Azze writes: > I suppose gnuplot can't "know" the difference between a real > interval between points and one where something has broken. You can tell gnuplot that the data is missing and it will not connect the end points of the data on either side of the gap. You can either do that by inserting a blank record or indicating the missing data by some string for individual datapoints. See "set datafile missing" in the documentation. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Achim! On Sun, 23 Oct 2016 19:04:47 +0200 Achim Gratz wrote: > Jason Azze writes: > > I suppose gnuplot can't "know" the difference between a > > real interval between points and one where something has broken. > > You can tell gnuplot that the data is missing and it will not connect > the end points of the data on either side of the gap. You can either > do that by inserting a blank record or indicating the missing data by > some string for individual datapoints. See "set datafile missing" in > the documentation. I already pushed a (slow) patch for this Friay night. Can anyone confirm it looks good> RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpFFXkyLiJml.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Eric! On Sun, 23 Oct 2016 12:44:16 -0400 "Eric S. Raymond" wrote: > Jason Azze : > > Whether or not it is worth a 10% slowdown is certainly debatable. > > Let's step back a bit. Why is it important for ntpviz to be fast? We already discussed this. The load on a RasPi distorts the timing so badly that it shows up on the ntpizv plots. The Heisenberg Uncertainty Principle in action: Mmasuring distorts the thing being measured. Look at the spikes in the red line on the first graph here: https://pi4.rellim.com/day/ Each one of those is ntpviz running. When ntpvis runs faster (finishes in less time) the spikes get smaller. The spikes used to me much, much, bigger. > I can see why it's important for ntpviz to use as few processor clocks > as possible, in order to avoid causing artifacts in the data. Is that > the same issue? Yes, plus it is a lot easier to debug a task that takes 36 seconds versus one that takes 5 minutes. Yes, it has a lot more feater now and is still that much faster. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp_yhOGVbvMm.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Jason! On Sun, 23 Oct 2016 12:39:42 -0400 Jason Azze wrote: > I like the change. Good. Not heard any complaints. > I've attached a sample image (I hope that's kosher > on the list), where you see peer "chilipepper" appear and disappear. I find it harder to find the fragments without the 'extra' lines, but I see it is more important to not confuse people that do not start at it all day. So I'll keep it as a fixed feature for now. > Whether or not it is worth a 10% slowdown is certainly debatable. I have some ideas to improve it. I have been thinking of re-arranging the data schema for other reasons. Most can stop reading here, only those that care about optimization and speed should continue. The big problem here is that it adds 4 more float() conversions per data point. That is a large overhead when plotting millions of points. I am also learning more about Python. Here is the hot code, executed millions of times: # a is a string of the current time, b is a float of the last time if 1024 < ( float( a ) - b): # do something b = float(a) So, two float conversions per cycle. If C you would do this to change a float to a store to make it much faster: # a is a string of the current time, b is a float of the last time # c is a float of the current time c = float (a ) if 1024 < ( c - b): # do something b = c In C that would be much faster, but in Python it is slower! In Python it seems to be slower to create a new variable than to convert a string to a float. Proving once, if you are looking for speed you need to profile, and expect to be surprised often. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpAclUHlRfAV.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python.h is broken on NetBSD and FreeBSD
Yo Hal! On Sat, 22 Oct 2016 21:18:57 -0700 Hal Murray wrote: > On Linux, it's in: > /usr/include/python2.7/Python.h Uh, not so fast, on my laptop: /usr/include/python3.5m/Python.h /usr/include/python3.4m/Python.h /usr/include/python2.7/Python.h Plus the /usr/local/include variants on another host compiled from source. The waf recipe will need to be creative. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpLgWnRiA6Bq.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Gary E. Miller writes: > We already discussed this. The load on a RasPi distorts the timing > so badly that it shows up on the ntpizv plots. The Heisenberg Uncertainty > Principle in action: Mmasuring distorts the thing being measured. The load itself actually doesn't distort the timing. It warms up the processor and RAM and finally the XO, however, and _that_ changes the timing. Also, the Heisenberg uncertainty principle is about the attainable precision of measurement of complementary pairs of variables, not about the disturbance of the system being measured by the act of measurement. For the rasPi 2B that I have here, I've logged the PPS arrival times together with the CPU temperature for two weeks now. After I've closed the box it's in, which made things much more stable and about 4K warmer overall, the loop values can be predicted by a kernel density filtered CPU temp measurement quite nicely and I end up with almost exactly linear 200ppb/K response over the 4K temperature range of the last week. Now, kernel density is a non-causal filter and can't be used in a feed forward loop and I haven't yet looked into a causal filter that gets rid of the noise and short spikes and can still predict correctly where the loop should be. But it seems possible to do that with only a small error, which would allow the main NTP FLL/PLL to just deal with the oscillator drift and the residual noise from the environment, which should become about an order of magnitude smaller, maybe a bit better even. > Look at the spikes in the red line on the first graph here: > > https://pi4.rellim.com/day/ > > Each one of those is ntpviz running. When ntpvis runs faster (finishes > in less time) the spikes get smaller. The spikes used to me much, much, > bigger. Sorry, but you're doing that wrong. Pull the data from the machine with rsync (using -rsh=ssh) and process it offline. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Terratec KOMPLEXER: http://Synth.Stromeko.net/Downloads.html#KomplexerWaves ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
ja...@azze.org said: > In the interest of keeping longer technical discussions on the devel list, > we have this: Thanks. > hmm, some people use up to poll 1024, I could preprocess the plot > file so that any gap longer than 1024 seconds does not get a connecting > line. Sound good? Or would people prefer to see just dots first? If the polling interval is X, there may be up to 8X gaps in peerstats. I think it helps to avoid the long lines across missing data. Lines vs points is an interesting tangle. Lines work well when the data is smooth. The same data with points turns into a fat line if the points are dense. (I consider the single dot type of point to be useless since I can't see an isolated dot on my screen unless I know where to look.) If you have occasional outliers, lines mode draws a line up to that sample and back down. That makes it easy to spot the outliers. If you have more than occasional outliers that turns into a lot of clutter on the screen and points mode is probably better. If the data is not smooth, and the samples are dense, line mode turns into a broad swath of up and down lines. Points mode has the same problem - the swath is made up of points rather than line segments. You can solve the problem by spreading things out in X, or Y if in points mode. Things get more interesting if you try to plot several sets of data that mostly overlap. I haven't found a good way to automate the decision process. I've been doing it by hand. A config file to guide that might be interesting. It could also specify which graphs to plot and/or what to put on a graph. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
g...@rellim.com said: >> Let's step back a bit. Why is it important for ntpviz to be fast? > We already discussed this. The load on a RasPi distorts the timing so badly > that it shows up on the ntpizv plots. The Heisenberg Uncertainty Principle > in action: Mmasuring distorts the thing being measured. Have you tried doing the work on another system? I use rsync. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Gary E. Miller writes: > So, two float conversions per cycle. If C you would do this to > change a float to a store to make it much faster: > > # a is a string of the current time, b is a float of the last time > # c is a float of the current time > c = float (a ) > if 1024 < ( c - b): > # do something > b = c > > In C that would be much faster, but in Python it is slower! In Python > it seems to be slower to create a new variable than to convert a string > to a float. So don't create and destroy variable c each time round your loop. Create it once outside the loop and use it for assignment inside the loop and you might see the expected speedup. Or not, depending on how clever Python gets when it sees that you use the same expression multiple times on the same argument. I'm sure Python has ways to just read in all that data into a float array before you dive into the looping business. You'd generally be much better off using such a facility if it exists instead of trying to parse the data file line by line. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Achim! On Sun, 23 Oct 2016 21:07:57 +0200 Achim Gratz wrote: > Gary E. Miller writes: > > We already discussed this. The load on a RasPi distorts the timing > > so badly that it shows up on the ntpizv plots. The Heisenberg > > Uncertainty Principle in action: Mmasuring distorts the thing being > > measured. > > The load itself actually doesn't distort the timing. It warms up the > processor and RAM and finally the XO, however, and _that_ changes the > timing. Ah, sorry. I disgree with your analysys. Check out the temp graph on the page to see that temps are unrelated. On other hosts I've measured up to 6 temps (cpu, disk, room, cpu, north brodge, etc.) and found zero correlation. The problem is that a loaded CPU takes longer to repond to an interrupt. > Also, the Heisenberg uncertainty principle is about the > attainable precision of measurement of complementary pairs of > variables, not about the disturbance of the system being measured by > the act of measurement. You have a common misunderstanding. I guess your college professor will have to call up and complain to mine. I'll let my A in that class speak for itself. But I'll not go off shearing that Yak with you. > For the rasPi 2B that I have here, I've logged the PPS arrival times > together with the CPU temperature for two weeks now. Two weeks, I have temps going back seveal months now. > After I've > closed the box it's in, which made things much more stable and about > 4K warmer overall, the loop values can be predicted by a kernel > density filtered CPU temp measurement quite nicely and I end up with > almost exactly linear 200ppb/K response over the 4K temperature range > of the last week. I also see the effect you speak of. That is on my graphs, but only loosely related to the spikes. They correlate, but are lagged. That is, the CPU load is cause to both the temp and frequency. The load come first, then the freq spike, then a smaller temp spike. The temp is the dependent variable on load spikes, not the independent variable. Feel free to post your graphs here, after you are runnning ntpviz in a crontab too. But since you are not running ntpviz on your ntpd host we are not even talking about the same experiment. > > Look at the spikes in the red line on the first graph here: > > > > https://pi4.rellim.com/day/ > > > > Each one of those is ntpviz running. When ntpvis runs faster > > (finishes in less time) the spikes get smaller. The spikes used to > > me much, much, bigger. > > Sorry, but you're doing that wrong. Pull the data from the machine > with rsync (using -rsh=ssh) and process it offline. Wrong? Just different. Pretty hard to build a standalone NTP server when you need two servers. I'm runnning so many test ntpd servers that is not a practical solution. Plus, it providea a nice standard load. It keeps visibility on the real problem: that ntpd is too load sensitive. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpt06NCXD8w1.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Hal! On Sun, 23 Oct 2016 12:08:40 -0700 Hal Murray wrote: > ja...@azze.org said: > > In the interest of keeping longer technical discussions on the > > devel list, we have this: > > Thanks. > > > hmm, some people use up to poll 1024, I could preprocess > > the plot file so that any gap longer than 1024 seconds does not get > > a connecting line. Sound good? Or would people prefer to see just > > dots first? > > If the polling interval is X, there may be up to 8X gaps in peerstats. Ah, lost me. Say what? > I think it helps to avoid the long lines across missing data. Another vote for the new patch. > Lines vs points is an interesting tangle. Yes, easy to change, but looks terrible to me, and everyone I showed it too. > I haven't found a good way to automate the decision process. I've > been doing it by hand. You said similar on the plot clipping. Have you looked at the new --clip option? Does that look good to you? Maybe something to tweak. > A config file to guide that might be > interesting. It could also specify which graphs to plot and/or what > to put on a graph. If you run ntpviz with the new -D 2 option, the plot files then are left in your system temp directory. Yuu can then trivially edit them. Say change 'with lines' to 'with linespoints'. If you can some up with a 'more pleasing' graph I can make it an option. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpp6HnZk5OYA.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Hal! On Sun, 23 Oct 2016 12:11:40 -0700 Hal Murray wrote: > g...@rellim.com said: > >> Let's step back a bit. Why is it important for ntpviz to be > >> fast? > > We already discussed this. The load on a RasPi distorts the timing > > so badly that it shows up on the ntpizv plots. The Heisenberg > > Uncertainty Principle in action: Mmasuring distorts the thing being > > measured. > > Have you tried doing the work on another system? I use rsync. Of course. But now I have a much more complex sysadmin problem. I can't expect most people to do that. Dedicating just one host to ntp is asking a lot of people. Plus I am resistant to this 'cloud' way of thinking. More machines to debug and fail in more ways. I will do it offline when I have an event in the data I want to freeze in time. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpS_3Ck3C9Ax.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Achim! On Sun, 23 Oct 2016 21:16:00 +0200 Achim Gratz wrote: > Gary E. Miller writes: > > So, two float conversions per cycle. If C you would do this to > > change a float to a store to make it much faster: > > > > # a is a string of the current time, b is a float of the > > last time # c is a float of the current time > > c = float (a ) > > if 1024 < ( c - b): > > # do something > > b = c > > > > In C that would be much faster, but in Python it is slower! In > > Python it seems to be slower to create a new variable than to > > convert a string to a float. > > So don't create and destroy variable c each time round your loop. Right, that was my initial hack. > Create it once outside the loop and use it for assignment inside the > loop and you might see the expected speedup. Yes, the next logical step. But that float subtract/compare is also painful. I'm thinking of going integer milliseconds instead of floats. But that adds a float multiply, so it may it work out. Lot's to try. Feel free to run some of your own profiles and share. > Or not, depending on how > clever Python gets when it sees that you use the same expression > multiple times on the same argument. Not an issue here. Each loop has new unique values. > I'm sure Python has ways to just read in all that data into a float > array before you dive into the looping business. PHP does a lot better than Python. If you find a better way please forward the patch and the profile to show the improvement. > You'd generally be > much better off using such a facility if it exists instead of trying > to parse the data file line by line. Certainly, and PHP does much better than Python on this regard. I found that an unpleasant surprise, I had thought Python was supposed to be better at arrays than PHP, but not true. Now if I can get Eric to code up some new Python functions in C for me? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpSo20_Ptlb8.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
g...@rellim.com said: >> If the polling interval is X, there may be up to 8X gaps in peerstats. > Ah, lost me. Say what? Look at the time differences between samples for an IP Address in peerstats. If the polling interval is X, there will be a line in rawstats for each packet received. If none are lost, the spacing will be X. There is a filter between the raw packets and the ones that come out of peerstats. It has a buffer of the last 8 samples. It only uses the one with the lowest round trip time. So if you get 8 samples in a row with increasing RTT, the last 7 will get saved until the a new sample bumps the first one out of the buffer. Refclocks samples always get through. I assume it takes new samples when the RTT is equal but there might be a simple special case test. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
g...@rellim.com said: >> Lines vs points is an interesting tangle. > Yes, easy to change, but looks terrible to me, and everyone I showed it too. Are you graphing any systems that are only using the pool? That will get some less good servers with ugly data. To my eye, sometimes it looks much better in points mode. Another interesting test case is the NIST servers in Gaithersburg MD. They have a routing quirk that adds a 30ms offset as well as the noise from being heavily loaded. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Hal! On Sun, 23 Oct 2016 13:39:32 -0700 Hal Murray wrote: > g...@rellim.com said: > >> If the polling interval is X, there may be up to 8X gaps in > >> peerstats. > > Ah, lost me. Say what? > > Look at the time differences between samples for an IP Address in > peerstats. > > If the polling interval is X, there will be a line in rawstats for > each packet received. If none are lost, the spacing will be X. Where X is by default 64 sec, and by default can be ramped up to 1024 sec. So the current line break hack does not plot a line longer than 1024 sec. Lines of 1024 sec are plotted. That will not work for folks setting maxpoll over 1024 sec. > There is a filter between the raw packets and the ones that come out > of peerstats. It has a buffer of the last 8 samples. It only uses > the one with the lowest round trip time. So if you get 8 samples in > a row with increasing RTT, the last 7 will get saved until the a new > sample bumps the first one out of the buffer. Why do we care? Don't we still get a sample, of some sort, every X? > Refclocks samples always get through. I assume it takes new samples > when the RTT is equal but there might be a simple special case test. Aren't reclocks local? Thus no rtt? Is there anything to try in real plots. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpoKuPDwWhTE.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Hal! On Sun, 23 Oct 2016 13:53:32 -0700 Hal Murray wrote: > g...@rellim.com said: > >> Lines vs points is an interesting tangle. > > Yes, easy to change, but looks terrible to me, and everyone I > > showed it too. > > Are you graphing any systems that are only using the pool? No, I never use the pool. > That will > get some less good servers with ugly data. To my eye, sometimes it > looks much better in points mode. As I said, use -D 2 and play. If we can fugure out an option or hueristic we can try it in nptviz. > Another interesting test case is the NIST servers in Gaithersburg > MD. They have a routing quirk that adds a 30ms offset as well as the > noise from being heavily loaded. I get that a lot. The plot looks good to me. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpMuIEs4YUbl.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
> Why do we care? Don't we still get a sample, of some sort, every X? Yes if you look in rawstats. No if you look in peerstats. That was the whole point of this discussion. (as well as good background) > So the current line break hack does not plot a line longer than 1024 sec. > Lines of 1024 sec are plotted. That will not work for folks setting maxpoll > over 1024 sec. It will also get occasional bogus breaks with the default maxpoll of 1024 when the peer logic filters out several samples. > Aren't reclocks local? Thus no rtt? The data from a refclock gets merged into the main flow. It has to have some rtt. 0 seems like a good number. > Is there anything to try in real plots. You should setup a system using the pool. That will give you lots of interesting data to look at. If you setup a local known-good server (or several) to use it as a noselect server, you can compare the offset they see with its claimed offset. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
g...@rellim.com said: >> Have you tried doing the work on another system? I use rsync. > Of course. But now I have a much more complex sysadmin problem. This discussion probably depends upon the big picture goals. You seem to have set things up so that a cron job on a system makes a set of graphs that you like and packages them up as a web page. I take a different approach. A script collects all the data on one system. Another script makes a bunch of graphs. I look at them directly. No web stuff. Many of those graphs involve data from more than one system. It's not uncommon that I tweak the collection of graphs I look at. The most common tweak is roughly your clipping. I want to see both the unclipped and the zoomed in versions. -- Some of our different opinions about points and lines may be because I'm looking at much larger graphs than you are making and/or at the unclipped data. I'm using most of a large screen, roughly 4 times the area of your web graphs. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Hal! On Sun, 23 Oct 2016 14:30:18 -0700 Hal Murray wrote: > > Why do we care? Don't we still get a sample, of some sort, every > > X? > > Yes if you look in rawstats. No if you look in peerstats. That was > the whole point of this discussion. (as well as good background) > > > > So the current line break hack does not plot a line longer than > > 1024 sec. Lines of 1024 sec are plotted. That will not work for > > folks setting maxpoll over 1024 sec. > > It will also get occasional bogus breaks with the default maxpoll of > 1024 when the peer logic filters out several samples. Not the current logic. As long as the peerstats stride is 1024 or less the data is plotted as 'with lines'. > > Aren't reclocks local? Thus no rtt? > > The data from a refclock gets merged into the main flow. It has to > have some rtt. 0 seems like a good number. It is always zero, and ntpviz always ignores it. > > Is there anything to try in real plots. > > You should setup a system using the pool. That will give you lots of > interesting data to look at. I already have 5 test systems, and I don't care about the pool. But I will take patches. > If you setup a local known-good server (or several) to use it as a > noselect server, you can compare the offset they see with its claimed > offset. Even better, I have 5 test servers in a cluster with PPS. Plus some production servers. They all lock to the 'preferred' local PPS. https://rellim.com https://pi.rellim.com https://pi2.rellim.com https://pi3.rellim.com https://pi4.rellim.com I gotta enough problems I care about to track down for the near future. Feel free to scrathc you itch. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp4FYWhoe8s8.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
Yo Hal! On Sun, 23 Oct 2016 14:31:38 -0700 Hal Murray wrote: > g...@rellim.com said: > >> Have you tried doing the work on another system? I use rsync. > > Of course. But now I have a much more complex sysadmin problem. > > This discussion probably depends upon the big picture goals. Yup. > You seem to have set things up so that a cron job on a system makes a > set of graphs that you like and packages them up as a web page. Yup. > I take a different approach. A script collects all the data on one > system. Another script makes a bunch of graphs. I look at them > directly. No web stuff. Many of those graphs involve data from more > than one system. Sure. You do your thing, I do mine. Mine is a lot simpler for a newbie to setup, one line in a crontab: 6 0-23/3 * * * cd /usr/local/src/NTP/ntpsec/ntpstats; ./ntpviz @week/optionfile > It's not uncommon that I tweak the collection of graphs I look at. > The most common tweak is roughly your clipping. I want to see both > the unclipped and the zoomed in versions. That is then two lines in your crontab... You gotta admit you are not our majority target audience. :-) > Some of our different opinions about points and lines may be because > I'm looking at much larger graphs than you are making and/or at the > unclipped data. I'm using most of a large screen, roughly 4 times > the area of your web graphs. In 2016, most screen are 1288x768, or smaller. Then subtract the browser frame zize. That is what I am designing too. If you plot for bigger then 1/2 the people can't user it. I'm not gonna change a default to something wring for most user. But I would consier an option for screen plot size if you would use it. The @optionfile makes it easy to use a bunch of tweaks. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpHM3cV7FwwU.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python.h is broken on NetBSD and FreeBSD
Gary E. Miller : > Yo Hal! > > On Sat, 22 Oct 2016 21:18:57 -0700 > Hal Murray wrote: > > > On Linux, it's in: > > /usr/include/python2.7/Python.h > > Uh, not so fast, on my laptop: > > /usr/include/python3.5m/Python.h > /usr/include/python3.4m/Python.h > /usr/include/python2.7/Python.h > > Plus the /usr/local/include variants on another host compiled from source. > > The waf recipe will need to be creative. There's now a waf fix in head that should find Python.h portably. -- http://www.catb.org/~esr/";>Eric S. Raymond signature.asc Description: PGP signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Python.h is broken on NetBSD and FreeBSD
> There's now a waf fix in head that should find Python.h portably. Builds on a quick try on NetBSD and FreeBSD. Thanks. What does it mean to "compile" a header file? Why is it compiling pthon stuff twice? [111/139] Compiling ntpkeygen/ntpkeygen.c [112/139] Compiling ntptime/ntptime.c [113/139] Compiling include/ntp_control.h [114/139] Compiling include/ntp.h [115/139] Compiling pylib/__init__.py [116/139] Compiling pylib/__init__.py [117/139] Compiling pylib/ntp_control.py [118/139] Compiling pylib/ntp_control.py [119/139] Compiling pylib/ntp_magic.py [120/139] Compiling pylib/ntp_magic.py -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz - Don't plot a line during data absence
> Sure. You do your thing, I do mine. Mine is a lot simpler for a > newbie to setup, one line in a crontab: Don't forget to setup your web server. (and keep it secure) > You gotta admit you are not our majority target audience. :-) Well, neither are you. If you are targeting the majority, I think you need to setup a system using the pool so you will have more typical data to work with. > In 2016, most screen are 1288x768, or smaller. That's probably correct for laptops. My old/dead one was 1280x800. My new one is 1366x768. HD TV is 1920 x 1080. That's a sweet spot in the price/quality space. I got mine a couple of years ago when my old tube display died. I should have done it much earlier. I think I paid $150. Frys has some for under $100. I think there are enough people with bigger displays that an option to target for them would see a lot of use. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
