Objectives for the next year
Developers, please weigh in on what you think the NTPSec project's
goals for the next year ought to be. These goals can be coding
projects ("Move the Python code to Go") process goals ("Halve the size
of the issue list") or project infrastructure goals ("Build a hardware
lab so we can live-test supported clocks.") Or anything else you
can think of.
These goals can be large or small. They can be old issues we haven't
afddressed well enough. For purposes of the document I'm trying to put
together, clear definition of a goal matters more than how anbitious
it is.
My only request is that you not argue priorities when you see other
peoples' suggestions in the replies - not yet. The initial phase of
this discussion should be brainstorming. At some point I'll summarize
the suggestions that seem viable (clearly enough defined, possible to
complete wityhin a year) and we can start rank orering.
I'll start the ball rolling with this big one: It's time to move out of C.
C is a terrible implementation language for any project that wants to
bne secure and reliable, due to wild-pointer bugs. Our client code,
being in Python, is memory-safe; the daemon code, the most crucial
part, is not. Let's fix that.
My choice for a language to move to would be Go. Possibly one of you
can argue for a different choice, though if you agree that Go is a
suitable target I would find that information interesting.
--
>>esr>>
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
> I'll start the ball rolling with this big one: It's time to move out of C. I want to threadify things, and taking advantage of that, I want to run at full wire speed on a gigabit link with a modest server class CPU. I have test code running. I'm pretty sure it will work. But my test code is in C. Are other people happy with threads? I think threadifying things will allow us to clean up the code a lot. I think that may be as important for writing safe/clean code as moving out of C. > My choice for a language to move to would be Go. The only other choice I've seen seriously mentioned is Rust. How do they compare for thread support and for CPU cycles? Do they have full access to all the options for network I/O? (for example recv time stamps) My first reaction is that I don't want reference counted stuff involved with network I/O. But maybe that is bogus. It's roughy 5 microseconds for a recv/send pair in C. We should write some test code so we can get some real timing numbers. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
As long as we are in blue sky mode... What was the name for your attempt to get a GPSD style replay of old data? Did we ever figure out why that didn't work? The GPSD code is one way: Input => output. There is no back and forth, no request => response which changes internal state. Does that pattern make things too complicated? If we plan ahead, could we do something that would help? ... -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
On Fri, Jun 18, 2021, at 6:08 AM Hal Murray via devel wrote: > > As long as we are in blue sky mode... > > What was the name for your attempt to get a GPSD style replay of old data? > Did we ever figure out why that didn't work? It was called testframe IIRC... https://blog.ntpsec.org/2017/02/22/testframe-the-epic-failure.html > The GPSD code is one way: Input => output. There is no back and forth, no > request => response which changes internal state. Does that pattern make > things too complicated? If we plan ahead, could we do something that would > help? ... Maybe there are some huge opaque blocks of code that do not act reliably. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
Are there any C to golang or rust transpilers that work reasonably well? The last time I checked the best rust transpiler generated rs files that were just shallow glosses and the golang transpiler was somewhat inadequate and verbose. An early-started long-lived tread for each of DNS, clock- manipulation, and client I/O. Probably with several for refclocks and server functionality. Linux seccomp can work per thread but only subtractively. It would be great if we had a sandbox test tool that hits everything NTPsec uses in a few minutes. Then we could update the sandboxing so that downstream would not have to. Maybe break up some of the less testable code blocks into more manageable pieces and have tests for them. Tests deeper than just getting the version string. Tests on platforms that are not GNU/Linux w/ GCC. Tests on other architectures, see if there is a way to update docker for the runners. re: Gentoo and recent Ubuntu failures. sort out everything in my scrap piles and get the good bits merged. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
Is it worthwhile improving the current C code to a 'hardened' programming standard?� Example - Joint Strike Fighter standards https://www.stroustrup.com/JSF-AV-rules.pdf - NASA JPL standards https://andrewbanks.com/wp-content/uploads/2019/07/JPL_Coding_Standard_C.pdf - MISRA https://misra.org.uk/LinkClick.aspx?fileticket=vfArSqzP1d0%3d&tabid=57 What effort would be required for 'hardening'? If not a full 'hardening', is it worthwhile to use the hardening/vulnerability/guideline-fail reporting tools to identify and fix the worst vulnerabilities or to grab the low-hanging fruit? Anyone with experience with 'hardening' C code? (I don't) But first, what's the problem with the ntpsec C code. Is there an issue with vulnerabilities in the current C code, uncertainty with possible unknown vulnerabilities in the current code, or is the concern one of introducing vulnerabilities in the future as the C code is maintained or new functionality added? Or is the answer to that "yes". Is 'hardening' a solution or just an improvement? I assume you're still vulnerable where the hardening guidelines failed or weren't ideally followed? Is moving to a new language the better solution? If moving to another language is inevitable, if that move is selected as a goal for the next year, is 'hardening' the ntpsec C code still worthwhile? - Could 'hardening' be done and in place before the move to another language is complete. For what benefit. - Or would the 'hardened' C code be replaced weeks later by code in a new language. Or would new language code be in place in the same or similar time (sooner?), if 'hardening' efforts were instead put on moving. - If a full 'hardening' isn't worthwhile, is some 'hardening' effort worthwhile. Regards, Michael ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
Hal Murray : > > I'll start the ball rolling with this big one: It's time to move out of C. > > I want to threadify things, and taking advantage of that, I want to run at > full wire speed on a gigabit link with a modest server class CPU. > > I have test code running. I'm pretty sure it will work. But my test code is > in C. > > Are other people happy with threads? I am now. Working with Go for the last couple of years has nrought me up to speed on that kind of programming. > I think threadifying things will allow us to clean up the code a lot. I > think > that may be as important for writing safe/clean code as moving out of C. I think you are right. However, I'm pretty sure we should change languages before we threadify. Both Rust and (especially) Go have concurrency primitives that are *far* more tractable than their rough C equivalents. > > My choice for a language to move to would be Go. > > The only other choice I've seen seriously mentioned is Rust. How do they > compare for thread support and for CPU cycles? Do they have full access to > all the options for network I/O? (for example recv time stamps) > > My first reaction is that I don't want reference counted stuff involved with > network I/O. But maybe that is bogus. It's roughy 5 microseconds for a > recv/send pair in C. We should write some test code so we can get some real > timing numbers. There are two major issues with Rust: 1. We have at least two people who are expert Go programmers - Ian and myself. We have nobody, AFAIK, who is up to speed on Rust. Moving the code will be a large amount of work - I don't think any good purpose is siolved by adding "learn to be fluent in an entire new language" on top of that. 2. I don't think Rust is as yet stable enough for our purposes. The language and core libraries are still in some flux - we can't yet count on a feature we're relying on not disappearing over the next decade. This is a very srtark contrast with Go's ironclad forward-compatibility guarantee. For me, issue #2 is the real dealbreaker. As for your questions: Both languages have reasonable threading support, far better than C's. Rust would be more abstemious of processor cycles than Go is, but I dont believe the difference is significant for our deployment. Full access to all aptioons for betwork I/O: I believe pure Go can be beaten into this, but that corner of the API is very poorly documented so I don't know exactly how yet. We have the option of using cgo and writing small C extensions to gety at things the Go API doesan't suppport. I don't knpw of anuy direct support for things like recv time stamps in Rust. But Rust also has a facility to call C extensions. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
Hal Murray : > What was the name for your attempt to get a GPSD style replay of old data? > Did we ever figure out why that didn't work? I did. There's a blog post about it: https://blog.ntpsec.org/2017/02/22/testframe-the-epic-failure.html -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
Yo MLewis! On Fri, 18 Jun 2021 20:17:56 -0400 MLewis via devel wrote: > Is it worthwhile improving the current C code to a 'hardened' > programming standard?� > > Example > - Joint Strike Fighter standards > https://www.stroustrup.com/JSF-AV-rules.pdf That is for C++. And anything JSF (F-35) does, we should do the opposite. > - NASA JPL standards > https://andrewbanks.com/wp-content/uploads/2019/07/JPL_Coding_Standard_C.pdf Dated 2009, I dont see much in there that would help NTPsec. > - MISRA > https://misra.org.uk/LinkClick.aspx?fileticket=vfArSqzP1d0%3d&tabid=57 More of a process guide for beginners than a coding guide. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can't measure it, you can't improve it." - Lord Kelvin pgpTeNlGQWQ9P.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
James Browning via devel : > Are there any C to golang or rust transpilers that work > reasonably well? The last time I checked the best rust > transpiler generated rs files that were just shallow glosses > and the golang transpiler was somewhat inadequate and > verbose. This is still the state of play, alas. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Objectives for the next year
MLewis : >Is it worthwhile improving the current C code to a 'hardened' programming >standard?� > >Example >- Joint Strike Fighter standards >[1]https://www.stroustrup.com/JSF-AV-rules.pdf >- NASA JPL standards > > [2]https://andrewbanks.com/wp-content/uploads/2019/07/JPL_Coding_Standard_C.pdf >- MISRA >[3]https://misra.org.uk/LinkClick.aspx?fileticket=vfArSqzP1d0%3d&tabid=57 > >What effort would be required for 'hardening'? > >If not a full 'hardening', is it worthwhile to use the >hardening/vulnerability/guideline-fail reporting tools to identify and fix >the worst vulnerabilities or to grab the low-hanging fruit? > >Anyone with experience with 'hardening' C code? (I don't) > >But first, what's the problem with the ntpsec C code. Is there an issue >with vulnerabilities in the current C code, uncertainty with possible >unknown vulnerabilities in the current code, or is the concern one of >introducing vulnerabilities in the future as the C code is maintained or >new functionality added? Or is the answer to that "yes". Is 'hardening' a >solution or just an improvement? I assume you're still vulnerable where >the hardening guidelines failed or weren't ideally followed? Is moving to >a new language the better solution? > >If moving to another language is inevitable, if that move is selected as a >goal for the next year, is 'hardening' the ntpsec C code still worthwhile? >- Could 'hardening' be done and in place before the move to another >language is complete. For what benefit. >- Or would the 'hardened' C code be replaced weeks later by code in a new >language. Or would new language code be in place in the same or similar >time (sooner?), if 'hardening' efforts were instead put on moving. >- If a full 'hardening' isn't worthwhile, is some 'hardening' effort >worthwhile. > >Regards, > >Michael > > References > >Visible links >1. https://www.stroustrup.com/JSF-AV-rules.pdf >2. > https://andrewbanks.com/wp-content/uploads/2019/07/JPL_Coding_Standard_C.pdf >3. https://misra.org.uk/LinkClick.aspx?fileticket=vfArSqzP1d0%3d&tabid=57 Here is my judgment: > Is there an issue >with vulnerabilities in the current C code, uncertainty with possible >unknown vulnerabilities in the current code, or is the concern one of >introducing vulnerabilities in the future as the C code is maintained or >new functionality added? All of the above. I *do* have experience with hardening C code - a substabtiual amount of it has been on this project, and there was GPSD before that. In truth, we've already done most of the best practices. More effort would have rapidly diminishing returns. But the real problem is at a deeper level. C is fundamentally unsafe; hardening - either as we've achieved it or as a hypothetical ideal - can only be an improvement, not a solution. I think our time would be better spent moving to a memory-safe language than trying to harden the C code further. -- http://www.catb.org/~esr/";>Eric S. Raymond ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
PLL (was Re: Objectives for the next year)
e...@thyrsus.com said: > I did. There's a blog post about it: > https://blog.ntpsec.org/2017/02/22/testframe-the-epic-failure.html >From there: > One was what in discussion on the mailing list I later tagged "the code-path > split". There are two kinds of NTP hosts; one uses a kernel facility known as > the "PLL" for doing fine adjustments to the tick speed of the system clock, > the other kind does not. Most of our potential deployment targets have PLLs; > an undismissable minority do not. What do you mean by PLL? Which systems don't have it? A PLL requires 2 things. One is a knob/switch to adjust something. The other is sensor to compare something against a target setting and use that to control the knob. A thermostat and heater is the normal example. The kernels I'm familiar with have the knob but not the sensor. They actually do have a sensor if you build the kernel with the right option but none of the distros I'm familiar with ship with that option enabled. The bottom line is that every time you use "PLL" like that you toss a layer of confusion into the discussion. (or at least my part of it) -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
