Re: Logging with threads

2025-07-06 Thread James Browning via devel
On Sunday, July 6, 2025 10:27:17 AM Pacific Daylight Time Hal Murray wrote: > Interesting. Thanks. > > > Currently it adds 16 bytes per line logged. > > I think 16 is a bit too much. > > > The code is pretty simple and I think it should be portable. > > + fprintf(term_fil

Re: Logging with threads

2025-07-06 Thread Hal Murray via devel
Interesting. Thanks. > Currently it adds 16 bytes per line logged. I think 16 is a bit too much. > The code is pretty simple and I think it should be portable. > + fprintf(term_file, "%s[%d.%ld]: ", prog, pid, > (long int)pthread_self()); The Linux man page says that

Re: Logging with threads

2025-07-06 Thread James Browning via devel
On Saturday, July 5, 2025 10:40:16 PM Pacific Daylight Time Hal Murray via devel wrote: > I'd like msyslog to indicate which thread is doing the logging. > > The main thread in ntpd currently creates up to 3 threads. > One for DNS, and a NTS thread for each of IPv4 and IPv6.

Re: Logging with threads

2025-07-05 Thread Gary E. Miller via devel
Yo Hal! On Sat, 05 Jul 2025 22:40:16 -0700 Hal Murray via devel wrote: > I'd like msyslog to indicate which thread is doing the logging. Nice idea. > Anybody thought about this area? Any good ideas? None from me, but I like your idea. RGDS GARY --

Logging with threads

2025-07-05 Thread Hal Murray via devel
I'd like msyslog to indicate which thread is doing the logging. The main thread in ntpd currently creates up to 3 threads. One for DNS, and a NTS thread for each of IPv4 and IPv6. The DNS thread gets created and joined, but there is only one at any time. We may need more NTS threads i

Re: Threads

2021-04-07 Thread Hal Murray via devel
27;s using 1/4 of the wire. Modern CPUs have lots of cores. With 4 threads... The more I think about it, the more I think that threads will clean up the code. -- These are my opinions. I hate spam. ___ devel mailing list devel@ntpsec.org http:

Re: Threads

2021-04-06 Thread Sanjeev Gupta via devel
not be able to process packets at full wire > speed. > 3) The kernel's input dispatcher thread might run out of CPU cycles. > 4) The client threads might run out of CPU cycles. > > I don't have a good setup to demonstrate the Ethernet chip being the > limiting > fac

Threads

2021-04-06 Thread Hal Murray via devel
There are 4 places that might be the limiting factor. 1) The wire might be full 2) The Ethernet chip might not be able to process packets at full wire speed. 3) The kernel's input dispatcher thread might run out of CPU cycles. 4) The client threads might run out of CPU cycles. I

Re: TSC warp, Threads

2021-03-12 Thread Achim Gratz via devel
Hal Murray via devel writes: > Can you say more. Is there any good Intel documentation that says "Xeon v3 > and up"? Not that I know of, although it's surely buried inside some file someplace on ARK. > Or anything that describes which families or chips will/won't do what > I want? The trouble

Re: TSC warp, Threads

2021-03-12 Thread Hal Murray via devel
Thanks. > That's an IvyBridge I think, you will want Haswell or later. > Haswell and later are OK (Xeon v3 and up) AFAIK Can you say more. Is there any good Intel documentation that says "Xeon v3 and up"? Or anything that describes which families or chips will/won't do what I want? My search p

Re: TSC warp, Threads

2021-03-11 Thread ASSI via devel
Hal Murray via devel writes: > Does anybody have access to server class systems? Not with bare metal Linux at the moment… > I'm trying to setup a test environment for the thread work. Dell high end > workstations use Xeon chips. I have a T5610, but it has TSC warp so doesn't > use the TSC for

Re: TSC warp, Threads

2021-03-10 Thread Mike via devel
On 3/10/21 8:25 PM, Hal Murray via devel wrote: Does anybody have access to server class systems? Or know somebody who can run a quick test for me? [    0.00] tsc: Fast TSC calibration using PIT [    0.00] tsc: Detected 2100.063 MHz processor [    0.481983] TSC deadline timer available

Re: TSC warp, Threads

2021-03-10 Thread Gary E. Miller via devel
Yo Hal! On Wed, 10 Mar 2021 17:25:26 -0800 Hal Murray via devel wrote: > Just run "dmesg | grep tsc -i" If you get the 3 lines above, it's warped. None of mine: > You can get the CPU chip from /proc/cpuinfo model name : AMD Ryzen 5 3600 6-Core Processor model name : Intel(R) Core(T

TSC warp, Threads

2021-03-10 Thread Hal Murray via devel
een debugging test code. I have a multi threaded echo server with a knob to spin for N uSec before replying. I also have a test harness to drive clients on several systems, each with several threads/sockets so several low-end systems can generate enough traffic to saturate a high end system. If

threads, locks, atomic

2021-01-14 Thread Hal Murray via devel
Here is what I think we want to end up with: N server threads processing client requests M threads processing NTS-KE requests 1 main thread to process ntp.conf and timers a client thread per server from server/pool in ntp.conf a thread for each refclock We already have M=2

Re: More thoughts on threads...

2020-12-16 Thread Gary E. Miller via devel
Yo Hal! On Wed, 16 Dec 2020 20:21:55 -0800 Hal Murray via devel wrote: > But maybe I'm fooling myself. Are people comfortable with threads > and locks? (I've been working with them since 1976.) In theory: vey much yes. In practice: the devil is in the det

More thoughts on threads...

2020-12-16 Thread Hal Murray via devel
I'm still scheming on this topic. Here is what I'd like to end up with: N server threads, each with their own socket listening on UDP 123 a client thread for each server we are using (with own socket) a thread for each refclock the main thread for whatever is left Getting

Re: Thoughts on networking and threads...

2020-12-03 Thread James Browning via devel
t RFC to have the client use something other than port > 123. > Splitting the client and server gets that. > > Using threads will clean up the packet processing at the cost of adding > locks. > I think the locks will be simple -- after we understand where they > interact. &

Thoughts on networking and threads...

2020-11-28 Thread Hal Murray via devel
FC to have the client use something other than port 123. Splitting the client and server gets that. Using threads will clean up the packet processing at the cost of adding locks. I think the locks will be simple -- after we understand where they interact. The client can stay single threaded. I

Re: Various cleanups: threads, STA_NANO

2019-01-24 Thread Hal Murray via devel
> I think we covered this in https://gitlab.com/NTPsec/ntpsec/issues/421 I think that's a different issue. OpenBSD doesn't support that API at all. The STA_NANO case I'm interested in is several ifdef-s in ntp_loopcontrol No big deal, just some clutter we might be able to remove. The questio

Re: Various cleanups: threads, STA_NANO

2019-01-23 Thread Eric S. Raymond via devel
Matthew Selsky : > > There is some point in listing known-good platforms (Linux is the > > biggie), but no point in trying to be exhaustive about it, because > > anybody with enough chops to build this on a Unix variant we don't > > know about is necessarily clued in enough to apply this creterion

Re: Various cleanups: threads, STA_NANO

2019-01-23 Thread Matthew Selsky via devel
On Wed, Jan 23, 2019 at 01:36:33AM -0800, Hal Murray wrote: > > e...@thyrsus.com said: > > In particular, some of our targets (some BSDs, I don't remember which) have > > only microsecond, not nanosecond resolution in the clock-adjustment calls. > > NetBSD and FreeBSD have NANO. > > Matthew Sel

Re: Various cleanups: threads, STA_NANO

2019-01-23 Thread Matthew Selsky via devel
On Wed, Jan 23, 2019 at 12:43:53AM -0500, Eric S. Raymond wrote: > That's because our actual support target is this: > > This software should build on any operating system conformant to > POSIX.1-2001 and ISO/IEC 9899:1999 (C99). In addition, the operating > system must have an ntp_a

Re: Various cleanups: threads, STA_NANO

2019-01-23 Thread Hal Murray via devel
e...@thyrsus.com said: > In particular, some of our targets (some BSDs, I don't remember which) have > only microsecond, not nanosecond resolution in the clock-adjustment calls. NetBSD and FreeBSD have NANO. Matthew Selsky suggested Solaris. Can anybody confirm that? Anybody know of others?

Re: Various cleanups: threads, STA_NANO

2019-01-22 Thread Eric S. Raymond via devel
Matthew Selsky via devel : > We added the STA_NANO checks to support systems that have some STA_ > symbols, but not all. See > https://lists.ntpsec.org/pipermail/vc/2017-December/003604.html In particular, some of our targets (some BSDs, I don't remember which) have only microsecond, not nanoseco

Re: Various cleanups: threads, STA_NANO

2019-01-22 Thread Hal Murray via devel
ibthr is a FreeBSD thing... Thanks. On a FreeBSD system, I just commented out these two lines: #ctx.check_cc(lib="thr", mandatory=False, # comment="thr library, required by some operating systems.") And it still builds and ends up using libthr. The man page

Re: Various cleanups: threads, STA_NANO

2019-01-22 Thread Matthew Selsky via devel
On Tue, Jan 22, 2019 at 07:37:18PM -0800, Hal Murray via devel wrote: > > We have various cruft associated with threads. Can we add POSIX threads to > our list of requirements? Or is it already included in POSIX.1-2001 and > ISO/IEC 9899:1999 (C99)? POSIX threads are optional

Various cleanups: threads, STA_NANO

2019-01-22 Thread Hal Murray via devel
We have various cruft associated with threads. Can we add POSIX threads to our list of requirements? Or is it already included in POSIX.1-2001 and ISO/IEC 9899:1999 (C99)? The idea is to remove HAVE_PTHREAD_H and HAVE_PTHREAD from config.h and remove most of wafhelpers/check_pthread.py

Re: seccomp ratsnets: crypto using threads

2017-11-26 Thread Kurt Roeckx via devel
_get_digestbyname () >from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.1 So you've upgraded for an OpenSSL 1.0.2 to 1.1.0. In 1.1.0 we made changes to the thread support, and assume applications might be using threads instead of relying on them to set things like locking up properly. This

Re: seccomp ratsnets: crypto using threads

2017-11-26 Thread Kurt Roeckx via devel
On Sat, Nov 25, 2017 at 05:09:02AM -0800, Hal Murray wrote: > > k...@roeckx.be said: > > This means that when we initialize a global variable we use the > > pthread_once() function, which internally uses the futex to do that. It's > > not using threads itself, it&#

Re: seccomp ratsnets: crypto using threads

2017-11-25 Thread Hal Murray via devel
k...@roeckx.be said: > This means that when we initialize a global variable we use the > pthread_once() function, which internally uses the futex to do that. It's > not using threads itself, it's just making sure that if you use threads > things work properly. Thanks. D

seccomp ratsnets: crypto using threads

2017-11-25 Thread Hal Murray via devel
An interesting tangle. I'm cleaning up odds and ends, including upgrading Raspberry Pi-s to Stretch. One of them dies with a SIGSYS trap. It's dieing on 240 which is futex. That's locks, only needed if you are using threads. This system was built with --disable_dns_l

Re: Killing threads

2016-09-27 Thread Gary E. Miller
Yo Daniel! On Tue, 27 Sep 2016 15:34:46 -0400 Daniel Franke wrote: > On 9/27/16, Gary E. Miller wrote: > > On Tue, 27 Sep 2016 15:16:55 -0400 > > Daniel Franke wrote: > >> But in that case, what's the point of using signals? > > > > So the user can also niely terminate a thread. Or so a s

Re: Killing threads

2016-09-27 Thread Daniel Franke
On 9/27/16, Gary E. Miller wrote: > On Tue, 27 Sep 2016 15:16:55 -0400 > Daniel Franke wrote: >> But in that case, what's the point of using signals? > > So the user can also niely terminate a thread. Or so a system shutdown > can nicely terminate the thread.Or if the msater thread already crash

Re: Killing threads

2016-09-27 Thread Eric S. Raymond
Hal Murray : > man 7 signal has a long list of what you can do in a signal handler. (all > other system calls are unsafe) > > Is there something similar that applies to killing threads? If so, where is > it documented? > > For example, if I kill a thread that is in the

Re: Killing threads

2016-09-27 Thread Hal Murray
> There are no such guarantees. Killing threads is basically never safe. Thanks. I should have looked at the man page. pthead_kill sends a signal. ... I didn't see a serious warning on that man page. If I didn't know about the longjmp mess, I probably wouldn't be suspicious

Re: Killing threads

2016-09-27 Thread Gary E. Miller
Yo Daniel! On Tue, 27 Sep 2016 15:16:55 -0400 Daniel Franke wrote: > On 9/27/16, Gary E. Miller wrote: > > There is more than one way to 'kill' a thread. The nice way is to > > 'kill -HUP', which politely tells the thread to commit seppuku in > > the cleanest way the thread knows how. > > S

Re: Killing threads

2016-09-27 Thread Gary E. Miller
Yo Daniel! On Tue, 27 Sep 2016 15:03:34 -0400 Daniel Franke wrote: > Hal's question was about *killing* threads. Even if a thread only 3 > ever makes thread-safe calls, it's still unsafe to kill, because > doing so can leave its mutexes locked and the data they control i

Re: Killing threads

2016-09-27 Thread Daniel Franke
On 9/27/16, Gary E. Miller wrote: > There is more than one way to 'kill' a thread. The nice way is to > 'kill -HUP', which politely tells the thread to commit seppuku in the > cleanest way the thread knows how. Sure, but the default signal handler for that just terminates the thread immediately

Re: Killing threads

2016-09-27 Thread Eric S. Raymond
Daniel Franke : > Hal's question was about *killing* threads. Even if a thread only 3 > ever makes thread-safe calls, it's still unsafe to kill, because doing so > can leave its mutexes locked and the data they control in an inconsistent > state. Oh, you're righ

Re: Killing threads

2016-09-27 Thread Daniel Franke
On Sep 27, 2016 15:00, "Eric S. Raymond" wrote: > > Hal Murray : > > man 7 signal has a long list of what you can do in a signal handler. (all > > other system calls are unsafe) > > > > Is there something similar that applies to killing threads? If so,

Re: Killing threads

2016-09-27 Thread Daniel Franke
On Sep 27, 2016 14:51, "Hal Murray" wrote: > > man 7 signal has a long list of what you can do in a signal handler. (all > other system calls are unsafe) > > Is there something similar that applies to killing threads? If so, where is > it documented? > > For e

Killing threads

2016-09-27 Thread Hal Murray
man 7 signal has a long list of what you can do in a signal handler. (all other system calls are unsafe) Is there something similar that applies to killing threads? If so, where is it documented? For example, if I kill a thread that is in the middle of malloc, are things guaranteed to work