netstat showing multiple lines for each listening socket
Hello, we have been running some BIND nameservers on Debian-based systems for many years. Until (including) Debian 10 with BIND 9.11.5, netstat always showed only one line per listening socket, e.g. tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named We noticed that with Debian 11 and 12 (BIND 9.16.48 / 9.18.24), netstat instead shows multiple (on some systems four, on others up to 20) completely identical lines for each listening socket, like this: tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named We wonder what is causing this and if this is intended behaviour? - Thomas  -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: netstat showing multiple lines for each listening socket
Hi Robert, it's the same PID for all lines, parent process is systemd. The lines in the netstat output are exact duplicates (same IP, port and PID). Other tools like ss show the same, so it's not a problem with netstat. It's the same bahaviour on different machines, some upgraded from Debian < 11 and others installed from scratch with Debian 11 or 12. I also set up a test VM and started BIND with the default configuration shipped with Debian. Same behaviour: All lines are shown twice. It looks like on machines with only two interfaces (lo / eth0) the lines are shown twice while on machines with more interfaces (active or not) there are up to 20 duplicate lines. - Thomas On 08.07.24 12:10, Robert Wagner wrote: Some diagnostics is needed. When you reboot, does it show it up multiple binds to the same port? Can your run netstat -tP to identify the process ID (are they the same or different). There may also be other options to provide more diagnostics. -Trying to determine if you are really binding the service four times to the same port or this is just a ghost in the netstat program... Most systems are designed to prevent binding multiple applications to the same ip/port, but a service can spawn multiple threads on the same ip/port. You may be seeing the threads and not unique service instances. Looking at the process ID, you may be able to track back to the root process and determine if these are just service threads. Robert Wagner From: bind-users on behalf of Thomas Hungenberg via bind-users Sent: Monday, July 8, 2024 4:52 AM To: bind-users@lists.isc.org Subject: netstat showing multiple lines for each listening socket This email originated from outside of TESLA Do not click links or open attachments unless you recognize the sender and know the content is safe. Hello, we have been running some BIND nameservers on Debian-based systems for many years. Until (including) Debian 10 with BIND 9.11.5, netstat always showed only one line per listening socket, e.g. tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named We noticed that with Debian 11 and 12 (BIND 9.16.48 / 9.18.24), netstat instead shows multiple (on some systems four, on others up to 20) completely identical lines for each listening socket, like this: tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 10.x.x.x:53 0.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named tcp0 0 127.0.0.1:530.0.0.0:* LISTEN 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 10.x.x.x:53 0.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named udp0 0 127.0.0.1:530.0.0.0:* 1234/named We wonder what is causing this and if this is intended behaviour? - Thomas  -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: netstat showing multiple lines for each listening socket
On 08.07.24 15:59, Lee wrote: How many cpus does your machine have? I'm running bind at home; not a whole lot of traffic to named so it seemed like all those threads were a waste. So pretend there's only one cpu: $ grep bind /etc/default/named # OPTIONS="-u bind " OPTIONS="-u bind -n 1" Thanks! I can confirm netstat and ss show only one line per socket when starting named with option "-n 1". However, according to the manpage there should be "*two* threads per each CPU present": = -n #cpus This option controls the number of CPUs that named assumes the presence of. If not specified, named tries to determine the number of CPUs present automatically; if it fails, a single CPU is assumed to be present. named creates two threads per each CPU present (one thread for receiving and sending client traffic and another thread for sending and receiving resolver traffic) and then on top of that a single thread for handling time-based events. = When running named without setting "-n" on a test VM with a single CPU assigned, I see two threads per socket which matches what the manpage says. When starting named with "-n 1" I would expect to see two threads as well but there is only one in the netstat / ss output. And on a small embedded system with a single CPU, it creates *four* threads per socket. Hmmm... - Thomas -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: netstat showing multiple lines for each listening socket
On 10.07.24 14:20, Tom Marcoen (EXT) wrote: My server has four (virtual; it runs on vSphere) CPUs and also shows four lines in `ss` output. The `ps` command shows the `-U` which - I assume - is set automatically triggered by the number of CPUs. # ps -elf | grep named 5 S named23769 1 9 80 0 - 251941 do_sig 07:12 ? 00:39:02 /usr/local/sbin/named -U4 -u named -c /usr/local/etc/namedb/named.conf The manpage says: -U #listeners This option tells named the number of #listeners worker threads to listen on, for incoming UDP packets on each address. If not specified, named calculates a default value based on the number of detected CPUs: 1 for 1 CPU, and the number of detected CPUs minus one for machines with more than 1 CPU. So if not specified, the value of "-U" should be set to 3 with four CPUs. Also, the parameter "-U" usually does not show up in the ps output if not specified. So in your case it looks more like named is specifically started with "-U4"? - Thomas -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
