--- Begin Message ---
Package: release.debian.org
Severity: normal
Tags: bullseye
User: release.debian....@packages.debian.org
Usertags: pu
X-Debbugs-Cc: r...@debian.org
This is a future unblock request before I upload
djbdns-1:1.05-13+deb11u1 to fix an RC bug in stable.
[ Reason ]
See #996807 for more information; in short, the tinydns server
program may stop responding to DNS queries because it has
reached the process data size limit. The reason is that recent
versions of glibc appear to need more memory, so the value of
the limit that has been enough for many years now is no longer
adequate. Note that this only happens on some Debian architectures.
[ Impact ]
The authoritative server (tinydns) from the djbdns package will
stop responding to queries, thus not fulfilling its basic purpose.
[ Tests ]
The attached debdiff also backports some improvements to
the autopkgtest suite, including a new test that sends a lot of
queries to the tinydns server to make sure it keeps responding.
[ Risks ]
The fix itself should not pose any risk at all, except maybe when
running the tinydns server in extremely resource-contstrained
environments when the additional 100K now available to the tinydns
process may eat into some other processes' memory. However, it is
my belief that people running tinydns in such environments will
use their own configuration files or at least be mindful of
changes in the Debian packages.
The changes to the autopkgtest suite should also not pose any
risk at all: they are mostly code clean-up and simplification.
If, however, it is your opinion that they are too extensive,
I can prepare another upload that drops them and only adds
the "raise the process data size limit" patch itself.
[ Checklist ]
[x] *all* changes are documented in the d/changelog
[x] I reviewed all changes and I approve them
[x] attach debdiff against the package in (old)stable
[x] the issue is verified as fixed in unstable
[ Changes ]
The data limit for the djbdns services is raised by an additional
100,000 bytes to reflect the programs' needs imposed by glibc.
The autopkgtest suite's Python code is cleaned up.
The autopkgtest suite performs an additional check: run a lot of
queries against a tinydns server to make sure it keeps responding.
Thanks in advance for looking at this, and keep up the great work!
G'luck,
Peter
diff -Nru djbdns-1.05/debian/changelog djbdns-1.05/debian/changelog
--- djbdns-1.05/debian/changelog 2020-07-28 00:03:21.000000000 +0300
+++ djbdns-1.05/debian/changelog 2022-02-26 19:29:35.000000000 +0200
@@ -1,3 +1,18 @@
+djbdns (1:1.05-13+deb11u1) bullseye; urgency=medium
+
+ * Add the 0011-datalimit patch to catch up with recent versions of
+ glibc generating larger executable files. Closes: #996807
+ * Several improvements to the Python tinytest autopkgtest tool:
+ - use "with subprocess.Popen()"
+ - simplify the command-line parsing a bit
+ - minor import statement fixes
+ - add a tox.ini file to make it easier to run static code checkers
+ - turn a class into a dataclass
+ - send a lot of queries to tinydns to make sure that the fix for
+ #996807 actually works
+
+ -- Peter Pentchev <r...@debian.org> Sat, 26 Feb 2022 19:29:35 +0200
+
djbdns (1:1.05-13) unstable; urgency=medium
* Do not even consider running tinydns during the package build.
diff -Nru djbdns-1.05/debian/patches/0011-datalimit.patch
djbdns-1.05/debian/patches/0011-datalimit.patch
--- djbdns-1.05/debian/patches/0011-datalimit.patch 1970-01-01
02:00:00.000000000 +0200
+++ djbdns-1.05/debian/patches/0011-datalimit.patch 2022-02-26
19:29:35.000000000 +0200
@@ -0,0 +1,38 @@
+Description: Raise the axfrdns, dnscache, and tinydns data limit.
+Bug-Debian: https://bugs.debian.org/996807
+Author: Peter Pentchev <r...@ringlet.net>
+Last-Update: 2021-11-13
+
+--- a/axfrdns-conf.c
++++ b/axfrdns-conf.c
+@@ -50,7 +50,7 @@
+
+ start("run");
+ outs("#!/bin/sh\nexec 2>&1\nexec envdir ./env sh -c '\n exec envuidgid ");
outs(user);
+- outs(" softlimit -d300000 tcpserver -vDRHl0 -x tcp.cdb -- \"$IP\" 53 ");
++ outs(" softlimit -d400000 tcpserver -vDRHl0 -x tcp.cdb -- \"$IP\" 53 ");
+ outs(auto_home); outs("/sbin/axfrdns\n'\n");
+ finish();
+ perm(0755);
+--- a/dnscache-conf.c
++++ b/dnscache-conf.c
+@@ -118,7 +118,7 @@
+ seed_addtime(); perm(0644);
+ seed_addtime(); start("env/CACHESIZE"); outs("1000000\n"); finish();
+ seed_addtime(); perm(0644);
+- seed_addtime(); start("env/DATALIMIT"); outs("3000000\n"); finish();
++ seed_addtime(); start("env/DATALIMIT"); outs("4000000\n"); finish();
+ seed_addtime(); perm(0644);
+ seed_addtime(); start("run");
+ outs("#!/bin/sh\nexec 2>&1\nexec <seed\nexec envdir ./env sh -c '\n exec
envuidgid "); outs(user);
+--- a/tinydns-conf.c
++++ b/tinydns-conf.c
+@@ -46,7 +46,7 @@
+
+ start("run");
+ outs("#!/bin/sh\nexec 2>&1\nexec envuidgid "); outs(user);
+- outs(" envdir ./env softlimit -d300000 ");
++ outs(" envdir ./env softlimit -d400000 ");
+ outs(auto_home); outs("/sbin/tinydns\n");
+ finish();
+ perm(0755);
diff -Nru djbdns-1.05/debian/patches/series djbdns-1.05/debian/patches/series
--- djbdns-1.05/debian/patches/series 2020-07-28 00:03:21.000000000 +0300
+++ djbdns-1.05/debian/patches/series 2022-02-26 19:29:35.000000000 +0200
@@ -7,3 +7,4 @@
0007-dnscache-merge-similar-outgoing-udp-packets.patch
0008-Cache-SOA-records.patch
0009-usr-sbin.patch
+0011-datalimit.patch
diff -Nru djbdns-1.05/debian/service/dnscache/conf/DATALIMIT
djbdns-1.05/debian/service/dnscache/conf/DATALIMIT
--- djbdns-1.05/debian/service/dnscache/conf/DATALIMIT 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/service/dnscache/conf/DATALIMIT 2022-02-26
19:29:35.000000000 +0200
@@ -1 +1 @@
-3000000
+4000000
diff -Nru djbdns-1.05/debian/service/tinydns/conf/DATALIMIT
djbdns-1.05/debian/service/tinydns/conf/DATALIMIT
--- djbdns-1.05/debian/service/tinydns/conf/DATALIMIT 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/service/tinydns/conf/DATALIMIT 2022-02-26
19:29:35.000000000 +0200
@@ -1 +1 @@
-3000000
+4000000
diff -Nru djbdns-1.05/debian/tests/pyproject.toml
djbdns-1.05/debian/tests/pyproject.toml
--- djbdns-1.05/debian/tests/pyproject.toml 1970-01-01 02:00:00.000000000
+0200
+++ djbdns-1.05/debian/tests/pyproject.toml 2022-02-26 19:29:35.000000000
+0200
@@ -0,0 +1,2 @@
+[tool.black]
+line-length = 79
diff -Nru djbdns-1.05/debian/tests/setup.cfg djbdns-1.05/debian/tests/setup.cfg
--- djbdns-1.05/debian/tests/setup.cfg 1970-01-01 02:00:00.000000000 +0200
+++ djbdns-1.05/debian/tests/setup.cfg 2022-02-26 19:29:35.000000000 +0200
@@ -0,0 +1,6 @@
+[flake8]
+ignore = E203,W503
+
+[mypy]
+python_version = 3.7
+strict = True
diff -Nru djbdns-1.05/debian/tests/tinytest/defs.py
djbdns-1.05/debian/tests/tinytest/defs.py
--- djbdns-1.05/debian/tests/tinytest/defs.py 2020-07-28 00:03:21.000000000
+0300
+++ djbdns-1.05/debian/tests/tinytest/defs.py 2022-02-26 19:29:35.000000000
+0200
@@ -4,7 +4,7 @@
import pathlib
import subprocess
-from typing import Dict, List, Union
+from typing import Dict, List, Union # noqa: H301
@dataclasses.dataclass(frozen=True)
diff -Nru djbdns-1.05/debian/tests/tinytest/__main__.py
djbdns-1.05/debian/tests/tinytest/__main__.py
--- djbdns-1.05/debian/tests/tinytest/__main__.py 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/tests/tinytest/__main__.py 2022-02-26
19:29:35.000000000 +0200
@@ -29,14 +29,14 @@
parser.add_argument(
"-b",
"--bindir",
- type=str,
+ type=pathlib.Path,
required=True,
help="the path to the djbdns user tools",
)
parser.add_argument(
"-s",
"--sbindir",
- type=str,
+ type=pathlib.Path,
required=True,
help="the path to the djbdns system tools",
)
@@ -60,11 +60,11 @@
subenv["LANGUAGE"] = "en"
return Config(
- bindir=pathlib.Path(args.bindir),
- sbindir=pathlib.Path(args.sbindir),
- skip_run_test=bool(args.skip_run_test),
+ bindir=args.bindir,
+ sbindir=args.sbindir,
+ skip_run_test=args.skip_run_test,
subenv=subenv,
- verbose=bool(args.verbose),
+ verbose=args.verbose,
)
@@ -80,8 +80,9 @@
if not cfg.skip_run_test:
test_run.test_tinydns_run(cfg, tempd)
+ test_run.test_tinydns_run_udp(cfg, tempd)
else:
- print("\n==== Skipping test_tinydns_run")
+ print("\n==== Skipping test_tinydns_run*")
print("\n==== All fine!")
diff -Nru djbdns-1.05/debian/tests/tinytest/test_conf.py
djbdns-1.05/debian/tests/tinytest/test_conf.py
--- djbdns-1.05/debian/tests/tinytest/test_conf.py 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/tests/tinytest/test_conf.py 2022-02-26
19:29:35.000000000 +0200
@@ -5,7 +5,7 @@
import pwd
import sys
-from typing import List, Union
+from typing import List, Union # noqa: H301
from . import defs
diff -Nru djbdns-1.05/debian/tests/tinytest/test_edit.py
djbdns-1.05/debian/tests/tinytest/test_edit.py
--- djbdns-1.05/debian/tests/tinytest/test_edit.py 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/tests/tinytest/test_edit.py 2022-02-26
19:29:35.000000000 +0200
@@ -1,10 +1,10 @@
"""Test the `tinydns-edit` tool and the generated data file."""
-import pathlib
import os
+import pathlib
import sys
-from typing import List, Union
+from typing import List, Union # noqa: H301
from . import defs
diff -Nru djbdns-1.05/debian/tests/tinytest/test_get.py
djbdns-1.05/debian/tests/tinytest/test_get.py
--- djbdns-1.05/debian/tests/tinytest/test_get.py 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/tests/tinytest/test_get.py 2022-02-26
19:29:35.000000000 +0200
@@ -5,7 +5,7 @@
import re
import sys
-from typing import List, Union
+from typing import List, Union # noqa: H301
from . import defs
diff -Nru djbdns-1.05/debian/tests/tinytest/test_run.py
djbdns-1.05/debian/tests/tinytest/test_run.py
--- djbdns-1.05/debian/tests/tinytest/test_run.py 2020-07-28
00:03:21.000000000 +0300
+++ djbdns-1.05/debian/tests/tinytest/test_run.py 2022-02-26
19:29:35.000000000 +0200
@@ -4,16 +4,31 @@
import enum
import os
import pathlib
+import socket
+import struct
import subprocess
import sys
import time
-from typing import Callable, Dict, List, Tuple, Union
+from typing import (
+ Callable, # noqa: H301
+ Dict,
+ List,
+ Tuple,
+ Union,
+ TYPE_CHECKING,
+)
from . import defs
from . import test_get
+if TYPE_CHECKING:
+ PopenStr = subprocess.Popen[str] # pylint: disable=unsubscriptable-object
+else:
+ PopenStr = subprocess.Popen
+
+
class ProcessFDType(str, enum.Enum):
"""File descriptor types, enough for this very limited test."""
@@ -54,22 +69,15 @@
files: List[ProcessFDInfo]
+@dataclasses.dataclass
class ParseProcessInfo:
"""Parse the output of `lsof -F` for our very limited testing needs."""
cfg: defs.Config
- cproc: Dict[str, str]
- cfile: Dict[str, str]
- cfiles: List[ProcessFDInfo]
- result: Dict[int, ProcessInfo]
-
- def __init__(self, cfg: defs.Config) -> None:
- """Initialize an empty object."""
- self.cfg = cfg
- self.cproc = {}
- self.cfile = {}
- self.cfiles = []
- self.result = {}
+ cproc: Dict[str, str] = dataclasses.field(default_factory=dict)
+ cfile: Dict[str, str] = dataclasses.field(default_factory=dict)
+ cfiles: List[ProcessFDInfo] = dataclasses.field(default_factory=list)
+ result: Dict[int, ProcessInfo] = dataclasses.field(default_factory=dict)
def finish_file_ipv4(self) -> ProcessFDInfo:
"""Store the information about a socket."""
@@ -188,6 +196,72 @@
return data.proto == "UDP" and data.address == ipaddr and data.port == 53
+def run_tinydns(
+ cfg: defs.Config,
+ tempd: pathlib.Path,
+ callback: Callable[[PopenStr, str], None],
+) -> None:
+ """Test the `tinydns` and `dnsq` programs."""
+ ipaddr = defs.RECORDS[0].address
+ svcdir = tempd / defs.SVCDIR
+ if not svcdir.is_dir():
+ sys.exit(f"Expected {svcdir} to be a directory")
+ os.chdir(svcdir)
+
+ with subprocess.Popen(
+ ["./run"],
+ shell=False,
+ env=cfg.subenv,
+ bufsize=0,
+ encoding=defs.MINENC,
+ stdout=subprocess.PIPE,
+ ) as proc:
+ try:
+ print(f"- spawned process {proc.pid}")
+ assert proc.stdout is not None
+ print("- waiting for the 'starting tinydns' line")
+ line = proc.stdout.readline()
+ print(f"- got line {line!r}")
+ if line != "starting tinydns\n":
+ sys.exit(f"Unexpected first line from tinydns: {line!r}")
+
+ lines = cfg.check_output(
+ [
+ "lsof",
+ "-a",
+ "-n",
+ "-P",
+ "-p",
+ str(proc.pid),
+ "-F",
+ "-i4udp:53",
+ ]
+ ).splitlines()
+ cfg.diag(f"lsof output: {lines!r}")
+ processes = ParseProcessInfo(cfg).parse(lines)
+ if not check_single_udp_socket(processes, proc.pid, ipaddr):
+ sys.exit(
+ f"tinydns should listen on a single UDP socket: "
+ f"{processes!r}"
+ )
+ print(f"- tinydns is listening at {ipaddr}:53")
+ callback(proc, ipaddr)
+ finally:
+ print("- we spawned a process, checking if it has exited")
+ res = proc.poll()
+ if res is None:
+ print(f"Terminating process {proc.pid}")
+ proc.terminate()
+ time.sleep(0.5)
+ res = proc.poll()
+ if res is None:
+ print(f"Killing process {proc.pid}")
+ proc.kill()
+
+ res = proc.wait()
+ print(f"Process {proc.pid}: exit code {res}")
+
+
def test_tinydns_run(cfg: defs.Config, tempd: pathlib.Path) -> None:
"""Test the `tinydns` and `dnsq` programs."""
print("\n==== test_tinydns_run")
@@ -199,40 +273,9 @@
if not dnsq.is_file() or not os.access(dnsq, os.X_OK):
sys.exit(f"Not an executable file: {dnsq}")
- ipaddr = defs.RECORDS[0].address
- svcdir = tempd / defs.SVCDIR
- if not svcdir.is_dir():
- sys.exit(f"Expected {svcdir} to be a directory")
- os.chdir(svcdir)
-
- proc = None
- try:
- proc = subprocess.Popen(
- ["./run"],
- shell=False,
- env=cfg.subenv,
- bufsize=0,
- encoding=defs.MINENC,
- stdout=subprocess.PIPE,
- )
- print(f"- spawned process {proc.pid}")
+ def test_dnsq(proc: PopenStr, ipaddr: str) -> None:
+ """Run `dnsq` against the specified `tinydns` instance."""
assert proc.stdout is not None
- print("- waiting for the 'starting tinydns' line")
- line = proc.stdout.readline()
- print(f"- got line {line!r}")
- if line != "starting tinydns\n":
- sys.exit(f"Unexpected first line from tinydns: {line!r}")
-
- lines = cfg.check_output(
- ["lsof", "-a", "-n", "-P", "-p", str(proc.pid), "-F", "-i4udp:53"]
- ).splitlines()
- cfg.diag(f"lsof output: {lines!r}")
- processes = ParseProcessInfo(cfg).parse(lines)
- if not check_single_udp_socket(processes, proc.pid, ipaddr):
- sys.exit(
- f"tinydns should listen on a single UDP socket: {processes!r}"
- )
- print(f"- tinydns is listening at {ipaddr}:53")
for rec in defs.RECORDS:
rdef = defs.TYPES[rec.rtype]
@@ -256,17 +299,68 @@
print(" - there should be a single line of output from tinydns")
line = proc.stdout.readline()
cfg.diag(f"line: {line!r}")
- finally:
- if proc is not None:
- print("- we spawned a process, checking if it has exited")
- res = proc.poll()
- if res is None:
- print(f"Terminating process {proc.pid}")
- proc.terminate()
- time.sleep(0.5)
- res = proc.poll()
- if res is None:
- print(f"Killing process {proc.pid}")
- proc.kill()
- res = proc.wait()
- print(f"Process {proc.pid}: exit code {res}")
+
+ run_tinydns(cfg, tempd, test_dnsq)
+
+
+def test_tinydns_run_udp(cfg: defs.Config, tempd: pathlib.Path) -> None:
+ """Test the `tinydns` program by sending UDP packets."""
+ print("\n==== test_tinydns_run_udp")
+ if os.geteuid() != 0:
+ print("- not running as root, skipped")
+ return
+
+ def test_dnsq(proc: PopenStr, ipaddr: str) -> None:
+ """Run `dnsq` against the specified `tinydns` instance."""
+ assert proc.stdout is not None
+
+ pkt = struct.pack(
+ ">HHHHHH8p4p1pHH",
+ 17,
+ 0,
+ 1,
+ 0,
+ 0,
+ 0,
+ "example".encode("us-ascii"),
+ "com".encode("us-ascii"),
+ b"",
+ 1,
+ 1,
+ )
+ print(f"- query packet: {pkt!r}")
+
+ sock = socket.socket(
+ socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP
+ )
+ sock.connect((ipaddr, 53))
+
+ for idx in range(1000000):
+ pkt = struct.pack(">H", (idx + 17) % 65536) + pkt[2:]
+ sent = sock.send(pkt)
+ if sent != len(pkt):
+ sys.exit(f"Only sent {sent} of {len(pkt)} bytes")
+
+ line = proc.stdout.readline()
+ if "+" not in line:
+ sys.exit(
+ f"Query {idx + 1}: expected a '+' in "
+ f"the tinydns log line {line!r}"
+ )
+
+ resp = sock.recv(4096)
+ if resp[:2] != pkt[:2]:
+ sys.exit(
+ f"Query {idx + 1}: unexpected ID: "
+ f"query {pkt!r} response {resp!r}"
+ )
+ if resp[4:8] != b"\x00\x01\x00\x01":
+ sys.exit(
+ f"Query {idx + 1}: unexpected query/response count: "
+ f"query {pkt!r} response {resp!r}"
+ )
+
+ if (idx % 10000) == 9999:
+ print(f"- sent {idx + 1} queries to {ipaddr}")
+
+ run_tinydns(cfg, tempd, test_dnsq)
diff -Nru djbdns-1.05/debian/tests/tox.ini djbdns-1.05/debian/tests/tox.ini
--- djbdns-1.05/debian/tests/tox.ini 1970-01-01 02:00:00.000000000 +0200
+++ djbdns-1.05/debian/tests/tox.ini 2022-02-26 19:29:35.000000000 +0200
@@ -0,0 +1,55 @@
+[tox]
+envlist =
+ black
+ pep8
+ pep8h
+ mypy
+ pylint
+skipsdist = True
+
+[defs]
+pyfiles =
+ tinytest
+
+[testenv:black]
+skipinstall = True
+deps =
+ black >= 21b0, < 22b0
+commands =
+ black --check {[defs]pyfiles}
+
+[testenv:black-reformat]
+skipinstall = True
+deps =
+ black >= 21b0, < 22b0
+commands =
+ black {[defs]pyfiles}
+
+[testenv:pep8]
+skipinstall = True
+deps =
+ flake8
+commands =
+ flake8 {[defs]pyfiles}
+
+[testenv:pep8h]
+skipinstall = True
+deps =
+ flake8
+ hacking >= 4
+commands =
+ flake8 {[defs]pyfiles}
+
+[testenv:mypy]
+skipinstall = True
+deps =
+ mypy
+commands =
+ mypy {[defs]pyfiles}
+
+[testenv:pylint]
+skipinstall = True
+deps =
+ pylint
+commands =
+ pylint {[defs]pyfiles}
signature.asc
Description: PGP signature
--- End Message ---
