On Mon, 14 Mar 2022 23:58:37 +0100 "Steinar H. Gunderson" <[email protected]>
wrote:
Package: libunbound8
Version: 1.13.1-1
Severity: normal
Hi,
We were investigating a performance regression in production that crept
in at some point (we noticed by accident that something had become very slow
and started investigating). It turns out the culprit was libunbound; we do
a series of DNS lookups against localhost using ub_resolve(), and each of them
now takes a bit over 6 ms, which is huge for sending a UDP packet and getting
an answer from cache.
It turns out that some of this is because libunbound in Debian is now built
against libnettle (it wasn't when we built the system). The libnettle code in
util/random.c goes through a very slow reseeding phase; and worse, it does it
for both creating a context (we create a new one for each call, because
Reasons(TM)) and for each and every query (because ub_resolve() starts its own
worker, which reseeds). This reseeding is responsible for 60% of the CPU usage
or so, according to perf.
Wug. This is awful.
According to pkg-config --libs libunbound, it seems one links to OpenSSL anyway,
This, if true, is a bug. libunbound itself does not link to openssl.
On my system with 1.13.1-1 version of libunbound, pkg-config does not
list openssl libs for it:
$ pkg-config --libs libunbound
-lunbound
so perhaps the simplest solution is to stop linking against libnettle?
Well, you already noted above the "Reasons(TM)". If you want to know
the particular reason for this, see #828699 which forced us to build
libunbound with nettle. It indeed is "Reasons(TM)" :)
Maybe someone from the Debian DNS team can add something here.
But it looks like we're sort of stuck here and should address
this on the nettle side. Is it at all possible? Or should
we reopen #828699?
BTW, Robert, do you remember why you had to split libunbound
build? The changelog/comments says this build is here to have
less dependencies, - which dependencies these are?
Optionally, one can use getentropy() (which calls getrandom()) unconditionally
on Linux, at least with modern kernels.
Doing the latter, and also reusing contexts (which is a pain for us, and I
don't think we had to do it before?) takes it down to a more reasonable 0.5 ms.
Well, these are workarounds, it seems...
Thanks!
/mjt
