Re: remote ssh login fail "Authenticator provider $SSH_SK_PROVIDER did not resolve" (macOS)
Thus said Sylvain Saboua on Sat, 28 Dec 2024 07:07:18 +0100: > debug1: Connecting to saboua.xyz port 22. > ssh: connect to host saboua.xyz port 22: Operation timed out Have you observed with tcpdump what traffic is flowing? Also, there was a recent discussion about SSH and Mac OS which may be of interest, though not likely the same cause since you're hitting timeouts: https://marc.info/?t=17342023562&r=1&w=2 Andy
acme-client challenges
Hey, misclings,
I'm standing up a new domain on a server that's been deployed for a
while. Big thanks to Gilles Chehade for his guide (
https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/
), as well as for smtpd. As part of this, I'm finally trying to adopt
TLS everywhere. So I have everything that I need running using
self-signed certs (it's a vanity domain, so all the users can be relied
upon to accept self-signed certs), but that's increasingly less
acceptable.
Short form: with httpd working, contents of /var/www/acme reachable
(but not the empty dir) (as-installed perms on everything in /var/www),
stock configuration (and reduced forms of it to simplify), A records
pointing to simmonpatch.com and tserig.simmonpatch.com (the main and
one alternative used in minimal test), default/simplified configuration
of /etc/acme-client.conf and /etc/httpd.conf, using the let's encrypt
staging server, I get this:
$ doas acme-client -vvv simmonpatch.com
acme-client: /etc/ssl/private/leo-simmonpatch.com.key: loaded domain key
acme-client: /etc/acme/letsencrypt-staging-privkey.pem: loaded account
key
acme-client: https://acme-staging.api.letsencrypt.org/directory:
directories
acme-client: https://acme-staging.api.letsencrypt.org/directory: bad
comm
acme-client: bad exit: netproc(39958): 1
This is a later run; in earlier runs I removed keys after every
failure, until I realized that keys (and their names) are cheap. But
this is the same error seen before, multiple times. The extra v (which
has no effect) was there to see if maybe that would help. I can't find
any logs, and there don't seem to be any other switches/knobs to turn
up verbosity more. As it is, it's rather cryptic. 'directories' may be
an error, or just informational (like the two key loading lines, which
are not errors). 'bad comm' is bad error messaging (not from
acme-client, I think, but from the staging server, which is reporting
what it got).
I'm hoping someone is going to point out something that will make me
feel stupid, but that I can't see until it's pointed out.
Minimal httpd.conf and acme-client.conf:
server "simmonpatch.com" {
listen on * port 80
alias "www.simmonpatch.com"
location "/.well-known/acme-challenge/*" {
root "/acme"
request strip 2
}
location * {
block return 302 "https://$HTTP_HOST$REQUEST_URI";
}
}
(tested with an index.html in /var/www/acme: it's reachable with only
this, though 403 for /.well-known/acme-challenge/, which is probably
the point, so I made sure it was empty when invoking acme-client)
authority letsencrypt-staging {
api url "https://acme-staging.api.letsencrypt.org/directory";
account key "/etc/acme/letsencrypt-staging-privkey.pem"
}
domain simmonpatch.com {
alternative names { tserig.simmonpatch.com,
mail.simmonpatch.com, www.simmonpatch.com }
domain key "/etc/ssl/private/leo-simmonpatch.com.key"
domain certificate "/etc/ssl/leo-simmonpatch.com.crt"
domain full chain certificate
"/etc/ssl/leo-simmonpatch.com.fullchain.pem"
sign with letsencrypt-staging
}
(everything else got removed to test; yay /etc/examples!)
No records (not available from hosting provider), no DNSSEC (I
could do that, but it's said to be not required and to sometimes cause
problems). httpd works with the tls bloc installed pointing at my
self-signed cert. I got a different error once when the request used
the same key+cert name as self-signed.
Also (double-checking logs): there is no attempted connection from
anyone except me (my ip address) trying the acme-challenge url.
Presumably, then, let's encrypt fails before that?
Thanks for reading this far, at least. :-)
Amy!
--
Amelia A. Lewisamyzing {at} talsever.com
It's odd that people think of programming as precise and methodical.
*Computers* are precise and methodical.
Hacking is something you do with a gleeful laugh.
-- Paul Graham, "The Word 'Hacker'"
Re: acme-client challenges
On Tue, 31 Dec 2024 01:44:15 +0100, Amelia A Lewis wrote: > > $ doas acme-client -vvv simmonpatch.com > acme-client: /etc/ssl/private/leo-simmonpatch.com.key: loaded domain key > acme-client: /etc/acme/letsencrypt-staging-privkey.pem: loaded account > key > acme-client: https://acme-staging.api.letsencrypt.org/directory: > directories > acme-client: https://acme-staging.api.letsencrypt.org/directory: bad > comm > acme-client: bad exit: netproc(39958): 1 > can you run host acme-staging.api.letsencrypt.org on the same machine? I'm asking because as far as I understand the code at 3:20 am, the only case when you see 'bad comm' without additional errors / warnings is DNS related. I mean that acme-client can't resolve acme-staging.api.letsencrypt.org. -- wbr, Kirill
Re: acme-client challenges
Quick response, thank you!
On Tue, 31 Dec 2024 03:22:14 +0100, Kirill A. Korinsky wrote:
> On Tue, 31 Dec 2024 01:44:15 +0100,
> Amelia A Lewis wrote:
>>
>> $ doas acme-client -vvv simmonpatch.com
>> acme-client: /etc/ssl/private/leo-simmonpatch.com.key: loaded domain key
>> acme-client: /etc/acme/letsencrypt-staging-privkey.pem: loaded account
>> key
>> acme-client: https://acme-staging.api.letsencrypt.org/directory:
>> directories
>> acme-client: https://acme-staging.api.letsencrypt.org/directory: bad
>> comm
>> acme-client: bad exit: netproc(39958): 1
>>
>
> can you run host acme-staging.api.letsencrypt.org on the same machine?
$ host acme-staging.api.letsencrypt.org
Host acme-staging.api.letsencrypt.org not found: 3(NXDOMAIN)
Well, that seems dispositive.
$ dig acme-staging.api.letsencrypt.org
; <<>> dig 9.10.8-P1 <<>> acme-staging.api.letsencrypt.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 46315
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;acme-staging.api.letsencrypt.org. IN A
;; AUTHORITY SECTION:
letsencrypt.org.1784IN SOA owen.ns.cloudflare.com.
dns.cloudflare.com. 2360987238 1 2400 604800 1800
;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Dec 30 21:51:48 EST 2024
;; MSG SIZE rcvd: 123
H. okay. That's a confirmation.
> I'm asking because as far as I understand the code at 3:20 am, the only case
> when you see 'bad comm' without additional errors / warnings is DNS related.
Ah. I went looking in netproc.c (after I sent the email; should've
looked first), and localized it to nfreq() < 0, but couldn't focus
enough after a day working to figure which of the return -1's in there
were the bad ones. Which reminds me: I owe the devs an apology (but
"comm" is unnecessarily cryptic; google keeps suggesting that I must
mean "bad community" and while I suspect it ought to mean "bad command"
or "bad communication channel" or something like that, I'm not certain:
"bad connection" [sic]?). But it is an acme-client bit, and it was rude
of me to snark on it. Teach me to check the code before asking
questions.
> I mean that acme-client can't resolve acme-staging.api.letsencrypt.org.
Seems so. Neither acme-staging nor acme.
$ host -t ns letsencrypt.org
letsencrypt.org name server owen.ns.cloudflare.com.
letsencrypt.org name server vera.ns.cloudflare.com.
acme.api.letsencrypt.org is also nxdomain.
host acme{,-staging}.api.letsencrypt.org vera.ns.cloudflare.com ditto.
Pretty much nx-ed. Lemme try a couple things. Same from other server in
colo (they're sharing the same lookup mechanism, not dispositive).
Okay, trying host acme.api.letsencrypt.org $nameserver and substituting
the IP address of a nameserver selected from a list of public resolving
servers in unbound.conf (an older copy has a longer list than new
installs) remains consistent. NXDOMAIN from everywhere I can start
from. Is there a problem at le.o? Just me? But if it's just me, then
directly querying 8.8.8.8 or similar from my home machine _should_
resolve, no? Not understanding how it can be filtered for both my home
machines checking public resolvers and for the servers in colo, but not
the rest of the world.
FWIW, letsencrypt.org responds positively, and I've been trying to
browse their docs since yesterday for something that looked like this
problem. But no one else has reported it? Or at least the available
hits for a search on 'acme-client "directories" "bad comm" -ai' turns
up nothing really pointing at dns (or else I missed it).
But wait! I found some curl commands on the net that someone offered
for debugging, which downloaded json from a staging server, on my
machine.
host acme-staging-v02.api.letsencrypt.org
acme-staging-v02.api.letsencrypt.org is an alias for
staging.api.letsencrypt.org.
staging.api.letsencrypt.org is an alias for
56a5f4b0bc8146689ec3e272c43525f9.pacloudflare.com.
56a5f4b0bc8146689ec3e272c43525f9.pacloudflare.com has address
172.65.46.172
56a5f4b0bc8146689ec3e272c43525f9.pacloudflare.com has IPv6 address
2606:4700:60:0:f41b:d4fe:4325:6026
Ah, now *that's* encouraging.
$ host staging.api.letsencrypt.org
staging.api.letsencrypt.org is an alias for
56a5f4b0bc8146689ec3e272c43525f9.pacloudflare.com.
56a5f4b0bc8146689ec3e272c43525f9.pacloudflare.com has address
172.65.46.172
56a5f4b0bc8146689ec3e272c43525f9.pacloudflare.com has IPv6 address
2606:4700:60:0:f41b:d4fe:4325:6026
lessee, delete an 'acme-' ...
$ doas acme-client -vv simmonpatch.com
acme-client: /etc/acme/letsencrypt-staging-privkey.pem: loaded account
key
acme-client: /etc/ssl/private/leo-simmonpatch.com.key: loaded domain key
acme-client: https://staging.api.letsencrypt.org/directory: directories
acme-client: staging.api.letsencrypt.org: DNS: 172.65.46.172
acme-client: 172.65.46.172: tl
Re: acme-client challenges
On Mon, Dec 30, 2024 at 07:44:15PM -0500, Amelia A Lewis wrote:
> authority letsencrypt-staging {
> api url "https://acme-staging.api.letsencrypt.org/directory";
> account key "/etc/acme/letsencrypt-staging-privkey.pem"
> }
>
> domain simmonpatch.com {
> alternative names { tserig.simmonpatch.com,
> mail.simmonpatch.com, www.simmonpatch.com }
> domain key "/etc/ssl/private/leo-simmonpatch.com.key"
> domain certificate "/etc/ssl/leo-simmonpatch.com.crt"
> domain full chain certificate
> "/etc/ssl/leo-simmonpatch.com.fullchain.pem"
> sign with letsencrypt-staging
> }
According to /etc/examples/acme-client.conf,
authority letsencrypt-staging {
api url "https://acme-staging-v02.api.letsencrypt.org/directory";
account key "/etc/acme/letsencrypt-staging-privkey.pem"
}
and acme-staging-v02.api.letsencrypt.org does resolve.
Re: acme-client challenges
On 2024-12-30 19:44 -05, Amelia A Lewis wrote: > acme-client: https://acme-staging.api.letsencrypt.org/directory: bad > comm > acme-client: bad exit: netproc(39958): 1 your acme-client.conf is about 5.5 years too old: https://cvsweb.openbsd.org/src/etc/examples/acme-client.conf#rev1.2 The correct staging url is https://acme-staging-v02.api.letsencrypt.org/directory If your acme-client is the same vintage you probably need to update, I don't think it will be able to talk to a v2 api endpoint. -- In my defence, I have been left unsupervised.
Re: acme-client challenges
updating with a little more info on most-recent error.
On Mon, 30 Dec 2024 22:54:52 -0500, Amelia A Lewis wrote:
[snip]
> lessee, delete an 'acme-' ...
>
> $ doas acme-client -vv simmonpatch.com
> acme-client: /etc/acme/letsencrypt-staging-privkey.pem: loaded account
> key
> acme-client: /etc/ssl/private/leo-simmonpatch.com.key: loaded domain key
> acme-client: https://staging.api.letsencrypt.org/directory: directories
> acme-client: staging.api.letsencrypt.org: DNS: 172.65.46.172
> acme-client: 172.65.46.172: tls_write: name
> `staging.api.letsencrypt.org' not present in server certificate
> acme-client: 172.65.46.172: tls_read: name
> `staging.api.letsencrypt.org' not present in server certificate
> acme-client: https://staging.api.letsencrypt.org/directory: bad comm
> acme-client: bad exit: netproc(18286): 1
>
[snip some more; i talk too much]
>
> Thanks for the quick reply and pointers! Have you any idea what the
> tls_write tls_read errors are? They're not triggering off pretend pear
> x1 and bogus broccoli x2 are they?
Call stack for the tls_read/tls_write bit:
netproc.c/nreq() -> http.c/http_get() -> http.c/http_alloc() \
-> http.c/dotlsread() -> http.c/tls_read()
-> http.c/dotlswrite() -> http.c/tls_write()
There are three calls of http_get() in netproc, one directly in nreq()
(most likely?, line 203), and two more in sreq(), which isn't called by
nreq() directly, but is called ten times by various dosomething()
functions, and it's too late for me to continue tracing (prolly easier
to instrument, but reading (somebody else's) code does make it easier
to sleep).
I'm not familiar enough with certificate contents and the protocol's
expectations of contents to decipher which server is supposed to have
staging.api in it and doesn't. Prolly the one delivered by staging.api
to identify itself? which seems ... weird. One would expect a server
providing certificate chains to remember to add its own link to its own
cert chain. Very absent minded to forget such a thing, like getting all
dressed up to go out for dinner, only to realize on arrival that one is
not wearing shoes. Not impossible, but rather unexpected. And with that
horribly strained analogy, I'm out for the night.
Amy!
Amelia A. Lewisamyzing {at} talsever.com
Love?
A joke, that. Love was the problem, not the solution. Being hit by a
car was better than love.
-- Steven Brust, PJF, "Cowboy Feng's Space Bar and Grille"
