Issue running "dig txt rs.dns-oarc.net" on 9.12
I'm sure I'm doing something wrong, but for the life of me I can't figure out
what. I'm running named 9.12 in a simple recursive setup (built from source on
CentOS 7).
In named.conf I've set:
dnssec-enable yes;
dnssec-validation auto;
When I try to run "dig txt rs.dns-oarc.net" I get SERVFAIL. The logs show:
validating rs.dns-oarc.net/CNAME: starting
validating rs.dns-oarc.net/CNAME: attempting insecurity proof
validating rs.dns-oarc.net/CNAME: checking existence of DS at 'net'
validating net/DS: starting
validating net/DS: attempting positive response validation
validating net/DS: keyset with trust secure
validating net/DS: verify rdataset (keyid=41824): success
validating net/DS: marking as secure, noqname proof not needed
validating rs.dns-oarc.net/CNAME: in dsfetched2: success
validating rs.dns-oarc.net/CNAME: resuming proveunsecure
validating rs.dns-oarc.net/CNAME: checking existence of DS at 'dns-oarc.net'
validating dns-oarc.net/DS: starting
validating dns-oarc.net/DS: attempting positive response validation
validating net/DNSKEY: starting
validating net/DNSKEY: attempting positive response validation
validating net/DNSKEY: verify rdataset (keyid=35886): success
validating net/DNSKEY: marking as secure (DS)
validating dns-oarc.net/DS: in fetch_callback_validator
validating dns-oarc.net/DS: keyset with trust secure
validating dns-oarc.net/DS: resuming validate
validating dns-oarc.net/DS: verify rdataset (keyid=25733): success
validating dns-oarc.net/DS: marking as secure, noqname proof not needed
validating rs.dns-oarc.net/CNAME: in dsfetched2: success
validating rs.dns-oarc.net/CNAME: resuming proveunsecure
validating rs.dns-oarc.net/CNAME: checking existence of DS at 'rs.dns-oarc.net'
validating rs.dns-oarc.net/CNAME: continuing validation would lead to deadlock:
aborting validation
validating rs.dns-oarc.net/CNAME: deadlock found (create_fetch)
Jan 26 15:06:59 red named[3036]: no valid RRSIG resolving
'rs.dns-oarc.net/TXT/IN': 64.191.0.133#53
However if I run "dig txt rs.dns-oarc.net +trace" and then "dig txt
rs.dns-oarc.net" the query completes as expected. It continues to complete as
expected until I restart named.
If I alter named.conf to say
dnssec-enable yes;
dnssec-validation yes;
The running "dig txt rs.dns-oarc.net" works immediately, but of course that
breaks verification of the root zone, so DNSSEC is worthless.
My named.conf is super simple:
key "rndc-key" {
algorithm hmac-sha256;
secret "$KEY";
};
controls {
inet 127.0.0.1 port 953
allow { 127.0.0.1; } keys { "rndc-key"; };
};
acl "NNEX" {
127.0.0.1;
aaa.bbb.ccc.0/22;
};
options {
listen-on port 53 { 127.0.0.1; };
listen-on port 53 { aaa.bbb.ccc.d; };
directory "/var/named";
dnssec-enable yes;
dnssec-validation auto;
allow-recursion { nnex; };
allow-query { nnex; };
recursion yes;
};
logging {
channel dnssec_log {
file "/var/log/dnssec.log";
severity debug 3;
};
category dnssec { dnssec_log; };
};
Thank you,
-Nick
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
from this list
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
RE: Issue running "dig txt rs.dns-oarc.net" on 9.12
Good thought but no luck, it doesn’t matter how many times I run “dig txt rs.dns-oarc.net” or how long I wait it continues to SERVFAIL until I run "dig txt rs.dns-oarc.net +trace" Interestingly I've found that running "dig txt dns-oarc.net +trace" isn't enough to fix it, I actually have to run "dig txt rs.dns-oarc.net +trace" before things start working. [...] There's an insecure delegation (NS set, and NSEC proving the nonexistence of a DS set) from dns-oarc.net to rs.dns-oarc.net. However, there's disagreement between the parent and child about what name servers actually serve rs.dns-oarc.net, and at least some of them are refusing to answer TCP. It's likely your recursive server is, for whatever reason, being drawn to the ones failing to respond, and not getting good data elsewhere fast enough to answer your query. [...] I agree, from my limited understanding this seems to describe what is happening well. The thing I'm wondering is why? I'm running older visions of named (9.9.4, yum provided RPM on CentOS 6) that seem immune to this issue. I've been digging through release notes and can't find any setting that has changed between the versions that might explain it (I know 9.9.4 to 9.12 is a big jump, so I'm sure I'm missing something) Thank you! -Nick ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: Issue running "dig txt rs.dns-oarc.net" on 9.12
[…] If you want to understand why your resolver is failing, again I'd have a look at the 'resolver' log channel. It should have some detail about what's resulting in the SERVFAIL message. […] I took a look at the ‘resolver’ log channel. I didn’t find any useful information there, just: fetch: rs.dns-oarc.net/TXT fetch: sns-pb.isc.org/A fetch: ns.isc.afilias-nst.info/A fetch: net/DS fetch: dns-oarc.net/DS fetch: net/DNSKEY I started trying different releases and found this query works consistently for me all the way up to bind-9.12.0rc1. As soon as I try bind-9.12.0rc3 the queries start failing. I’m using the exact same config and server for both the working rc1 and not working rc3 (both complied from source). Any thoughts on any differences between RC1 and RC3 that might explain this or any other logs I should be checking? The ‘resolver’ log channel on rc1 (which works) shows: fetch: rs.dns-oarc.net/TXT fetch: sns-pb.isc.org/A fetch: ns.isc.afilias-nst.info/A fetch: net/DS fetch: dns-oarc.net/DS fetch: net/DNSKEY fetch: rs.dns-oarc.net/DS fetch: dns-oarc.net/DNSKEY fetch: rst.x487.rs.dns-oarc.net/TXT fetch: rst.x461.x487.rs.dns-oarc.net/TXT fetch: rst.x466.x461.x487.rs.dns-oarc.net/TXT Looking at the ‘dnssec’ log channel I see this on RC1: validating rs.dns-oarc.net/CNAME: starting validating rs.dns-oarc.net/CNAME: attempting insecurity proof validating rs.dns-oarc.net/CNAME: checking existence of DS at 'net' validating net/DS: starting validating net/DS: attempting positive response validation validating net/DS: keyset with trust secure validating net/DS: verify rdataset (keyid=41824): success validating net/DS: marking as secure, noqname proof not needed validating rs.dns-oarc.net/CNAME: in dsfetched2: success validating rs.dns-oarc.net/CNAME: resuming proveunsecure validating rs.dns-oarc.net/CNAME: checking existence of DS at 'dns-oarc.net' validating dns-oarc.net/DS: starting validating dns-oarc.net/DS: attempting positive response validation validating net/DNSKEY: starting validating net/DNSKEY: attempting positive response validation validating net/DNSKEY: verify rdataset (keyid=35886): success validating net/DNSKEY: marking as secure (DS) validating dns-oarc.net/DS: in fetch_callback_validator validating dns-oarc.net/DS: keyset with trust secure validating dns-oarc.net/DS: resuming validate validating dns-oarc.net/DS: verify rdataset (keyid=25733): success validating dns-oarc.net/DS: marking as secure, noqname proof not needed validating rs.dns-oarc.net/CNAME: in dsfetched2: success validating rs.dns-oarc.net/CNAME: resuming proveunsecure validating rs.dns-oarc.net/CNAME: checking existence of DS at 'rs.dns-oarc.net' validating rs.dns-oarc.net/DS: starting validating rs.dns-oarc.net/DS: attempting negative response validation validating dns-oarc.net/SOA: starting validating dns-oarc.net/SOA: attempting positive response validation validating dns-oarc.net/DNSKEY: starting validating dns-oarc.net/DNSKEY: attempting positive response validation validating dns-oarc.net/DNSKEY: verify rdataset (keyid=20899): success validating dns-oarc.net/DNSKEY: marking as secure (DS) validating dns-oarc.net/SOA: in fetch_callback_validator validating dns-oarc.net/SOA: keyset with trust secure validating dns-oarc.net/SOA: resuming validate validating dns-oarc.net/SOA: verify rdataset (keyid=12093): success validating dns-oarc.net/SOA: marking as secure, noqname proof not needed validating rs.dns-oarc.net/DS: in authvalidated validating rs.dns-oarc.net/DS: resuming nsecvalidate validating rs.dns-oarc.net/NSEC: starting validating rs.dns-oarc.net/NSEC: attempting positive response validation validating rs.dns-oarc.net/NSEC: keyset with trust secure validating rs.dns-oarc.net/NSEC: verify rdataset (keyid=12093): success validating rs.dns-oarc.net/NSEC: marking as secure, noqname proof not needed validating rs.dns-oarc.net/DS: in authvalidated validating rs.dns-oarc.net/DS: looking for relevant NSEC validating rs.dns-oarc.net/DS: nsec proves name exists (owner) data=0 validating rs.dns-oarc.net/DS: resuming nsecvalidate validating rs.dns-oarc.net/DS: nonexistence proof(s) found validating rs.dns-oarc.net/CNAME: in dsfetched2: ncache nxrrset validating rs.dns-oarc.net/CNAME: marking as answer (dsfetched2) validating rst.x476.rs.dns-oarc.net/CNAME: starting validating rst.x476.rs.dns-oarc.net/CNAME: attempting insecurity proof validating rst.x476.rs.dns-oarc.net/CNAME: checking existence of DS at 'net' validating rst.x476.rs.dns-oarc.net/CNAME: checking existence of DS at 'dns-oarc.net' validating rst.x476.rs.dns-oarc.net/CNAME: checking existence of DS at 'rs.dns-oarc.net' validating rst.x476.rs.dns-oarc.net/CNAME: marking as answer (proveunsecure (4)) validating rst.x461.x476.rs.dns-oarc.net/CNAME: starting validating rst.x461.x476.rs.dns-oarc.net/CNAME: attempting insecurity proof validating rst.x461.x476.rs.dns-oarc.net/CNAME: checking existence of DS at 'net' validating rst.x4
RE: Issue running "dig txt rs.dns-oarc.net" on 9.12
Just wanted to follow up 9.12.1b1 fixes this issue for me. Thanks, -Nick ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: Issue running "dig txt rs.dns-oarc.net" on 9.12
I apologize, I sent the previous email in error. This is what I get for having so many revisions of named sitting around on one server, I was not running 9.12.1b1 when I tested this. I will be more careful in the future. I just tested this with 9.12.1b1 and it still fails, the same as 9.12. I do understand that the function of rs.dns-oarc.net is to test things that can no longer be tested in 9.12, but regardless I should still be able to resolve the nameservers for rs.dns-oarc.net, right? Is anyone on 9.12 able to do this? Is this just my problem? Thank you again, -Nick -Original Message- From: NNEX Support Sent: Thursday, February 22, 2018 8:21 AM To: 'bind-users@lists.isc.org' Subject: RE: Issue running "dig txt rs.dns-oarc.net" on 9.12 Just wanted to follow up 9.12.1b1 fixes this issue for me. Thanks, -Nick ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: Issue running "dig txt rs.dns-oarc.net" on 9.12
I'm sorry to keep replying to myself but I believe I've found the line of code that is causing this issue. Looking at validator.c, in the check_deadlock function, 9.12.0rc1 says: ... if (parent->event != NULL && parent->event->type == type && dns_name_equal(parent->event->name, name) && ... 9.12.0rc3 and above says: ... if (parent->event != NULL && (parent->event->type == type || parent->event->type == dns_rdatatype_cname) && dns_name_equal(parent->event->name, name) && ... By removing "parent->event->type == dns_rdatatype_cname)" (and adjusting the rest of the if statement appropriately) the query "dig ns rs.dns-oarc.net" works. I see this commit related to this line of code: https://gitlab.isc.org/isc-projects/bind9/commit/2b51d5874c49ac823890b88824290fbf1c18f2cc I'm sure this line of code is important, otherwise it wouldn't be there and I don't know enough to be removing random bits of code, so of course I'd never run this in production. Still I want to understand why this is happening and if it’s a bug or me not understanding DNS properly. As always thank you for your time! -Nick ___ Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
RE: Issue running "dig txt rs.dns-oarc.net" on 9.12
Thanks for the information Cathy. I've always run the Red Hat provided packages in the past, this is the first time I've ever tried running the newest release direct. Mostly I'm just feeling extra cautious since this is something I've never done before and admittedly I don't know as much about DNS as I should so I really appreciate you taking the time to break down what is happening. Based on your explanation it sounds like this isn't something I'll ever run into other than this one special case so I'll stop worrying about it. Thank you! -Nick -Original Message- From: bind-users [mailto:bind-users-boun...@lists.isc.org] On Behalf Of Cathy Almond Sent: Tuesday, February 27, 2018 4:29 AM To: bind-users@lists.isc.org Subject: Re: Issue running "dig txt rs.dns-oarc.net" on 9.12 On 22/02/2018 16:44, NNEX Support wrote: > I'm sorry to keep replying to myself but I believe I've found the line of > code that is causing this issue. Looking at validator.c, in the > check_deadlock function, 9.12.0rc1 says: > > ... > > if (parent->event != NULL && > parent->event->type == type && > dns_name_equal(parent->event->name, name) && > > ... > > 9.12.0rc3 and above says: > > ... > > if (parent->event != NULL && > (parent->event->type == type || > parent->event->type == dns_rdatatype_cname) && > dns_name_equal(parent->event->name, name) && > > ... > > By removing "parent->event->type == dns_rdatatype_cname)" (and adjusting the > rest of the if statement appropriately) the query "dig ns rs.dns-oarc.net" > works. > > I see this commit related to this line of code: > https://gitlab.isc.org/isc-projects/bind9/commit/2b51d5874c49ac823890b > 88824290fbf1c18f2cc > > I'm sure this line of code is important, otherwise it wouldn't be there and I > don't know enough to be removing random bits of code, so of course I'd never > run this in production. Still I want to understand why this is happening and > if it’s a bug or me not understanding DNS properly. Good sleuthing - though apart from understanding why the query now fails, I don't think there's any code defect that needs to be addressed. This line of code belongs with these changes between RC1 and RC3. They are kinda important (note the CVE reference): 4859. [bug] A loop was possible when attempting to validate unsigned CNAME responses from secure zones; this caused a delay in returning SERVFAIL and also increased the chances of encountering CVE-2017-3145. [RT #46839] 4858. [security] Addresses could be referenced after being freed in resolver.c, causing an assertion failure. (CVE-2017-3145) [RT #46839] The debug log you pointed to was also specific about why the validation stopped: validating rs.dns-oarc.net/CNAME: checking existence of DS at 'rs.dns-oarc.net' validating rs.dns-oarc.net/CNAME: continuing validation would lead to deadlock: aborting validation validating rs.dns-oarc.net/CNAME: deadlock found (create_fetch) The rs.dns-oarc.net zone is broken because it returns a CNAME for queries at the apex. Observe the delegation (I'm querying one of the servers auth for dns-oarc.net): ; <<>> DiG 9.11.2 <<>> +norec +dnssec @64.191.0.65 rs.dns-oarc.net NS ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43571 ;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 3, ADDITIONAL: 3 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 4096 ; COOKIE: 47d4eddbbbde6fd18616a25b5a952d35767788ad0b03038f (good) ;; QUESTION SECTION: ;rs.dns-oarc.net. IN NS ;; AUTHORITY SECTION: rs.dns-oarc.net.3600IN NS ns00.rs.dns-oarc.net. rs.dns-oarc.net.3600IN NSECrs4.dns-oarc.net. NS RRSIG NSEC rs.dns-oarc.net.3600IN RRSIG NSEC 8 3 3600 20180328101103 20180226091103 12093 dns-oarc.net. floDmByYaxmh+QQWou7PtICj4tnpW6/ea1WzatUfAEMvPOSmm54CJ467 KWpnf5XADFgFrcHOr0gYLlbFVJrwEB5n6R+SvXOTx9zwgva3SY37Vgq8 ZMwdNPdGxmVLOz1Ou5tByfZV2ZLpueF+hBB12wft+wNCysjMuwtx4U2D a64= ;; ADDITIONAL SECTION: ns00.rs.dns-oarc.net. 3600IN A 64.191.0.133 ns00.rs.dns-oarc.net. 3600IN 2620:ff:c000:0:2::133 Then look at the query response for a DS RRset that the BIND validator is receiving from ns00.rs.dns-oarc.net: ; <<>> DiG 9.11.2 <<>> +nore
