It's surprising that more organizations don't fix this--it can be a
serious DoS vulnerability if the record is important enough. Anyone
know of tools that, given a zone or a set of labels, will test for this
behavior?
John
On 05/30/2014 11:42 AM, David A. Evans wrote:
To my question of how many more are lurking out there. It
looks like quite a few. I am not sure we are going to be able to
continue with RPZ's and NSDNAME's.
xserv.dell.com is my newest main stream web site having the
issue.
I is behaving the same way as www.rackspace.com. They have
DNS servers answering NXDOMAIN when they should be answering NODATA.
*David A. Evans*
*Enterprise IP/DNS Management*
*Network Infrastructure Tools and Services*
*(309) 675-9700 *
*_Evans_David_A@cat.com_* <mailto:evans_davi...@cat.com>
bind-users-boun...@lists.isc.org wrote on 05/28/2014 03:09:54 PM:
> From: "David A. Evans" <evans_davi...@cat.com>
> To: bind-us...@isc.org
> Date: 05/28/2014 03:10 PM
> Subject: Re: RPZ and www.rackspace.com
> Sent by: bind-users-boun...@lists.isc.org
>
> CATERPILLAR SECURITY ALERT: The email address in the sender line
> does not match the account that sent the email. This can be an
> indication of phishing. Do not click links or open attachments
> unless you are certain it is from a safe source. Learn more at
> security.cat.com/phishing
> Rack Space appears to have fixed the issue. "dig
> www.wip.rackspace.com NS" now returns NO DATA instead of NXDOMAIN.
>
> I wonder how many more are lurking out there.
>
>
> We are still getting a trickle in of complaints about
> slowness and failures that appear to be related to the RPZ and the
> amount of time it takes to complete all the extra queries for the
> NSDNAME checks. When we research these they seem to fit into 2 groups.
>
>
> 1. DNS zones with "slightly" broken infrastructure. These
> would be domains with either slow response from one or more name
> servers or not responding name servers. A recursive resolver
> without RPZ loaded can work though the issue but the extra lookups
> required primarily for the NSDNAME increases the query time to the
> point where DNS times out from the client perspective. I can't
> really see a fix here, the issue does reside with the domain owner,
> we are simply more susceptible to the issue because of the RPZ's.
>
> 2. DNS zones with a large number of NS records and the name
> servers have FQDN's in several different DNS zones.
>
> www.domain.com NS ns1.domain1.com
> NS ns2.domain2.com
> NS ns3.domain3.net .....
>
> These are the most frustrating as there is really nothing
> wrong with this setup in my opinion. This is just going to
> generate a huge number of DNS lookups to do a full NSDNAME check.
> These are hard to explain away as they "work from home" and "work
> from my phone".
>
>
> Have others found similar issues when implementing RPZ's?
>
>
>
>
>
>
> David A. Evans
> Enterprise IP/DNS Management
> Network Infrastructure Tools and Services
>
>
>
> Mark Andrews <ma...@isc.org> wrote on 05/07/2014 10:44:05 AM:
>
> > From: Mark Andrews <ma...@isc.org>
> > To: "David A. Evans" <evans_davi...@cat.com>
> > Cc: bind-us...@isc.org
> > Date: 05/07/2014 10:44 AM
> > Subject: Re: RPZ and www.rackspace.com
> >
> >
> > In message <OFDC3C86D9.D668B707-ON86257CD1.005339FC-86257CD1.
> > 00543...@notes.cat.com>, "David A. Evans" writes:
> > >
> > > I've done some more troubleshooting with info from people
that
> > > responded directly to me and not to the list. This can be
reproduced
> > > without any RPZ loaded by mimicking the behavior of the RPZ lookups
> > > required to validate NSDNAME lines.
> > >
> > > Issue these 'digs' within 30 second of each other.
> > >
> > > dig www.wip.rackspace.com
> > > www.wip.rackspace.com. 30 IN A 173.203.44.116
> > >
> > > dig www.wip.rackspace.com NS
> > > (NXDOMAIN)
> > >
> > > dig www.wip.rackspace.com
> > > (NXDOMAIN)
> > >
> > >
> > > I think this is another case of miss configured load
balancers.
> > > Shouldn't the NS record lookup respond with a NODATA response and
not
> > > NXDOMAIN?
> >
> > Yes. The name exists.
> >
> > > That still doesn't really answer why a site as big as
> > > www.rackspace.comisn't getting hit with support issues on
theirweb site.
> > > It only took us about 4 hours into our first production day with
> > > NSDNAME's in our RPZ to get a call about www.rackspace.comnot
loading.
> >
> > Because NS queries are not common with normal DNS lookups. For
> > some reason people that deploy load balancers think they don't need
> > to fix issues like this. Send something other than a A record and
> > you get:
> >
> > - NXDOMAIN being returned when the name exists.
> > - NOTIMP being returned.
> > (Really you can't just send NODATA?)
> > - REFUSED being returned.
> > (Really you don't want to tell us the record does not exist?)
> > - the wrong SOA being returned.
> > - malformed RDATA with the content being the A record content.
> >
> > Mark
> >
> > > David A. Evans
> > > Enterprise IP/DNS Management
> > > Network Infrastructure Tools and Services
> > > evans_davi...@cat.com
> > >
> > >
> > >
> > > From: "David A. Evans" <evans_davi...@cat.com>
> > > To: bind-users@lists.isc.org
> > > Date: 05/07/2014 09:11 AM
> > > Subject: RPZ and www.rackspace.com
> > > Sent by: bind-users-boun...@lists.isc.org
> > >
> > >
> > >
> > > CATERPILLAR SECURITY ALERT: The email address in the sender
linedoes not
> > > match the account that sent the email. This can be an indication of
> > > phishing. Do not click links or open attachments unless you are
> certain it
> > > is from a safe source. Learn more at security.cat.com/phishing
> > > We have just enabled RPZ with some NSDNAME checks and are
seeing
> > > an issue resolving www.rackspace.com.
> > >
> > > The first lookup is successful and returns both the CNAME
and the
> > > A record. The second query, within a second of the first, will only
> > > return the CNAME. It will only return the CNAME until the TTL of
the A
> > > record times out. The first query, when it actually has to go
out and do
> > > recursion will always work. Answering from cache will always
fail. When
> > > you inspect the cache during the time that it is only returning
> the CNAME,
> > > the record in cache is "www.wip.rackspace.com type ANY
> NXDOMAIN". This
> > > only happens with RPZ's enabled and query hitting a RPZ zone with a
> > > NSDNAME line. Turning off RPZ or whitelisting the lookup via
RPZ before
> > > it hits a RPZ with NSDNAME allows the query to be successful 100%
of the
> > > time.
> > >
> > >
> > > Can anyone else verify this behavior? What is going on
with
> > > www.rackspace.com? If this is a miss configuration on
Rackspace's DNS
> > > servers how are they not getting hit with support calls like crazy?
> > >
> > >
> > >
> > > dig @redacted.cat.com www.rackspace.com
> > >
> > > ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>>
@redacted.cat.com
> > > www.rackspace.com
> > > ; (1 server found)
> > > ;; global options: +cmd
> > > ;; Got answer:
> > > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30337
> > > ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
> > >
> > > ;; QUESTION SECTION:
> > > ;www.rackspace.com. IN A
> > >
> > > ;; ANSWER SECTION:
> > > www.rackspace.com. 300 IN CNAME www.wip.rackspace.com.
> > > www.wip.rackspace.com. 30 IN A 173.203.44.116
> > >
> > > ;; Query time: 193 msec
> > > ;; SERVER: redacted
> > > ;; WHEN: Wed May 7 08:53:08 2014
> > > ;; MSG SIZE rcvd: 73
> > >
> > >
> > >
> > > dig @redacted.cat.com www.rackspace.com
> > >
> > > ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.17.rc1.el6_4.6 <<>>
@redacted.cat.com
> > > www.rackspace.com
> > > ; (1 server found)
> > > ;; global options: +cmd
> > > ;; Got answer:
> > > ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 25905
> > > ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0
> > >
> > > ;; QUESTION SECTION:
> > > ;www.rackspace.com. IN A
> > >
> > > ;; ANSWER SECTION:
> > > www.rackspace.com. 298 IN CNAME www.wip.rackspace.com.
> > >
> > > ;; AUTHORITY SECTION:
> > > wip.rackspace.com. 58 IN SOA www-gtm-ord1.rackspace.com.
> > > hostmaster.305181-GTM1.rackspace.com. 86 10800 3600 604800 60
> > >
> > > ;; Query time: 2 msec
> > > ;; SERVER: redacted
> > > ;; WHEN: Wed May 7 08:53:10 2014
> > > ;; MSG SIZE rcvd: 129
> > >
> > >
> > > David A. Evans
> > > Enterprise IP/DNS Management
> > > Network Infrastructure Tools and Services
> > > evans_davi...@cat.com _______________________________________________
> > > Please visit https://lists.isc.org/mailman/listinfo/bind-usersto
> > > unsubscribe from this list
> > >
> > > bind-users mailing list
> > > bind-users@lists.isc.org
> > > https://lists.isc.org/mailman/listinfo/bind-users
> > >
> > > --=_alternative 0054318186257CD1_=
> > > Content-Type: text/html; charset="US-ASCII"
> > > _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-usersto
> unsubscribe from this list
>
> bind-users mailing list
> bind-users@lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
from this list
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe
from this list
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
