On 6/9/2014 3:11 PM, David F. Skoll wrote:
On Mon, 9 Jun 2014 11:51:21 -0700 (PDT)
John Hardin <jhar...@impsec.org> wrote:
So there is merit in building a distributed look-up system using SA.
Distributed lookup of *what*, though? Can you clarify that part of
your idea? Are you referring to distributed whois queries for a
domain name, to determine its age?
Well, here's how it could be done. Imagine someone runs a DNS zone
for "newdomain.example.net". You want to see if "example.org" is a new
domain, so you look up a TXT record for example.org.newdomain.example.net.
The DNS software that serves the zone newdomain.example.net runs
the following pseudo-code when "example.org" is looked up:
IF example.org is in my database
THEN
return the TXT record associated with example.org
update the last-looked-up time for example.org
ELSE
generate a TXT record of the form YYYYMMDDHHMMSS corresponding to current
time (UTC)
insert it in the database
return it
ENDIF
A background job will periodically clean out domains that haven't been
queried in a long time.
The clever part is that once lots of sites begin using this in their
SA setups, we'll very quickly build up quite an accurate database of
newly-seen domains that's completely independent of any registrar for
a data source.
Yes, spammers can poison it by specifically looking up a domain,
waiting a couple of days, and then spamming. But I think most won't bother
(witness how effective greylisting still is.)
Furthermore, you can ignore all but the first few hundred lookups before you
enter the TXT record in the database; this will make it more expensive
for spammers to poison the data. Or you could not enter a record in the
database until it has been looked up from 100 different IP addresses... I
can think of a few other countermeasures.
So.... who's volunteering to do this? :)
Thank you for elegantly writing my idea out though there is a bit more
to it.
So yes, effectively it's a system that can leverage registry provided
creation data and it can build it's own as domains are seen in emails
using SA installations as nodes to spread the whois load around so as to
get around whois server bans,
And I think as someone pointed out, it needs to also report the
registrar seen at the time the record is created. I'm not sure about
DNS servers but we could try logging that as well. Perhaps the whole
whois record could be stored and parsed later.
