Continuing to fine-tune this service - thank you to everyone testing it.
Some updates were pushed out yesterday:
* Initial new domain "grace period" reduced to 8 minutes (down from 15
mins) - 4 attempts are made within this time to get a valid HTTP response
* Mozilla browser spoofing is implemented to avoid problems with
websites that block HttpClient requests
* Fixes to NXDOMAIN negative result caching appear to be working well now
Some lessons learned in the meantime as well. Turns out that letting the
HTTP test run though an email server IP is a terrible idea as it will
put the IP on some blocklists for attempting to make HTTP connections to
botnet command & control honeypot servers if someone happens to query
one of those domains, LOL.
A couple people asked about me posting the code/service so they could
run it on their own systems but I'm currently leaning away from that. I
don't think there is any benefit to doing that instead of just utilizing
the centralized service. The whole thing works better if everyone using
it queries a central service and helps avoid people easily making bad
mistakes like the one above and then spending hours scrambling to try to
find non-existent botnet infections on their network while mail bounces
because they are on a blocklisted :( If someone has a good reason for
making the service locally installable let me know though, haha.
