On 02-09-15 10:44, Reindl Harald wrote: > > > Am 02.09.2015 um 10:23 schrieb Axb: >> On 09/02/15 09:51, Olivier Nicole wrote: >>> Hi, >>> >>> I am looking at malware patrol, but they offer a list of over 300,000 >>> rules, that is way too big. >>> >>> So I was considering using it in a URIDNSBL type of way, but including >>> the full URL, not only the host part. It should be able to accept things >>> like foo.example.com:81/directory/foo?something >>> >>> Does that exist already? >> >> that doesn't exist, publicly... >> >> There are many reasons why running this isn't trivial either. >> >> - tracking IDs/unique identifiers in URLs >> - *can* cause massive scanning overhead >> - depending on special cases, DNS spec limitations. >> etc, etc.. >> >> What problem are you trying to solve which cannot be solved with "known" >> methods? > > on example would be a URL for masshosting / freehosting in the way of > http://hosterdomain/username/ where URIBL over the whole domain is not > correct just because one user account was hacked and malware placed there > > in general it would make sense at least be specific to subdomains and > the first folder in some cases on the listing side, drawback would be > more URIBL requests for each possible variant >
I implemented this at my previous $workjob, using a webservice in stead of a DNSBL, and limiting the amount of requests by selecting a max of 10 URLs picked randomly from the message. Client-side performance was OK, but the dataset in the BL was meager, making the test results not very useful. Using a DNSBL in stead of a webservice should be fairly easy, by using a hash values of the URIs in stead of the URI itself; I've seen implementations of that technique before. Regards, Tom