Cloudflare's cache timeout is set to 5 seconds. So, I would doubt that Cloudflare's cache is the issue, it may be an ISP thing in the middle doing the caching, which is what Paul is guessing at this point, if I am following the thread correctly.
Out of an abundance of caution I did a worldwide flush of daily.cvd yesterday. Which caused everyone to get a new copy if it didn't match what they had. This resulted in about 3TB of traffic in 10 minutes, but after that it settled back down. We're still a bit higher than normal, as I eased some of the "you're going to fast" restrictions. (I have a rate limiter set up, if you are downloading 100 cdiffs in 10 seconds, to rate limit the offender...) I've disabled this for now We're up to about 71TB a day right now and it seems to be holding steady. Give it a couple more days and see if it comes back down. -- Joel Esler Manager, Communities Division Cisco Talos Intelligence Group http://www.talosintelligence.com > On Dec 10, 2018, at 9:56 PM, Eric Tykwinski <eric-l...@truenet.com> wrote: > > Paul, > > Sorry some of this confusion is probably my fault trying to help without > going back to the whole thread. > >> On Dec 10, 2018, at 9:34 PM, Paul Kosinski <clamav-us...@iment.com> wrote: >> >> We ARE using freshclam to perform the actual update. And always have >> been! >> >> We've only been using curl (not wget, if that matters) to pull the first >> few bytes of the cvd to see if its version number matches what the DNS >> TXT query said. >> >> We do this because, after the conversion to Cloudflare, we were getting >> lots of FAILURES where *freshclam* said things were out of sync (and >> eventually disabled all the mirrors). > > Have you tried what I did below? I.E. curl/wget/telnet whatever your flavor > of the day, and pull the newest cdiff? > If you’re getting a 404, that’s definitely an issue. > > My guess is that it’s actually timing out though, and could be more of an > issue troubleshooting. > Is it local, ie an IDP getting stuck scanning the files, or remotely > freshclam itself is timing out on BOS pulling the update from ClamAV and > caching it before you can download it. > >> And we have recently seen that our Web server sometimes can get the new >> updates (from IAD) *hours* before our main LAN does (from BOS). > > Those hours before are only checking the CVDs, which can and probably are > cached on CloudFlare so not up to date. > My guess is that there are just more people in Boston using Clam, so the > cache last the longest. > >> P.S. It's been quite frustrating getting some replies seemingly based on >> assumptions that we are doing things we shouldn't, when we aren't in >> fact doing those things. (Like not using freshclam.) > > I would agree, this has gone on a long time from my recollection, which is > why I jumped in and started looking at it. > Definitely, I did hop on without all the facts and was just trying to figure > out on the fly what’s going on, so my bad on that. > > When in doubt, I usually pull a pcap on a server. There’s a lot of factors > that can come into play, but actually with clam only using http, this > actually makes it a lot easier. > > Sincerely, > > Eric Tykwinski > > > _______________________________________________ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
