Okay, so then it seems like 1) ClamAV’s origin server periodically serves an old version of a file after the DNS TXT record is updated, or 2) Cloudflare returns a cached resource from the wrong URL, or 3) Someone is making a request to new ?version URLs before the DNS TXT record is updated (and such would be visible in the origin server’s HTTP request log).
What is the URL format that is used? I don’t see an obvious example in the conf man pages for the fully constructed URL, and I’m not near a full computer to figure it out. I was hoping to throw a few HTTP requests at it and see if the headers give any clues. I have no way to prove or test #1, but #2 would be a major and fairly obvious issue that would cause an impact to virtually all Cloudflare customers. While not impossible, this seems unlikely. #3 would certainly be possible, but would be moderately straightforward to identify on the web server hosting the original files — Or could be avoided if the origin web server includes a cache-control: no-cache (or maybe max-age=300) for version numbers greater than the current, while still returning whatever version is actually current, so that the requesting client still gets something valid, but I’m not clear what, if any, smarts are contained on the origin server. Either way, perhaps “cache-control: max-age=3600, must-revalidate” would make sense so that the problem has the opportunity to clear itself faster than the current 43200 seconds? As long as the origin server supports last-modified and similar, the impact would be relatively minimal in terms of the number of bytes delivered, although the number of requests making it to the origin would increase somewhat, but still well within the capabilities of a modest server. I’m sure smarter minds than I have looked at this, but it seems like a relatively small set of possibilities, and it just seems unlikely to me that it would go unnoticed if Cloudflare were regularly returning cached content from a different URL. On Sun, Dec 13, 2020, at 19:57, Joel Esler (jesler) via clamav-users wrote: > Both of those things are done as well. > > Sent from my iPhone > > > On Dec 13, 2020, at 19:24, Dave Warren via clamav-users > > <clamav-users@lists.clamav.net> wrote: > > > > On 2020-12-11 08:51, Paul Kosinski via clamav-users wrote: > >> "The whole CVD filename is not versioned (always "daily.cvd") which is > >> why the CloudFlare caching issue may result in serving the previous > >> version." > >> HTML filenames for Web pages are not versioned either. Does this mean > >> that CDNs like Cloudflare often serve up obsolete Web pages? If so, does > >> nobody notice (and complain)? > >> A delay of an hour could have an adverse effect on online commerce, > >> especially during the busy holiday season. > > > > By default Cloudflare does not cache HTML. Cloudflare also respects > > cache-control headers, which is the normal mechanism used for websites > > which want caching, but only to a point. > > > > Cloudflare also has an API to clear the cache (at least by URI, or > > everything, and possibly more depending on the particular options offered > > by your plan). But in practice clearing the cache is not completely > > reliable and seems to be intended for cases where it is strictly needed and > > not for every "I updated this file" situation. I have the impression that > > this applies when using Cloudflare's tiered caching, my idle speculation > > wonders if perhaps this is a timing issue, where server #1 clears the > > cache, processes a request for the file which it obtains from server #2 all > > before server #2 clears the file from cache and then processes a request by > > pulling it from server #1. > > > > From a ClamAV perspective, one solution to solve this would be to call > > daily.cvd?version=26013 -- Note that the underlying web server could ignore > > the version parameter completely, but this would ensure that each > > Cloudflare cache retrieves a fresh version of the file and negates the need > > to push a cache clear message at all. If ClamAV's server serves an outdated > > version of the file then it would still get cached, but this would defeat > > any caching within Cloudflare for new versions as they're released. > > > > > > _______________________________________________ > > > > clamav-users mailing list > > clamav-users@lists.clamav.net > > https://lists.clamav.net/mailman/listinfo/clamav-users > > > > > > Help us build a comprehensive ClamAV guide: > > https://github.com/vrtadmin/clamav-faq > > > > http://www.clamav.net/contact.html#ml > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml > > > *Attachments:* > * smime.p7s
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
