Yes to all of those concerns, Paul. As far as the reduced signature sets, it’s possible to run multiple instances on the same box in order to speed each individual reload. For example, you could have one instance that only loads the ClamAV databases and only reloads a couple of times a day and you could have one instance that only loads a custom database that changes frequently. This way you’re not reloading the ClamAV databases every time you need to reload the custom one.
You could also have one instance that does check PUAs but doesn’t quarantine/ remove them but does notify you while other instances that only look at specific other signatures. With regard to the limited memory cases, there are tradeoffs that need to be evaluated in each individual situation so limited memory configurations may be able get around issues with reduced signatures by having files scanned at a different point in the architecture. With respect to the email/upload situations those were examples of specific use cases. I wouldn’t suggest that scanning on access be the only time to scan for exactly the reason you point out. Integration with mail / ftp are great ways to block known things from getting in or out. But any change in “known” means that you need to rescan everything to make sure it didn’t get in while it was unknown. Performance is only part of the equation, risk analysis / acceptance also has to be factored into any architectural design. Maarten Sent from a tiny keyboard > On Mar 17, 2018, at 21:49, Paul Kosinski <clamav-us...@iment.com> wrote: > > 1. With regard to separating signature sets: > > Isn't there always a danger that scanning with a reduced signature set > misses malware that was not "expected" in a particular context? For > example, except in highly restrictive work environments, people tend to > visit all sorts of Web sites, any of which might be a source of malware > of various kinds. > > 2. With regard to scanning incoming email, uploads & general downloads: > > Scanning only upon upload or download runs a risk with files containing > 0-day malware. Since there is no signature yet, it will be passed, but > if it were scanned again later, the malware would be found. This > problem occurs with file server scanning as well, and any other scheme > which scans only once and then marks the data clean. Sadly, doing a > really thorough job means lots and lots of repeated scanning. This > could be mitigated a bit if signatures were marked as to their > criticality. For example, a new signature for what was a 0-day found > "in the wild" would pretty much demand rescanning of everything. But at > least the rescan could be done with a very tiny signature set. > > 3. With regard to reducing the delay when a new signature set is loaded: > > I don't know the internals of ClamAV, but it seems that it ought to be > possible to restructure it to overlap (in a different thread) the > loading of signatures into memory with the running of the clamd scan > engine. Then, when new signatures have been loaded, a simple pointer > swap or three (mutexed, of course) would cause subsequent scans to use > the new signatures. After the swap, the memory for the old signatures > would be released by the loader thread. This would take more memory > during signature update, but it might be a worthwhile option. > > > On Sat, 17 Mar 2018 17:17:17 -0400 > Maarten Broekman <maarten.broek...@gmail.com> wrote: > >> Some considerations: >> - the longest “delay” will occur when reloading signature databases. >> If reducing the delay is important, run multiple instances with >> smaller signatures in each. ESPECIALLY, if you’re going to writing >> your own story signatures or using databases that change often. >> >> - scanning speed will be limited most by filesystem access speed. If >> you schedule scans, limit the slow file systems to off hours or only >> scan them occasionally. >> >> - on memory-constrained systems separating the signature sets is even >> more important so that you can limit the memory usage in addition to >> the reload speed. >> >> - where possible, have clamd running and use clamdscan instead of >> spawning clamscan processed for each scan. >> >> - depending on the workloads, it might be more efficient to run >> ClamAV running everywhere with a certain set of signatures and run a >> single system for specific targeted scans using larger signature sets >> (such as a mailserver or ftp server). >> >> - proftpd has pretty good integration with ClamAV as a plug-in for >> upload scanning. >> >> Maarten > _______________________________________________ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
