That 2 hour you mention seems to explain why it's always around the same time and seems to come earlier or later by 2 hours. I do not know if it literally coincides with that refresh (not sure if that's logged on my system).
Having the database reloaded in chunks looks a lot more efficient to me. Ok 7xxMB memory is quite a small machine. But e.g. on a 16GB system the standard memory usage is still almost 10% and so when reloading about 20% (very rough numbers I admit). Of course I don't know how the database is implemented so perhaps it's easier said than done. In that regards "just twice" still is a lot ;) On Wed, 2 Dec 2020 at 11:00, Frans de Boer <fr...@fransdb.nl> wrote: > On 02/12/2020 09:34, PenguinWhispererThe via clamav-users wrote: > > Hi, > > I have a webserver with 4GB of memory that also functions as a mailserver. > The mail volume is rather low (perhaps a few hundred mails/day). > Almost every day around the same time I get a swap usage warning and once > in a while clamd crashes because it has no more swap space available > blocking mails from being processed. > > Right now it uses about 1.3GB and all is fine. I'm using FreeBSD. I'm > trying to see the logic why every day around the same time clamd decides to > need so much more memory. It's not like it needs it to process the emails. > > I've searched and read that clamd uses a lot of memory (30% is indeed > quite a lot). But nowhere I see these kinds of numbers (multiple > gigabytes). Having it use 60% memory (at least 2.3GB when it crashes) is > getting ridiculous. Since all mails are being processed just fine when > clamd uses 1.3GB I don't want to just increase the memory as it might start > using 60GB at some point for no clear reason. > > I didn't modify cronjobs recently so I'm unclear on why this seems to be a > periodic thing. I had it like weeks in a row at around 15:45. Then it > seemed to have switched to 17:45 and had it now once at 19:45. There seems > to be this 2 hour change in it or it's something that happens every 2 hours > and circumstances get "just right" later and later. > > Anyone experienced the same? Knows what's going on? Has a solution to this > (not looking for "don't run clamAV as a daemon")? > > Thanks in advance! > > I have not seen this before, but let me ask if the extra memory usage > coincide with reloading the signatures? freshclam does a default 2 hour > database check. > > If so, it is because of the design decision made by the clamav team. When > a database is reloaded, it is using newly allocated memory. To allow > continues virus scanning, the old contents is only thrashed after reloading > and processing continues with the freshly loaded databases. This approach > is expected when the databases are kept growing - now consumes around 1.2GB > memory - and server memory is expected to grow also. > > My situation is similar as yours, and as such I switch from a 32-bit > system with 750 MB to 64-bit and 16GB. The switch was needed because some > emails where not scanned due to the limited memory and processing time by > clamd. > > Of course, running clamd is needed to avoid the cost of reloading clamscan > with every email and the latency incurred due to accessing the database on > disk. > > An alternative would be to use compressed databases. Although I am not > sure if they get decompressed when loading the databases into memory. If > not decompressed, the processor speed might be a bottleneck. > > As for your 60GB fear: Unless the design changes, it will keep on using > just twice the "normal" memory size. So as always, adding some GB's to your > system memory - if possible - is better as well as the cheapest way. > > --- Frans. > > -- > A: Yes, just like that A: Ja, net zo > Q: Oh, Just like reading a book backwards Q: Oh, net als een boek > achterstevoren lezen > A: Because it upsets the natural flow of a story A: Omdat het de natuurlijke > gang uit het verhaal haalt > Q: Why is top-posting annoying? Q: Waarom is Top-posting zo > irritant? > > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml >
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
