On Tue, Aug 10, 2010 at 13:58, Jorge <jne...@gmail.com> wrote: > Hello, I'm also seeing large memory usage of freshclam in Gentoo x86-64, > there is a memory graph here: http://imagebin.ca/view/V-BshuFl.html In the > graph you can see the big memory usage of freshclam around June and > subsequent OOM kill. Now it's happening again. For the moment I will keep > freshclam alive just in case someone needs some debug/dump/info, but if no > solution is found I will setup a weekly cron to restart freshclam. At this > moment is at >1.5 gigabyte and slowly growing: > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND > 14360 clamav 39 19 1532m 1.1g 5236 S 0 29.7 22:17.62 > freshclam > 14347 clamav 23 3 395m 266m 5388 S 0 6.7 49:49.39 > clamd > > Our config is: > > # egrep '^[^#]' /etc/freshclam.conf > UpdateLogFile /var/log/clamav/freshclam.log > LogFileMaxSize 4M > LogTime yes > PidFile /var/run/clamav/freshclam.pid > DatabaseOwner clamav > AllowSupplementaryGroups yes > DNSDatabaseInfo current.cvd.clamav.net > DatabaseMirror db.es.clamav.net > DatabaseMirror database.clamav.net > ScriptedUpdates yes > Checks 48 > NotifyClamd /etc/clamd.conf > SafeBrowsing yes > > # clamconf -n > Checking configuration files in /etc > > Config file: clamd.conf > ----------------------- > LogFile = "/var/log/clamav/clamd.log" > LogTime = "yes" > PidFile = "/var/run/clamav/clamd.pid" > LocalSocket = "/var/run/clamav/clamd.sock" > User = "clamav" > AllowSupplementaryGroups = "yes" > > Config file: freshclam.conf > --------------------------- > LogFileMaxSize = "4194304" > LogTime = "yes" > PidFile = "/var/run/clamav/freshclam.pid" > AllowSupplementaryGroups = "yes" > UpdateLogFile = "/var/log/clamav/freshclam.log" > Checks = "48" > DatabaseMirror = "db.es.clamav.net", "database.clamav.net" > SafeBrowsing = "yes" > > clamav-milter.conf not found > > Software settings > ----------------- > Version: 0.96.1 > Optional features supported: MEMPOOL CLAMUKO AUTOIT_EA06 BZIP2 RAR JIT > Database directory: /var/lib/clamav > main.cvd: version 52, sigs: 704727, built on Mon Feb 15 15:54:51 2010 > daily.cld: version 11525, sigs: 109522, built on Tue Aug 10 10:18:29 2010 > safebrowsing.cld: version 22992, sigs: 725991, built on Tue Aug 10 11:45:41 > 2010 > bytecode.cld: version 33, sigs: 8, built on Tue Aug 10 07:57:52 2010 > > Platform information > -------------------- > uname: Linux 2.6.28-hardened-r9 #1 SMP Thu Feb 4 13:34:23 CET 2010 x86_64 > OS: linux-gnu, ARCH: x86_64, CPU: x86_64 > zlib version: 1.2.3 (1.2.3), compile flags: a9 > > Build information > ----------------- > GNU C: 4.3.4 (4.3.4) > GNU C++: 4.3.4 (4.3.4) > CPPFLAGS: > CFLAGS: -O2 -march=athlon64 -pipe -ggdb -fno-strict-aliasing > CXXFLAGS: -O2 -march=athlon64 -pipe -ggdb > LDFLAGS: -Wl,-O1 > Configure: '--prefix=/usr' '--build=x86_64-pc-linux-gnu' > '--host=x86_64-pc-linux-gnu' '--mandir=/usr/share/man' > '--infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc' > '--localstatedir=/var/lib' '--libdir=/usr/lib64' '--enable-bzip2' > '--disable-ipv6' '--enable-clamdtop' '--disable-milter' '--with-iconv' > '--disable-experimental' '--enable-id-check' '--disable-zlib-vcheck' > '--with-dbdir=/var/lib/clamav' 'build_alias=x86_64-pc-linux-gnu' > 'host_alias=x86_64-pc-linux-gnu' 'CFLAGS=-O2 -march=athlon64 -pipe -ggdb' > 'LDFLAGS=-Wl,-O1' > > As I have seen in the archives one solution could be to disable > SafeBrowsing, but in this case it's of valuable use to us. I have setup a > second freshclam in daemon mode and different directory without SafeBrowsing > to test for anomalies, it has just completed the first initial setup and > it's using ~51Mb : > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND > 12356 clamav 20 0 51608 7140 5240 S 0 0.2 0:06.35 > freshclam > > The contents of the directories now are: > > /var/lib/clamav: > total 65M > -rw-r--r-- 1 clamav clamav 82K Aug 10 08:33 bytecode.cld > -rw-r--r-- 1 clamav clamav 6.4M Aug 10 10:35 daily.cld > -rw-r--r-- 1 clamav clamav 22M Jul 26 14:38 main.cvd > -rw------- 1 clamav clamav 208 Aug 10 13:36 mirrors.dat > -rw-r--r-- 1 clamav clamav 36M Aug 10 13:06 safebrowsing.cld > > /var/lib/clamav-debug: > total 25M > -rw-r--r-- 1 clamav clamav 24K Aug 10 13:46 bytecode.cvd > -rw-r--r-- 1 clamav clamav 2.5M Aug 10 13:46 daily.cvd > -rw-r--r-- 1 clamav clamav 22M Aug 10 13:45 main.cvd > -rw------- 1 clamav clamav 52 Aug 10 13:46 mirrors.dat > > Will see how it evolves. > > -- > Jorge Nerín > <jne...@gmail.com> >
Just a quick update after one day: freshclam+safebrowsing has incremented its virtual memory size from 1532m to 1545m (Δ ~13m/day) freshclam without safebrowsing has remained in exactly the same virtual memory size 51608 as yesterday. The contents of the directories have changed somewhat since yesterday: /var/lib/clamav: total 36M -rw-r--r-- 1 clamav clamav 82K Aug 10 08:33 bytecode.cld -rw-r--r-- 1 clamav clamav 6.4M Aug 11 12:42 daily.cld -rw-r--r-- 1 clamav clamav 22M Jul 26 14:38 main.cvd -rw------- 1 clamav clamav 208 Aug 11 13:42 mirrors.dat -rw-r--r-- 1 clamav clamav 7.2M Aug 10 18:10 safebrowsing.cvd /var/lib/clamav-debug: total 29M -rw-r--r-- 1 clamav clamav 24K Aug 10 13:46 bytecode.cvd -rw-r--r-- 1 clamav clamav 6.4M Aug 11 12:46 daily.cld -rw-r--r-- 1 clamav clamav 22M Aug 10 13:45 main.cvd -rw------- 1 clamav clamav 208 Aug 11 13:46 mirrors.dat daily.cld from clamav-debug (without safebrowsing) was 2.5M and now is at 6.4M, and the most surprising thing is that the 36M safebrowsing.cld has disapeared and now I have a 7.2M safebrowsing.cvd. Checking the feshclam.log I found that freshclam at ~Aug 10 18:10 tried to download safebrowsing-22996.cdiff from various servers (db.es.clamav.net: 150.214.142.197, 194.65.79.153, 82.194.71.224) failed and downloaded a different file, safebrowsing.cvd. Perhaps the memory leak lies somewhere when changing files freshclam.log: Tue Aug 10 18:07:55 2010 -> Received signal: wake up Tue Aug 10 18:07:55 2010 -> ClamAV update process started at Tue Aug 10 18:07:55 2010 Tue Aug 10 18:07:55 2010 -> main.cvd is up to date (version: 52, sigs: 704727, f-level: 44, builder: s ven) Tue Aug 10 18:07:55 2010 -> daily.cld is up to date (version: 11525, sigs: 109522, f-level: 53, builde r: ccordes) Tue Aug 10 18:07:55 2010 -> Trying host db.es.clamav.net (82.159.137.16)... Tue Aug 10 18:08:25 2010 -> nonblock_connect: connect timing out (30 secs) Tue Aug 10 18:08:25 2010 -> Can't connect to port 80 of host db.es.clamav.net (IP: 82.159.137.16) Tue Aug 10 18:08:25 2010 -> Trying host db.es.clamav.net(150.214.142.197)... Tue Aug 10 18:08:25 2010 -> WARNING: getfile: safebrowsing-22996.cdiff not found on remote server (IP: 150.214.142.197) Tue Aug 10 18:08:25 2010 -> WARNING: getpatch: Can't download safebrowsing-22996.cdiff from db.es.clam av.net Tue Aug 10 18:08:25 2010 -> Trying host db.es.clamav.net (82.159.137.16)... Tue Aug 10 18:08:55 2010 -> nonblock_connect: connect timing out (30 secs) Tue Aug 10 18:08:55 2010 -> Can't connect to port 80 of host db.es.clamav.net (IP: 82.159.137.16) Tue Aug 10 18:08:55 2010 -> Trying host db.es.clamav.net (82.194.71.224)... Tue Aug 10 18:08:55 2010 -> WARNING: getfile: safebrowsing-22996.cdiff not found on remote server (IP: 82.194.71.224) Tue Aug 10 18:08:55 2010 -> WARNING: getpatch: Can't download safebrowsing-22996.cdiff from db.es.clam av.net Tue Aug 10 18:08:55 2010 -> Trying host db.es.clamav.net (82.159.137.16)... Tue Aug 10 18:09:25 2010 -> nonblock_connect: connect timing out (30 secs) Tue Aug 10 18:09:25 2010 -> Can't connect to port 80 of host db.es.clamav.net (IP: 82.159.137.16) Tue Aug 10 18:09:25 2010 -> Trying host db.es.clamav.net (194.65.79.153)... Tue Aug 10 18:09:26 2010 -> WARNING: getfile: safebrowsing-22996.cdiff not found on remote server (IP: 194.65.79.153) Tue Aug 10 18:09:26 2010 -> WARNING: getpatch: Can't download safebrowsing-22996.cdiff from db.es.clamav.net Tue Aug 10 18:09:26 2010 -> WARNING: Incremental update failed, trying to download safebrowsing.cvd Tue Aug 10 18:09:26 2010 -> Trying host db.es.clamav.net (82.159.137.16)... Tue Aug 10 18:09:56 2010 -> nonblock_connect: connect timing out (30 secs) Tue Aug 10 18:09:56 2010 -> Can't connect to port 80 of host db.es.clamav.net (IP: 82.159.137.16) Tue Aug 10 18:09:56 2010 -> Trying host db.es.clamav.net (194.65.79.153)... Tue Aug 10 18:10:14 2010 -> Downloading safebrowsing.cvd [100%] Tue Aug 10 18:10:16 2010 -> safebrowsing.cvd updated (version: 22997, sigs: 319245, f-level: 53, builder: google) Tue Aug 10 18:10:16 2010 -> bytecode.cld is up to date (version: 33, sigs: 8, f-level: 53, builder: edwin) Tue Aug 10 18:10:16 2010 -> Database updated (1133502 signatures) from db.es.clamav.net (IP: 194.65.79.153) Tue Aug 10 18:10:16 2010 -> Clamd successfully notified about the update. Notice that the previous run reported about safebrowsing: Tue Aug 10 17:37:55 2010 -> safebrowsing.cld is up to date (version: 22995, sigs: 728272, f-level: 53, builder: google) Version updated from 22995 to 22997 when downloading safebrowsing.cvd but number of sigs changed from 728272 to 319245 (lost 409027 sigs) Safebrowsing was updated previously succesfully: Tue Aug 10 15:07:51 2010 -> Downloading safebrowsing-22995.cdiff [100%] Tue Aug 10 15:07:54 2010 -> safebrowsing.cld updated (version: 22995, sigs: 728272, f-level: 53, build er: google) Does something ring a bell to someone? -- Jorge Nerín <jne...@gmail.com> _______________________________________________ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
