If you don’t mind my asking – are you using a large number of third party databases? Our official databases have grown quite a bit this year – but I wouldn’t expect anywhere near 5 minutes for load time. On my laptop this morning I see around 45 seconds load time for clamd.
Every now and then it’s prudent to groom the database and remove problematic signatures, or consolidate them. We do this on occasion, and have an ongoing effort to replace hash-based signatures with logical signatures that detect more than one file per signature. I wonder if any of the unofficial databases have similar efforts to keep the volume and quality of signatures in check. Regards, Micah From: clamav-users <clamav-users-boun...@lists.clamav.net> on behalf of Robert Kudyba <rkud...@fordham.edu> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net> Date: Wednesday, July 31, 2019 at 10:29 AM To: Reio Remma <r...@mrstuudio.ee>, "clamav-users@lists.clamav.net" <clamav-users@lists.clamav.net> Subject: Re: [clamav-users] ***Spam 3.041*** clamd using 100% CPU in Fedora 30 with sendmail & clamav-milter, : Probe for slot 1 returned: failed Sorry forgot to include the hive in my responses. So increasing the timeout value to 900 did work. I didn’t time it but it definitely seems like 4-5 minutes to finally start. We rebooted and it started fine. Should a big report be created? Would this be in Fedora’s Bugzilla, or Clamav’s bug tracker? Are there any other optimization settings? On Jul 31, 2019, at 2:47 AM, Reio Remma <r...@mrstuudio.ee<mailto:r...@mrstuudio.ee>> wrote: Just curious, did you note how long it actually took to fully load clamd afterwards? It might be worth taking this to CentOS devs, because the signatures database keeps growing and clamd loading time with it. But it's really an issue with older machines like the one I have here. :D Good luck! Reio On 30/07/2019 23:30, Robert Kudyba wrote: I did but then I also increased from 600 to 900 and that started the daemon. Any idea why this wouldn't be considered a bug? Thanks for the response. On Tue, Jul 30, 2019 at 3:48 PM Reio Remma <r...@mrstuudio.ee<mailto:r...@mrstuudio.ee>> wrote: Did you do "systemctl daemon-reload" before restarting the service again? On 30.07.2019 22:23, Robert Kudyba wrote: No luck: systemd[1]: Starting Generic clamav scanner daemon... journalctl -xe -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_systemd-2Ddevel&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=HeyWNpOta-zU4iUgsT6qIc2Inh2JoVpyP8g7tjAJKuc&e=> -- -- An ExecStart= process belonging to unit clamd@scan.service<mailto:clamd@scan.service> has exited. -- -- The process' exit code is 'killed' and its exit status is 15. Jul 30 15:20:21 storm.cis.fordham.edu<http://storm.cis.fordham.edu/> systemd[1]: clamd@scan.service<mailto:clamd@scan.service>: Failed with result 'timeout'. -- Subject: Unit failed -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_systemd-2Ddevel&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=HeyWNpOta-zU4iUgsT6qIc2Inh2JoVpyP8g7tjAJKuc&e=> -- -- The unit clamd@scan.service<mailto:clamd@scan.service> has entered the 'failed' state with result 'timeout'. Jul 30 15:20:21 storm.cis.fordham.edu<http://storm.cis.fordham.edu/> systemd[1]: Failed to start Generic clamav scanner daemon. -- Subject: A start job for unit clamd@scan.service<mailto:clamd@scan.service> has failed -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_mailman_listinfo_systemd-2Ddevel&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=HeyWNpOta-zU4iUgsT6qIc2Inh2JoVpyP8g7tjAJKuc&e=> -- -- A start job for unit clamd@scan.service<mailto:clamd@scan.service> has finished with a failure. -- -- The job identifier is 331899 and the job result is failed. It's as if clamd continues to try to start as running 'top' shows 100% CPU: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4949 root 20 0 774044 727648 7736 R 93.8 1.5 1:16.88 clamd status shows it's still trying to start: systemctl status clamd@scan.service<mailto:clamd@scan.service> * clamd@scan.service<mailto:clamd@scan.service> - Generic clamav scanner daemon Loaded: loaded (/usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>; enabled; vendor preset: disabled) Active: activating (start) since Tue 2019-07-30 15:21:52 EDT; 26s ago Docs: man:clamd(8) man:clamd.conf(5) https://www.clamav.net/documents/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.clamav.net_documents_&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=2whNina19JXLnZkRg0ik9mRMN2az0Rq8MSra-kr4rGo&e=> Cntrl PID: 5175 (clamd) Tasks: 1 (limit: 4915) Memory: 244.0M CGroup: /system.slice/system-clamd.slice/clamd@scan.service<mailto:system.slice/system-clamd.slice/clamd@scan.service> `-5175 /usr/sbin/clamd -c /etc/clamd.d/scan.conf Jul 30 15:21:52 ourdomain systemd[1]: Starting Generic clamav scanner daemon... And just to be sure: cat /lib/systemd/system/clamd@.service [Unit] Description = clamd scanner (%i) daemon Documentation=man:clamd(8) man:clamd.conf(5) https://www.clamav.net/documents/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.clamav.net_documents_&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=qdbDc4JHdK2uYYGA7aB9lPX-XVVvrrkt2cBbMiopJX4&s=2whNina19JXLnZkRg0ik9mRMN2az0Rq8MSra-kr4rGo&e=> # Check for database existence # ConditionPathExistsGlob=@DBDIR@/main.{c[vl]d,inc} # ConditionPathExistsGlob=@DBDIR@/daily.{c[vl]d,inc} After = syslog.target nss-lookup.target network.target [Service] Type = forking ExecStart = /usr/sbin/clamd -c /etc/clamd.d/%i.conf Restart = on-failure TimeoutSec=600 On Tue, Jul 30, 2019 at 3:12 PM Reio Remma via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote: I suspect it's might be the same issue I had a few days back. Check out the thread "Clamd fails to start with daily.cvd". As suggested by user Axb: in file clamd.service to section: [Service] add TimeoutSec=900 restart clamd service I personally increased the limit to 300 seconds. :) I suspect systemd is killing the process because it goes over the timeout threshold when loading the signatures. Good luck! Reio On 30.07.2019 21:58, Robert Kudyba wrote: rpm -qa clamav-milter clamav-milter-0.101.2-2.fc30.x86_64 rpm -qa clamd clamd-0.101.2-2.fc30.x86_64 See some logs and statuses below. clamd takes up all of the CPU. clamd does appear to start based on the ps command but you can see the status shows no running; PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 26618 root 20 0 214188 207576 7996 R 99.0 0.4 0:10.76 clamd Tue Jul 30 14:30:17 2019 -> WARNING: No clamd server appears to be available Tue Jul 30 14:31:16 2019 -> Failed to establish a connection to clamd Tue Jul 30 14:31:16 2019 -> Probe for slot 1 returned: failed Tue Jul 30 14:31:16 2019 -> WARNING: No clamd server appears to be available Tue Jul 30 14:32:15 2019 -> Failed to establish a connection to clamd Tue Jul 30 14:32:15 2019 -> Probe for slot 1 returned: failed Tue Jul 30 14:32:15 2019 -> WARNING: No clamd server appears to be available ps -auwx|grep clam clamav 2538 0.0 0.0 18348 3156 ? Ss Jul29 0:00 /usr/bin/freshclam -d -c 4 clamav 24692 0.0 0.0 19852 10044 ? Ss 14:10 0:00 /usr/lib/systemd/systemd --user clamav 24697 0.0 0.0 181296 5200 ? S 14:10 0:00 (sd-pam) clamav 24717 0.0 0.0 113064 3312 ? Ss 14:10 0:00 /bin/sh -c [ -x /usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash /usr/local/sbin/clamav-unofficial-sigs.sh > /dev/null clamav 24718 0.0 0.0 113848 3908 ? S 14:10 0:00 /usr/bin/bash /usr/local/sbin/clamav-unofficial-sigs.sh clamilt 26222 0.0 0.0 88488 588 ? Ssl 14:18 0:00 /usr/sbin/clamav-milter -c /etc/mail/clamav-milter.conf root 26227 99.6 0.5 263348 251924 ? Rs 14:18 0:20 /usr/sbin/clamd -c /etc/clamd.d/scan.conf clamav 26360 1.8 0.0 126316 12992 ? S 14:18 0:00 /usr/bin/wget --no-check-certificate --quiet --connect-timeout=60 --random-wait --tries=3 --timeout=180 --output-document=/var/lib/clamav-unofficial-sigs/dbs-si/securiteinfo.hdb https://www.securiteinfo.com/get/signatures/6651194e2baf9979742029c715d7dd90c94e25355ca57fdf22c81828f6fe7a3fc01bfbee6c9a20efa17559c52a04cc4aab1cbe6810596bb16afae8518a9400d1/securiteinfo.hdb\<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.securiteinfo.com_get_signatures_6651194e2baf9979742029c715d7dd90c94e25355ca57fdf22c81828f6fe7a3fc01bfbee6c9a20efa17559c52a04cc4aab1cbe6810596bb16afae8518a9400d1_securiteinfo.hdb-255C&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=1r9FHRsyaDucqFLyd9_yj6gSAcLUUdcJpWaCvwUs0Lk&e=> systemctl status clamd@scan.service<mailto:clamd@scan.service> * clamd@scan.service<mailto:clamd@scan.service> - Generic clamav scanner daemon Loaded: loaded (/usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>; enabled; vendor preset: disabled) Active: inactive (dead) since Mon 2019-07-29 13:24:11 EDT; 24h ago Docs: man:clamd(8) man:clamd.conf(5) https://www.clamav.net/documents/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.clamav.net_documents_&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=hP9lw6TqqvMhxIcjF0PBc9GJj8nl0wNqMlhW1kf4Y7c&e=> Jul 29 13:24:09 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. Jul 29 13:24:11 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: clamd@scan.service<mailto:clamd@scan.service>: Control process exited, code=killed, status=15/TERM Jul 29 13:24:11 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: clamd@scan.service<mailto:clamd@scan.service>: Succeeded. Jul 29 13:24:11 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: Stopped Generic clamav scanner daemon. Jul 30 04:53:06 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. Jul 30 11:13:50 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. Jul 30 11:19:10 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. Jul 30 14:05:05 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. Jul 30 14:05:07 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. Jul 30 14:05:08 ourdomain.edu<https://urldefense.proofpoint.com/v2/url?u=http-3A__ourdomain.edu&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=6m-MquKuRi80BkM8EQ7xbHz37fXGqKYU2rpZtozbpfo&e=> systemd[1]: /usr/lib/systemd/system/clamd@scan.service<mailto:usr/lib/systemd/system/clamd@scan.service>:1: .include directives are deprecated, and support for them will be removed in a future version of systemd. Please use drop-in files instead. systemctl status clamav-milter * clamav-milter.service - Milter module for the Clam Antivirus scanner Loaded: loaded (/usr/lib/systemd/system/clamav-milter.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2019-07-29 13:23:46 EDT; 24h ago Main PID: 4350 (clamav-milter) Tasks: 3 (limit: 4915) Memory: 2.6M CGroup: /system.slice/clamav-milter.service `-4350 /usr/sbin/clamav-milter -c /etc/mail/clamav-milter.conf Jul 29 13:23:45 ourserver systemd[1]: Starting Milter module for the Clam Antivirus scanner... Jul 29 13:23:46 ourserver systemd[1]: Started Milter module for the Clam Antivirus scanner. Tue Jul 30 14:20:11 2019 -> +++ Started at Tue Jul 30 14:20:11 2019 Tue Jul 30 14:20:11 2019 -> Received 0 file descriptor(s) from systemd. Tue Jul 30 14:20:11 2019 -> clamd daemon 0.101.2 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64) Tue Jul 30 14:20:11 2019 -> Log file size limited to 1048576 bytes. Tue Jul 30 14:20:11 2019 -> Reading databases from /var/lib/clamav Tue Jul 30 14:20:11 2019 -> Not loading PUA signatures. Tue Jul 30 14:20:11 2019 -> Bytecode: Security mode set to "TrustSigned". The uncommented directives in /etc/clamd.d/scan.conf are: LogFile /var/log/clamd.scan LogTime yes LogSyslog yes DatabaseDirectory /var/lib/clamav TCPSocket 3310 TCPAddr 127.0.0.1 I had to disable it in sendmail where I had this in sendmail.mc<https://urldefense.proofpoint.com/v2/url?u=http-3A__sendmail.mc&d=DwMDaQ&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=O-HFjqRKx8lUg5kjTbP_aR07F4djoMOoFcS0SywdWWk&s=Nm05g7xhCGkbvqrPNmiCxoaZIINskP8j9ub976PYskg&e=>: INPUT_MAIL_FILTER(`clamav-milter', `S=inet:6666@127.0.0.1<mailto:inet%3A6666@127.0.0.1>, F=, T=S:4m;R:4m')dnl This all starting happening after a reboot. Any ideas what may be wrong?
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
