W dniu 2018-01-26 15:26, Per Jessen napisał(a):
Per Jessen wrote:
Dianne Skoll wrote:
Hi,
Something went badly wrong with clamd recently; it's stuck with
hundreds/thousands of open files per process and interrupting mail
flow.
When a scanning thread finishes, I see this in the strace output.
(I ran clamdscan /etc/hosts as a test):
[pid 3707] 02:11:01 sendto(295, "/etc/hosts: OK\n", 15, 0, NULL, 0)
[= 15
[pid 3707] 02:11:01 shutdown(295, SHUT_RDWR) = 0
[pid 3707] 02:11:01 close(295) = 0
[pid 3707] 02:11:01 futex(0x1933c3c,
[FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 387, {1516950691,
[0}, ffffffff) = -1 ETIMEDOUT (Connection timed out)
[pid 3707] 02:11:31 futex(0x1933c10, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 3707] 02:11:31 madvise(0x7fae6affe000, 8368128, MADV_DONTNEED)
[= 0
[pid 3707] 02:11:31 _exit(0) = ?
[pid 3707] 02:11:31 +++ exited with 0 +++
So it scans the file, says it's OK. and then hangs in the futex for
30 seconds.
HELP! This is causing major outages for many of our customers.
We're seeing something similar - this morning with 0.99.2, later with
0.99.3. I am guessing there is something wrong in the database?
We run our own daemon using the clamav libraries - in /tmp I see
directories like this:
clamav-168beed7241e57cd439e26cd5fd96eac.tmp/
they seem to be piling up and our daemon eventually runs out of file
descriptors.
Same here. clamd 0.99.2
I have clamd processes running on separate servers.
Each running clamd keeps 1024 file descriptors open (all that it can):
# ls -l /proc/29866/fd
total 0
lr-x------ 1 root root 64 Jan 26 15:44 0 -> pipe:[4007352]
l-wx------ 1 root root 64 Jan 26 15:44 1 -> pipe:[4007353]
l-wx------ 1 root root 64 Jan 26 15:44 10 -> pipe:[4006371]
lrwx------ 1 root root 64 Jan 26 15:44 100 ->
/tmp/clamav-10911737c186fa7dc6ef20e590b66015.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1000 ->
/tmp/clamav-66ef5c3821ee0de410a09cb56158637c.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1001 ->
/tmp/clamav-35ef3a074c015870de87bca44ccbae69.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1002 ->
/tmp/clamav-400a243c437eb8f030a095b885586b45.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1003 ->
/tmp/clamav-509983ba9c3f50435f102bfe19b8149b.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1004 ->
/tmp/clamav-018c4e6e767f7d994fb343a4d0dd6a45.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1005 ->
/tmp/clamav-1381c37f81ff0a4426c4648f9b88cdd0.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1006 ->
/tmp/clamav-20bf11a62d664d167ccb743277404413.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1007 ->
/tmp/clamav-da1a0ea777e27fa22156e17c9ae082bd.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1008 ->
/tmp/clamav-ca777e11ae7a58542f5dfa07176ce58f.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1009 ->
/tmp/clamav-e0f872aed2725b9eeae6420ad6f11cb3.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 101 ->
/tmp/clamav-74354d2674a0e4003b6c781e0976f177.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1010 ->
/tmp/clamav-f28e4189d6933861c92a0f714bced25f.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1011 ->
/tmp/clamav-784cdb452ed3fb2acec5c02326cf45e7.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1012 ->
/tmp/clamav-6be671cee5cc4c7f159f88f3582bf745.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1013 ->
/tmp/clamav-55f5e509b4d6c995cda89cf68c02dad4.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1014 ->
/tmp/clamav-c60608f4e98ad8774bf0ced9c7d4c7b3.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1015 ->
/tmp/clamav-848055158702437590e6f30378576dac.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1016 ->
/tmp/clamav-0456073a9443ab3e3f6990306cc82666.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1017 ->
/tmp/clamav-9132c07058a0046bb845ed5887b77b92.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1018 ->
/tmp/clamav-9ba25f2151327a7daaf44914467d9fb2.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1019 ->
/tmp/clamav-55ac8c33daaf55d3dda527f55bfc71b3.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 102 ->
/tmp/clamav-9f49783a5c8797427c5ba00693c2c103.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1020 ->
/tmp/clamav-ca3051fda967d90873121fcc62023102.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1021 ->
/tmp/clamav-6b38a0aa9921810680a2b7dbcb1b1405.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1022 ->
/tmp/clamav-bb3ecaa7e3702cf0e828709df363a04d.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 1023 ->
/tmp/clamav-4da381032e4a1f382b67e234c33d8003.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 103 ->
/tmp/clamav-4646c34f92eb8dec155b96408a325e80.tmp (deleted)
lrwx------ 1 root root 64 Jan 26 15:44 104 ->
/tmp/clamav-179e91d62f159efb7d8c278a3724c17c.tmp (deleted)
...
clamd can't accept more connections and is logging "-> ERROR: accept()
failed:"
Problem started today about 6 am CET (with higher mail traffic).
Regards,
Jacek
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
http://www.clamav.net/contact.html#ml
