On Tuesday 02 September 2014 23:12, Steven Morgan wrote:
> Hi J. David,
>
> Thanks for the additional analysis and information. I've been looking at
> this for a bit today. I have opened a ticket in the ClamAV bugzilla system
> to track the issue. The ticket number is 11089. Hope to have an answer
> soon.
>
Have been doing a bit of hacking around with the code.
Have not been able to get the logging functions built in to libclamd
functions such as "cli_dbgmsg" to log reliably.
freshcam and clamd appear to use a call back to intercept and route
their output via their own function "logg". However clamav-milter
seeems to be missing the necessary bits to do this.
I ended up putting direct calls in to fprintf and fflush.
problem is definitely in cl_hash_data. As a work round i have patched
the function cli_md5buff in libclamav/others_common.c to use the
random data directly to create the file name when the call to
cl_hash_data fails.
Diagnostic output from my current version below.
Note it is the call to EVP_get_digestbyname from cli_hash_data that fails.
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-cc4ad42fc23e93aaf82a1acc428307d9.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-cc4ad42fc23e93aaf82a1acc428307d9.tmp: 11
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-6296cff0b0f8e0d462e2a82ede5885ac.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-6296cff0b0f8e0d462e2a82ede5885ac.tmp: 13
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-40f9e43c4dc6f6213d604a6c642bb738.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-40f9e43c4dc6f6213d604a6c642bb738.tmp: 11
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-45633b0b40207939c2acdc5d752f0862.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-45633b0b40207939c2acdc5d752f0862.tmp: 12
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-876a73b889eb4308c4773efa6818933d.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-876a73b889eb4308c4773efa6818933d.tmp: 12
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-6ed0e7970718b603f4125c9c920aecee.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-6ed0e7970718b603f4125c9c920aecee.tmp: 11
LibClamAV Info: cli_hash_data: Called
LibClamAV Error: cli_hash_data: Call to EVP_get_digestbyname failed
cli_md5buff:fprint Call to cl_hash_data failed to return hash
cli_gentemp: filename /tmp/clamav-0e1024e48ddf5d9245ec42cc366bba55.tmp
LibClamAV Error: cli_gentempfd: Createed temporary file
/tmp/clamav-0e1024e48ddf5d9245ec42cc366bba55.tmp: 12
> Steve
>
> On Sun, Aug 31, 2014 at 5:52 AM, J. David Rye <d....@roadtech.co.uk> wrote:
> > On Thu, 2014-08-21 at 19:22 -0400, Steven Morgan wrote:
> > > Hi Urban,
> > >
> > > I took a look at this code. The real problem is the inability to
> > > create a
> > > temporary file. The second message just results from the return code
> > > of the
> > > function that attempts to create the temp file. We need to find out
> > > why the
> > > temp file creation fails. There should also be a clamav error message
> > > written from: cli_errmsg("cli_gentempfd: Can't create temporary file
> > > %s:
> > > %s\n", *name, strerror(errno)); Can you find this message?
> > >
> > > Otherwise, it is a memory allocation failure for space for the temp
> > > file
> > > name, which seems unlikely.
> > >
> > > Steve
> >
> > I am also seeing this issue. Mostly intermitant but see further down.
> >
> > cli_errmsg wont work if clamav-milter has daemonezed.
> > it only writes to STDERR and the function daemonize closes standard
> > error even if you recompile with CL_DEBUG set.
> >
> > Only way to get is to get the error messages from cli_gentempfd seams
> > to be to uncomment the line
> >
> > "#Foreground yes"
> >
> > In clamav-milter.conf, then run in foreground from command line.
> >
> > As an aside I wonder why cli_gentempfd does not use the function logg()
> > and output to file or syslog depending on configuration file.
> >
> > I am running clamav-milter on a VM. OS is CentoOS 6.5
> > VM has 4 vcpu, and 2GB RAM
> > clamav-milter is version 0.98.4-1.el6.rf installed from rpmforge
> > repository.
> >
> > Looking at he logs if time stamps in syslog for calls to clamav-milter
> > are two seconds or more apart the problem never shows.
> >
> > However if 4 or more messages arrive in two seconds problem always shows
> > up, the failure to create temp file is usually time stamped 2 seconds
> > after the first message in the burst that triggered it.
> >
> > On a sustained burst of traffic pretty much all the messages trip the
> > issue.
> > In a 1 hour period last week when I had a lot of messages due to a
> > different issue. I had 20,000 temp file failures, and 23 messages
> > delivered.
> >
> > [root@mailhost-c6 etc]# clamav-milter
> > --config-file=/etc/clamav-milter.conf.foreground
> > Local socket unix:/var/run/clamav/clamd.sock added to the pool (slot 1)
> > Probe for slot 1 returned: success
> > LibClamAV Error: cli_gentempfd: Can't create temporary
> > file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
> > ERROR: Failed to create temporary file
> > ERROR: Failed to initiate streaming/fdpassing
> > LibClamAV Error: cli_gentempfd: Can't create temporary
> > file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
> > ERROR: Failed to create temporary file
> > ERROR: Failed to initiate streaming/fdpassing
> > LibClamAV Error: cli_gentempfd: Can't create temporary
> > file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
> > ERROR: Failed to create temporary file
> > ERROR: Failed to initiate streaming/fdpassing
> > LibClamAV Error: cli_gentempfd: Can't create temporary
> > file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
> > ERROR: Failed to create temporary file
> > ERROR: Failed to initiate streaming/fdpassing
> > LibClamAV Error: cli_gentempfd: Can't create temporary
> > file /tmp/clamav-0000000000000000626683ff3a000000.tmp: File exists
> > ERROR: Failed to create temporary file
> > ERROR: Failed to initiate streaming/fdpassing
> > Message from <n0r3ply812...@scotland117.wanadoo.co.uk> to
> > <stevensonbros> infected by Heuristics.Phishing.Email.SpoofedDomain
> > Message from <n0r3ply620...@aughamullan.dungannon.ni.sch.uk> to
> > <brett01> infected by Heuristics.Phishing.Email.SpoofedDomain
> > Probe for slot 1 returned: success
> >
> >
> > I think
> >
> > clamav-milter does a lot of initialization, including setting up a
> > structure with a list of function entry points it then calls smfi_main.
> >
> > smfi_main in turn forks one thread for each message, and calls the entry
> > points in the context of the thread.
> >
> > call back to function clamfi_header
> > that calls sendchunk
> > which calls nc_connect_rand
> > which calls cli_gentempfd which prints the EEXISTS errors to stderr
> > shown above.
> >
> > File name looks like it is supposed to be based on a 16 byte MD5 digest
> > printed in hex.
> >
> > cli_gentempfd builds the name by calling cli_gentemp
> > which adds 32 bytes of random data from cli_rndnum to a 16 byte
> > seed
> > passes a pointer the the 48 buffer to cli_md5buff
> > which in turn calls cl_hash_data
> > digest returned is used to create the file name,
> > and update the seed for the next name.
> >
> >
> > Note cli_gentemp is common to all programs in the clam set.
> >
> > Note cl_hash_data is calling openssl library functions to calculate the
> > digest.
> >
> > Note cli_md5buff does not check for an error in cl_hash_data
> >
> > _______________________________________________
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/vrtadmin/clamav-faq
> > http://www.clamav.net/support/ml
>
> _______________________________________________
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> http://www.clamav.net/support/ml
_______________________________________________
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
