Hi Kevin, Very well explained. Thank you so much.
In the case of Case #6 I did not see any trace of "UNKNOWN COMMAND" either in clamd terminal logs or strace. Here is the terminal logs from clamd in debug mode. $fds_poll_recv: timeout after 600 seconds $Received POLLIN|POLLHUP on fd 6 $Got new connection, FD 11 $Received POLLIN|POLLHUP on fd 7 $fds_poll_recv: timeout after 5 seconds $Received POLLIN|POLLHUP on fd 11 $Receveived a file descriptor: 12 $got command FILDES (7, 11), argument: $RECVTH: FILDES command complete $THRMGR: queue (single) crossed low threshold -> signaling $THRMGR: queue (bulk) crossed low threshold -> signaling $mode -> MODE_WAITREPLY LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16) LibClamAV debug: Recognized ASCII text LibClamAV debug: cache_check: 43146aa1f85ee7fabf908998d8d8c4a0 is negative $Breaking command loop, mode is no longer MODE_COMMAND LibClamAV debug: hashtab: Freeing hashset, elements: 0, capacity: 0 LibClamAV debug: in cli_scanscript() LibClamAV debug: cli_scanscript: saving normalized file to /var/tmp/clamav-014eb26dadbb9124c9a4f342fc4a3bfc.tmp $Consumed entire command LibClamAV debug: cli_magic_scandesc: returning 0 at line 2470 LibClamAV debug: cache_add: 43146aa1f85ee7fabf908998d8d8c4a0 (level 0) $Number of file descriptors polled: 1 fds $fds_poll_recv: timeout after 600 seconds $Closed fd 12 $Finished scanthread $Scanthread: connection shut down (FD 11) $THRMGR: queue (single) crossed low threshold -> signaling $THRMGR: queue (bulk) crossed low threshold -> signaling $Received POLLIN|POLLHUP on fd 6 $Got new connection, FD 11 $Received POLLIN|POLLHUP on fd 7 $fds_poll_recv: timeout after 5 seconds $Received POLLIN|POLLHUP on fd 11 $Receveived a file descriptor: 12 $got command FILDES (7, 11), argument: $RECVTH: FILDES command complete $mode -> MODE_WAITREPLY $Breaking command loop, mode is no longer MODE_COMMAND $Consumed entire command $Number of file descriptors polled: 1 fds $fds_poll_recv: timeout after 600 seconds $THRMGR: queue (single) crossed low threshold -> signaling $THRMGR: queue (bulk) crossed low threshold -> signaling LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16) LibClamAV debug: Recognized ASCII text LibClamAV debug: Matched signature for file type HTML data at 51 LibClamAV debug: cache_check: 8c8b4137fd8974c6be208a43461e19a6 is positive LibClamAV debug: cli_magic_scandesc: returning 0 at line 2681 (no post, no cache) $Closed fd 12 $Finished scanthread $Scanthread: connection shut down (FD 11) $THRMGR: queue (single) crossed low threshold -> signaling $THRMGR: queue (bulk) crossed low threshold -> signaling $fds_poll_recv: timeout after 600 seconds And the temporary file written as. Note: I removed the actual virus signature with "<EICAR SIGNATURE>" ------------------------------df07680798a9 content-disposition: form-data; name="attachment"; filename="eicar.com" content-type: application/octet-stream <EICAR SIGNATURE> ------------------------------df07680798a9-- [r If clamd doesn't support parsing HTTP Messages then that explains all NOT WORKING cases, I think. What is your thought about this? Regards Manoj Ramakrishnan On 18/02/15 6:09 AM, "Kevin Lin" <k...@sourcefire.com> wrote: >There are a number of reasons for the differences in the detection cases. > >The first of which is how ClamAV identifies the file type of file being >scanned. ClamAV determines the file type of a scanned file using the 'ftm' >signature files. The important signatures follow: > >type:offset:magic:rtype:type > >0:0:504b0304:ZIP:CL_TYPE_ANY:CL_TYPE_ZIP >0:0:504b3030504b0304:ZIP:CL_TYPE_ANY:CL_TYPE_ZIP >1:*:504b0304:ZIP-SFX:CL_TYPE_ANY:CL_TYPE_ZIPSFX > >0:0:1f8b:GZip:CL_TYPE_ANY:CL_TYPE_GZ > >There ZIP archive file type signatures, two of which look for a specific >magic at offset 0. However, the last signature uses a '*' offset which >indicates the magic can be located anywhere within the file. Do note that >the signature is meant to detect the specific variant of ZIPSFX. > >The GZ file, on the other hand, only has one magic that only triggers if >found at offset 0. > >While an argument could be to extend the GZ file type signature file to >search the entire file, there are a number of important counter arguments: > > 1. The GZ file magic is only 2 bytes long, this means that the >extension > over the whole file would result in a large number of false positives > 2. In theory, by modifying the original GZ file, the file may no longer > be a valid GZ file. Thus it's likely that ClamAV would not be able to > correctly parse the file. > >Argument 2 may also result in the lack of detection as the file may not be >possible to parse with modifications. > > >As for the reason for the curl POST issue in case #6, can I ask how you >what response you get back from clamd when you upload the file using curl >POST? > >clamd is designed to handle a specific of commands that are described in >the clamdoc.pdf that comes with the ClamAV source distribution. From what >I >can see, clamd does not natively support parsing HTTP messages. When I >send >a file to scan to clamd using curl, clamd fails to understand the message >and sends back the message: > >UNKNOWN COMMAND > >-Kevin > >On Tue, Feb 17, 2015 at 1:23 PM, Noel Jones <njo...@megan.vbhcs.org> >wrote: > >> On 2/17/2015 12:11 AM, Manoj Ramakrishnan wrote: >> > Hi Al, >> > >> > Thanks for replying. >> > It is exactly what I thought. But why is it different from ZIP file? >> > I added extra characters in the beginning of the ZIP file but no >>issues >> in >> > scanning that and finding eicar signature. >> >> zip and gzip are very different formats. I suppose you added your >> random character at a point where unzip ignored it. >> >> >> > >> > Also curious to see why is it not working in case #4 and #6? >> >> Either broke the eicar file with leading or trailing characters, or >> maybe the squid plugin didn't recognize the file as a gzip. Use the >> clam debug tools to examine the files extracted and scanned. >> >> The eicar signature is *very* specific, anchored at both the >> beginning and end allowing only for a few extra spaces at the end of >> the payload, no other extra characters. >> _______________________________________________ >> Help us build a comprehensive ClamAV guide: >> https://github.com/vrtadmin/clamav-faq >> >> http://www.clamav.net/contact.html#ml >> >_______________________________________________ >Help us build a comprehensive ClamAV guide: >https://github.com/vrtadmin/clamav-faq > >http://www.clamav.net/contact.html#ml _______________________________________________ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
