Hi Mark, TL;DR: The type detection mismatch is fixed in the current daily + 0.103.1. The issue was with the signature. We didn't know about it because of the mismatch. You should've found that the offending signature was dropped on Saturday morning.
Details: 0.103.1 introduced CL_TYPE_TIFF and changed TIFF file type recognition from: 0:0:49492a00:TIFF Little Endian:CL_TYPE_ANY:CL_TYPE_GRAPHICS 0:0:4d4d:TIFF Big Endian:CL_TYPE_ANY:CL_TYPE_ GRAPHICS to: 0:0:49492a00:TIFF Little Endian:CL_TYPE_ANY:CL_TYPE_TIFF 0:0:4d4d:TIFF Big Endian:CL_TYPE_ANY:CL_TYPE_TIFF When FTM signatures are loaded from daily.cvd, it overrides the built-in FTM signatures. So it turns out that daily's FTM file had been missing the original CL_TYPE_GRAPHICS detection of TIFF files all this time, which would've been required for Target:5 signatures to alert on TIFF files. As a result, the signature in question "worked" in testing (with a single LDB file, using built-in FTM), but never worked in worked during FP testing or in production (with a daily CVD file). When we added this to daily.ftm to support 0.103.1: 0:0:49492a00:TIFF Little Endian:CL_TYPE_ANY:CL_TYPE_TIFF:122 0:0:4d4d:TIFF Big Endian:CL_TYPE_ANY:CL_TYPE_TIFF:122 ... all of a sudden a signature which was written for TIFF files started alerting on TIFF files (as it should've) because the new CL_TYPE_TIFF also alerts on Target:5 (graphics) types. We never added the CL_TYPE_GRAPHICS variant for 0.103.0 and prior, which is why it appeared to be an issue with 0.103.1. Perhaps we should? I'll ask MRT about it. Anyways, this is basically a reminder that we need to make sure daily FTM and libclamav's FTM are in sync. -Micah > -----Original Message----- > From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On Behalf Of > Mark Allan > Sent: Saturday, February 13, 2021 3:35 PM > To: ClamAV Development <clamav-devel@lists.clamav.net> > Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1 > > Thanks. I've just found another one too > > BC.Img.Exploit.CVE_2017_11255-6335669-1 > > It's triggering on a file that's been part of macOS for many years. It's also > a tiff > file. I can submit this as well if necessary? > > Out of interest, is the type detection mismatch something that can be fixed > in daily.cvd or can I patch libclamav/filetypes_int.h to revert it to what it > was > at 0.103.0? > > Mark > > > On 12 Feb 2021, at 5:23 am, Micah Snyder (micasnyd) > <micas...@cisco.com> wrote: > > > > It appears to me to be an issue with the signature which is only evident in > 0.103.1 now that we're matching TIFFs with Target:5 signatures, like this one. > > > > There was apparently a mismatch for TIFF file type detection between the > file type magic signatures built-in to libclamav (libclamav/filetypes_int.h) > and > the .ftm sigs shipped with daily.cvd (which override the internal ones when > loaded). > > > > I'll ask to have the signature dropped and re-evaluated. > > > > -Micah > > > >> -----Original Message----- > >> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On Behalf > >> Of Micah Snyder (micasnyd) > >> Sent: Thursday, February 11, 2021 8:27 PM > >> To: ClamAV Development <clamav-devel@lists.clamav.net> > >> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1 > >> > >> Thank you Mark! We'll take a look. > >> > >> -Micah > >> > >>> -----Original Message----- > >>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On > Behalf > >>> Of Mark Allan > >>> Sent: Thursday, February 11, 2021 3:54 PM > >>> To: ClamAV Development <clamav-devel@lists.clamav.net> > >>> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1 > >>> > >>> Hi Micah, > >>> > >>> Yes of course! I've just uploaded a zip file (Archive.zip) to the FP > >>> page on clamav.net > >>> MD5 (Archive.zip) = 45229d954a884a1e03aba15b9f42168a > >>> > >>> Regards > >>> Mark > >>> > >>>> On 11 Feb 2021, at 7:12 pm, Micah Snyder (micasnyd) > >>> <micas...@cisco.com> wrote: > >>>> > >>>> Hi Mark, > >>>> > >>>> Do you think you could share a sample or two with me to test. I'm > >>>> really > >>> curious what changed and would like to debug each version with a > >>> sample or two. > >>>> > >>>> -Micah > >>>> > >>>>> -----Original Message----- > >>>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On > >>>>> Behalf Of Mark Allan > >>>>> Sent: Monday, February 8, 2021 3:04 AM > >>>>> To: ClamAV Development <clamav-devel@lists.clamav.net> > >>>>> Subject: [Clamav-devel] Issue with FP only on 0.103.1 > >>>>> > >>>>> Hi all, > >>>>> > >>>>> It looks like the additional image file type support in 0.103.1 > >>>>> has introduced an issue with a particular signature which has been > >>>>> in the database since 2018 > >>>>> > >>>>> Img.Exploit.CVE_2018_4904-6449838-0 > >>>>> > >>>>> It's flagging up thousands of known-good files. As far as I can > >>>>> tell, they're all TIFF files. > >>>>> > >>>>> I've added that signature to an ign2 file for now, but I'm > >>>>> wondering if there's something else that's maybe amiss somewhere > >>>>> either with the signature or the 0.103.1 update? > >>>>> > >>>>> Best regards, > >>>>> Mark > >>>>> > >>>>> _______________________________________________ > >>>>> > >>>>> clamav-devel mailing list > >>>>> clamav-devel@lists.clamav.net > >>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel > >>>>> > >>>>> Please submit your patches to our Github: > >>>>> https://github.com/Cisco- Talos/clamav-devel/pulls > >>>>> > >>>>> Help us build a comprehensive ClamAV guide: > >>>>> https://github.com/vrtadmin/clamav-faq > >>>>> > >>>>> http://www.clamav.net/contact.html#ml > >>>> _______________________________________________ > >>>> > >>>> clamav-devel mailing list > >>>> clamav-devel@lists.clamav.net > >>>> https://lists.clamav.net/mailman/listinfo/clamav-devel > >>>> > >>>> Please submit your patches to our Github: > >>>> https://github.com/Cisco-Talos/clamav-devel/pulls > >>>> > >>>> Help us build a comprehensive ClamAV guide: > >>>> https://github.com/vrtadmin/clamav-faq > >>>> > >>>> http://www.clamav.net/contact.html#ml > >>> > >>> _______________________________________________ > >>> > >>> clamav-devel mailing list > >>> clamav-devel@lists.clamav.net > >>> https://lists.clamav.net/mailman/listinfo/clamav-devel > >>> > >>> Please submit your patches to our Github: https://github.com/Cisco- > >>> Talos/clamav-devel/pulls > >>> > >>> Help us build a comprehensive ClamAV guide: > >>> https://github.com/vrtadmin/clamav-faq > >>> > >>> http://www.clamav.net/contact.html#ml > >> _______________________________________________ > >> > >> clamav-devel mailing list > >> clamav-devel@lists.clamav.net > >> https://lists.clamav.net/mailman/listinfo/clamav-devel > >> > >> Please submit your patches to our Github: https://github.com/Cisco- > >> Talos/clamav-devel/pulls > >> > >> Help us build a comprehensive ClamAV guide: > >> https://github.com/vrtadmin/clamav-faq > >> > >> http://www.clamav.net/contact.html#ml > > _______________________________________________ > > > > clamav-devel mailing list > > clamav-devel@lists.clamav.net > > https://lists.clamav.net/mailman/listinfo/clamav-devel > > > > Please submit your patches to our Github: > > https://github.com/Cisco-Talos/clamav-devel/pulls > > > > Help us build a comprehensive ClamAV guide: > > https://github.com/vrtadmin/clamav-faq > > > > http://www.clamav.net/contact.html#ml > > _______________________________________________ > > clamav-devel mailing list > clamav-devel@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-devel > > Please submit your patches to our Github: https://github.com/Cisco- > Talos/clamav-devel/pulls > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml _______________________________________________ clamav-devel mailing list clamav-devel@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-devel Please submit your patches to our Github: https://github.com/Cisco-Talos/clamav-devel/pulls Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
