I have 48472 and 48473. The 48474 I got was the gdb file that was downloaded as part of the cdiff. The freshclam process hung after downloading though. The order of the 48474 gdb file was no different from the order of the 48473 file.
Freshclam gets this far before hanging after the download. The gdb file listed there has the same format. Wed Mar 6 16:50:46 2019 -> *main.cvd version from DNS: 58 Wed Mar 6 16:50:46 2019 -> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr) Wed Mar 6 16:50:46 2019 -> *daily.cvd version from DNS: 25380 Wed Mar 6 16:50:46 2019 -> daily.cvd is up to date (version: 25380, sigs: 1503528, f-level: 63, builder: raynman) Wed Mar 6 16:50:46 2019 -> *safebrowsing.cvd version from DNS: 48474 LibClamAV debug: in cli_untgz() LibClamAV debug: cli_untgz: Unpacking /home/logins/mbroekman/analysis/tmp/clamav-317041d4b9d853e83b60005464dd098c.tmp/clamav-b4a94beaae2191e11c7805c6e49be7e6.tmp/COPYING LibClamAV debug: cli_untgz: Unpacking /home/logins/mbroekman/analysis/tmp/clamav-317041d4b9d853e83b60005464dd098c.tmp/clamav-b4a94beaae2191e11c7805c6e49be7e6.tmp/ safebrowsing.info LibClamAV debug: cli_untgz: Unpacking /home/logins/mbroekman/analysis/tmp/clamav-317041d4b9d853e83b60005464dd098c.tmp/clamav-b4a94beaae2191e11c7805c6e49be7e6.tmp/safebrowsing.gdb LibClamAV debug: in cli_untgz_cleanup() Wed Mar 6 16:50:49 2019 -> *Retrieving http://db.US.clamav.net/safebrowsing-48474.cdiff Wed Mar 6 16:50:49 2019 -> nonblock_connect: connect(): fd=4 errno=101: Network is unreachable Wed Mar 6 16:50:49 2019 -> Can't connect to port 80 of host db.US.clamav.net (IP: 2606:4700::6810:da54) Wed Mar 6 16:50:49 2019 -> *Trying to download http://db.US.clamav.net/safebrowsing-48474.cdiff (IP: 104.16.219.84) Wed Mar 6 16:50:49 2019 -> Downloading safebrowsing-48474.cdiff [100%] The last time I ran freshclam, I was stuck at 100% on the download for 40 minutes before I killed the process. The info file in the tmpdir shows: ClamAV-VDB:06 Mar 2019 13-24 -0500:48474:3232286:63:X:X:google:1551896655 safebrowsing.gdb:132636452:7f6645b8d865de3992be1ad5de215afd848acee4c021eed4818fdb760f76b57e DSIG:NxsTJGIb7EQ9e71CjIH2QJYzp+BhrH0qK1Mb0Ef5BQfO5WZnm8qZSqj/y6vstcjAOUfWwLG8ba3RemesF+KxIuk/HMkDgRCJep+shVvz8nAccajvbBN1ZnmpTkf1T0QgTsDbuBK9cTItdlQWupKfuiV1aKKdF1jSLvtRJU4zoZl+B3/qgIAPi7sqmkh8W5qKplYdsICdfmDLxK5dDwCkGmdtXZol5pHHXTQb1/LJqml8SORrFydkYizuVl07/uuc332dk5Uk1NfZrDj94wG0dIIloWiwfPzj563Vl5e7GvCvCdMR1Gfq3EGYZGSPftR7a/K7TashvsoWP2Uma0Fq/ On Wed, Mar 6, 2019 at 5:47 PM David Raynor <dray...@sourcefire.com> wrote: > That's strange, the 48474 I have should have the sorting changed and has > the improved loading time we're talking about. > > $ sigtool --info safebrowsing.cvd > File: safebrowsing.cvd > Build time: 06 Mar 2019 13:24 -0500 > Version: 48474 > Signatures: 3232286 > Functionality level: 63 > Builder: google > MD5: 70c61f41e52b5a2134ff7e272f5a6df1 > > SHA256 (safebrowsing.gdb) = > 7f6645b8d865de3992be1ad5de215afd848acee4c021eed4818fdb760f76b57e > > Something must be different. > > Dave R. > > On Wed, Mar 6, 2019 at 5:39 PM Maarten Broekman via clamav-users < > clamav-users@lists.clamav.net> wrote: > >> The new safebrowsing cvd (starting with version 48473) seems to be sorted >> in a way that increases the load time of that file by several orders of >> magnitude. >> >> I have a previous version from February where the entries in the gdb >> section are sorted like this: >> >> S2:F:0000917787cff7b0993917209809ff3d94bec7e1de7188b323d9b88e0273cb71 >> S2:F:000149794d90dc5bce4f685deed6076d00c9209bd81cef4cbdf8a4e41f0a2153 >> S2:F:00042c895c912fd567afa35450cfe5d321d0d68eb3833156925c4e27d2c29aa2 >> S2:F:0006d4dcb0d939d725e676a9e68aaeb303e04478e6861d2a77469d1b6a0a0f7d >> S2:F:0007bf7c1808d12177f0ae90d336d60c5a7a3d89703806955b75c56f898dd919 >> ... >> S2:P:00009177 >> S2:P:00014979 >> S2:P:00042c89 >> S2:P:0006d4dc >> S2:P:0007bf7c >> ... >> S:F:00000860493997b798861956e06d3d3606f82384259b971bb922f94f886a4b55 >> S:F:00000bddafae162a7a2f1249b3b38c8e4b6d3cb8bf0c30c26cc354ebcba16b37 >> S:F:000046cad35fbecbcc8dd4ebb244bd08aa6dbf1078279115c82f8e21b2cf8478 >> S:F:0000684200da7b11f38a6f4719bda4ec6c6ae8b2be1f7e12a16605b2d3a5d490 >> S:F:000072f3f33e47a2f97b8711d240267462aa3f0a5f8130845b119a2ad3798292 >> ... >> S:P:00000860 >> S:P:00000bdd >> S:P:000046ca >> S:P:00006842 >> S:P:000072f3 >> >> >> That loads into clamd (and clamscan) in under 5 seconds for the 3041760 >> entries in it. >> >> Version 48473 and 48474 are sorted like this: >> >> S2:P:00009177 >> S2:F:0000917787cff7b0993917209809ff3d94bec7e1de7188b323d9b88e0273cb71 >> S2:P:00014979 >> S2:F:000149794d90dc5bce4f685deed6076d00c9209bd81cef4cbdf8a4e41f0a2153 >> ... >> >> >> That version loads in 50+ seconds for the 3229612 entries in it. >> >> If I flip the order of the entries so the :F: entries comes before the >> corresponding :P: entry, it loads the same number of entries in 5 - 10 >> seconds. >> If I reorder the entire file so that _all_ the :F: entries for each >> section (S or S2) come before the :P: entries for that section, it loads in >> under 5 seconds again. >> >> Earlier today it was mentioned that 'the next version of the CVD' would >> fix it (when 48473 was the current version). That seems to have not been >> the case since 48474 didn't fix it. Is there a plan to fix it? Or will we >> have to live with the enormous load times for this database? >> >> --Maarten >> >> >> >> _______________________________________________ >> >> clamav-users mailing list >> clamav-users@lists.clamav.net >> https://lists.clamav.net/mailman/listinfo/clamav-users >> >> >> Help us build a comprehensive ClamAV guide: >> https://github.com/vrtadmin/clamav-faq >> >> http://www.clamav.net/contact.html#ml >> > > > -- > --- > Dave Raynor > Talos Security Intelligence and Research Group > dray...@sourcefire.com > > _______________________________________________ > > clamav-users mailing list > clamav-users@lists.clamav.net > https://lists.clamav.net/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml >
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
