It sounds like at some point the freshclam process being used to populate the private mirror failed to delete the CVD file after applying a CDIFF and generating the CLD.
The client in private mirror mode will look for both a CVD and a CLD and download the file with the correct version number after doing a HEAD for the metadata of the file. I'd start by deleting all the data from the mirror and re-initializing it with freshclam. After 4 hours or so run freshclam again on the master to make sure it downloads and apples the CDIFF correctly and generates the CLD and removes the CLD. It's possible there is/was a permissions issue not allowing freshclam to delete the original CVD. Tom > On Jan 18, 2018, at 3:45 PM, Al Varnell <alvarn...@mac.com> wrote: > > As I mentioned before, I don't use a private mirror, so I've dropped out of > the conversation. Someone who does use option 2 needs to double-check your > settings here. > > -Al- > > On Thu, Jan 18, 2018 at 10:07 AM, Hugo Deprez wrote: >> Hello, >> >> I don't think I'm the only one having this issue. It seems that I used the >> official configuration from http://www.clamav.net/doc/mirrors-faq.html >> <http://www.clamav.net/doc/mirrors-faq.html> >> Should I report the issue somewhere ? >> >> The only way I see to avoid this issue is to create a shell script and to >> push the updates my self on all servers, but this is not the proper way to >> do this... >> >> Hugo >> >> On 15 January 2018 at 11:44, Hugo Deprez <hugo.dep...@gmail.com >> <mailto:hugo.dep...@gmail.com>> wrote: >> >>> Hello, >>> >>> by using this freshclam.conf : >>> >>> AllowSupplementaryGroups false >>> Bytecode true >>> Checks 24 >>> CompressLocalDatabase no >>> ConnectTimeout 30 >>> DNSDatabaseInfo current.cvd.clamav.net <http://current.cvd.clamav.net/> >>> DatabaseDirectory /var/lib/clamav >>> DatabaseMirror mirror.recette.local >>> DatabaseOwner clamav >>> Debug false >>> Foreground false >>> LogFacility LOG_LOCAL6 >>> LogFileMaxSize 0 >>> LogRotate true >>> LogSyslog true >>> LogTime yes >>> LogVerbose false >>> MaxAttempts 5 >>> PidFile /var/run/clamav/freshclam.pid >>> #PrivateMirror mirror.recette.local >>> ReceiveTimeout 30 >>> ScriptedUpdates no >>> TestDatabases yes >>> UpdateLogFile /var/log/clamav/freshclam.log >>> >>> I got the old issue : >>> >>> root@server:/var/lib/clamav# rm -f * >>> root@server:/var/lib/clamav# freshclam >>> ClamAV update process started at Mon Jan 15 11:38:48 2018 >>> Downloading main.cvd [100%] >>> main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr) >>> Downloading daily.cvd [100%] >>> daily.cvd updated (version: 24223, sigs: 1825336, f-level: 63, builder: >>> neo) >>> Downloading bytecode.cvd [100%] >>> bytecode.cvd updated (version: 315, sigs: 75, f-level: 63, builder: >>> raynman) >>> Database updated (6391660 signatures) from mirror.recette.local (IP: >>> 192.168.1.15) >>> root@raadmin:/var/lib/clamav# freshclam >>> ClamAV update process started at Mon Jan 15 11:40:00 2018 >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: >>> sigmgr) >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, >>> builder: neo) >>> Downloading bytecode.cvd [100%] >>> WARNING: Mirror 192.168.8.15 is not synchronized. >>> Trying again in 5 secs... >>> ClamAV update process started at Mon Jan 15 11:40:05 2018 >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: >>> sigmgr) >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, >>> builder: neo) >>> WARNING: Can't download bytecode.cvd from mirror.recette.local >>> Trying again in 5 secs... >>> ClamAV update process started at Mon Jan 15 11:40:10 2018 >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: >>> sigmgr) >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, >>> builder: neo) >>> WARNING: Can't download bytecode.cvd from mirror.recette.local >>> Trying again in 5 secs... >>> ClamAV update process started at Mon Jan 15 11:40:15 2018 >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: >>> sigmgr) >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, >>> builder: neo) >>> WARNING: Can't download bytecode.cvd from mirror.recette.local >>> Trying again in 5 secs... >>> ClamAV update process started at Mon Jan 15 11:40:20 2018 >>> main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, builder: >>> sigmgr) >>> daily.cvd is up to date (version: 24223, sigs: 1825336, f-level: 63, >>> builder: neo) >>> ERROR: Can't download bytecode.cvd from mirror.recette.local >>> Giving up on mirror.recette.local ... >>> Update failed. Your network may be down or none of the mirrors listed in >>> /etc/clamav/freshclam.conf is working. Check http://www.clamav.net/doc/ >>> <http://www.clamav.net/doc/> >>> mirrors-faq.html for possible reasons. >>> >>> >>> >>> On 14 January 2018 at 10:36, Al Varnell <alvarn...@mac.com >>> <mailto:alvarn...@mac.com>> wrote: >>> >>>> >>>> On Sat, Jan 13, 2018 at 04:12 AM, Hugo Deprez wrote: >>>>> >>>>> hello, >>>>> >>>>> sorry I wasn't clear : >>>>> >>>>> - I used solution number 3 or more than one year, but freshclam on >>>> clients >>>>> was randomly reporting error on downloading cvd files. I never found the >>>>> solution : >>>>> >>>>> *Tue Dec 1 17:54:34 2015 -> WARNING: Can't download daily.cvd from * >>>>> >>>>> *mirror* >>>>> >>>>> - So now I switch to solution number 2 from >>>>> https://www.clamav.net/documents/private-local-mirrors >>>>> <https://www.clamav.net/documents/private-local-mirrors> >>>>> >>>>> My "/etc/clamav/freshclam.conf" is : >>>>> >>>>> *AllowSupplementaryGroups falseBytecode trueChecks >>>>> 24CompressLocalDatabase noConnectTimeout 30DNSDatabaseInfo >>>>> current.cvd.clamav.net >>>>> <http://current.cvd.clamav.net>DatabaseDirectory >>>>> /var/lib/clamavDatabaseMirror mirror.recette.localDatabaseOwner >>>>> clamavDebug falseForeground falseLogFacility LOG_LOCAL6LogFileMaxSize >>>>> 0LogRotate trueLogSyslog trueLogTime yesLogVerbose falseMaxAttempts >>>>> 5PidFile /var/run/clamav/freshclam.pidPrivateMirror >>>>> mirror.recette.localReceiveTimeout 30ScriptedUpdates noTestDatabases >>>>> yesUpdateLogFile /var/log/clamav/freshclam.log* >>>>> >>>>> But when I run freshclam, it still downloading the cld files : >>>> >>>> Because you changed PrivateMirror to mirror.recette.local. See >>>> freshclam.conf: >>>> >>>> # This option allows you to easily point freshclam to private mirrors. >>>> # If PrivateMirror is set, freshclam does not attempt to use DNS >>>> # to determine whether its databases are out-of-date, instead it will >>>> # use the If-Modified-Since request or directly check the headers of the >>>> # remote database files. For each database, freshclam first attempts >>>> # to download the CLD file. If that fails, it tries to download the >>>> # CVD file. This option overrides DatabaseMirror, DNSDatabaseInfo >>>> # and ScriptedUpdates. It can be used multiple times to provide >>>> # fall-back mirrors. >>>> # Default: disabled >>>> #PrivateMirror mirror1.mynetwork.com <http://mirror1.mynetwork.com/> >>>> >>>> Try commenting that line out. >>>> >>>>> *# freshclam* >>>>> >>>>> *ClamAV update process started at Sat Jan 13 13:07:47 2018Reading CVD >>>>> header (main.cld): OK (IMS)Reading CVD header (main.cvd): OK >>>>> (IMS)main.cvd is up to date (version: 58, sigs: 4566249, f-level: 60, >>>>> builder: sigmgr)Reading CVD header (daily.cld): OKDownloading >>>>> daily.cld [100%]daily.cld updated (version: 24217, sigs: 1823727, >>>>> f-level: 63, builder: neo)Reading CVD header (bytecode.cld): >>>>> OKbytecode.cvd is up to date (version: 319, sigs: 75, f-level: 63, >>>>> builder: neo)[LibClamAV] Detected duplicate databases >>>>> /var/lib/clamav/daily.cvd and /var/lib/clamav/daily.cld. The >>>>> /var/lib/clamav/daily.cvd database is older and will not be loaded, >>>>> you should manually remove it from the database directory.Database >>>>> updated (6390051 signatures) from mirror.recette.local (IP: >>>>> 192.168.1.15)*If I remove cld files from my private mirror : >>>>> >>>>> *# freshclamClamAV update process started at Sat Jan 13 13:10:18 >>>>> 2018Downloading main.cld [100%]main.cld updated (version: 58, sigs: >>>>> 4566249, f-level: 60, builder: sigmgr)Downloading daily.cld >>>>> [100%]daily.cld updated (version: 24217, sigs: 1823727, f-level: 63, >>>>> builder: neo)WARNING: getfile: bytecode.cld not found on >>>>> mirror.recette.local (IP: 192.168.1.15)WARNING: Can't download >>>>> bytecode.cld from mirror.recette.localDownloading bytecode.cvd >>>>> [100%]bytecode.cvd updated (version: 319, sigs: 75, f-level: 63, >>>>> builder: neo)Database updated (6390051 signatures) from >>>>> mirror.recette.local (IP: 192.168.1.15)* >>>>> >>>>> I just want to run freshclam without any error. >>>>> >>>>> * Any idea ? * >>>>> >>>>> On 11 January 2018 at 10:37, Al Varnell <alvarn...@mac.com >>>>> <mailto:alvarn...@mac.com>> wrote: >>>>> >>>>>> On Thu, Jan 11, 2018 at 01:15 AM, Hugo Deprez wrote: >>>>>>> Hello, >>>>>>> >>>>>>> thank you for the answer. >>>>>>> I don't think my freshclam is trying to download the cdiff file >>>> because I >>>>>>> use : ScriptedUpdates no >>>>>>> >>>>>>> Is this parameter you are talking about incremental updates ? >>>>>> >>>>>> Yes, that's what I was referring to. >>>>>> >>>>>>> I know this is not bandwith efficient, but I had to many issues with >>>>>>> freshclam and my local repository made by clamdownloader.pl >>>>>> >>>>>> So you are using an option 3 private local mirror as described in < >>>>>> https://www.clamav.net/documents/private-local-mirrors >>>>>> <https://www.clamav.net/documents/private-local-mirrors>>. Make sure you >>>>>> also change freshclam.conf "DatabaseMirror machine1.mylan" where >>>>>> machine1.mylan is the name of your mirror server. >>>>>> >>>>>> Also make sure you are using the latest version of the >>>> clamavdownloader.pl >>>>>> <https://github.com/akissa/clamav-faq/blob/master/ >>>>>> <https://github.com/akissa/clamav-faq/blob/master/> >>>>>> mirrors/clamdownloader.pl> >>>>>> >>>>>> The existing clamdownloader.pl script does not have any error >>>> correction >>>>>> it simply bails out if a downloaded file is not valid and is unable to >>>>>> retry different mirrors if one fails. That is the most likely reason >>>> for >>>>>> those 404 errors. >>>>>> >>>>>> I have no experience with private local mirrors, but you might have >>>> better >>>>>> luck with the clamavmirror script <https://pypi.python.org/pypi/ >>>>>> <https://pypi.python.org/pypi/> >>>>>> clamavmirror/0.0.3>. >>>>>> >>>>>> -Al- >>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> On 10 January 2018 at 10:20, Al Varnell <alvarn...@mac.com >>>>>>> <mailto:alvarn...@mac.com> <mailto: >>>>>> alvarn...@mac.com <mailto:alvarn...@mac.com>>> wrote: >>>>>>> >>>>>>>> The first time freshclam pulls down a daily.cdiff file, your >>>> daily.cvd >>>>>>>> file will be decompressed to daily.cld and the .cdiff file added to >>>> it. >>>>>>>> From that point on you should only have the daily.cld file. One >>>>>> exception >>>>>>>> is that if for some reason freshclam is unable to find needed .cdiff >>>>>> files >>>>>>>> on a mirror, it will download a new .cvd file which will start the >>>>>> process >>>>>>>> again. >>>>>>>> >>>>>>>> It is possible to reconfigure freshclam to download .cvd files by >>>>>>>> disabling incremental updates, but that would be very inefficient >>>> use of >>>>>>>> bandwidth and mirror server time, so unless you have a overriding >>>> need >>>>>> for >>>>>>>> that, I don't recommend it. >>>>>>>> >>>>>>>> -Al- >>>>>>>> >>>>>>>> On Wed, Jan 10, 2018 at 01:12 AM, Hugo Deprez wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> I have a question about daily.cvd and daily.cld files. If I >>>> understood >>>>>>>>> correctly, those two files are almost the same : one is compressed, >>>> the >>>>>>>>> other is not. >>>>>>>>> Still I have an issue in my setup : >>>>>>>>> >>>>>>>>> If I put both filtes in apache2 server (which act as PrivateMirror) >>>> : >>>>>>>>> >>>>>>>>> -rw-r--r-- 1 www-data www-data 117892267 07.06.2017 23:38 main.cvd >>>>>>>>> -rw-r--r-- 1 www-data www-data 153228 07.12.2017 03:17 >>>> bytecode.cvd >>>>>>>>> lrwxrwxrwx 1 root root 8 05.01.2018 10:14 main.cld >>>> -> >>>>>>>>> main.cvd >>>>>>>>> lrwxrwxrwx 1 root root 9 05.01.2018 10:14 daily.cld >>>> -> >>>>>>>>> daily.cvd >>>>>>>>> lrwxrwxrwx 1 root root 12 05.01.2018 10:14 >>>> bytecode.cld >>>>>> -> >>>>>>>>> bytecode.cvd >>>>>>>>> -rw-r--r-- 1 www-data www-data 43804052 10.01.2018 06:17 daily.cvd >>>>>>>>> >>>>>>>>> On the client side I have this Warning : >>>>>>>>> >>>>>>>>> LibClamAV Warning: Detected duplicate databases >>>>>> /var/lib/clamav/daily.cvd >>>>>>>>> and /var/lib/clamav/daily.cld, please manually remove one of them >>>>>>>>> >>>>>>>>> >>>>>>>>> But If I remove the *.cld files on my PrivateMirror I got multiples >>>> 404 >>>>>>>>> errors from the Freshclam clients. >>>>>>>>> >>>>>>>>> Is there a way to configure Freshclam in order to grabe only the cvd >>>>>>>> files >>>>>>>>> ? >>>>>>>>> >>>>>>>>> Best regards, > _______________________________________________ > clamav-users mailing list > clamav-users@lists.clamav.net > http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users > > > Help us build a comprehensive ClamAV guide: > https://github.com/vrtadmin/clamav-faq > > http://www.clamav.net/contact.html#ml
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
