Hello, I'm maintainer of FreeBSD ClamAV port. Today I recieved problem report from our user that download of source archive fails with 403 forbitten. I checked it and confirmed it really happens as following.
---------------------------------------------------------------------- yasu@rolling-vm-freebsd2[1058]% make fetch ===> License GPLv2 accepted by the user ===> clamav-0.104.1,1 depends on file: /usr/local/sbin/pkg - found => clamav-0.104.1.tar.gz doesn't seem to exist in /net/freebsd/ports/distfiles/. => Attempting to fetch https://www.clamav.net/downloads/production/clamav-0.104.1.tar.gz fetch: https://www.clamav.net/downloads/production/clamav-0.104.1.tar.gz: Forbidden => Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/clamav-0.104.1.tar.gz fetch: http://distcache.FreeBSD.org/ports-distfiles/clamav-0.104.1.tar.gz: Not Found => Couldn't fetch it - please try to retrieve this => port manually into /net/freebsd/ports/distfiles/ and try again. *** Error code 1 Stop. make: stopped in /usr/ports/security/clamav yasu@rolling-vm-freebsd2[1059]% ---------------------------------------------------------------------- In the base system of FreeBSD there is a program named 'fetch'. Fetch is a utility to retieve a file by URL, similar one to famous GNU Wget and curl. FreeBSD ports framework uses fetch to download files through network. And 403 forbitten happens when I tried to download source archive with fetch. ---------------------------------------------------------------------- yasu@rolling-vm-freebsd2[1066]% fetch https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz fetch: https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz: Forbidden yasu@rolling-vm-freebsd2[1067]% ---------------------------------------------------------------------- I also tried download with GNU Wget and curl and it also fails with same reason. ---------------------------------------------------------------------- yasu@rolling-vm-freebsd2[1069]% wget https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz --2021-11-15 08:53:12-- https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz Resolving www.clamav.net (www.clamav.net)... 104.16.218.84, 104.16.219.84, 2606:4700::6810:da54, ... Connecting to www.clamav.net (www.clamav.net)|104.16.218.84|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2021-11-15 08:53:12 ERROR 403: Forbidden. yasu@rolling-vm-freebsd2[1070]% curl -i https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz HTTP/2 403 date: Sun, 14 Nov 2021 23:53:17 GMT content-type: text/plain; charset=UTF-8 content-length: 16 x-frame-options: SAMEORIGIN referrer-policy: same-origin cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 expires: Thu, 01 Jan 1970 00:00:01 GMT expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"; strict-transport-security: max-age=15552000 x-content-type-options: nosniff server: cloudflare cf-ray: 6ae4288c495f206b-NRT error code: 1020% yasu@rolling-vm-freebsd2[1071]% ---------------------------------------------------------------------- But download succeeds if I use web browser such as Chrome, Edge and Firefox. it seems server checks User-Agent header of HTTP request. So I tried to confirm it. Our fetch utility has '--user-agent' options that changes the value of User-Agent header. And if I specify 'Mozilla/5.0' as it, then source archive can be downloaded successfully. ---------------------------------------------------------------------- yasu@rolling-vm-freebsd2[1072]% fetch --user-agent=Mozilla/5.0 https://www.clamav.net/downloads/production/clamav-0.103.4.tar.gz clamav-0.103.4.tar.gz 15 MB 2092 kBps 07s yasu@rolling-vm-freebsd2[1073]% ---------------------------------------------------------------------- These results means server checks User-Agent header of HTTP request and returns 403 forbitten if the value doesn't look like that of web browser. Then is it intened change? Best Regards. --- Yasuhiro Kimura _______________________________________________ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
