Yeah, this is even happening over http/2: Host cygwin.com:443 was resolved. * IPv6: (none) * IPv4: 8.43.85.97 * Trying 8.43.85.97:443... * Connected to cygwin.com (8.43.85.97) port 443 > GET /setup-x86_64.exe HTTP/2 > Host: cygwin.com > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off < HTTP/2 200 < date: Tue, 22 Oct 2024 08:25:16 GMT < server: Apache/2.4.37 (Red Hat Enterprise Linux) OpenSSL/1.1.1k mod_qos/11.74 mod_wsgi/4.6.4 Python/3.6 mod_perl/2.0.12 Perl/v5.26.3 < vary: User-Agent < last-modified: Tue, 16 Apr 2024 18:39:04 GMT < etag: "157a13-6163b0d6e8405" < accept-ranges: bytes < content-length: 1407507 < cache-control: max-age=0 < expires: Tue, 22 Oct 2024 08:25:16 GMT < content-security-policy: default-src 'self' http: https: < strict-transport-security: max-age=16070400 < x-cache: MISS from cygwin.com < content-type: application/octet-stream < ... > GET /setup-x86_64.exe HTTP/2 > Host: cygwin.com > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0< HTTP/2 200 < date: Tue, 22 Oct 2024 08:25:24 GMT < server: Apache/2.4.37 (Red Hat Enterprise Linux) OpenSSL/1.1.1k mod_qos/11.74 mod_wsgi/4.6.4 Python/3.6 mod_perl/2.0.12 Perl/v5.26.3 < last-modified: Tue, 16 Apr 2024 18:39:04 GMT < content-length: 0 < vary: User-Agent < etag: "157a13-6163b0d6e8405" < accept-ranges: bytes < cache-control: max-age=0 < expires: Tue, 22 Oct 2024 08:25:24 GMT < content-security-policy: default-src 'self' http: https: < strict-transport-security: max-age=16070400 < x-cache: REVALIDATE from cygwin.com < content-type: application/octet-stream
I suspect this is mod_qos (https://mod-qos.sourceforge.net/) that is doing something like rate-limiting, but instead of giving you a 403 Forbidden, it gives you a zero content-length. Looks like it would be best to contact the webmaster of cygwin.com <http://cygwin.com/>? It's all hosted on sourceware.org <http://sourceware.org/>, as far as I can see. -Dimitry > On 22 Oct 2024, at 10:21, David Allsopp via Cygwin <cygwin@cygwin.com> wrote: > > On Tue, 22 Oct 2024 at 09:02, Michele Locati via Cygwin > <cygwin@cygwin.com> wrote: >> > > More data-points - running from a couple of networks here in the UK, > but adapting your script to display the x-cache header as well: > > Downloading https://cygwin.com/setup-x86_64.exe multiple times (press > Ctrl+C to stop) > x-cache: MISS from cygwin.com > Actual size: 1407507 > > x-cache: MISS from cygwin.com > Actual size: 1407507 > > x-cache: REVALIDATE from cygwin.com > Actual size: 1407507 > > x-cache: REVALIDATE from cygwin.com > Actual size: 0 > > <10 more REVALIDATE with Actual size 0> > > x-cache: MISS from cygwin.com > Actual size: 1407507 > > x-cache: REVALIDATE from cygwin.com > Actual size: 1407507 > > x-cache: REVALIDATE from cygwin.com > Actual size: 0 > > x-cache: MISS from cygwin.com > Actual size: 1407507 > > x-cache: REVALIDATE from cygwin.com > Actual size: 1407507 > > x-cache: REVALIDATE from cygwin.com > Actual size: 0 > > so there seems to be a pattern. Isn't that x-cache header coming from > Apache's mod_cache (i.e. something at the cygwin.com/sourceware end, > not any CDN sat in the way)? Having _never_ seen anything like this > from cygwin.com before, and the fact it's happening on diverse > networks, my debugging intuition would be to find out if sourceware > reconfigured anything last week before trying to engage in the futile > task of contacting any network providers...... > > All best, > > > David > > >> Il giorno mar 22 ott 2024 alle ore 06:28 Brian Inglis via Cygwin >> <cygwin@cygwin.com> ha scritto: >>> I typically use wget -N -nv for downloads, without any issues for decades, >>> on >>> any service. >> >> Hello Brian, >> >> thanks for the reply! >> >> It seems the problem occurs when we require the same URL from the same >> IP multiple times. >> This occurs for example in GitHub Actions, where the cygwin server >> sees just one or few IPs for the whole GitHub system). >> >> You can "simulate" it by using curl (or wget or any tool you prefer) >> by forcing the download of the same file multiple times. >> Consider for example the following script: >> >> >> #!/bin/bash >> set -o errexit >> set -o nounset >> set -o pipefail >> url=https://cygwin.com/setup-x86_64.exe >> filename="$(basename "$url")" >> printf 'Downloading %s multiple times (press Ctrl+C to stop)\n' "$url" >> while :; do >> wget -N -nv $url >> size=$(cat "$filename" | wc -c) >> rm "$filename" >> printf 'Actual size: %s\n\n' $size >> sleep 1 >> done >> >> Please remark that I used wget like you wrote, with the difference >> that I delete the downloaded file before performing another wget call. >> This "simulates" different wget calls from different PCs, but seen by >> cygwin.com always with the same IP. >> >> I always have results similar to the following: >> >> Downloading https://cygwin.com/setup-x86_64.exe multiple times (press >> Ctrl+C to stop) >> 2024-10-22 09:52:10 URL:https://cygwin.com/setup-x86_64.exe >> [1407507/1407507] -> "setup-x86_64.exe" [1] >> Actual size: 1407507 >> >> 2024-10-22 09:52:14 URL:https://cygwin.com/setup-x86_64.exe >> [1407507/1407507] -> "setup-x86_64.exe" [1] >> Actual size: 1407507 >> >> 2024-10-22 09:52:16 URL:https://cygwin.com/setup-x86_64.exe [0/0] -> >> "setup-x86_64.exe" [1] >> Actual size: 0 >> >> 2024-10-22 09:52:18 URL:https://cygwin.com/setup-x86_64.exe [0/0] -> >> "setup-x86_64.exe" [1] >> Actual size: 0 >> >> 2024-10-22 09:52:20 URL:https://cygwin.com/setup-x86_64.exe [0/0] -> >> "setup-x86_64.exe" [1] >> Actual size: 0 >> >> 2024-10-22 09:52:22 URL:https://cygwin.com/setup-x86_64.exe [0/0] -> >> "setup-x86_64.exe" [1] >> Actual size: 0 >> >> 2024-10-22 09:52:24 URL:https://cygwin.com/setup-x86_64.exe [0/0] -> >> "setup-x86_64.exe" [1] >> Actual size: 0 >> >> 2024-10-22 09:52:26 URL:https://cygwin.com/setup-x86_64.exe [0/0] -> >> "setup-x86_64.exe" [1] >> Actual size: 0 >> >> ^C >> >> >> I tried the script above on: >> - WSL from my home PC >> - WSL from my office PC >> - bare metal Linux located in my office LAN >> - DigitalOcean droplet located in New York >> - DigitalOcean droplet located in Amsterdam >> - VPS hosted on Aruba located in Italy >> - Amazon Lightsail located in Frankfurt >> >> And I always have similar results... >> >> Ciao, >> Michele >> >> -- >> Problem reports: https://cygwin.com/problems.html >> FAQ: https://cygwin.com/faq/ >> Documentation: https://cygwin.com/docs.html >> Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
