Hi, I just had a look at this again. It seems the issue comes from our varnish setup, as things are fine when hitting apache directly. But the combination of streaming and a mix of small and large files in varnish's cache leads to things like the below, where varnish gives up after hitting nuke_limit (50) files to make space in the cache.
Cheers, Julien > jcristau@sallinen:~$ sudo varnishlog -g session -q 'ReqUrl eq > "/file/51288439bde385157c889f23483b467608586aad"' > * << Session >> 3937381 > - Begin sess 0 HTTP/1 > - SessOpen 2a01:cb08:88c7:5b00:6798:1a76:ef57:4714 58440 a1 > 2001:630:206:4000:1a1a:0:c13e:ca1b 80 1628066418.824244 26 > - Link req 3937382 rxreq > - SessClose REM_CLOSE 2.846 > - End > ** << Request >> 3937382 > -- Begin req 3937381 rxreq > -- Timestamp Start: 1628066418.824321 0.000000 0.000000 > -- Timestamp Req: 1628066418.824321 0.000000 0.000000 > -- ReqStart 2a01:cb08:88c7:5b00:6798:1a76:ef57:4714 58440 a1 > -- ReqMethod GET > -- ReqURL /file/51288439bde385157c889f23483b467608586aad > -- ReqProtocol HTTP/1.1 > -- ReqHeader Host: snapshot.debian.org > -- ReqHeader User-Agent: curl/7.74.0 > -- ReqHeader Accept: */* > -- ReqHeader X-Forwarded-For: 2a01:cb08:88c7:5b00:6798:1a76:ef57:4714 > -- VCL_call RECV > -- VCL_return hash > -- VCL_call HASH > -- VCL_return lookup > -- VCL_call MISS > -- VCL_return fetch > -- Link bereq 3937383 fetch > -- Timestamp Fetch: 1628066418.826448 0.002127 0.002127 > -- RespProtocol HTTP/1.1 > -- RespStatus 200 > -- RespReason OK > -- RespHeader Date: Wed, 04 Aug 2021 08:40:18 GMT > -- RespHeader Server: Apache > -- RespHeader X-Content-Type-Options: nosniff > -- RespHeader X-Frame-Options: sameorigin > -- RespHeader Referrer-Policy: no-referrer > -- RespHeader X-Xss-Protection: 1 > -- RespHeader Permissions-Policy: interest-cohort=() > -- RespHeader Last-Modified: Fri, 20 Apr 2018 20:58:57 GMT > -- RespHeader ETag: "2c0cb0ca-56a4df4e5684a" > -- RespHeader Content-Length: 739029194 > -- RespHeader X-Clacks-Overhead: GNU Terry Pratchett > -- RespHeader Cache-Control: max-age=31536000, public > -- RespHeader X-Varnish: 3937382 > -- RespHeader Age: 0 > -- RespHeader Via: 1.1 varnish (Varnish/6.1) > -- VCL_call DELIVER > -- RespHeader connection: close > -- VCL_return deliver > -- Timestamp Process: 1628066418.826480 0.002160 0.000032 > -- RespHeader Accept-Ranges: bytes > -- Timestamp Resp: 1628066421.667356 2.843036 2.840876 > -- ReqAcct 128 0 128 512 19857408 19857920 > -- End > *** << BeReq >> 3937383 > --- Begin bereq 3937382 fetch > --- Timestamp Start: 1628066418.824419 0.000000 0.000000 > --- BereqMethod GET > --- BereqURL /file/51288439bde385157c889f23483b467608586aad > --- BereqProtocol HTTP/1.1 > --- BereqHeader Host: snapshot.debian.org > --- BereqHeader User-Agent: curl/7.74.0 > --- BereqHeader Accept: */* > --- BereqHeader X-Forwarded-For: 2a01:cb08:88c7:5b00:6798:1a76:ef57:4714 > --- BereqHeader Accept-Encoding: gzip > --- BereqHeader X-Varnish: 3937383 > --- VCL_call BACKEND_FETCH > --- VCL_return fetch > --- BackendOpen 32 default 127.0.0.1 80 127.0.0.1 56612 > --- BackendStart 127.0.0.1 80 > --- Timestamp Bereq: 1628066418.824718 0.000299 0.000299 > --- Timestamp Beresp: 1628066418.826314 0.001895 0.001596 > --- BerespProtocol HTTP/1.1 > --- BerespStatus 200 > --- BerespReason OK > --- BerespHeader Date: Wed, 04 Aug 2021 08:40:18 GMT > --- BerespHeader Server: Apache > --- BerespHeader X-Content-Type-Options: nosniff > --- BerespHeader X-Frame-Options: sameorigin > --- BerespHeader Referrer-Policy: no-referrer > --- BerespHeader X-Xss-Protection: 1 > --- BerespHeader Permissions-Policy: interest-cohort=() > --- BerespHeader Upgrade: h2,h2c > --- BerespHeader Connection: Upgrade > --- BerespHeader Last-Modified: Fri, 20 Apr 2018 20:58:57 GMT > --- BerespHeader ETag: "2c0cb0ca-56a4df4e5684a" > --- BerespHeader Accept-Ranges: bytes > --- BerespHeader Content-Length: 739029194 > --- BerespHeader X-Clacks-Overhead: GNU Terry Pratchett > --- BerespHeader Cache-Control: max-age=31536000, public > --- TTL RFC 31536000 10 0 1628066419 1628066419 1628066418 0 > 31536000 cacheable > --- VCL_call BACKEND_RESPONSE > --- VCL_return deliver > --- Filters > --- Storage file s0 > --- Fetch_Body 3 length stream > --- ExpKill LRU_Cand p=0x7fa236c79b00 f=0x0 r=1 > --- ExpKill LRU x=3873725 > --- ExpKill LRU_Cand p=0x7fa236c6f700 f=0x0 r=1 > --- ExpKill LRU x=3696172 > --- ExpKill LRU_Cand p=0x7fa238320d40 f=0x0 r=1 > --- ExpKill LRU x=3873726 > --- ExpKill LRU_Cand p=0x7fa23e73c640 f=0x0 r=1 > --- ExpKill LRU x=3627041 > --- ExpKill LRU_Cand p=0x7fa2388080c0 f=0x0 r=1 > --- ExpKill LRU x=3696176 > --- ExpKill LRU_Cand p=0x7fa236864680 f=0x0 r=1 > --- ExpKill LRU x=3902662 > --- ExpKill LRU_Cand p=0x7fa2402ca500 f=0x0 r=1 > --- ExpKill LRU x=3696177 > --- ExpKill LRU_Cand p=0x7fa238807dc0 f=0x0 r=1 > --- ExpKill LRU x=3696178 > --- ExpKill LRU_Cand p=0x7fa23eee1080 f=0x0 r=1 > --- ExpKill LRU x=3902663 > --- ExpKill LRU_Cand p=0x7fa23e73c580 f=0x0 r=1 > --- ExpKill LRU x=3627042 > --- ExpKill LRU_Cand p=0x7fa236c7a280 f=0x0 r=1 > --- ExpKill LRU x=3627043 > --- ExpKill LRU_Cand p=0x7fa23fa9a440 f=0x0 r=1 > --- ExpKill LRU x=3696185 > --- ExpKill LRU_Cand p=0x7fa236463200 f=0x0 r=1 > --- ExpKill LRU x=3873739 > --- ExpKill LRU_Cand p=0x7fa23faf6840 f=0x0 r=1 > --- ExpKill LRU x=3627044 > --- ExpKill LRU_Cand p=0x7fa23ca36780 f=0x0 r=1 > --- ExpKill LRU x=3873740 > --- ExpKill LRU_Cand p=0x7fa23ceb0f80 f=0x0 r=1 > --- ExpKill LRU x=3784856 > --- ExpKill LRU_Cand p=0x7fa2361702c0 f=0x0 r=1 > --- ExpKill LRU x=3457616 > --- ExpKill LRU_Cand p=0x7fa237402700 f=0x0 r=1 > --- ExpKill LRU x=3784857 > --- ExpKill LRU_Cand p=0x7fa23fe35a40 f=0x0 r=1 > --- ExpKill LRU x=3901352 > --- ExpKill LRU_Cand p=0x7fa236d032c0 f=0x0 r=1 > --- ExpKill LRU x=3624875 > --- ExpKill LRU_Cand p=0x7fa23689e2c0 f=0x0 r=1 > --- ExpKill LRU x=3696197 > --- ExpKill LRU_Cand p=0x7fa238808300 f=0x0 r=1 > --- ExpKill LRU x=3873741 > --- ExpKill LRU_Cand p=0x7fa23e746c00 f=0x0 r=1 > --- ExpKill LRU x=3696198 > --- ExpKill LRU_Cand p=0x7fa2364d7900 f=0x0 r=1 > --- ExpKill LRU x=3696199 > --- ExpKill LRU_Cand p=0x7fa236c777c0 f=0x0 r=1 > --- ExpKill LRU x=3873742 > --- ExpKill LRU_Cand p=0x7fa23fa99b40 f=0x0 r=1 > --- ExpKill LRU x=3696200 > --- ExpKill LRU_Cand p=0x7fa236c86880 f=0x0 r=1 > --- ExpKill LRU x=3873743 > --- ExpKill LRU_Cand p=0x7fa2368a30c0 f=0x0 r=1 > --- ExpKill LRU x=3696201 > --- ExpKill LRU_Cand p=0x7fa23fa9aa40 f=0x0 r=1 > --- ExpKill LRU x=3460107 > --- ExpKill LRU_Cand p=0x7fa23610b040 f=0x0 r=1 > --- ExpKill LRU x=3460108 > --- ExpKill LRU_Cand p=0x7fa23cea5f40 f=0x0 r=1 > --- ExpKill LRU x=3724090 > --- ExpKill LRU_Cand p=0x7fa23da433c0 f=0x0 r=1 > --- ExpKill LRU x=3724091 > --- ExpKill LRU_Cand p=0x7fa23611f140 f=0x0 r=1 > --- ExpKill LRU x=3627063 > --- ExpKill LRU_Cand p=0x7fa23ceac000 f=0x0 r=1 > --- ExpKill LRU x=3724092 > --- ExpKill LRU_Cand p=0x7fa23616f900 f=0x0 r=1 > --- ExpKill LRU x=3720845 > --- ExpKill LRU_Cand p=0x7fa236c6ef80 f=0x0 r=1 > --- ExpKill LRU x=2487312 > --- ExpKill LRU_Cand p=0x7fa238809bc0 f=0x0 r=1 > --- ExpKill LRU x=3627066 > --- ExpKill LRU_Cand p=0x7fa238320e00 f=0x0 r=1 > --- ExpKill LRU x=3724093 > --- ExpKill LRU_Cand p=0x7fa2364d8800 f=0x0 r=1 > --- ExpKill LRU x=3902697 > --- ExpKill LRU_Cand p=0x7fa24022cc00 f=0x0 r=1 > --- ExpKill LRU x=3460117 > --- ExpKill LRU_Cand p=0x7fa23fa999c0 f=0x0 r=1 > --- ExpKill LRU x=3460118 > --- ExpKill LRU_Cand p=0x7fa238320140 f=0x0 r=1 > --- ExpKill LRU x=3724098 > --- ExpKill LRU_Cand p=0x7fa236cd6340 f=0x0 r=1 > --- ExpKill LRU x=3627069 > --- ExpKill LRU_Cand p=0x7fa236109180 f=0x0 r=1 > --- ExpKill LRU x=3724099 > --- ExpKill LRU_Cand p=0x7fa23ceac0c0 f=0x0 r=1 > --- ExpKill LRU x=3724100 > --- ExpKill LRU_Cand p=0x7fa23ce69c80 f=0x0 r=1 > --- ExpKill LRU x=3460122 > --- ExpKill LRU_Cand p=0x7fa238320200 f=0x0 r=1 > --- ExpKill LRU x=3724101 > --- ExpKill LRU_Cand p=0x7fa23f205540 f=0x0 r=1 > --- ExpKill LRU x=3724102 > --- ExpKill LRU_Cand p=0x7fa23611f080 f=0x0 r=1 > --- ExpKill LRU x=3627070 > --- ExpKill LRU_Cand p=0x7fa236128880 f=0x0 r=1 > --- ExpKill LRU x=3724103 > --- ExpKill LRU reached nuke_limit > --- FetchError Could not get storage > --- BackendClose 32 default > --- BereqAcct 229 0 229 471 106479616 106480087 > --- End
