janhoy opened a new issue, #104:
URL: https://github.com/apache/solr-orbit/issues/104
An interrupted distribution download leaves a partial tarball at the final
cache path, and every later run then treats it as a complete download and fails
with an opaque error:
```
File "solrorbit/builder/solr_provisioner.py", line 172, in install
File ".../tarfile.py", line 2569, in makefile
copyfileobj(source, target, tarinfo.size, ReadError, bufsize)
File ".../gzip.py", line 338, in read
EOFError: Compressed file ended before the end-of-stream marker was reached
[ERROR] ❌ Cannot run. This test_run ended with a fatal crash.
```
Nothing in that message points at the cache, and the only cure is knowing to
delete `~/.solr-orbit/cache/solr-<version>.tgz` by hand.
Reproduce by interrupting a download (Ctrl-C, SIGTERM, a dropped connection,
a CI timeout) and running again. I hit it three times in a row while working on
the integration tests, with 17 MB, 185 MB and 204 MB leftovers of a 389 MB
tarball.
**Cause:** `SolrProvisioner.download`
(`solrorbit/builder/solr_provisioner.py:140`) writes straight to the final path
with `urllib.request.urlretrieve(url, dest)`, and the `os.path.exists(dest)`
check at line 131 then reports it as cached. The `except` branch does remove a
partial file, but it only runs for an exception — not when the process is
killed.
**Expected:** an interrupted download cannot be mistaken for a complete one.
**Suggested fix:** download to a temporary file in the cache directory and
`os.replace()` it into place on success, so the final path only ever appears
complete. Verifying the size against `Content-Length`, or the published
checksum, would catch a silently truncated response too.
Worth noting for anyone reproducing: `downloads.apache.org` only carries the
current releases (today 9.11.0 and 10.0.0), so older versions come from
`archive.apache.org` and are a much longer download — which is what made this
easy to trip over.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
