On 2020-10-29 11:37 a.m., Allen Wittenauer wrote:
On Oct 28, 2020, at 11:57 PM, Chris Lambertus <c...@apache.org> wrote:
Infra would LOVE a smarter way to clean the cache. We have to use a heavy
hammer because there are 300+ projects that want a piece of it, and who don’t
clean up.. We are not build engineers, so we rely on the community to advise us
in dealing with the challenges we face. I would be very happy to work with you
on tooling to improve the cleanup if it improves the experience for all
projects.
I'll work on YETUS-1063 so that things make more sense. But in short, Yetus'
"docker-cleanup --sentinel" will purge container images if they are older than
a week, then kill stuck containers after 24 hours. That order prevents running jobs from
getting into trouble. But it also means that in some cases it doesn't look very clean
until two or three days later. But that's ok: it is important to remember that an empty
cache is a useless cache. Those values came from experiences with Hadoop and HBase, but
we can certainly add some way to tune them. Oh, and unlike the docker tools, it pretty
much ignores labels. It does _not_ do anything with volumes, probably something we need
to add.
(Sidebar about the script's details)
I tried to read the shell script, but I'm not in the headspace to fully
parse it at the moment. If I'm understanding correctly, this will still
catch CouchDB's CI docker images if they haven't changed in a week,
which happens often enough, negating the cache.
As a project, we're kind of stuck between a rock and a hard place. We
want to force a docker pull on the base CI image if it's out of date or
the image is corrupted. Otherwise we want to cache forever, not just for
a week. I can probably manage the "do we need to re-pull?" bit with some
clever CI scripting (check for the latest image hash locally, validate
the local image, pull if either fails) but I don't understand how the
script resolves the latter.
Can a exemption list be passed to the script so that images matching a
certain regex are excluded? You say the script ignores labels entirely,
so perhaps not...
-Joan
