> On Oct 28, 2020, at 9:01 PM, Joan Touzet <woh...@apache.org> wrote:
>
> Even for those of us lucky enough to have sponsorship for dedicated CI
> workers, it's still a problem. Infra has scripts to wipe all
> not-currently-in-use Docker containers off of each machine every 24
> hours (or did, last I looked).
Argh. I really hope this isn't happening again, at least on the
machines where Apache Yetus' test-patch runs regularly. It can manage the
local cache just fine (which is why after we implemented the docker cache
cleanup code, the Hadoop nodes rarely if ever had docker space problems...). I
did separate that part of the code out, so if infra wants a _smarter_ way to
clean the cache on nodes where test-patch and friends aren't getting used, the
docker-cleanup utility from Yetus is an option. (Although, to be fair, that
utility is poorly documented. Maybe I'll work on that this week if there is
interest. )
>
> 2. Infra provides their own Docker registry. Projects that need images
> can host them there. These will be automatically exempt. Infra will have
> to plan for sufficient storage (this will get big *fast*) and bandwidth
> (same). They will also have to firewall it off from non-Apache projects.
Given that Apache Yetus is about to launch a Github Action to the
marketplace that uses docker, I've been thinking more and more about pursuing
access to the ASF's access to github's registry due to all of the fallout of
Docker, Inc. flailing. Needless to say, firewalls aren't an option for what I'm
needing.
