> On Oct 28, 2020, at 11:47 PM, Allen Wittenauer <a...@effectivemachines.com>
> wrote:
>
>
>
>> On Oct 28, 2020, at 9:01 PM, Joan Touzet <woh...@apache.org> wrote:
>>
>> Even for those of us lucky enough to have sponsorship for dedicated CI
>> workers, it's still a problem. Infra has scripts to wipe all
>> not-currently-in-use Docker containers off of each machine every 24
>> hours (or did, last I looked).
>
> Argh. I really hope this isn't happening again, at least on the
> machines where Apache Yetus' test-patch runs regularly. It can manage the
> local cache just fine (which is why after we implemented the docker cache
> cleanup code, the Hadoop nodes rarely if ever had docker space problems...).
> I did separate that part of the code out, so if infra wants a _smarter_ way
> to clean the cache on nodes where test-patch and friends aren't getting used,
> the docker-cleanup utility from Yetus is an option. (Although, to be fair,
> that utility is poorly documented. Maybe I'll work on that this week if
> there is interest. )
Infra would LOVE a smarter way to clean the cache. We have to use a heavy
hammer because there are 300+ projects that want a piece of it, and who don’t
clean up.. We are not build engineers, so we rely on the community to advise us
in dealing with the challenges we face. I would be very happy to work with you
on tooling to improve the cleanup if it improves the experience for all
projects.
-C
