> On Oct 29, 2020, at 12:19 AM, Jarek Potiuk <jarek.pot...@polidea.com> wrote:
>
> Got the message too and also got very concerned.
>
> There is an easy (I think) solution though. The new GitHub Container
> Registry is a good replacement, providing that it is enabled by INFRA. I've
> opened the ticket few weeks ago as we would love to switch to it from the
> current (deprecated) GitHub Docker Registry:
> https://issues.apache.org/jira/projects/INFRA/issues/INFRA-20959
As I mentioned in the referenced ticket, the Github Container Registry (“GCR”)
is not something Infra can immediately support due to Github’s limitations. The
Board collectively with Infra needs to determine how or if the GCR fits our
needs from both operational and fiscal perspectives. Additionally, Infra does
not have the ability to enable this beta feature at a repository level, it is
an org-wide setting. Infra appreciates that this is of interest to some
projects, and will continue to review it.
-C
>
> We are already using the Docker Registry in Airflow for quite a while and
> very extensively to cache our build images and we have little complaints
> (one serious problem that could be workaround). Apparently, GCR is a much
> better replacement - it has all the management options (including the
> possibility for image retention - lacking in the current Docker Registry).
>
> Most importantly - the images can be marked as publicly available and
> without limits of Docker. It's free for public repositories.
>
> I think the only problem to solve is the management of access - by default,
> the image published to the registry is owned and accessible by the
> individual who uploads it (apparently), and any other write access can be
> given only explicitly (via API as well). But from the support ticket, I
> opened to Github Support - they are open to change it. You can link the
> image to a specific repository and if this leads to giving write access to
> all the committers for that repository, I think that would solve our
> problems entirely.
>
> Even without such "default" access, we could add a simple Action that could
> automatically assign the right access to such uploaded Images - I already
> wrote a few of those - happy to write another one.
>
> J.
>
>
>
> On Thu, Oct 29, 2020 at 7:57 AM Chris Lambertus <c...@apache.org> wrote:
>
>>
>>
>>> On Oct 28, 2020, at 11:47 PM, Allen Wittenauer <a...@effectivemachines.com>
>> wrote:
>>>
>>>
>>>
>>>> On Oct 28, 2020, at 9:01 PM, Joan Touzet <woh...@apache.org> wrote:
>>>>
>>>> Even for those of us lucky enough to have sponsorship for dedicated CI
>>>> workers, it's still a problem. Infra has scripts to wipe all
>>>> not-currently-in-use Docker containers off of each machine every 24
>>>> hours (or did, last I looked).
>>>
>>> Argh. I really hope this isn't happening again, at least on the
>> machines where Apache Yetus' test-patch runs regularly. It can manage the
>> local cache just fine (which is why after we implemented the docker cache
>> cleanup code, the Hadoop nodes rarely if ever had docker space
>> problems...). I did separate that part of the code out, so if infra wants
>> a _smarter_ way to clean the cache on nodes where test-patch and friends
>> aren't getting used, the docker-cleanup utility from Yetus is an option.
>> (Although, to be fair, that utility is poorly documented. Maybe I'll work
>> on that this week if there is interest. )
>>
>> Infra would LOVE a smarter way to clean the cache. We have to use a heavy
>> hammer because there are 300+ projects that want a piece of it, and who
>> don’t clean up.. We are not build engineers, so we rely on the community to
>> advise us in dealing with the challenges we face. I would be very happy to
>> work with you on tooling to improve the cleanup if it improves the
>> experience for all projects.
>>
>> -C
>>
>>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
