Hi Piotr That's a good point.
As DockerHub is managed by The ASF, I think it's worth it to have docker images hosted there at least. That said, I don't see a problem with publishing on GH Packages. Regards JB On Thu, Dec 5, 2024 at 2:57 PM Piotr Findeisen <piotr.findei...@gmail.com> wrote: > > Hi, > > Sorry for coming late here. > Did we consider GitHub packages as a home of the Apache docker images? > We already use GitHub for development and GitHub packages are better > integrated with GitHub. > In my personal opinion github packages are also less likely to be rate > limited. > > Best > Piotr > > > > > On Fri, 22 Nov 2024 at 19:03, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: >> >> Hi >> >> That's correct: in Sung's PR, I can see the secret.DOCKERHUB_USER and >> secret.DOCKERHUB_TOKEN. >> So, we should be able to publish docker images via this GitHub action ;) >> >> Regards >> JB >> >> On Fri, Nov 22, 2024 at 6:16 PM Fokko Driesprong <fo...@apache.org> wrote: >> > >> > I think Sung beat you to it: https://github.com/apache/iceberg/pull/11632 >> > >> > As mentioned earlier it would be awesome if we could have a nightly build >> > so we can test all the different languages against the nightly. In this >> > case, when there are changes or new features, we can test/implement them >> > right away. >> > >> > Kind regards, >> > Fokko >> > >> > Op vr 22 nov 2024 om 18:11 schreef Kevin Liu <kevin.jq....@gmail.com>: >> >> >> >> Thanks for setting this up, JB! It looks like PR #11283 is close to being >> >> merged. >> >> >> >> What is the deployment strategy for the Docker image? Ideally, this >> >> process could be fully automated using GitHub and GitHub Actions. >> >> >> >> I’d love to hear everyone’s thoughts on this! >> >> >> >> Best regards, >> >> Kevin Liu >> >> >> >> >> >> On Fri, Nov 22, 2024 at 6:06 AM Jean-Baptiste Onofré <j...@nanthrax.net> >> >> wrote: >> >>> >> >>> Hi folks, >> >>> >> >>> I created the iceberg repo on DockerHub (in the Apache org): >> >>> >> >>> https://hub.docker.com/r/apache/iceberg >> >>> >> >>> I created an "Iceberg team" on DockerHub. >> >>> >> >>> I created DOCKERHUB_USER and DOCKERHUB_TOKEN credentials for the >> >>> Iceberg repo. That will allow us to directly push on DockerHub repo >> >>> from GitHub Action. >> >>> I also added Fokko to the repo. >> >>> >> >>> If you are a committer and you want to get permission on the Iceberg >> >>> DockerHub repo, please let me know, I will add your DockerHub account >> >>> to the "iceberg team". >> >>> >> >>> Thanks ! >> >>> >> >>> Regards >> >>> JB >> >>> >> >>> On Fri, Nov 15, 2024 at 7:39 PM Kevin Liu <kevin.jq....@gmail.com> wrote: >> >>> > >> >>> > +1 to Iceberg REST TCK docker image. Thanks, JB for driving this and >> >>> > Ajantha for setting up the docker image. >> >>> > We already found a bug in PyIceberg [1] from integrating with the TCK >> >>> > docker image. It would be great to have a nightly build, perhaps we >> >>> > can set up a Github Action to automate the docker image publishing. >> >>> > >> >>> > Best, >> >>> > Kevin Liu >> >>> > >> >>> > >> >>> > [1] https://github.com/apache/iceberg-python/pull/1321 >> >>> > >> >>> > On Fri, Nov 15, 2024 at 1:36 AM Fokko Driesprong <fo...@apache.org> >> >>> > wrote: >> >>> >> >> >>> >> +1 — excited to see this happen! >> >>> >> >> >>> >> For the TCK, I think we can release this with the Java together, and >> >>> >> have a nightly build (tag the container with nightly Dockerhub). This >> >>> >> way we can already test out (and start implementing) the new features >> >>> >> in the related projects. Thoughts on that? >> >>> >> >> >>> >>> Regarding the Kafka Connect Docker image, I believe that if we >> >>> >>> maintain it, we could also manage other integration images, such as >> >>> >>> those for Spark and Trino with Iceberg. We should have a separate >> >>> >>> discussion on which integration images Iceberg should officially >> >>> >>> support. >> >>> >> >> >>> >> >> >>> >> Let's split out that discussion. My take on that is that we want to >> >>> >> defer that to the query engines. In an ideal situation, the Iceberg >> >>> >> integration should be part of the project itself (e.g. with Hive 4 >> >>> >> where it is maintained by Hive itself). For Spark itself, it only >> >>> >> requires a runtime to be added through the packages argument, and >> >>> >> would love to see if we can avoid maintaining images for that. >> >>> >> >> >>> >> Kind regards, >> >>> >> Fokko >> >>> >> >> >>> >> >> >>> >> Op do 14 nov 2024 om 18:16 schreef Christian Thiel >> >>> >> <christ...@hansetag.com.invalid>: >> >>> >>> >> >>> >>> +1 for this as well – for us especially the REST TCK image would be >> >>> >>> nice. >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> From: Bryan Keller <brya...@gmail.com> >> >>> >>> Date: Thursday, 14. November 2024 at 17:13 >> >>> >>> To: dev@iceberg.apache.org <dev@iceberg.apache.org> >> >>> >>> Subject: Re: [PROPOSAL] Create Iceberg DockerHub repository >> >>> >>> >> >>> >>> +1 this would be great! Thanks JB. >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> -Bryan >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> On Nov 14, 2024, at 8:30 AM, Ajantha Bhat <ajanthab...@gmail.com> >> >>> >>> wrote: >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> +1 for setting up the DockerHub repo, >> >>> >>> >> >>> >>> We discussed about this already in >> >>> >>> https://www.mail-archive.com/dev@iceberg.apache.org/msg07888.html >> >>> >>> >> >>> >>> Now that the Docker image PR is ready for the REST catalog adapter, >> >>> >>> we can proceed with setting up the DockerHub repository. >> >>> >>> >> >>> >>> Regarding the Kafka Connect Docker image, I believe that if we >> >>> >>> maintain it, we could also manage other integration images, such as >> >>> >>> those for Spark and Trino with Iceberg. We should have a separate >> >>> >>> discussion on which integration images Iceberg should officially >> >>> >>> support. >> >>> >>> >> >>> >>> For now, maintaining the REST catalog adapter image has already been >> >>> >>> approved in earlier discussions, so let’s start with that. >> >>> >>> >> >>> >>> - Ajantha >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> On Thu, Nov 14, 2024 at 9:45 PM Sung Yun <sun...@apache.org> wrote: >> >>> >>> >> >>> >>> Hi JB, >> >>> >>> >> >>> >>> That sounds great!! >> >>> >>> >> >>> >>> The REST TCK /adapter docker image will be super useful for the >> >>> >>> Iceberg subprojects as it will ensure that they have access to a >> >>> >>> light-weight REST Catalog Server image with the latest features to >> >>> >>> run integration tests against. >> >>> >>> >> >>> >>> Sung >> >>> >>> >> >>> >>> On 2024/11/14 15:41:04 Jean-Baptiste Onofré wrote: >> >>> >>> > Hi folks, >> >>> >>> > >> >>> >>> > While reviewing https://github.com/apache/iceberg/pull/11283, we >> >>> >>> > discussed having a DockerHub repository for Iceberg. >> >>> >>> > >> >>> >>> > I can create this repository, similar to other Apache projects >> >>> >>> > (like >> >>> >>> > for example https://hub.docker.com/r/apache/activemq-classic, >> >>> >>> > https://hub.docker.com/r/apache/airflow, etc). >> >>> >>> > I can create an iceberg group (on DockerHub), and committers can >> >>> >>> > ask >> >>> >>> > to join (in order to be able to push docker images). >> >>> >>> > >> >>> >>> > For now, the purpose of this DockerHub repo is to host: >> >>> >>> > - Iceberg REST TCK docker images >> >>> >>> > - Iceberg Kafka Connect docker images >> >>> >>> > >> >>> >>> > Thoughts ? >> >>> >>> > >> >>> >>> > Regards >> >>> >>> > JB >> >>> >>> > >> >>> >>> >> >>> >>>
