Hi, Sorry for coming late here. Did we consider GitHub packages as a home of the Apache docker images? We already use GitHub for development and GitHub packages are better integrated with GitHub. In my personal opinion github packages are also less likely to be rate limited.
Best Piotr On Fri, 22 Nov 2024 at 19:03, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi > > That's correct: in Sung's PR, I can see the secret.DOCKERHUB_USER and > secret.DOCKERHUB_TOKEN. > So, we should be able to publish docker images via this GitHub action ;) > > Regards > JB > > On Fri, Nov 22, 2024 at 6:16 PM Fokko Driesprong <fo...@apache.org> wrote: > > > > I think Sung beat you to it: > https://github.com/apache/iceberg/pull/11632 > > > > As mentioned earlier it would be awesome if we could have a nightly > build so we can test all the different languages against the nightly. In > this case, when there are changes or new features, we can test/implement > them right away. > > > > Kind regards, > > Fokko > > > > Op vr 22 nov 2024 om 18:11 schreef Kevin Liu <kevin.jq....@gmail.com>: > >> > >> Thanks for setting this up, JB! It looks like PR #11283 is close to > being merged. > >> > >> What is the deployment strategy for the Docker image? Ideally, this > process could be fully automated using GitHub and GitHub Actions. > >> > >> I’d love to hear everyone’s thoughts on this! > >> > >> Best regards, > >> Kevin Liu > >> > >> > >> On Fri, Nov 22, 2024 at 6:06 AM Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >>> > >>> Hi folks, > >>> > >>> I created the iceberg repo on DockerHub (in the Apache org): > >>> > >>> https://hub.docker.com/r/apache/iceberg > >>> > >>> I created an "Iceberg team" on DockerHub. > >>> > >>> I created DOCKERHUB_USER and DOCKERHUB_TOKEN credentials for the > >>> Iceberg repo. That will allow us to directly push on DockerHub repo > >>> from GitHub Action. > >>> I also added Fokko to the repo. > >>> > >>> If you are a committer and you want to get permission on the Iceberg > >>> DockerHub repo, please let me know, I will add your DockerHub account > >>> to the "iceberg team". > >>> > >>> Thanks ! > >>> > >>> Regards > >>> JB > >>> > >>> On Fri, Nov 15, 2024 at 7:39 PM Kevin Liu <kevin.jq....@gmail.com> > wrote: > >>> > > >>> > +1 to Iceberg REST TCK docker image. Thanks, JB for driving this and > Ajantha for setting up the docker image. > >>> > We already found a bug in PyIceberg [1] from integrating with the > TCK docker image. It would be great to have a nightly build, perhaps we can > set up a Github Action to automate the docker image publishing. > >>> > > >>> > Best, > >>> > Kevin Liu > >>> > > >>> > > >>> > [1] https://github.com/apache/iceberg-python/pull/1321 > >>> > > >>> > On Fri, Nov 15, 2024 at 1:36 AM Fokko Driesprong <fo...@apache.org> > wrote: > >>> >> > >>> >> +1 — excited to see this happen! > >>> >> > >>> >> For the TCK, I think we can release this with the Java together, > and have a nightly build (tag the container with nightly Dockerhub). This > way we can already test out (and start implementing) the new features in > the related projects. Thoughts on that? > >>> >> > >>> >>> Regarding the Kafka Connect Docker image, I believe that if we > maintain it, we could also manage other integration images, such as those > for Spark and Trino with Iceberg. We should have a separate discussion on > which integration images Iceberg should officially support. > >>> >> > >>> >> > >>> >> Let's split out that discussion. My take on that is that we want to > defer that to the query engines. In an ideal situation, the Iceberg > integration should be part of the project itself (e.g. with Hive 4 where it > is maintained by Hive itself). For Spark itself, it only requires a runtime > to be added through the packages argument, and would love to see if we can > avoid maintaining images for that. > >>> >> > >>> >> Kind regards, > >>> >> Fokko > >>> >> > >>> >> > >>> >> Op do 14 nov 2024 om 18:16 schreef Christian Thiel < > christ...@hansetag.com.invalid>: > >>> >>> > >>> >>> +1 for this as well – for us especially the REST TCK image would > be nice. > >>> >>> > >>> >>> > >>> >>> > >>> >>> From: Bryan Keller <brya...@gmail.com> > >>> >>> Date: Thursday, 14. November 2024 at 17:13 > >>> >>> To: dev@iceberg.apache.org <dev@iceberg.apache.org> > >>> >>> Subject: Re: [PROPOSAL] Create Iceberg DockerHub repository > >>> >>> > >>> >>> +1 this would be great! Thanks JB. > >>> >>> > >>> >>> > >>> >>> > >>> >>> -Bryan > >>> >>> > >>> >>> > >>> >>> > >>> >>> On Nov 14, 2024, at 8:30 AM, Ajantha Bhat <ajanthab...@gmail.com> > wrote: > >>> >>> > >>> >>> > >>> >>> > >>> >>> +1 for setting up the DockerHub repo, > >>> >>> > >>> >>> We discussed about this already in > https://www.mail-archive.com/dev@iceberg.apache.org/msg07888.html > >>> >>> > >>> >>> Now that the Docker image PR is ready for the REST catalog > adapter, we can proceed with setting up the DockerHub repository. > >>> >>> > >>> >>> Regarding the Kafka Connect Docker image, I believe that if we > maintain it, we could also manage other integration images, such as those > for Spark and Trino with Iceberg. We should have a separate discussion on > which integration images Iceberg should officially support. > >>> >>> > >>> >>> For now, maintaining the REST catalog adapter image has already > been approved in earlier discussions, so let’s start with that. > >>> >>> > >>> >>> - Ajantha > >>> >>> > >>> >>> > >>> >>> > >>> >>> On Thu, Nov 14, 2024 at 9:45 PM Sung Yun <sun...@apache.org> > wrote: > >>> >>> > >>> >>> Hi JB, > >>> >>> > >>> >>> That sounds great!! > >>> >>> > >>> >>> The REST TCK /adapter docker image will be super useful for the > Iceberg subprojects as it will ensure that they have access to a > light-weight REST Catalog Server image with the latest features to run > integration tests against. > >>> >>> > >>> >>> Sung > >>> >>> > >>> >>> On 2024/11/14 15:41:04 Jean-Baptiste Onofré wrote: > >>> >>> > Hi folks, > >>> >>> > > >>> >>> > While reviewing https://github.com/apache/iceberg/pull/11283, we > >>> >>> > discussed having a DockerHub repository for Iceberg. > >>> >>> > > >>> >>> > I can create this repository, similar to other Apache projects > (like > >>> >>> > for example https://hub.docker.com/r/apache/activemq-classic, > >>> >>> > https://hub.docker.com/r/apache/airflow, etc). > >>> >>> > I can create an iceberg group (on DockerHub), and committers can > ask > >>> >>> > to join (in order to be able to push docker images). > >>> >>> > > >>> >>> > For now, the purpose of this DockerHub repo is to host: > >>> >>> > - Iceberg REST TCK docker images > >>> >>> > - Iceberg Kafka Connect docker images > >>> >>> > > >>> >>> > Thoughts ? > >>> >>> > > >>> >>> > Regards > >>> >>> > JB > >>> >>> > > >>> >>> > >>> >>> >
