Thanks for setting this up, JB! It looks like PR #11283 <https://github.com/apache/iceberg/pull/11283> is close to being merged.
What is the deployment strategy for the Docker image? Ideally, this process could be fully automated using GitHub and GitHub Actions. I’d love to hear everyone’s thoughts on this! Best regards, Kevin Liu On Fri, Nov 22, 2024 at 6:06 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi folks, > > I created the iceberg repo on DockerHub (in the Apache org): > > https://hub.docker.com/r/apache/iceberg > > I created an "Iceberg team" on DockerHub. > > I created DOCKERHUB_USER and DOCKERHUB_TOKEN credentials for the > Iceberg repo. That will allow us to directly push on DockerHub repo > from GitHub Action. > I also added Fokko to the repo. > > If you are a committer and you want to get permission on the Iceberg > DockerHub repo, please let me know, I will add your DockerHub account > to the "iceberg team". > > Thanks ! > > Regards > JB > > On Fri, Nov 15, 2024 at 7:39 PM Kevin Liu <kevin.jq....@gmail.com> wrote: > > > > +1 to Iceberg REST TCK docker image. Thanks, JB for driving this and > Ajantha for setting up the docker image. > > We already found a bug in PyIceberg [1] from integrating with the TCK > docker image. It would be great to have a nightly build, perhaps we can set > up a Github Action to automate the docker image publishing. > > > > Best, > > Kevin Liu > > > > > > [1] https://github.com/apache/iceberg-python/pull/1321 > > > > On Fri, Nov 15, 2024 at 1:36 AM Fokko Driesprong <fo...@apache.org> > wrote: > >> > >> +1 — excited to see this happen! > >> > >> For the TCK, I think we can release this with the Java together, and > have a nightly build (tag the container with nightly Dockerhub). This way > we can already test out (and start implementing) the new features in the > related projects. Thoughts on that? > >> > >>> Regarding the Kafka Connect Docker image, I believe that if we > maintain it, we could also manage other integration images, such as those > for Spark and Trino with Iceberg. We should have a separate discussion on > which integration images Iceberg should officially support. > >> > >> > >> Let's split out that discussion. My take on that is that we want to > defer that to the query engines. In an ideal situation, the Iceberg > integration should be part of the project itself (e.g. with Hive 4 where it > is maintained by Hive itself). For Spark itself, it only requires a runtime > to be added through the packages argument, and would love to see if we can > avoid maintaining images for that. > >> > >> Kind regards, > >> Fokko > >> > >> > >> Op do 14 nov 2024 om 18:16 schreef Christian Thiel < > christ...@hansetag.com.invalid>: > >>> > >>> +1 for this as well – for us especially the REST TCK image would be > nice. > >>> > >>> > >>> > >>> From: Bryan Keller <brya...@gmail.com> > >>> Date: Thursday, 14. November 2024 at 17:13 > >>> To: dev@iceberg.apache.org <dev@iceberg.apache.org> > >>> Subject: Re: [PROPOSAL] Create Iceberg DockerHub repository > >>> > >>> +1 this would be great! Thanks JB. > >>> > >>> > >>> > >>> -Bryan > >>> > >>> > >>> > >>> On Nov 14, 2024, at 8:30 AM, Ajantha Bhat <ajanthab...@gmail.com> > wrote: > >>> > >>> > >>> > >>> +1 for setting up the DockerHub repo, > >>> > >>> We discussed about this already in > https://www.mail-archive.com/dev@iceberg.apache.org/msg07888.html > >>> > >>> Now that the Docker image PR is ready for the REST catalog adapter, we > can proceed with setting up the DockerHub repository. > >>> > >>> Regarding the Kafka Connect Docker image, I believe that if we > maintain it, we could also manage other integration images, such as those > for Spark and Trino with Iceberg. We should have a separate discussion on > which integration images Iceberg should officially support. > >>> > >>> For now, maintaining the REST catalog adapter image has already been > approved in earlier discussions, so let’s start with that. > >>> > >>> - Ajantha > >>> > >>> > >>> > >>> On Thu, Nov 14, 2024 at 9:45 PM Sung Yun <sun...@apache.org> wrote: > >>> > >>> Hi JB, > >>> > >>> That sounds great!! > >>> > >>> The REST TCK /adapter docker image will be super useful for the > Iceberg subprojects as it will ensure that they have access to a > light-weight REST Catalog Server image with the latest features to run > integration tests against. > >>> > >>> Sung > >>> > >>> On 2024/11/14 15:41:04 Jean-Baptiste Onofré wrote: > >>> > Hi folks, > >>> > > >>> > While reviewing https://github.com/apache/iceberg/pull/11283, we > >>> > discussed having a DockerHub repository for Iceberg. > >>> > > >>> > I can create this repository, similar to other Apache projects (like > >>> > for example https://hub.docker.com/r/apache/activemq-classic, > >>> > https://hub.docker.com/r/apache/airflow, etc). > >>> > I can create an iceberg group (on DockerHub), and committers can ask > >>> > to join (in order to be able to push docker images). > >>> > > >>> > For now, the purpose of this DockerHub repo is to host: > >>> > - Iceberg REST TCK docker images > >>> > - Iceberg Kafka Connect docker images > >>> > > >>> > Thoughts ? > >>> > > >>> > Regards > >>> > JB > >>> > > >>> > >>> >
