+1 to Iceberg REST TCK docker image. Thanks, JB for driving this and Ajantha for setting up the docker image. We already found a bug in PyIceberg [1] from integrating with the TCK docker image. It would be great to have a nightly build, perhaps we can set up a Github Action to automate the docker image publishing.
Best, Kevin Liu [1] https://github.com/apache/iceberg-python/pull/1321 On Fri, Nov 15, 2024 at 1:36 AM Fokko Driesprong <fo...@apache.org> wrote: > +1 — excited to see this happen! > > For the TCK, I think we can release this with the Java together, and have > a nightly build (tag the container with nightly Dockerhub). This way we can > already test out (and start implementing) the new features in the related > projects. Thoughts on that? > > Regarding the Kafka Connect Docker image, I believe that if we maintain >> it, we could also manage other integration images, such as those for Spark >> and Trino with Iceberg. We should have a separate discussion on which >> integration images Iceberg should officially support. > > > Let's split out that discussion. My take on that is that we want to defer > that to the query engines. In an ideal situation, the Iceberg integration > should be part of the project itself (e.g. with Hive 4 where it is > maintained by Hive itself). For Spark itself, it only requires a runtime to > be added through the packages argument, and would love to see if we can > avoid maintaining images for that. > > Kind regards, > Fokko > > > Op do 14 nov 2024 om 18:16 schreef Christian Thiel > <christ...@hansetag.com.invalid>: > >> +1 for this as well – for us especially the REST TCK image would be nice. >> >> >> >> *From: *Bryan Keller <brya...@gmail.com> >> *Date: *Thursday, 14. November 2024 at 17:13 >> *To: *dev@iceberg.apache.org <dev@iceberg.apache.org> >> *Subject: *Re: [PROPOSAL] Create Iceberg DockerHub repository >> >> +1 this would be great! Thanks JB. >> >> >> >> -Bryan >> >> >> >> On Nov 14, 2024, at 8:30 AM, Ajantha Bhat <ajanthab...@gmail.com> wrote: >> >> >> >> +1 for setting up the DockerHub repo, >> >> We discussed about this already in >> https://www.mail-archive.com/dev@iceberg.apache.org/msg07888.html >> >> Now that the Docker image PR is ready for the REST catalog adapter, we >> can proceed with setting up the DockerHub repository. >> >> Regarding the Kafka Connect Docker image, I believe that if we maintain >> it, we could also manage other integration images, such as those for Spark >> and Trino with Iceberg. We should have a separate discussion on which >> integration images Iceberg should officially support. >> >> For now, maintaining the REST catalog adapter image has already been >> approved in earlier discussions, so let’s start with that. >> >> - Ajantha >> >> >> >> On Thu, Nov 14, 2024 at 9:45 PM Sung Yun <sun...@apache.org> wrote: >> >> Hi JB, >> >> That sounds great!! >> >> The REST TCK /adapter docker image will be super useful for the Iceberg >> subprojects as it will ensure that they have access to a light-weight REST >> Catalog Server image with the latest features to run integration tests >> against. >> >> Sung >> >> On 2024/11/14 15:41:04 Jean-Baptiste Onofré wrote: >> > Hi folks, >> > >> > While reviewing https://github.com/apache/iceberg/pull/11283, we >> > discussed having a DockerHub repository for Iceberg. >> > >> > I can create this repository, similar to other Apache projects (like >> > for example https://hub.docker.com/r/apache/activemq-classic, >> > https://hub.docker.com/r/apache/airflow, etc). >> > I can create an iceberg group (on DockerHub), and committers can ask >> > to join (in order to be able to push docker images). >> > >> > For now, the purpose of this DockerHub repo is to host: >> > - Iceberg REST TCK docker images >> > - Iceberg Kafka Connect docker images >> > >> > Thoughts ? >> > >> > Regards >> > JB >> > >> >> >> >
