Personally, I’m unsure about what good a nightly Docker image build helps with. If we’re backing up to the same Docker Hub repository, I think it could make some sense as long as we manage the “latest” tag properly to only ever tag releases since the proposed effort and maintenance overhead is low - but if we are going through the effort to create a whole new nightly/snapshot repo, what does this help us achieve? Do we see a proper use case for customers to ever really need that?
On the question that started this thread about the Polaris Admin Tool, I’m onboard with Option 3 - let’s separate out the Admin Tool and the Server. We should be careful to ensure that these move together in terms of versioning - but I don’t see that as a big enough concern to refute this option. Best, Adnan Hemani > On May 22, 2025, at 12:12 PM, Dmitri Bourlatchkov <di...@apache.org> wrote: > > good point about Renovate! > > On Wed, May 21, 2025 at 4:49 AM Robert Stupp <sn...@snazy.de> wrote: > >> How would tools like dependabot and Renovate distinguish work then? >> >> I think having separate `-unstable` repos is the safest way forward. >> >> >> On 20.05.25 19:10, Jean-Baptiste Onofré wrote: >>> Yes but we can use tag that way “by convention”. >>> >>> Le mar. 20 mai 2025 à 10:18, Robert Stupp <sn...@snazy.de> a écrit : >>> >>> There's no notion of "snapshot" or "nightly" for image tags. >>> That's why >>> we're pushing for the separate repos. >>> >>> On 20.05.25 05:18, Jean-Baptiste Onofré wrote: >>>> Yes agree. Latest would be a valid tag only for release images. >>>> Snapshot images will be just … snapshot tag ;) (not latest). >>>> >>>> Regards >>>> JB >>>> >>>> Le lun. 19 mai 2025 à 21:40, Dmitri Bourlatchkov >>> <di...@apache.org> a >>>> écrit : >>>> >>>>> If we put nightlies in the same repo, we should be careful >>> with the >>>>> "latest" tag. >>>>> >>>>> I suppose users will expect "latest" to track only officially >>> released >>>>> images in that case. >>>>> >>>>> It might even be worth _not_ using the "latest" tag at all. >>>>> >>>>> Cheers, >>>>> Dmitri. >>>>> >>>>> On Mon, May 19, 2025 at 3:24 PM Jean-Baptiste Onofré >>> <j...@nanthrax.net> >>>>> wrote: >>>>> >>>>>> I'm not super convinced by a repository dedicated to nightly >>> or snapshot. >>>>>> >>>>>> A nightly or a SNAPSHOT is a version/tag of an image. So, I would >>>>>> expect to have it in the same repository as the released images. >>>>>> >>>>>> For instance, you would have apache/polaris:x.y.z and >>>>>> apache/polaris:x.y.z-SNAPSHOT >>>>>> >>>>>> Some projects do that (for instance >>>>>> https://www.google.com/url?q=https://hub.docker.com/r/apache/gravitino/tags&source=gmail-imap&ust=1748545963000000&usg=AOvVaw32_izMPTTmU8H9LY8DFkqU). >>>>>> >>>>>> I would propose to have apache/polaris and >>> apache/polaris-admin-tool >>>>> repos. >>>>>> Thoughts ? >>>>>> >>>>>> Regards >>>>>> JB >>>>>> >>>>>> On Mon, May 19, 2025 at 8:43 PM Dmitri Bourlatchkov >>> <di...@apache.org> >>>>>> wrote: >>>>>>> Using a different repo name for nightlies / unstable sounds >>> good to me, >>>>>>> >>>>>>> Cheers, >>>>>>> Dmitri. >>>>>>> >>>>>>> On Mon, May 19, 2025 at 12:19 PM Alex Dutra >>>>>> <alex.du...@dremio.com.invalid> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> To be honest my preference would be Option 4: a different >>> image name >>>>>>>> for everything that is "nightly" or "unstable". For example, >>>>>>>> "apache/polaris-unstable" or >>> "apache/polaris-admin-tool-nightly". >>>>>>>> >>>>>>>> My reasoning is simple: make it almost impossible for a user to >>>>>>>> accidentally deploy an unstable version of the server into >>>>> production, >>>>>>>> by confusing an unstable tag with a production-ready one. >>>>>>>> >>>>>>>> Otherwise, I can also live with Option 3, especially if we >>> use tags >>>>>>>> that are "scary" enough to dissuade people from using them in >>>>>>>> production. >>>>>>>> >>>>>>>> Option 1 would be really bad for DevOps workflows: for >>> example, I >>>>>>>> think running Kubernetes Jobs to bootstrap or purge realms >>> (or doing >>>>>>>> other administrative tasks) will become a common practice; >>> but in >>>>> this >>>>>>>> case, the tool must be available as a Docker image. >>>>>>>> >>>>>>>> Option 2 is also bad: users expect Docker binaries to >>> contain the >>>>> same >>>>>>>> "thing" regardless of tags, so it would be confusing to mix >>> binaries >>>>>>>> for the server and the tool under the same image. Not to >>> mention that >>>>>>>> mixing binaries would absolutely preclude the usage of the tag >>>>>>>> "latest" since we wouldn't know if "latest" contains the >>> server or >>>>> the >>>>>>>> tool. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Alex >>>>>>>> >>>>>>>> >>>>>>>> On Mon, May 19, 2025 at 5:53 PM Robert Stupp >>> <sn...@snazy.de> wrote: >>>>>>>>> Also +1 on option 3. >>>>>>>>> >>>>>>>>> Would also propose to push releases and snapshots to >>> separate image >>>>>>>>> repositories. >>>>>>>>> >>>>>>>>> On 19.05.25 17:46, Dmitri Bourlatchkov wrote: >>>>>>>>>> Option 2 looks very confusing to me. While it can technically >>>>>> work, I >>>>>>>> think >>>>>>>>>> most people expect the repository name to reflect the >>> nature of >>>>> the >>>>>>>> binary, >>>>>>>>>> so apache/polaris would mean "server" by default. >>>>>>>>>> >>>>>>>>>> I prefer option 3. >>>>>>>>>> >>>>>>>>>> I also think we should have an image for the admin tool >>> because >>>>> it >>>>>> is >>>>>>>>>> required for bootstrapping. >>>>>>>>>> >>>>>>>>>> If the server is in k8s, it would be natural to run the admin >>>>> tool >>>>>> in >>>>>>>> k8s >>>>>>>>>> too, hence it needs a docker image. By providing an official >>>>> image >>>>>> we >>>>>>>>>> greatly simplify users' workflows. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Dmitri. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, May 19, 2025 at 11:11 AM Jean-Baptiste Onofré < >>>>>> j...@nanthrax.net >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi folks, >>>>>>>>>>> >>>>>>>>>>> Right now, as part of the release and nightly build, we >>> plan to >>>>>> push >>>>>>>>>>> the Polaris server docker image (on >>>>>>>>>>> https://www.google.com/url?q=https://hub.docker.com/r/apache/polaris&source=gmail-imap&ust=1748545963000000&usg=AOvVaw2XdKjt_AvlaM7mSP0rHDof). >>>>>>>>>>> Concretely, it means we push Polaris server >>>>>>>>>>> >>>>>>>>>>> As part of RC3 release prep, I pushed >>>>>>>>>>> apache/polaris:0.10.0-beta-incubating-rc3 image >>> (corresponding >>>>> to >>>>>>>>>>> Polaris server). >>>>>>>>>>> >>>>>>>>>>> The question is regarding the Polaris Admin Tool container >>>>> image. >>>>>> We >>>>>>>>>>> have basically 3 options: >>>>>>>>>>> 1. We only push Polaris server image on DockerHub (no admin >>>>> tool): >>>>>>>>>>> it's what I do in RC3 prep and also what I proposed in >>>>>>>>>>> https://www.google.com/url?q=https://github.com/apache/polaris/pull/1593&source=gmail-imap&ust=1748545963000000&usg=AOvVaw1BisjFCR64xVl636VC0Kwt >>>>>>>>>>> 2. We push Polaris Admin Tool in apache/polaris using a tag >>>>> format >>>>>>>>>>> (like apache/polaris:admin-tool-x.y.z) >>>>>>>>>>> 3. I create a dedicated repository apache/polaris-admin-tool >>>>>> where we >>>>>>>>>>> push only admin tool images >>>>>>>>>>> >>>>>>>>>>> Personally, I don't like (2), and I wonder if it makes >>> sense to >>>>>> push >>>>>>>>>>> admin tool image. If yes, I would propose (3) and I will be >>>>> happy >>>>>> to >>>>>>>>>>> create the corresponding Docker repository. >>>>>>>>>>> >>>>>>>>>>> Thoughts ? >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> JB >>>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Robert Stupp >>>>>>>>> @snazy >>>>>>>>> >>> -- >>> Robert Stupp >>> @snazy >>> >> -- >> Robert Stupp >> @snazy >>
