@Matt Sicker <boa...@gmail.com> -> agree on both, central chart repo, and per-project sources for charts. Though packaging is a bit different than Docker though. The packages contain tar.gz sources of the chart (which usually includes quite complex go templating logic, not just simple yaml files) and those templates include names of the images used. This makes the Helm chart much more close to the -src.tar.gz files we release now than to Docker Images we publish. So IMHO the .tgz part should follow the normal release process with voting @Kaxilnaik <kaxiln...@apache.org>. Unless those are the very same sources of product that are already formally released. But then @Gaetan Semet previously pointed out as bad practice to bind the two together. IMHO that means that we have to vote on each Chart release. But I'd love what others think - knowing this packaging details.
@ermanndimb- my point for images pointed to by the helm chart is not about copying full sources (I think from this discussion I realised where the main confusion was) . This is not needed. I just made a PR to show how this can be done with the pgbouncer-exporter. This is quite a good example to base our discussion on: https://github.com/apache/airflow/pull/10759 The point is that we MUST give the users possibility to (re)build those images on their own if they want. They should be able to do that using: a) official base images (DockerHub has the "official image program" for platforms. openjdk, python,debian, alpine - those are all official images in DockerHub terms b) having access to properly licensed source code with some guarantees that it will not disappear ("official" repo, multiple forks in Github, heavily used by others, fork under "apache" organisation) . c) instructions where to find the sources and build the images (Dockerfile + necessary scripts to build the image if in case it is not straightforward). Similarly as we provide instructions on building the "product". If the first two are available but it's not obvious how the image was built - IMHO we need to provide the user instructions on how to build those images (ideally a script that does it automatically). This is all fulfilled by the PR above as an example: a) based on official, alpine image b) the pgbouncer code is properly licenced and we forked the sources several times to be sure it won't disappear c) we have a script that builds and publishes the image (image is published under apache/airflow DockerHub) IMHO - that's a very good example to show the kind of approach we can take, and shows that it isn't difficult to handle it well at all. The alternative that I am afraid of is - that we require our users to use binaries which are not "official" but maintained and controlled by 3rd-party (without a straightforward way of building the binary from official images + properly licensed sources + instructions). If we do not give those to the user but we simply point to a 3rd-party binary bimage - we both endorse the 3rd party, and introduce dependency on the 3rd-party. The user then has to rely on the 3rd-party to use the Airflow Helm Chart - which I think is a very bad idea. J. On Sat, Sep 5, 2020 at 7:42 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > Does voting needs to happen for "releasing" the Helm chart ? Do we any > guidelines from the ASF around releasing Helm Chart & Dockerfile? > > On Sat, Sep 5, 2020, 17:36 Matt Sicker <boa...@gmail.com> wrote: > > > If packaging is similar to Docker, then our Helm charts will still > > need some third party base images and such for GPL licensed system > > dependencies (besides the kernel itself, there's OpenJDK which is > > definitely used in plenty of projects here). The bits we build on top > > of external components becomes the Apache project that is ALv2 and is > > distributed through source archives with convenience binaries (in this > > case, the image layers and YAML files). > > > > I do like the idea of having a central Apache Helm chart repo where > > releases can be published. That makes it simpler for users to combine > > multiple Apache charts together, too, based on my past experience with > > using Helm (especially with the new decentralized charts distribution > > model they've switched to). I do think it makes sense to try and > > maintain the sources and such for those charts in the individual > > projects since each PMC may have different workflows on how that's > > created, and it makes it easier to release alongside the normal > > project's releases. > > > > On Sat, 5 Sep 2020 at 11:13, Daniel Imberman <dimber...@apache.org> > wrote: > > > > > > While I understand the necessity for users to have access to all source > > code, I'm a bit concerned about the complexity this places on the > > open-source project. Let's take Airflow as an example. If we want to use > > pgbouncer in our home chart are we expected to be able to build this > > external project? > > > > > > I think that if this is ASF policy, then maybe we need to modify the > > policy to meet the current ecosystem. Helm charts are made to handle > > complex deployments, and expecting a project to own every other piece of > > their ecosystem is a pretty heavy burden. Is there some compromise we can > > come to? Can we store certain pieces such as source code, Dockerfile, and > > docker binary in some Apache cold storage to ensure they are > reproduceable? > > > > > > Ideally I'd just like to keep the burden on project maintainers to a > > minimum. This could heavily discourage projects from creating official > helm > > charts. > > > > > > > > > On 2020/09/04 15:05:56, Scott Rigby <sc...@r6by.com> wrote: > > > > Jarek, this is fantastic! Please let me know if I/we can help with > > this. The Helm team has written tools to make some of these processes > > easier, including steps to preserve former git history, and GitHub > Actions > > for CI (chart testing) and CD (releasing chart version packages via > GitHub > > release artifacts). > > > > > > > > Bertrand, many orgs are adopting a git repo naming convention > > "helm-charts" to make this explicit. > > > > > > > > Fun follow-up question: a helm repo is really just an index (YAML) > > file of metadata in a format the helm client expects, including links to > > packaged tarballs for each immutable version of a chart. The idea of > Apache > > mirror hosting is on the table, one issue we have is that the GCP object > > storage buckets that currently host all previous versions of stable and > > incubator repo charts will be deleted after support for these repos ends > on > > Nov 13th ( > > > https://github.com/helm/charts/blob/master/README.md#deprecation-timeline > ). > > Would it be possible for Apache to host a mirror of all these packages > > (note, they are all Apache licenced), or only chart packages related to > > Apache software (airflow, solr, spark, etc)? > > > > > > > > Scott > > > > > > > > On 2020/09/04 08:47:30, Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > Big +1 for doing this within existing ASF infrastructure. Maybe > > indeed > > > > > current SVN publishing a release snapshot + mirroring is the best > > way. > > > > > > > > > > IMHO such 'released' charts do not have to have commit history at > all > > > > > (except 'release' commits) - just release 'snapshots' is enough. > For > > all > > > > > the other development uses, whatever the source repository is, you > > will be > > > > > able too install 'development' chart from the sources. > > > > > > > > > > Still having clear guidelines on how to approach Docker images and > > > > > 'rebuildability' by the users from sources is an important aspect > > IMHO. > > > > > J. > > > > > > > > > > pt., 4 wrz 2020, 10:15 użytkownik Bertrand Delacretaz < > > > > > bdelacre...@apache.org> napisał: > > > > > > > > > > > Hi, > > > > > > > > > > > > On Fri, Sep 4, 2020 at 9:22 AM Jarek Potiuk <ja...@potiuk.com> > > wrote: > > > > > > > ...I think having ASF "apache/chart" for all "officially > > released" helm > > > > > > > charts is the best solution.... > > > > > > > > > > > > Sounds like a good idea but please use a more specific name, > > "charts" > > > > > > is very generic. > > > > > > > > > > > > If distributing those Helm charts does not requires more than a > web > > > > > > server (with some metadata files I suppose) that can probably go > > in a > > > > > > specific folder under > https://dist.apache.org/repos/dist/release/ > > > > > > > > > > > > The next question is what kind of traffic is expected there, and > > can > > > > > > the clients which will download those Helm charts handle > redirects > > to > > > > > > distribution mirrors? If yes the mechanism of > > > > > > http://www.apache.org/dev/release-download-pages.html comes to > > mind, > > > > > > if you can piggyback on that mirroring infrastructure that's a > big > > > > > > plus IMO. > > > > > > > > > > > > I guess the way to move forward is for the people interested in > > this > > > > > > to list the technical and licensing/legal requirements on a wiki > > page > > > > > > or equivalent, to discuss with the ASF infrastructure team how to > > best > > > > > > support them. > > > > > > > > > > > > -Bertrand > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > > > > > -- > > Matt Sicker <boa...@gmail.com> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org > > For additional commands, e-mail: dev-h...@community.apache.org > > > > > -- +48 660 796 129
