Hi Krishna,

Overall I'm supportive of having an official docker image.
I have a few questions:
- Can you clarify the process of selecting the Java version? Is the
proposal to only pick LTS versions? or to pick the highest version
supported by Kafka?
- Once a new Kafka version is released, what happens to the image
containing the previous release? Do we expect to still update it in
case of CVEs? If so for how long?
- How will we get notified that the base image has a CVE?
- Rather than having scripts PMC members have to run from their
machines, would it e possible to have a Jenkins job or GitHub action?

Thanks,
Mickael



On Fri, Oct 20, 2023 at 12:51 PM Vedarth Sharma
<vedarth.sha...@gmail.com> wrote:
>
> Hi Manikumar,
>
> Thanks for the feedback!
>
> 1. We propose the addition of a new directory named "docker" at the root of
> the repository, where all Docker-related code will be stored. A detailed
> directory structure has been added in the KIP.
> 2. We request the creation of an Apache Kafka repository (apache/kafka) on
> DockerHub, to be administered under the The Apache Software Foundation
> <https://hub.docker.com/u/apache>. The PMC members should have the
> necessary permissions for pushing updates to the docker repo.
>
> Thanks and regards,
> Vedarth
>
>
> On Fri, Oct 20, 2023 at 2:44 PM Manikumar <manikumar.re...@gmail.com> wrote:
>
> > Hi Krishna, Vedarth,
> >
> > Thanks for the KIP.
> >
> > 1. Can we add directory structure of Docker Image related files in Kafka
> > repo.
> >
> > 2. > Steps for the Docker image release will be included in the Release
> > Process doc of Apache Kafka
> >
> > Can we list down the requirements (repos, accounts) for releasing images to
> > docker hub. I am mainly asking because PMC needs to request docker hub
> > access/repos.
> > I can help in getting required repos/accounts.
> > https://infra.apache.org/docker-hub-policy.html
> >
> >
> > Thanks,
> > Manikumar
> >
> > On Thu, Oct 19, 2023 at 8:22 PM Krishna Agarwal <
> > krishna0608agar...@gmail.com> wrote:
> >
> > > Hi Viktor,
> > >
> > > I've noticed there are two types of custom jar configurations:
> > >
> > >    1. *Type 1*: In this case, only the class name is required(e.g
> > > *authorizer.class.name
> > >    <http://authorizer.class.name>**)* This can be configured by the
> > >    following steps:
> > >       - Mount the jar in the container.
> > >       - Configure the *CLASSPATH* environment variable (used by
> > >       *kafka-run-class.sh*) by providing the mounted path to it. This can
> > >       be passed as an environment variable to the docker container.
> > >    2. *Type 2*: Here, in addition to the class name, classpath can also
> > be
> > >    configured (eg *remote.log.metadata.manager.class.name
> > >    <http://remote.log.metadata.manager.class.name> *and
> > >    *remote.log.metadata.manager.class.path*). This can be configured by
> > the
> > >    following steps:
> > >       - Mount the jar in the container.
> > >       - Configure the respective *class.path* property.
> > >
> > > Regards,
> > > Krishna
> > >
> > > On Mon, Sep 25, 2023 at 11:41 PM Krishna Agarwal <
> > > krishna0608agar...@gmail.com> wrote:
> > >
> > > > Hi Viktor,
> > > > Thanks for the questions.
> > > >
> > > >    1. While the docker image outlined in KIP-975 is designed for
> > > >    production environments, it is equally suitable for development and
> > > testing
> > > >    purposes. We will furnish the docker image, allowing users the
> > > flexibility
> > > >    to employ it according to their specific needs.
> > > >    2. The configs will be injected into the docker container through
> > > >    environment variables. These environment variables will have a
> > prefix
> > > >    allowing for efficient parsing to extract the relevant
> > > properties.(Will add
> > > >    this implementation in the KIP as well once we converge on this.)
> > > >    3. Regarding this question, I'll conduct a test on my end after
> > > >    gaining a better understanding, and then provide you with a
> > response.
> > > >
> > > > Regards,
> > > > Krishna
> > > >
> > > >
> > > > On Tue, Sep 19, 2023 at 3:42 PM Viktor Somogyi-Vass
> > > > <viktor.somo...@cloudera.com.invalid> wrote:
> > > >
> > > >> Hi Ismael,
> > > >>
> > > >> I'm not trying to advocate against the docker image, I just pointed
> > out
> > > >> that the current scoping of the KIP may be a bit too generic and
> > thought
> > > >> that KIP-974 and KIP-975 were aiming for mostly the same thing and can
> > > be
> > > >> discussed under one umbrella. Apologies if this was rooted in a
> > > >> misunderstanding.
> > > >>
> > > >> Kirshna,
> > > >>
> > > >> I think we need to refine the KIP a bit more. I think there are some
> > > >> interfaces that we need to include in the KIP as Kafka has plugins in
> > > >> certain cases where users are expected to provide implementation and I
> > > >> think it's worth discussing this in the KIP as they're kind of
> > > interfaces
> > > >> for users. Here are my questions in order:
> > > >> 1. In what environments do you want the image to be used? As I
> > > understand
> > > >> it would replace the current testing image and serve as a basis for
> > > >> development, but would it aim at production use cases too
> > > (docker-compose,
> > > >> Kubernetes, etc.)?
> > > >> 2. How do you plan to forward configs to the broker? Do we expect a
> > > >> populated server.properties file placed in a certain location or
> > should
> > > >> the
> > > >> docker image create this file based on some input (like env vars)?
> > > >> 3. Certain parts can be pluggable, like metric reporters or remote log
> > > >> implementations that were just introduced by KIP-405. These manifest
> > in
> > > >> jar
> > > >> files that must be put on the classpath of Kafka while certain
> > > classnames
> > > >> have to be configured. How do you plan to implement this, how do we
> > > >> allow users to configure such things?
> > > >>
> > > >> Thanks,
> > > >> Viktor
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Thu, Sep 14, 2023 at 4:59 PM Kenneth Eversole
> > > >> <kevers...@cloudflare.com.invalid> wrote:
> > > >>
> > > >> > Hello,
> > > >> >
> > > >> > I think this would be a wonderful improvement to the ecosystem.
> > While
> > > >> > Viktor is correct that most Docker pipelines eventually lead to a
> > > >> > kubernetes deployment, that should not stop us from creating an
> > > >> > Official Docker Image. Creating a Docker image would allow us to
> > > ensure
> > > >> a
> > > >> > level of quality and support for people who want to deploy Kafka as
> > a
> > > >> > container on baremetal machines, it could allow us to create
> > > >> > a sandbox/developer environment for new contributors and developers
> > to
> > > >> test
> > > >> > and have a single agreed upon environment that kafka works in for
> > > future
> > > >> > KIPs and would most likely spawn more contributions from people
> > > wanting
> > > >> to
> > > >> > optimize kafka for k8s.
> > > >> >
> > > >> >
> > > >> > I am 100% for this and will gladly help if approved.
> > > >> >
> > > >> > Kenneth
> > > >> >
> > > >> > On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma <m...@ismaeljuma.com>
> > > wrote:
> > > >> >
> > > >> > > Hi Viktor,
> > > >> > >
> > > >> > > I disagree. Docker is a very popular deployment tool and it's not
> > > only
> > > >> > used
> > > >> > > with Kubernetes.
> > > >> > >
> > > >> > > Ismael
> > > >> > >
> > > >> > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass
> > > >> > > <viktor.somo...@cloudera.com.invalid> wrote:
> > > >> > >
> > > >> > > > Hi Krishna,
> > > >> > > >
> > > >> > > > I think you should merge this KIP and KIP-974
> > > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974> as
> > there
> > > >> are
> > > >> > overlaps as
> > > >> > > > Federico pointed out on KIP-974
> > > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974>. I
> > think
> > > >> you
> > > >> > should keep that one as it
> > > >> > > > has well defined goals (improve tests) while I feel this one is
> > > too
> > > >> > > > generic. Docker is usually just a tool for either testing or
> > > >> > Kubernetes,
> > > >> > > so
> > > >> > > > they have very well defined use-cases. In the case of Flink for
> > > >> > instance
> > > >> > > > the image is used for its kubernetes operator. The use case
> > would
> > > >> > > determine
> > > >> > > > a lot of things and I think a generic image would likely not fit
> > > the
> > > >> > > needs
> > > >> > > > of all use-cases.
> > > >> > > >
> > > >> > > > Best,
> > > >> > > > Viktor
> > > >> > > >
> > > >> > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal <
> > > >> > > > krishna0608agar...@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi,
> > > >> > > > > Apache Kafka does not have an official docker image currently.
> > > >> > > > > I want to submit a KIP to publish a docker image for Apache
> > > Kafka.
> > > >> > > > >
> > > >> > > > > KIP-975 <
> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975>:
> > > >> > Docker Image for Apache Kafka
> > > >> > > > > <
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > > Regards,
> > > >> > > > > Krishna
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >

Reply via email to