Hi Viktor, I've noticed there are two types of custom jar configurations:
1. *Type 1*: In this case, only the class name is required(e.g *authorizer.class.name <http://authorizer.class.name>**)* This can be configured by the following steps: - Mount the jar in the container. - Configure the *CLASSPATH* environment variable (used by *kafka-run-class.sh*) by providing the mounted path to it. This can be passed as an environment variable to the docker container. 2. *Type 2*: Here, in addition to the class name, classpath can also be configured (eg *remote.log.metadata.manager.class.name <http://remote.log.metadata.manager.class.name> *and *remote.log.metadata.manager.class.path*). This can be configured by the following steps: - Mount the jar in the container. - Configure the respective *class.path* property. Regards, Krishna On Mon, Sep 25, 2023 at 11:41 PM Krishna Agarwal < krishna0608agar...@gmail.com> wrote: > Hi Viktor, > Thanks for the questions. > > 1. While the docker image outlined in KIP-975 is designed for > production environments, it is equally suitable for development and testing > purposes. We will furnish the docker image, allowing users the flexibility > to employ it according to their specific needs. > 2. The configs will be injected into the docker container through > environment variables. These environment variables will have a prefix > allowing for efficient parsing to extract the relevant properties.(Will add > this implementation in the KIP as well once we converge on this.) > 3. Regarding this question, I'll conduct a test on my end after > gaining a better understanding, and then provide you with a response. > > Regards, > Krishna > > > On Tue, Sep 19, 2023 at 3:42 PM Viktor Somogyi-Vass > <viktor.somo...@cloudera.com.invalid> wrote: > >> Hi Ismael, >> >> I'm not trying to advocate against the docker image, I just pointed out >> that the current scoping of the KIP may be a bit too generic and thought >> that KIP-974 and KIP-975 were aiming for mostly the same thing and can be >> discussed under one umbrella. Apologies if this was rooted in a >> misunderstanding. >> >> Kirshna, >> >> I think we need to refine the KIP a bit more. I think there are some >> interfaces that we need to include in the KIP as Kafka has plugins in >> certain cases where users are expected to provide implementation and I >> think it's worth discussing this in the KIP as they're kind of interfaces >> for users. Here are my questions in order: >> 1. In what environments do you want the image to be used? As I understand >> it would replace the current testing image and serve as a basis for >> development, but would it aim at production use cases too (docker-compose, >> Kubernetes, etc.)? >> 2. How do you plan to forward configs to the broker? Do we expect a >> populated server.properties file placed in a certain location or should >> the >> docker image create this file based on some input (like env vars)? >> 3. Certain parts can be pluggable, like metric reporters or remote log >> implementations that were just introduced by KIP-405. These manifest in >> jar >> files that must be put on the classpath of Kafka while certain classnames >> have to be configured. How do you plan to implement this, how do we >> allow users to configure such things? >> >> Thanks, >> Viktor >> >> >> >> >> On Thu, Sep 14, 2023 at 4:59 PM Kenneth Eversole >> <kevers...@cloudflare.com.invalid> wrote: >> >> > Hello, >> > >> > I think this would be a wonderful improvement to the ecosystem. While >> > Viktor is correct that most Docker pipelines eventually lead to a >> > kubernetes deployment, that should not stop us from creating an >> > Official Docker Image. Creating a Docker image would allow us to ensure >> a >> > level of quality and support for people who want to deploy Kafka as a >> > container on baremetal machines, it could allow us to create >> > a sandbox/developer environment for new contributors and developers to >> test >> > and have a single agreed upon environment that kafka works in for future >> > KIPs and would most likely spawn more contributions from people wanting >> to >> > optimize kafka for k8s. >> > >> > >> > I am 100% for this and will gladly help if approved. >> > >> > Kenneth >> > >> > On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma <m...@ismaeljuma.com> wrote: >> > >> > > Hi Viktor, >> > > >> > > I disagree. Docker is a very popular deployment tool and it's not only >> > used >> > > with Kubernetes. >> > > >> > > Ismael >> > > >> > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass >> > > <viktor.somo...@cloudera.com.invalid> wrote: >> > > >> > > > Hi Krishna, >> > > > >> > > > I think you should merge this KIP and KIP-974 >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974> as there >> are >> > overlaps as >> > > > Federico pointed out on KIP-974 >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974>. I think >> you >> > should keep that one as it >> > > > has well defined goals (improve tests) while I feel this one is too >> > > > generic. Docker is usually just a tool for either testing or >> > Kubernetes, >> > > so >> > > > they have very well defined use-cases. In the case of Flink for >> > instance >> > > > the image is used for its kubernetes operator. The use case would >> > > determine >> > > > a lot of things and I think a generic image would likely not fit the >> > > needs >> > > > of all use-cases. >> > > > >> > > > Best, >> > > > Viktor >> > > > >> > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal < >> > > > krishna0608agar...@gmail.com> >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > Apache Kafka does not have an official docker image currently. >> > > > > I want to submit a KIP to publish a docker image for Apache Kafka. >> > > > > >> > > > > KIP-975 < >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975>: >> > Docker Image for Apache Kafka >> > > > > < >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka >> > > > > > >> > > > > >> > > > > Regards, >> > > > > Krishna >> > > > > >> > > > >> > > >> > >> >
