Hi Claus, I agree with you that we will need both, especially on these configurations that are related to Confluent platform which are ported back to the community as you mentioned. I have created a JIRA ticket for it here: https://issues.apache.org/jira/browse/CAMEL-14062, I will work on it once I am done from DBZ which is already on the final stage that I can create a PR for.
Regards, Omar On Sun, 13 Oct 2019 at 18:19, Claus Ibsen <[email protected]> wrote: > Hi Omar > > Yeah the maven plugin idea is really good. > But maybe we need both. > > The maven plugin will only get you the options of the kafka version > that was used for that release. > With having a way for end users to set extra options then they could > work against newer Kafka versions without Camel have to upgrade. > And maybe also to proprietary options on Confluent. > > > On Sun, Oct 13, 2019 at 2:31 PM Omar Al-Safi <[email protected]> wrote: > > > > Hi Claus, > > > > I also thought about this, however since Kafka producers/consumers > options > > increase and some of them may gets deprecated, I was thinking we can > have a > > maven plugin that generate these configurations for us without the need > to > > do it manually, it will generate using the configDef (e.g: > > > https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java#L555 > ). > > I did this for DBZ as I built a maven plugin that scrap all these config > > and generate the camel configurations, you can take a look at my progress > > here: > > > https://github.com/omarsmak/camel/blob/CAMEL-14009/components/camel-debezium/camel-debezium-maven-plugin/src/main/java/org/apache/camel/maven/TestMojo.java#L24 > . > > Please take a look and share your thoughts about it. > > > > Regards, > > Omar > > > > On Sun, 13 Oct 2019 at 14:12, Claus Ibsen <[email protected]> wrote: > > > > > Hi > > > > > > Yeah this has come up a few times that Kafka is adding new options, or > > > more particularly Confluent adds propriety options that they dont give > > > back to the community. > > > > > > Can you create a JIRA ticket. > > > > > > Then we can look at adding a way to add custom options. You are of > > > course welcome to try to implement this. We love contributions. > > > https://github.com/apache/camel/blob/master/CONTRIBUTING.md > > > > > > On Tue, Oct 8, 2019 at 9:57 AM Erik Boye <[email protected]> > wrote: > > > > > > > > Hi, > > > > > > > > Is there any way to configure custom Kafka producer/consumer > properties > > > on the Camel Kafka component/endpoint (without "breaking in")? > > > > I specifically want to configure the Confluent Kafka property with > key > > > 'value.subject.name.strategy': > > > > https://docs.confluent.io/current/schema-registry/serializer-formatter.html > > > > > > > > Other Camel components (i.e Camel ActiveMQ; destination.consumer) are > > > made extensible by providing users a way to set "additional > properties". > > > Why does not Camel Kafka provide that? Until schemaRegistryURL was > added as > > > a configurable property, we had to make our own Camel Kafka Component > to > > > make it work ("breaking in"). There should definitely be a way to set > > > additional/custom properties on Kafka producers/consumers. > > > > > > > > Just have a look at the Camel Kafka documentation and do a text > search > > > for Confluent: > > > https://camel.apache.org/components/latest/kafka-component.html ;-) > > > > Having "native" support for schemas on Kafka records is definitely a > > > good idea, and I am pretty sure Apache Kafka will move in that > direction at > > > some point.... > > > > > > > > Erik > > > > > > > > "This email with attachments is solely for the use of the individual > or > > > entity to which it is addressed. It may contain confidential or > privileged > > > information. If you are not the addressee, please notify the sender and > > > delete this message and all attachments from your files." > > > > > > > > > > > > -- > > > Claus Ibsen > > > ----------------- > > > http://davsclaus.com @davsclaus > > > Camel in Action 2: https://www.manning.com/ibsen2 > > > > > > > -- > Claus Ibsen > ----------------- > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 >
