Hi, Some ideas worth investigating:
1. Using a mix of current kafka client options (i.e.: like maxPollrecords, maxPollInterval, etc) along with aggregate EIP 2. Create your own KafkaClientFactory that wraps a custom Producer/Consumer wrapping the Spring Kafka Consumer. If none of this works, then I suggest opening a ticket with your suggestion. We have been pretty busy / overloaded with the work on Camel 4, so it's pretty easy for us to miss bug reports and interesting feature suggestions - as yours.If possible, try to provide a pseudo-code, a reproducer or a unit test that the community can look at and work with. Thanks On Wed, Mar 15, 2023 at 1:31 PM Mark Nuttall <mknutt...@gmail.com> wrote: > I know consuming Kafka messages in a batch is not currently supported. I > googled and I didn't find any real options. I looked at hacking the Camel > classes to implement it and decided that it was too risky. > > So for now we are falling back to using a Spring Kafka Consumer. The issue > with that is I still want to use Camel because we need to do some complex > EIP processing and republishing of messages. And since Spring does not know > about Camel Context startup i have to do a hack to make it wait if the > CamelContext is not in the "started" state. > > Questions: > > - Is it possible to use the Spring Kafka Consumer in the "From" and be > managed by Camel (I think not but i figured i would ask). > - Is implementing batch consumers in the Kafka component planned/in > progress? > - If none of these - any ideas to make it work with the Kafka component? > -- Otavio R. Piske http://orpiske.net
