[ANNOUNCE] Apache Kafka 2.3.1

2019-10-24 Thread David Arthur
The Apache Kafka community is pleased to announce the release for Apache Kafka 2.3.1 This is a bugfix release for Kafka 2.3.0. All of the changes in this release can be found in the release notes: https://www.apache.org/dist/kafka/2.3.1/RELEASE_NOTES.html You can download the source and binary r

Re: Kafka 2.2.1 with OpenJDK 11

2019-10-24 Thread Debraj Manna
Can you point me to the link where I have to check? On Thu 24 Oct, 2019, 7:54 PM M. Manna, wrote: > Have you checked the Kafka build 2.3.1 RC2 which everyone is currently > voting for ? It’s worth checking for your question... > > Regards. > On Thu, 24 Oct 2019 at 13:31, Debraj Manna > wrote: >

Re: Running a stand alone Mirror Maker 2.0 cluster and Issues

2019-10-24 Thread Vishal Santoshi
I might have created a build from the trunk, rather then the 2.4 branch , but will confirm. On Thu, Oct 24, 2019 at 4:44 PM Vishal Santoshi wrote: > The above may not be an issue as in it just uses the returned class > loader to resolve the Connector I think . What is not obvious, why it does

Re: NoClassDefFoundError: org/rocksdb/RocksDBException with 2.3.1-rc2

2019-10-24 Thread Mark Anderson
I've not had a chance to try 2.3.x yet. If suggested mvn dependency:tree as I think I had a conflict when trying to override the default spring boot dependency in the past. I've also had to do stuff like https://docs.spring.io/spring-kafka/docs/2.2.10.RELEASE/reference/html/#deps-for-21x in the p

Re: NoClassDefFoundError: org/rocksdb/RocksDBException with 2.3.1-rc2

2019-10-24 Thread Xiyuan Hu
Yes, I'm using spring boot. Version 2.3.0 doesn't work for me since the performance issue with cache. Thus I'd like to use the newer version with the fix. Are you able to import 2.3.1-rc2 in spring boot? Thanks! On Thu, Oct 24, 2019 at 4:21 PM Mark Anderson wrote: > > Are you using Spring Boot?

Re: Running a stand alone Mirror Maker 2.0 cluster and Issues

2019-10-24 Thread Vishal Santoshi
The above may not be an issue as in it just uses the returned class loader to resolve the Connector I think . What is not obvious, why it does not go ahead and consume .. [mm2-dev-749469cf68-vpm2l] [2019-10-24 20:26:07,571] INFO refreshing known target topics took 15 ms (org.apache.kafka.connec

Re: NoClassDefFoundError: org/rocksdb/RocksDBException with 2.3.1-rc2

2019-10-24 Thread Mark Anderson
Are you using Spring Boot? I know that the recent Spring Boot 2.2.0 release specifically updates their Kafka dependency to 2.3.0. Previous version used Kafka 2.1.x though I've used 2.2.x with it. Maybe running mvn dependency:tree would help see if there are multiple Kafka versions that could conf

Re: Kafka with Monolithic

2019-10-24 Thread Swen Moczarski
We integrate our micro services and the monolith via Kafka. One general issue when integrating with a legacy application might be when you plan to write in a database and send the data also to Kafka. That might lead to inconsistencies when something brakes in the middle. In that case its maybe wort

Re: NoClassDefFoundError: org/rocksdb/RocksDBException with 2.3.1-rc2

2019-10-24 Thread Matthias J. Sax
Hard to say, but maybe Spring does not support the new version yet, as the error say `org.springframework.beans.factory.BeanCreationException`. Can you reach out to the Spring community to see if they can help? -Matthias On 10/24/19 10:59 AM, Xiyuan Hu wrote: > Hi, > > I'm trying to build and

Re: Running a stand alone Mirror Maker 2.0 cluster and Issues

2019-10-24 Thread Vishal Santoshi
Hey Ryanne, Seeing the below ERROR in the logs and then, it seems the process does not consume ( it does not exit with any errors ) . And this is intermittent. As in do it enough times. that does relaunch :) Is this something a known bug [mm2-dev-58bf5df684-ln9k2] [2019-10-2

Re: Kafka with Monolithic

2019-10-24 Thread André Missaglia
In our company, we use Kafka to integrate dozens of services, including our main monolith web application. I don't think it is a problem to use a Monolithic Architecture with Kafka, just beware that different implementations have different delivery guarantees and message loss potential. Em ter, 15

NoClassDefFoundError: org/rocksdb/RocksDBException with 2.3.1-rc2

2019-10-24 Thread Xiyuan Hu
Hi, I'm trying to build and import 2.3.1-rc2 but getting below exceptions: Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/rocksdb/RocksDBException org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'imaItemProcessor':

Re: Kafka Streams TimeoutException and keeps rebalancing

2019-10-24 Thread Xiyuan Hu
Thanks Matthias, I think it's related to https://issues.apache.org/jira/browse/KAFKA-8802. Once I disabled the cache, bytes out goes up a lot. Before the bytes out are kept at 14MB/S no matter bytes in is high or low. I'm trying to download the 2.3.1-rc2 build and try it again. On Thu, Oct 24, 2

Re: Monitor Kafka connect jobs

2019-10-24 Thread KhajaAsmath Mohammed
Thanks Jamie. This is helpful. Do we need to start the job again if cluster is restarted due to maintenance or upgrades ? Sent from my iPhone > On Oct 24, 2019, at 11:58 AM, Jamie wrote: > > Hi Asmath > When you create a connector the request is forwarded onto the rest api of the > worker wh

Re: Monitor Kafka connect jobs

2019-10-24 Thread Jamie
Hi Asmath When you create a connector the request is forwarded onto the rest api of the worker which is the leader, it uses a Kafka producer to update the topic which stores your connector config. The other workers will continuously monitor this config topic and know if it changes then then they

Monitor Kafka connect jobs

2019-10-24 Thread KhajaAsmath Mohammed
Hi, We are using kafka connect in production and I have few questions about it. when we submit kafka connect job using rest api . Job gets continously running in the background and due to some issues, lets say we restarted kafka cluster. do we need to start manually all the jobs again? Is there

Re: Kafka 2.2.1 with OpenJDK 11

2019-10-24 Thread M. Manna
Have you checked the Kafka build 2.3.1 RC2 which everyone is currently voting for ? It’s worth checking for your question... Regards. On Thu, 24 Oct 2019 at 13:31, Debraj Manna wrote: > Hi > > Does Kafka work with OpenJDK 11? I have seen the below issue which is > resolved in 2.1. > > https://is

Re: [VOTE] 2.3.1 RC2

2019-10-24 Thread David Arthur
Thanks to everyone who voted! The vote for RC2 of the 2.3.1 release passes with the 6 +1s and no +0 or -1. +1 votes PMC Members: * Jason Gustafson * Guozhang Wang * Matthias Sax * Rajini Sivaram Committers: * Colin McCabe Community: * Jonathan Santilli 0 votes * No votes -1 votes * No votes

Kafka 2.2.1 with OpenJDK 11

2019-10-24 Thread Debraj Manna
Hi Does Kafka work with OpenJDK 11? I have seen the below issue which is resolved in 2.1. https://issues.apache.org/jira/browse/KAFKA-7264 But it does not mention about OpenJDK. Can someone confirm if Kafka 2.2.1 is supported with OpenJDK 11 also?

Re: [VOTE] 2.3.1 RC2

2019-10-24 Thread Rajini Sivaram
+1 (binding) Verified signatures, built source and ran tests, verified binary using broker, producer and consumer with security enabled. Regards, Rajini On Wed, Oct 23, 2019 at 11:37 PM Matthias J. Sax wrote: > +1 (binding) > > - downloaded and compiled source code > - verified signatures f

Re: Re: Keep getting expiring X record(s) for changelog topic:326554 ms has passed since batch creation

2019-10-24 Thread Jamie
Have you considered increasing max.block.ms or buffer.memory? Thanks,  Jamie  -Original Message- From: Matthias J. Sax To: users Sent: Thu, Oct 24, 2019 06:35 AM Subject: Re: Keep getting expiring X record(s) for changelog topic:326554 ms has passed since batch creation I would recom