kafka-reassign-partitions / JSON parsing broke between 2.1 and 2.2. Was : broken cluster / com.fasterxml.jackson.databind.ext.Java7Support.getDeserializerForJavaNioFilePath

2019-12-12 Thread Charles Bueche
Hello again, spending hours debugging this and having no clue... * Kafka 2.12-2.1.0 : kafka-reassign-partitions.sh is working fine * Anything newer (2.2.2, 2.3.1, trunk) breaks any operation involving JSON. Stack Dump below. Help please. Charles Le 11.12.19 à 16:58, Charles Bueche a éc

Re: kafka-reassign-partitions / JSON parsing broke between 2.1 and 2.2. Was : broken cluster / com.fasterxml.jackson.databind.ext.Java7Support.getDeserializerForJavaNioFilePath

2019-12-12 Thread Mark Anderson
Jackson was updated to 2.10 in the latest Kafka release. The method mentioned no longer exists in 2.10. Do you have multiple versions of Jackson on the ckasspath? On Thu, 12 Dec 2019, 11:09 Charles Bueche, wrote: > Hello again, > > spending hours debugging this and having no clue... > > * Kaf

Re: kafka-reassign-partitions / JSON parsing broke between 2.1 and 2.2. Was : broken cluster / com.fasterxml.jackson.databind.ext.Java7Support.getDeserializerForJavaNioFilePath

2019-12-12 Thread Charles Bueche
Mark, thanks for helping. I suspect as well something from the environment but can't identify any suspect trace. I use tarballs from the download pages, but I tried as well to build trunk and 2.2.2 and both approaches exhibit the issue as well. I deploy my Kafka using symlinks, eg /appl/kafka --

Re: Message batch & compression doesn't work as expected

2019-12-12 Thread M. Manna
Frank, On Thu, 12 Dec 2019 at 11:28, Frank Zhou wrote: > Hi, > > I am testing kafka client on message batch and compression. I have enabled > message batching along with compression, with batch.size set to 3M, > linger.ms set to 5000ms and compression.type set to gzip(Attached whole > producer c

Re: [VOTE] 2.4.0 RC4

2019-12-12 Thread Israel Ekpo
@Manikumar , I have rerun the latest release candidate (RC4) and all the tests have passed with no errors. Should we go ahead and close this issue regarding the Flaky test? - https://issues.apache.org/jira/browse/KAFKA-9283 On Tue, Dec 10, 2019 at 11:11 AM Adam Bellemare wrote: > - All PGP sig

Re: [VOTE] 2.4.0 RC4

2019-12-12 Thread Manikumar
Hi Israel, Thanks for testing the release candidate. We can use https://issues.apache.org/jira/browse/KAFKA-9283 for fixing the flaky test. Thanks, On Thu, Dec 12, 2019 at 8:18 PM Israel Ekpo wrote: > > @Manikumar , I have rerun the latest release > candidate (RC4) and all the tests have pass

Re: kafka-reassign-partitions / JSON parsing broke between 2.1 and 2.2. Was : broken cluster / com.fasterxml.jackson.databind.ext.Java7Support.getDeserializerForJavaNioFilePath

2019-12-12 Thread Charles Bueche
Hello again, issue found. I'm installing Kafka by an Ansible playbook. A supplemental module I installed was kafka-http-metrics-reporter-1.1.0.jar from https://github.com/pcross616/kafka-http-metrics-reporter to expose some JMX metrics by HTTP (because monitoring JMX from Nagios is so 1990...). M

Re: [VOTE] 2.4.0 RC4

2019-12-12 Thread Manikumar
Hi all, Successful Jenkins builds for the 2.4 branch: Unit/integration tests: https://builds.apache.org/blue/organizations/jenkins/kafka-2.4-jdk8/detail/kafka-2.4-jdk8/106/ System tests: http://confluent-kafka-2-4-system-test-results.s3-us-west-2.amazonaws.com/2019-12-12--001.1576143096--confluent

Re: [VOTE] 2.4.0 RC4

2019-12-12 Thread Mickael Maison
Thanks Manikumar for running this release I verified the signatures All tests passed without failures. We've also been running this release for a few days in a couple of environments without any issues. +1 (binding) On Thu, Dec 12, 2019 at 3:14 PM Manikumar wrote: > > Hi all, > > Successful Jen

Re: [VOTE] 2.4.0 RC4

2019-12-12 Thread Israel Ekpo
Thanks @Manikumar for running the release as well. In my latest run, all signature validations and tests passed successfully without any issues. +1 (non-binding) On Thu, Dec 12, 2019 at 10:25 AM Mickael Maison wrote: > Thanks Manikumar for running this release > > I verified the signatures >

Re: Reducing streams startup bandwidth usage

2019-12-12 Thread Sophie Blee-Goldman
Thanks for collecting all these metrics. It might be that as the length of the lists increases over time, the cache is able to hold fewer unique keys and eventually has to start evicting things. This would explain why the cache hit rate starts to decrease, and likely why latency starts to go up. Wh

Re: Message batch & compression doesn't work as expected

2019-12-12 Thread Frank Zhou
Hi, Definitely will check this out, thanks. We just started tuning recently, and we are quite new to kafka world. The problem we are facing is with batch.size and linger.ms both set, the patch sent out doesn't seem to meet either one condition. We tried with more "reasonable" values as well, it ju

Re: [VOTE] 2.4.0 RC4

2019-12-12 Thread Gwen Shapira
+1 (binding) Validated signatures, tests and ran some test workloads. Thank you so much for driving this. Mani. On Mon, Dec 9, 2019 at 9:32 AM Manikumar wrote: > > Hello Kafka users, developers and client-developers, > > This is the fifth candidate for release of Apache Kafka 2.4.0. > > This re