Topic stopped processing messages

2019-08-27 Thread Sebastian Schmitz
Hello, I just had a weird problem in one of my topics. It accepted messages from producers but consumers couldn't see those messages. The producers didn't get any errors when sending messages to the topic, but even my test-console-consumer didn't see anything in the topic. Also checking the t

Help in Configuring Kafka on my local Mac

2019-08-27 Thread Santohsh B
Hi Community, I am accessing events from the enterorise Kafka topics via Windows command line via keytab successfully. However, I have transitioned to macbook this week and the same set of commands are not working in Mac. Any documentation on how to connect to kafka topic via macbook and using k

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-08-27 Thread Manikumar
Hi all, I put together a draft release plan with Oct 2019 as the release month and a list of KIPs that have already been voted: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=125307901 Here are the dates: 1) KIP Freeze: Sep 25, 2019 (A KIP must be accepted by this date in ord

Re: kafka client 2.0.1 compatibility with kafka broker 0.10.0.1

2019-08-27 Thread Upendra Yadav
FYI: I changed the dependencies of brooklin to use kafka 0.10.0.1. But compilation is getting failed. 1st failure related test case where it trying to create KafkaServer, where constructor args mismatch. I have changed KafkaServer creation API to old one, but another set of issues: brooklin having

Byzantine Fault Tolerance Implementation

2019-08-27 Thread Nayak, Soumya R.
Hi Eliza, I used the wrong word stability. Rather it should be a byzantine fault tolerant network. Regards, Soumya -Original Message- From: Eliza Sent: Tuesday, August 27, 2019 12:55 PM To: users@kafka.apache.org Subject: Re: Byzantine Fault Tolerance Implementation Hi on 2019/8/27

Re: Metrics of BytesRejectedPerSec keeping rising

2019-08-27 Thread 李海军
Hi Liam, Thank you for you kind reply. I will check it out to see if this helps. There are amost 5K topics in our cluster with 6 nodes. We collect the metrics by collectd (https://collectd.org/) and visualize with Grafana. If all these per topic metrics are collected, will it affect the performan

Re: Metrics of BytesRejectedPerSec keeping rising

2019-08-27 Thread Liam Clarke
Hi, The broker exposes per topic error rates, that might help? https://docs.confluent.io/current/kafka/monitoring.html#per-topic-metrics Kind regards, Liam Clarke On Tue, 27 Aug. 2019, 8:07 pm 李海军, wrote: > Hi all, > > We have a kafka cluster consisted of 6 brokers with the version of > kaf

Metrics of BytesRejectedPerSec keeping rising

2019-08-27 Thread 李海军
Hi all, We have a kafka cluster consisted of 6 brokers with the version of kafka_2.11-0.10.2.1. These days we saw a continuous increasing on the metrics of "BytesRejectedPerSec" on broker 1. I already knew that this could happen when the message records were too large and raise a "RecordTooLargeEx

Re: Byzantine Fault Tolerance Implementation

2019-08-27 Thread Eliza
Hi on 2019/8/27 12:36, Nayak, Soumya R. wrote: Zookeeper uses a version of Paxos - Zookeeper atomic broadcast. Is there any plan in future or current in progress where zookeeper will be implemented with a BFT algorithm. This might help to have a more stable distributed environment when we hav