after add broker how to change partition leader to new broker

hi,all: on my cluster , some topic Under-replicated 100% , I have 7 broker and the topic have 7 partition ,but the topic now only use 5 broker as leader , how can I chang it to use all the broker as leader ? 2017-11-01 lk_kafka

Re: after add broker how to change partition leader to new broker

You can use kafka-reassign-partitions.sh to move some to new broker http://kafka.apache.org/documentation/#basic_ops_automigrate also check leader balancing: http://kafka.apache.org/documentation/#basic_ops_leader_balancing On Thu, Nov 2, 2017 at 2:24 PM, lk_kafka wrote: > hi,all: > on

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Congratulations for this milestone ! Thanks to Gouzhang for running the release ! Paolo Patierno Senior Software Engineer (IoT) @ Red Hat Microsoft MVP on Azure & IoT Microsoft Azure Advisor Twitter : @ppatierno Linkedin : paolopatierno

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Great Job! - Xin 2017-11-02 18:30 GMT+08:00 Paolo Patierno : > Congratulations for this milestone ! > > > Thanks to Gouzhang for running the release ! > > > Paolo Patierno > Senior Software Engineer (IoT) @ Red Hat > Microsoft MVP on Azure & IoT > Microsoft Azure Advisor > > Twitter : @ppatierno

Kafka Streams : CommitFailedException

Hi All, I am facing CommitFailedException in my streams application. As per the log I tried changing the max.poll.interval.ms and max.poll.records. But both didn't help. PFA the full stack trace of the exception and below is the streams configuration used. What else could be wrong? val props = Pr

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Thanks for running the release, Guozhang! Also thanks to all the contributors who made 1.0 possible. :) Ismael On 1 Nov 2017 2:27 pm, "Guozhang Wang" wrote: The Apache Kafka community is pleased to announce the release for Apache Kafka 1.0.0. This is a major release of the Kafka project, and i

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Thanks Guozhang! On Thu, 2 Nov 2017 at 11:42 Ismael Juma wrote: > Thanks for running the release, Guozhang! Also thanks to all the > contributors who made 1.0 possible. :) > > Ismael > > On 1 Nov 2017 2:27 pm, "Guozhang Wang" wrote: > > The Apache Kafka community is pleased to announce the rele

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Great news, Congratulations to the team ! On Thu, 2 Nov 2017 at 17:17 Damian Guy wrote: > Thanks Guozhang! > > On Thu, 2 Nov 2017 at 11:42 Ismael Juma wrote: > > > Thanks for running the release, Guozhang! Also thanks to all the > > contributors who made 1.0 possible. :) > > > > Ismael > > > >

Re: Help with Mirror Maker config to produce to all partitions in target cluster

Thanks Manikumar for the reply :) For errors in the logs, I have a few WARN messages about "Reconnect due to socked error: null", and an occasional "Connection timed out", but that's it. I definitely can turn on trace logs. Is there something specific you know of that I should look for? WRT key

Getting rid of Attempting to send response via channel for which there is no open connection

Hi, I wanted to know in which cases those warning arise and how to reduce them (getting rid of them is even better): Attempting to send response via channel for which there is no open connection I’m asking because I linked those warnings to data loss on my production cluster. Indee

Requesting for help Kafka

Hi, I would request to help me with this issue. I should not commit offset when any exception comes while processing a message. I am using below approach to manually comit offset. Can you please help me in getting uncommited offsets to re-process the message in later point of time. import org.a

documentation link broken

Hi I was trying to look at the documentation for the AdminClient API , but the link from page https://kafka.apache.org/documentation/#adminapi in the sentence "For more information about the AdminClient APIs, see the javadoc. " To the URL: https://kafka.apache.org/100/javadoc/index.html?org/apac

Apache Kafka TLV Meetup group

Hi, I am the organizer of Apache Kafka TLV Meetup in Tel-Aviv Israel, please add our meetup group to your global events list. https://www.meetup.com/Apache-Kafka-TLV/ BR, Ori Donner

Re: documentation link broken

See: https://github.com/apache/kafka/blob/1.0/clients/src/main/java/org/apache/kafka/clients/admin/AdminClient.java On Thu, Nov 2, 2017 at 5:51 AM, diane wrote: > Hi > > I was trying to look at the documentation for the AdminClient API , but > the link from page > https://kafka.apache.org/docume

Kafka as a broadcast tool?

Good afternoon - I am investigating using Kafka as a way to communicate directly with clients that will periodically come online. For example, I'm thinking of using a single topic and creating a new partition on that topic as the messaging channel for that specific client. And then if someone wan

Re: Getting rid of Attempting to send response via channel for which there is no open connection

Have you seen this thread ? http://search-hadoop.com/m/Kafka/uyzND1ppGvmNscWc?subj=Re+Data+loss+while+upgrading+confluent+3+0+0+kafka+cluster+to+confluent+3+2+2 On Thu, Nov 2, 2017 at 1:46 PM, Karim Lamouri wrote: > Hi, > > I wanted to know in which cases those warning arise and how to reduce t

clean leader election on kafka 0.10.2.1

We were on kafka 0.10.2.1. We tried to switch from unclean leader election to clean leader election and found it became very difficult to start up the whole cluster. It seems the hosts went into a deadlock situation during startup - broker A was a follower on partition 1 and waits for the lea

Re: documentation link broken

Hi Diane, Thanks for reporting this. I have fixed the links in `kafka-site`. Guozhang On Thu, Nov 2, 2017 at 2:53 PM, Ted Yu wrote: > See: > https://github.com/apache/kafka/blob/1.0/clients/src/ > main/java/org/apache/kafka/clients/admin/AdminClient.java > > On Thu, Nov 2, 2017 at 5:51 AM, d

Re: Apache Kafka TLV Meetup group

Hello Ori, I have added the meetup on the events list. Cheers, Guozhang On Thu, Nov 2, 2017 at 2:24 AM, ApacheKafka TLV wrote: > Hi, > > I am the organizer of Apache Kafka TLV Meetup in Tel-Aviv Israel, please > add our meetup group to your global events list. > > https://www.meetup.com/Apac

Need help understanding precedence of log.flush.interval.ms versus log.flush.scheduler.interval.ms

We ship Kafka on an appliance, so it's a production environment where I don't want to lose data, but I also don't have the luxury of replication. So I want to enable log flushing. However, I'm confusing reading the broker config docs... The docs seem to imply that I only need to set log.flush.int

Re: Kafka 0.11 broker running out of file descriptors

Hi, We also facing the same issue when we upgraded our cluste to 0.11.0.1. Can anyone please help us out. It happened in production cluster. Thanks Madhukar On Fri, 15 Sep 2017 at 9:07 PM, Lukas Lalinsky wrote: > Hello, > > I'm dealing with a strange issue in production and I'm running out of