Re: what need to be done if we increase the Kafka nodes from 3 to 5?

They will be members of the cluster, yes, but they won't be serving any partitions unless you create new topics where partitions might be assigned to the new brokers, or run things which automatically balance partitions (kafka-monitor does that, and cruise-control can). You'll need to run partitio

what need to be done if we increase the Kafka nodes from 3 to 5?

Hi All I want to increase the Kafka nodes from 3 to 5 and just wondering what need to be done. Will it work if we directly add the 2 kafka into the cluster without changing anything?( like the partition, replicas ) ?

Re: How much time does it usually take for a rolling upgrade to complete?

That's unusual from my perspective. Where is the time being spent? And what versions are you using? We have over 20 clusters between 3-9 brokers each, and I do rolling upgrades across all of them in an hour or two with the main delay being keeping an eye on client performance. I can push through

Re: RecordTooLargeException

You can consider increase `max.request.size` ​a little big (the default value is `1048576`.), after checking Kafka client source code, they count [`key size` + `value size` + `header size` + others] together, so it's possible the calculated size is a little bigger than the default value. please ch

RecordTooLargeException

Hi all, I get this error even when my records are smaller than the 112 byte limit: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept. How do I ensure that my producer doesn't send records that are t

RecordTooLargeException

Hi all, I get this error even when my records are smaller than the 112 byte limit: org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept. How do I ensure that my producer doesn't send records that are t

Re: [VOTE] 2.0.0 RC1

+1.  1) Ran unit tests  2) 3 node cluster , tested basic operations.  Thanks, Harsha On Mon, Jul 2nd, 2018 at 11:13 AM, "Vahid S Hashemian" wrote: > > > > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). > > > Minor: It seems this doc updat

Re: [VOTE] 2.0.0 RC1

+1 . * Ran unit tests * Verified signatures * Ran 3 node cluster with basic operations Thanks, Harsha On Mon, Jul 2nd, 2018 at 11:13 AM, "Vahid S Hashemian" wrote: > > > > +1 (non-binding) > > Built from source and ran quickstart successfully on Ubuntu (with Java 8). > > > Minor: It see

Re: [kafka-clients] [VOTE] 1.0.2 RC1

+1.      1) Ran unit tests 2) 3 node cluster , tested basic operations. Thanks, Harsha On Mon, Jul 2nd, 2018 at 11:57 AM, Jun Rao wrote: > > > > Hi, Matthias, > > Thanks for the running the release. Verified quickstart on scala 2.12 > binary. +1 > > Jun > > On Fri, Jun 29, 2018 at 10:02 P

How much time does it usually take for a rolling upgrade to complete?

The answer to this depends on several factors: 1. Broker specs, 2. Connectivity between cluster brokers, 3. Number of messages stored in each partition, etc. I'd still like to get a sense how long rolling upgrades take for deployments out there however. My setup: 1. 5-broker cluster with 10GBp

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

In fact, to push the idea further (which IIRC is what Matthias originally proposed), if we can accept "Suppression#finalResultsOnly" in my last email, then we could also consider whether to eliminate "suppressLateEvents" entirely. We could always add it later, but you've both expressed doubt that

Re: [kafka-clients] [VOTE] 1.0.2 RC1

Hi, Matthias, Thanks for the running the release. Verified quickstart on scala 2.12 binary. +1 Jun On Fri, Jun 29, 2018 at 10:02 PM, Matthias J. Sax wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 1.0.2. > > This is a b

Re: [VOTE] 2.0.0 RC1

+1 (non-binding) Built from source and ran quickstart successfully on Ubuntu (with Java 8). Minor: It seems this doc update PR is not included in the RC: https://github.com/apache/kafka/pull/5280 Guozhang seems to have wanted to cherry-pick it to 2.0. Thanks Rajini! --Vahid From: Rajini S

Re: [RESULTS] [VOTE] Release Kafka version 0.11.0.3

Correcting the vote breakdown. Harash should be listed as a committer. Sorry for the mistake! This vote passes with 8 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jun * Rajini * Ismael Committers: * Harash * Matthias Community: * Vahid * Manikumar * Ted 0 votes * No vo

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

Hi again, Guozhang ;) Here's the second part of my response... It seems like your main concern is: "if I'm a user who wants final update semantics, how complicated is it for me to get it?" I think we have to assume that people don't always have time to become deeply familiar with all the nuances

[RESULTS] [VOTE] Release Kafka version 0.11.0.3

This vote passes with 8 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jun * Rajini * Ismael Committers: * Matthias Community: * Vahid * Manikumar * Ted * Harash 0 votes * No votes -1 votes * No votes Vote thread: http://search-hadoop.com/m/Kafka/uyzND1yVsGqxIVN5?subj=+VO

Re: [kafka-clients] Re: [VOTE] 0.11.0.3 RC0

+1 -Matthias On 7/1/18 11:28 AM, Manikumar wrote: > +1 (non-binding)   Verified quick start, ran producer/consumer perf tests, > > Thanks > > > On Sun, Jul 1, 2018 at 8:29 PM Ismael Juma > wrote: > > +1 (binding) > > Verified signature of source artifact,

[RESULTS] [VOTE] Release Kafka version 0.10.2.2

This vote passes with 5 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Jason * Guozhang * Jun Committers: * Matthias Community: * Ted 0 votes * No votes -1 votes * No votes Vote thread: http://search-hadoop.com/m/Kafka/uyzND1Wt6721GMzOE1?subj=+VOTE+0+10+2+2+RC1 I'll cont

Re: Possible bug? Duplicates when searching kafka stream state store with caching

We're using the latest Kafka (1.1.0). I'd like to note that when we encounter duplicates, the window is the same as well. My original code was a bit simplifier -- we also insert into the store if iterator.hasNext() as well, before returning null. We're using a window store because we have a punctu

Re: [kafka-clients] [VOTE] 0.10.2.2 RC1

+1 -Matthias On 6/29/18 2:46 PM, Jun Rao wrote: > Hi, Matthias, > > Thanks for running the release. Verified quickstart on scala 2.12 binary. +1 > > Jun > > On Fri, Jun 22, 2018 at 6:43 PM, Matthias J. Sax > wrote: > > Hello Kafka users, developers and clien

Re: Change in email id for Subscription

It's self-service. Unsubscribe with your old email and subscribe with your new one: https://kafka.apache.org/contact On 7/1/18 9:19 PM, Malik, Shibha (GE Renewable Energy, consultant) wrote: > Hi, > > I want to change my email id for subscription. Is this the right group to > email to ? > > Tha

Re: [DISCUSS] KIP-328: Ability to suppress updates for KTables

Hi Guozhang, Thanks for the clarification. To answer your questions: 1. Yes, specifically Y < X makes sense and is by design. The scenario is to support IQ queries over windows that are closed but not evicted. For example, suppose we have a metrics application backed by Streams. Let's say we do

KIP 179 - AdminClient Partition Assignment

Hi I recently cam a cross a use case where I would like to have the ability to increase the replication factor of a topic using the admin client. This is currently not possible, and the KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-179+-+Change+ReassignPartitionsCommand+to+use+Admin