Yes, the rebalance should not happen in that case. That is a little bit strange. Could you try to launch a clean Kafka cluster with auto.leader.election disabled and try push data? When leader migration occurs, NotLeaderForPartition exception is expected.
Jiangjie (Becket) Qin On 3/6/15, 3:14 PM, "Zakee" <kzak...@netzero.net> wrote: >Yes, Jiangjie, I do see lots of these errors "Starting preferred replica >leader election for partitions” in logs. I also see lot of Produce >request failure warnings in with the NotLeader Exception. > >I tried switching off the auto.leader.relabalance to false. I am still >noticing the rebalance happening. My understanding was the rebalance will >not happen when this is set to false. > >Thanks >Zakee > > > >> On Feb 25, 2015, at 5:17 PM, Jiangjie Qin <j...@linkedin.com.INVALID> >>wrote: >> >> I don’t think num.replica.fetchers will help in this case. Increasing >> number of fetcher threads will only help in cases where you have a large >> amount of data coming into a broker and more replica fetcher threads >>will >> help keep up. We usually only use 1-2 for each broker. But in your case, >> it looks that leader migration cause issue. >> Do you see anything else in the log? Like preferred leader election? >> >> Jiangjie (Becket) Qin >> >> On 2/25/15, 5:02 PM, "Zakee" <kzak...@netzero.net >><mailto:kzak...@netzero.net>> wrote: >> >>> Thanks, Jiangjie. >>> >>> Yes, I do see under partitions usually shooting every hour. Anythings >>>that >>> I could try to reduce it? >>> >>> How does "num.replica.fetchers" affect the replica sync? Currently have >>> configured 7 each of 5 brokers. >>> >>> -Zakee >>> >>> On Wed, Feb 25, 2015 at 4:17 PM, Jiangjie Qin >>><j...@linkedin.com.invalid> >>> wrote: >>> >>>> These messages are usually caused by leader migration. I think as long >>>> as >>>> you don¹t see this lasting for ever and got a bunch of under >>>>replicated >>>> partitions, it should be fine. >>>> >>>> Jiangjie (Becket) Qin >>>> >>>> On 2/25/15, 4:07 PM, "Zakee" <kzak...@netzero.net> wrote: >>>> >>>>> Need to know if I should I be worried about this or ignore them. >>>>> >>>>> I see tons of these exceptions/warnings in the broker logs, not sure >>>> what >>>>> causes them and what could be done to fix them. >>>>> >>>>> ERROR [ReplicaFetcherThread-3-5], Error for partition [TestTopic] to >>>>> broker >>>>> 5:class kafka.common.NotLeaderForPartitionException >>>>> (kafka.server.ReplicaFetcherThread) >>>>> [2015-02-25 11:01:41,785] ERROR [ReplicaFetcherThread-3-5], Error for >>>>> partition [TestTopic] to broker 5:class >>>>> kafka.common.NotLeaderForPartitionException >>>>> (kafka.server.ReplicaFetcherThread) >>>>> [2015-02-25 11:01:41,785] WARN [Replica Manager on Broker 2]: Fetch >>>>> request >>>>> with correlation id 950084 from client ReplicaFetcherThread-1-2 on >>>>> partition [TestTopic,2] failed due to Leader not local for partition >>>>> [TestTopic,2] on broker 2 (kafka.server.ReplicaManager) >>>>> >>>>> >>>>> Any ideas? >>>>> >>>>> -Zakee >>>>> ____________________________________________________________ >>>>> Next Apple Sensation >>>>> 1 little-known path to big profits >>>> >>>>> >>>>>http://thirdpartyoffers.netzero.net/TGL3231/54ee63b9e704b63b94061st03v >>>>>uc >>>> >>>> ____________________________________________________________ >>>> Extended Stay America >>>> Get Fantastic Amenities, low rates! Kitchen, Ample Workspace, Free >>>>WIFI >>>> >>>>http://thirdpartyoffers.netzero.net/TGL3255/54ee66f26da6f66f10ad4mp02du >>>>c >>>> >> >> >> ____________________________________________________________ >> Extended Stay America >> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed. >> http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13duc >><http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13duc>
