Thanks, Jiangjie for helping resolve the kafka controller migration driven
partition leader rebalance issue. The logs are much cleaner now.
There are a few incidences of Out of range offset even though there is no
consumers running, only producers and replica fetchers. I was trying to relate
to a cause, looks like compaction (log segment deletion) causing this. Not sure
whether this is expected behavior.
Broker-4:
[2015-03-14 07:46:52,338] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [Topic22kv,5] offset 1754769769 from
follower with correlation id 1645671. Possible cause: Request for offset
1754769769 but we only have log segments in the range 1400864851 to 1754769732.
(kafka.server.ReplicaManager)
Broker-3:
[2015-03-14 07:46:52,356] INFO The cleaning for partition [Topic22kv,5] is
aborted and paused (kafka.log.LogCleaner)
[2015-03-14 07:46:52,408] INFO Scheduling log segment 1400864851 for log
Topic22kv-5 for deletion. (kafka.log.Log)
…
[2015-03-14 07:46:52,421] INFO Compaction for partition [Topic22kv,5] is
resumed (kafka.log.LogCleaner)
[2015-03-14 07:46:52,517] ERROR [ReplicaFetcherThread-2-4], Current offset
1754769769 for partition [Topic22kv,5] out of range; reset offset to 1400864851
(kafka.server.ReplicaFetcherThread)
[2015-03-14 07:46:52,517] WARN [ReplicaFetcherThread-2-4], Replica 3 for
partition [Topic22kv,5] reset its fetch offset from 1400864851 to current
leader 4's start offset 1400864851 (kafka.server.ReplicaFetcherThread)
____________________________________________________________
Old School Yearbook Pics
View Class Yearbooks Online Free. Search by School & Year. Look Now!
http://thirdpartyoffers.netzero.net/TGL3231/5504a2032e49422021991st02vuc
on broker-4
[2015-03-14 07:46:52,338] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [Topic22kv,5] offset 1754769769 from
follower with correlation id 1645671. Possible cause: Request for offset
1754769769 but we only have log segments in the range 1400864851 to 1754769732.
(kafka.server.ReplicaManager)
[2015-03-14 07:46:52,759] INFO Closing socket connection to /19.10.4.143.
(kafka.network.Processor)
on broker-3
[2015-03-14 07:46:52,356] INFO The cleaning for partition [Topic22kv,5] is
aborted and paused (kafka.log.LogCleaner)
[2015-03-14 07:46:52,408] INFO Scheduling log segment 1400864851 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,408] INFO Scheduling log segment 1406227848 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,408] INFO Scheduling log segment 1411591123 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,408] INFO Scheduling log segment 1416954195 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1422317783 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1427680989 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1433044302 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1438407760 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1443770521 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1449133811 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,409] INFO Scheduling log segment 1454497169 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,410] INFO Scheduling log segment 1459860085 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,411] INFO Scheduling log segment 1465223478 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,411] INFO Scheduling log segment 1470586720 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,412] INFO Scheduling log segment 1475949659 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,412] INFO Scheduling log segment 1481312627 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,412] INFO Scheduling log segment 1486675299 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,412] INFO Scheduling log segment 1492038376 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,413] INFO Scheduling log segment 1497401497 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,413] INFO Scheduling log segment 1502764133 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,413] INFO Scheduling log segment 1508126631 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,413] INFO Scheduling log segment 1513489256 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,413] INFO Scheduling log segment 1518852045 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1524214801 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1529576936 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1534939512 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1540302166 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1545665202 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1551027837 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,414] INFO Scheduling log segment 1556390348 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,415] INFO Scheduling log segment 1561752847 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,415] INFO Scheduling log segment 1567115844 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,415] INFO Scheduling log segment 1572478219 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,415] INFO Scheduling log segment 1577840715 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,415] INFO Scheduling log segment 1583202882 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,415] INFO Scheduling log segment 1588565423 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,416] INFO Scheduling log segment 1593928097 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,416] INFO Scheduling log segment 1599290877 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,416] INFO Scheduling log segment 1604653392 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,416] INFO Scheduling log segment 1610015711 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,416] INFO Scheduling log segment 1615378342 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,416] INFO Scheduling log segment 1620740750 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1626103585 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1631466326 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1636828852 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1642191186 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1647554066 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1652916834 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,417] INFO Scheduling log segment 1658278968 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,418] INFO Scheduling log segment 1663641744 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,418] INFO Scheduling log segment 1669004451 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,418] INFO Scheduling log segment 1674367433 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,418] INFO Scheduling log segment 1679729882 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,418] INFO Scheduling log segment 1685092450 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1690455586 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1695818413 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1701181566 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1706544338 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1711906913 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1717269789 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,419] INFO Scheduling log segment 1722632729 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,420] INFO Scheduling log segment 1727994900 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,420] INFO Scheduling log segment 1733357931 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,420] INFO Scheduling log segment 1738720377 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,420] INFO Scheduling log segment 1744083107 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,420] INFO Scheduling log segment 1749446181 for log
Topic22kv-5 for deletion. (kafka.log.Log)
[2015-03-14 07:46:52,421] INFO Compaction for partition [Topic22kv,5] is
resumed (kafka.log.LogCleaner)
[2015-03-14 07:46:52,517] ERROR [ReplicaFetcherThread-2-4], Current offset
1754769769 for partition [Topic22kv,5] out of range; reset offset to 1400864851
(kafka.server.ReplicaFetcherThread)
[2015-03-14 07:46:52,517] WARN [ReplicaFetcherThread-2-4], Replica 3 for
partition [Topic22kv,5] reset its fetch offset from 1400864851 to current
leader 4's start offset 1400864851 (kafka.server.ReplicaFetcherThread)
Thanks
Zakee
> On Mar 9, 2015, at 12:18 PM, Zakee <kzak...@netzero.net> wrote:
>
> No broker restarts.
>
> Created a kafka issue: https://issues.apache.org/jira/browse/KAFKA-2011
> <https://issues.apache.org/jira/browse/KAFKA-2011>
>
>>> Logs for rebalance:
>>> [2015-03-07 16:52:48,969] INFO [Controller 2]: Resuming preferred replica
>>> election for partitions: (kafka.controller.KafkaController)
>>> [2015-03-07 16:52:48,969] INFO [Controller 2]: Partitions that completed
>>> preferred replica election: (kafka.controller.KafkaController)
>>> …
>>> [2015-03-07 12:07:06,783] INFO [Controller 4]: Resuming preferred replica
>>> election for partitions: (kafka.controller.KafkaController)
>>> ...
>>> [2015-03-07 09:10:41,850] INFO [Controller 3]: Resuming preferred replica
>>> election for partitions: (kafka.controller.KafkaController)
>>> ...
>>> [2015-03-07 08:26:56,396] INFO [Controller 1]: Starting preferred replica
>>> leader election for partitions (kafka.controller.KafkaController)
>>> ...
>>> [2015-03-06 16:52:59,506] INFO [Controller 2]: Partitions undergoing
>>> preferred replica election: (kafka.controller.KafkaController)
>>>
>>> Also, I still see lots of below errors (~69k) going on in the logs since
>>> the restart. Is there any other reason than rebalance for these errors?
>>>
>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-2-5], Error for
>>> partition [Topic-11,7] to broker 5:class
>>> kafka.common.NotLeaderForPartitionException
>>> (kafka.server.ReplicaFetcherThread)
>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-1-5], Error for
>>> partition [Topic-2,25] to broker 5:class
>>> kafka.common.NotLeaderForPartitionException
>>> (kafka.server.ReplicaFetcherThread)
>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-2-5], Error for
>>> partition [Topic-2,21] to broker 5:class
>>> kafka.common.NotLeaderForPartitionException
>>> (kafka.server.ReplicaFetcherThread)
>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-1-5], Error for
>>> partition [Topic-22,9] to broker 5:class
>>> kafka.common.NotLeaderForPartitionException
>>> (kafka.server.ReplicaFetcherThread)
>
>
>> Could you paste the related logs in controller.log?
> What specifically should I search for in the logs?
>
> Thanks,
> Zakee
>
>
>
>> On Mar 9, 2015, at 11:35 AM, Jiangjie Qin <j...@linkedin.com.INVALID
>> <mailto:j...@linkedin.com.INVALID>> wrote:
>>
>> Is there anything wrong with brokers around that time? E.g. Broker restart?
>> The log you pasted are actually from replica fetchers. Could you paste the
>> related logs in controller.log?
>>
>> Thanks.
>>
>> Jiangjie (Becket) Qin
>>
>> On 3/9/15, 10:32 AM, "Zakee" <kzak...@netzero.net
>> <mailto:kzak...@netzero.net>> wrote:
>>
>>> Correction: Actually the rebalance happened quite until 24 hours after
>>> the start, and thats where below errors were found. Ideally rebalance
>>> should not have happened at all.
>>>
>>>
>>> Thanks
>>> Zakee
>>>
>>>
>>>
>>>> On Mar 9, 2015, at 10:28 AM, Zakee <kzak...@netzero.net
>>>> <mailto:kzak...@netzero.net>> wrote:
>>>>
>>>>> Hmm, that sounds like a bug. Can you paste the log of leader rebalance
>>>>> here?
>>>> Thanks for you suggestions.
>>>> It looks like the rebalance actually happened only once soon after I
>>>> started with clean cluster and data was pushed, it didn’t happen again
>>>> so far, and I see the partitions leader counts on brokers did not change
>>>> since then. One of the brokers was constantly showing 0 for partition
>>>> leader count. Is that normal?
>>>>
>>>> Also, I still see lots of below errors (~69k) going on in the logs
>>>> since the restart. Is there any other reason than rebalance for these
>>>> errors?
>>>>
>>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-2-5], Error for
>>>> partition [Topic-11,7] to broker 5:class
>>>> kafka.common.NotLeaderForPartitionException
>>>> (kafka.server.ReplicaFetcherThread)
>>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-1-5], Error for
>>>> partition [Topic-2,25] to broker 5:class
>>>> kafka.common.NotLeaderForPartitionException
>>>> (kafka.server.ReplicaFetcherThread)
>>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-2-5], Error for
>>>> partition [Topic-2,21] to broker 5:class
>>>> kafka.common.NotLeaderForPartitionException
>>>> (kafka.server.ReplicaFetcherThread)
>>>> [2015-03-07 14:23:28,963] ERROR [ReplicaFetcherThread-1-5], Error for
>>>> partition [Topic-22,9] to broker 5:class
>>>> kafka.common.NotLeaderForPartitionException
>>>> (kafka.server.ReplicaFetcherThread)
>>>>
>>>>> Some other things to check are:
>>>>> 1. The actual property name is auto.leader.rebalance.enable, not
>>>>> auto.leader.rebalance. You’ve probably known this, just to double
>>>>> confirm.
>>>> Yes
>>>>
>>>>> 2. In zookeeper path, can you verify /admin/preferred_replica_election
>>>>> does not exist?
>>>> ls /admin
>>>> [delete_topics]
>>>> ls /admin/preferred_replica_election
>>>> Node does not exist: /admin/preferred_replica_election
>>>>
>>>>
>>>> Thanks
>>>> Zakee
>>>>
>>>>
>>>>
>>>>> On Mar 7, 2015, at 10:49 PM, Jiangjie Qin <j...@linkedin.com.INVALID
>>>>> <mailto:j...@linkedin.com.INVALID>>
>>>>> wrote:
>>>>>
>>>>> Hmm, that sounds like a bug. Can you paste the log of leader rebalance
>>>>> here?
>>>>> Some other things to check are:
>>>>> 1. The actual property name is auto.leader.rebalance.enable, not
>>>>> auto.leader.rebalance. You’ve probably known this, just to double
>>>>> confirm.
>>>>> 2. In zookeeper path, can you verify /admin/preferred_replica_election
>>>>> does not exist?
>>>>>
>>>>> Jiangjie (Becket) Qin
>>>>>
>>>>> On 3/7/15, 10:24 PM, "Zakee" <kzak...@netzero.net
>>>>> <mailto:kzak...@netzero.net>> wrote:
>>>>>
>>>>>> I started with clean cluster and started to push data. It still does
>>>>>> the
>>>>>> rebalance at random durations even though the auto.leader.relabalance
>>>>>> is
>>>>>> set to false.
>>>>>>
>>>>>> Thanks
>>>>>> Zakee
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Mar 6, 2015, at 3:51 PM, Jiangjie Qin <j...@linkedin.com.INVALID
>>>>>>> <mailto:j...@linkedin.com.INVALID>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Yes, the rebalance should not happen in that case. That is a little
>>>>>>> bit
>>>>>>> strange. Could you try to launch a clean Kafka cluster with
>>>>>>> auto.leader.election disabled and try push data?
>>>>>>> When leader migration occurs, NotLeaderForPartition exception is
>>>>>>> expected.
>>>>>>>
>>>>>>> Jiangjie (Becket) Qin
>>>>>>>
>>>>>>>
>>>>>>> On 3/6/15, 3:14 PM, "Zakee" <kzak...@netzero.net
>>>>>>> <mailto:kzak...@netzero.net>> wrote:
>>>>>>>
>>>>>>>> Yes, Jiangjie, I do see lots of these errors "Starting preferred
>>>>>>>> replica
>>>>>>>> leader election for partitions” in logs. I also see lot of Produce
>>>>>>>> request failure warnings in with the NotLeader Exception.
>>>>>>>>
>>>>>>>> I tried switching off the auto.leader.relabalance to false. I am
>>>>>>>> still
>>>>>>>> noticing the rebalance happening. My understanding was the rebalance
>>>>>>>> will
>>>>>>>> not happen when this is set to false.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Zakee
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Feb 25, 2015, at 5:17 PM, Jiangjie Qin
>>>>>>>>> <j...@linkedin.com.INVALID <mailto:j...@linkedin.com.INVALID>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I don’t think num.replica.fetchers will help in this case.
>>>>>>>>> Increasing
>>>>>>>>> number of fetcher threads will only help in cases where you have a
>>>>>>>>> large
>>>>>>>>> amount of data coming into a broker and more replica fetcher
>>>>>>>>> threads
>>>>>>>>> will
>>>>>>>>> help keep up. We usually only use 1-2 for each broker. But in your
>>>>>>>>> case,
>>>>>>>>> it looks that leader migration cause issue.
>>>>>>>>> Do you see anything else in the log? Like preferred leader
>>>>>>>>> election?
>>>>>>>>>
>>>>>>>>> Jiangjie (Becket) Qin
>>>>>>>>>
>>>>>>>>> On 2/25/15, 5:02 PM, "Zakee" <kzak...@netzero.net
>>>>>>>>> <mailto:kzak...@netzero.net>
>>>>>>>>> <mailto:kzak...@netzero.net <mailto:kzak...@netzero.net>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks, Jiangjie.
>>>>>>>>>>
>>>>>>>>>> Yes, I do see under partitions usually shooting every hour.
>>>>>>>>>> Anythings
>>>>>>>>>> that
>>>>>>>>>> I could try to reduce it?
>>>>>>>>>>
>>>>>>>>>> How does "num.replica.fetchers" affect the replica sync? Currently
>>>>>>>>>> have
>>>>>>>>>> configured 7 each of 5 brokers.
>>>>>>>>>>
>>>>>>>>>> -Zakee
>>>>>>>>>>
>>>>>>>>>> On Wed, Feb 25, 2015 at 4:17 PM, Jiangjie Qin
>>>>>>>>>> <j...@linkedin.com.invalid <mailto:j...@linkedin.com.invalid>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> These messages are usually caused by leader migration. I think as
>>>>>>>>>>> long
>>>>>>>>>>> as
>>>>>>>>>>> you don¹t see this lasting for ever and got a bunch of under
>>>>>>>>>>> replicated
>>>>>>>>>>> partitions, it should be fine.
>>>>>>>>>>>
>>>>>>>>>>> Jiangjie (Becket) Qin
>>>>>>>>>>>
>>>>>>>>>>> On 2/25/15, 4:07 PM, "Zakee" <kzak...@netzero.net
>>>>>>>>>>> <mailto:kzak...@netzero.net>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Need to know if I should I be worried about this or ignore them.
>>>>>>>>>>>>
>>>>>>>>>>>> I see tons of these exceptions/warnings in the broker logs, not
>>>>>>>>>>>> sure
>>>>>>>>>>> what
>>>>>>>>>>>> causes them and what could be done to fix them.
>>>>>>>>>>>>
>>>>>>>>>>>> ERROR [ReplicaFetcherThread-3-5], Error for partition
>>>>>>>>>>>> [TestTopic]
>>>>>>>>>>>> to
>>>>>>>>>>>> broker
>>>>>>>>>>>> 5:class kafka.common.NotLeaderForPartitionException
>>>>>>>>>>>> (kafka.server.ReplicaFetcherThread)
>>>>>>>>>>>> [2015-02-25 11:01:41,785] ERROR [ReplicaFetcherThread-3-5],
>>>>>>>>>>>> Error
>>>>>>>>>>>> for
>>>>>>>>>>>> partition [TestTopic] to broker 5:class
>>>>>>>>>>>> kafka.common.NotLeaderForPartitionException
>>>>>>>>>>>> (kafka.server.ReplicaFetcherThread)
>>>>>>>>>>>> [2015-02-25 11:01:41,785] WARN [Replica Manager on Broker 2]:
>>>>>>>>>>>> Fetch
>>>>>>>>>>>> request
>>>>>>>>>>>> with correlation id 950084 from client ReplicaFetcherThread-1-2
>>>>>>>>>>>> on
>>>>>>>>>>>> partition [TestTopic,2] failed due to Leader not local for
>>>>>>>>>>>> partition
>>>>>>>>>>>> [TestTopic,2] on broker 2 (kafka.server.ReplicaManager)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Any ideas?
>>>>>>>>>>>>
>>>>>>>>>>>> -Zakee
>>>>>>>>>>>> ____________________________________________________________
>>>>>>>>>>>> Next Apple Sensation
>>>>>>>>>>>> 1 little-known path to big profits
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> http://thirdpartyoffers.netzero.net/TGL3231/54ee63b9e704b63b94061
>>>>>>>>>>>> <http://thirdpartyoffers.netzero.net/TGL3231/54ee63b9e704b63b94061>
>>>>>>>>>>>> st0
>>>>>>>>>>>> 3v
>>>>>>>>>>>> uc
>>>>>>>>>>>
>>>>>>>>>>> ____________________________________________________________
>>>>>>>>>>> Extended Stay America
>>>>>>>>>>> Get Fantastic Amenities, low rates! Kitchen, Ample Workspace,
>>>>>>>>>>> Free
>>>>>>>>>>> WIFI
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> http://thirdpartyoffers.netzero.net/TGL3255/54ee66f26da6f66f10ad4m
>>>>>>>>>>> <http://thirdpartyoffers.netzero.net/TGL3255/54ee66f26da6f66f10ad4m>
>>>>>>>>>>> p02
>>>>>>>>>>> du
>>>>>>>>>>> c
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ____________________________________________________________
>>>>>>>>> Extended Stay America
>>>>>>>>> Official Site. Free WIFI, Kitchens. Our best rates here,
>>>>>>>>> guaranteed.
>>>>>>>>>
>>>>>>>>> http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13d
>>>>>>>>> <http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13d>
>>>>>>>>> uc
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <http://thirdpartyoffers.netzero.net/TGL3255/54ee80744cfa7747461mp13
>>>>>>>>> duc
>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ____________________________________________________________
>>>>>>> The WORST exercise for aging
>>>>>>> Avoid this "healthy" exercise to look & feel 5-10 years
>>>>>>> YOUNGER
>>>>>>>
>>>>>>> http://thirdpartyoffers.netzero.net/TGL3255/54fa40e98a0e640e81196mp07d
>>>>>>> <http://thirdpartyoffers.netzero.net/TGL3255/54fa40e98a0e640e81196mp07d>
>>>>>>> uc
>>>>>>
>>>>>
>>>>>
>>>>> ____________________________________________________________
>>>>> Seabourn Luxury Cruises
>>>>> Receive special offers from the World's Finest Small-Ship Cruise
>>>>> Line!
>>>>>
>>>>> http://thirdpartyoffers.netzero.net/TGL3255/54fbf3b0f058073b02901mp14duc
>>>>> <http://thirdpartyoffers.netzero.net/TGL3255/54fbf3b0f058073b02901mp14duc>
>>>>
>>>
>>
>>
>> ____________________________________________________________
>> Discover Seabourn
>> A journey as beautiful as the destination, request a brochure today!
>> http://thirdpartyoffers.netzero.net/TGL3255/54fdebfe6a2a36bfb0bb3mp10duc
>> <http://thirdpartyoffers.netzero.net/TGL3255/54fdebfe6a2a36bfb0bb3mp10duc>
>
>
> Thanks
> Zakee
>
>
>
> ____________________________________________________________
> Want to place your ad here?
> Advertise on United Online
> http://thirdpartyoffers.netzero.net/TGL3255/54fdf80bc575a780b0397mp05duc
