leader update partitions fail with KeeperErrorCode = BadVersion,kafka version=0.8.1.1

kafka version =0.8.1.1 i get the error log as follow: INFO Partition [Topic_Beacon_1,10] on broker 4: Shrinking ISR for partition [Topic_Beacon_1,10] from 4,7 to 4 (kafka.cluster.Partition) ERROR Conditional update of path /brokers/topics/Topic_Beacon_1/partitions/10/state with data {"control

Re: aborting a repartition assignment

Thx for the reply. Can't I just reassign the partition to the new broker in zookeeper manually? What zookeeper path should I change? TIA On Thursday, May 28, 2015, Lance Laursen wrote: > Hey, > > Try clearing out /admin/reassign_partitions on your zookeeper. > > Additionally, your best bet mig

Re: Waiting for replication factor to take effect...?

Thanks Joel, I am able to restart the brokers because this is a dev environment. That fixed it. I was trying to read the state-change.log and having difficultly interpreting the log. I haven't looked at these before. Is the only real fix to stop/start the brokers if this happens again? Maybe I w

IncompatibleClassChangeError

Hi, we are getting the following error on one of our producers, does the stack trace ring any bells for anyone? 2015-05-28 20:27:44 GMT - Failed to send messages java.lang.IncompatibleClassChangeError at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)

Re: KafkaException: Size of FileMessageSet has been truncated during write

Thank you! -- Andrey Yegorov On Wed, May 27, 2015 at 4:42 PM, Jiangjie Qin wrote: > This should be just a message fetch failure. The socket was disconnected > when broker was writing to it. There should not be data loss. > > Jiangjie (Becket) Qin > > On 5/27/15, 11:00 AM, "Andrey Yegoro

Re: Kafka Not Commiting Messages

I think you can also set this dynamically via an mbean (kafka.Log4jController) On Thu, May 28, 2015 at 08:26:00PM +, Jiangjie Qin wrote: > Actually the name should be log4j.logger.kafka.network.RequestChannel$ > It should be there in 0.8.2.1. Can you check it again? > > From: Charlie Mason ma

Re: aborting a repartition assignment

Hey, Try clearing out /admin/reassign_partitions on your zookeeper. Additionally, your best bet might be to bring up a new broker with the same broker ID as your failed broker. It'll join the cluster and carry on, though I'm not sure what effect having a now-empty partition is going to have. On

Re: Kafka Not Commiting Messages

Actually the name should be log4j.logger.kafka.network.RequestChannel$ It should be there in 0.8.2.1. Can you check it again? From: Charlie Mason mailto:charlie@gmail.com>> Reply-To: "charlie@gmail.com" mailto:charlie@gmail.com>> Date: Thursday, May 28, 2

Re: Kafka Not Commiting Messages

Hi Jiangjie, Thanks for you message. Unfortunately there doesn't appear to have that setting in log4j.properties in Kafka 0.8.2.1. I tried adding that property to the log file however it doesn't seem to have any effect. Is there a different logger I need to configure for 0.8.2.1? Thanks, Char

aborting a repartition assignment

Hi, I'm using kafka 0.8.2 & one of my kafka servers died (no way to recover the data on the disks). There is a topic, with replication of 1, with one of the partitions on the dead server. I thought a reassignment would move the metadata for that partition to a new server without needing the data,

Re: Waiting for replication factor to take effect...?

There are also various mbeans you can poke to check the replica fetcher lag but from your description it appears that the fetcher quit for some reason (since you mentioned the topic is 'pretty much' empty) BTW, the below is only for partition 0 > Do you have state change logs available? You shoul

Re: Waiting for replication factor to take effect...?

Do you have state change logs available? You should have a become follower state transition on 9255216 and a leader state transition at around the same time on 925537. The server log should also show 9255216 starting a replica fetcher thread to the leader. Do you have request logging available? Yo

RE: Java - High Level Consumer

You should receive only new messages if the auto.offset.reset is largest. How do you determine that the messages you are receiving are older? Are you checking the lag on your consumer by ConsumerOffsetChecker? Thanks, Aditya From: Panda, Samaresh [samare

Re: Waiting for replication factor to take effect...?

Nothing on this? Someone must surely have a need to add brokers and increase the replication factor. I waited a day and I still do not see in-sync replicas. :( On Wed, May 27, 2015 at 5:51 PM, Dillian Murphey wrote: > Hi Ho, > > I'm trying to increase my replication factor from 1 to 2. > >

Re: Java - High Level Consumer

Auto.offset.reset only comes into place when 1. the consumer fetches message from offset out of range - available offsets on broker side 2. The consumer group has no offset committed. So in your case, I guess high level consumer has already committed its offset before and that offset is not out of

Java - High Level Consumer

I'm following this page https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example and am able to consume messages using four threads. The threads keep receiving messages, which is good. However, I just want to receive only NEW messages and not the old ones. Since the default for