Re: Connection reset by peer

2013-03-24 Thread Yonghui Zhao
Hi Jun, I used kafka-server-start.sh to start kafka, there is only one jvm setting "-Xmx512M“ Do you have some recommend GC setting? Usually our sever has 32GB or 64GB RAM. 2013/3/22 Jun Rao > A typical reason for many rebalancing is the consumer side GC. If so, you > will see logs in the co

Re: commitOffsets java.lang.NullPointerException

2013-03-24 Thread Yonghui Zhao
Done, https://issues.apache.org/jira/browse/KAFKA-824 2013/3/25 Neha Narkhede > On the Kafka JIRA - https://issues.apache.org/jira/browse/KAFKA > > Thanks, > Neha > > On Sunday, March 24, 2013, Yonghui Zhao wrote: > > > Sure. no problem. Where should I file the bug? > > > > 2013/3/25 Neha Nark

Re: commitOffsets java.lang.NullPointerException

2013-03-24 Thread Neha Narkhede
On the Kafka JIRA - https://issues.apache.org/jira/browse/KAFKA Thanks, Neha On Sunday, March 24, 2013, Yonghui Zhao wrote: > Sure. no problem. Where should I file the bug? > > 2013/3/25 Neha Narkhede > > > Yes, I have. Unfortunately, I never quite around to fixing it. My guess > is > > that i

Re: commitOffsets java.lang.NullPointerException

2013-03-24 Thread Yonghui Zhao
Sure. no problem. Where should I file the bug? 2013/3/25 Neha Narkhede > Yes, I have. Unfortunately, I never quite around to fixing it. My guess is > that it is caused due to a race condition between the rebalance thread and > the offset commit thread when a rebalance is triggered or the client

Re: Kafka throw InvalidMessageException and lost data

2013-03-24 Thread 王国栋
Hi Neha, I am sorry, we couldn't find the detail reason of this bug. But we do believe that the bug is in java.nio package. Something bad may happen during sync-producer sends data in byteBuffer or broker receives data from the channel. The serialize/deserialize process is right, because we enabl

Re: commitOffsets java.lang.NullPointerException

2013-03-24 Thread Neha Narkhede
Yes, I have. Unfortunately, I never quite around to fixing it. My guess is that it is caused due to a race condition between the rebalance thread and the offset commit thread when a rebalance is triggered or the client is being shutdown. Do you mind filing a bug ? Thanks, Neha On Sunday, March 24

commitOffsets java.lang.NullPointerException

2013-03-24 Thread Yonghui Zhao
Have you ever seen this exception? 2013/03/25 12:08:32.020 WARN [ZookeeperConsumerConnector] [] 0_lu-ml-test10.bj-1364184411339-7c88f710 exception during commitOffsets java.lang.NullPointerException at org.I0Itec.zkclient.ZkConnection.writeData(ZkConnection.java:111) at org.I0Itec

Re: Kafka throw InvalidMessageException and lost data

2013-03-24 Thread Neha Narkhede
That's great. Would you know the details of the bug though ? It will be helpful for the community to understand. Thanks, Neha On Sunday, March 24, 2013, 王国栋 wrote: > Hi Neha & Jun, > > I think we have found the reason of this bug. > It is related to jdk versions. In the beginning, we use jdk1.6.

Re: Kafka throw InvalidMessageException and lost data

2013-03-24 Thread 王国栋
Hi Neha & Jun, I think we have found the reason of this bug. It is related to jdk versions. In the beginning, we use jdk1.6.20 to run the test. Yesterday, we used jdk1.6.38. Everything is OK now. Thanks a lot for all your kind help and answer. On Thu, Mar 21, 2013 at 1:05 PM, Neha Narkhede wr

Re: 0.8 best practices for migrating / electing leaders in failure situations?

2013-03-24 Thread Jun Rao
If you bring up 3 new brokers with different broker ids, you won't be able to use them on existing topics until after you have run the partition reassignment tool. Thanks, Jun On Fri, Mar 22, 2013 at 9:23 PM, Scott Clasen wrote: > Thanks! > > Would there be any difference if I instead delete