Re: rebalance issues from mirrormaker

2013-06-20 Thread Jun Rao
t: Monday, June 10, 2013 11:21 AM > To: 'users@kafka.apache.org' > Subject: rebalance issues from mirrormaker > > Hi, > > I come across several critical issues when benchmarking mirrormakers. > > 1 If a topic has N partitions on source cluster, after mirroring, on

RE: rebalance issues from mirrormaker

2013-06-20 Thread Yu, Libo
: Yu, Libo [ICG-IT] Sent: Monday, June 10, 2013 11:21 AM To: 'users@kafka.apache.org' Subject: rebalance issues from mirrormaker Hi, I come across several critical issues when benchmarking mirrormakers. 1 If a topic has N partitions on source cluster, after mirroring, on the destination

Re: rebalance issues from mirrormaker

2013-06-10 Thread Jun Rao
How many total #topics/partitions are you mirroring? If that number is large, you will need to increase the # of retries and backoff time in the consumer in MirrorMaker. Thanks, Jun On Mon, Jun 10, 2013 at 8:21 AM, Yu, Libo wrote: > Hi, > > I come across several critical issues when benchmark

rebalance issues from mirrormaker

2013-06-10 Thread Yu, Libo
Hi, I come across several critical issues when benchmarking mirrormakers. 1 If a topic has N partitions on source cluster, after mirroring, on the destination cluster it has only one partition. 2 To solve the issue in 1), a topic is created on source and destination at the same time with the s