I did a restart and the issue was gone. It could be that we changed the retention size and did not restart the brokers to pick up the change. Thanks for your help.
Regards, Libo -----Original Message----- From: Yu, Libo [ICG-IT] Sent: Friday, November 22, 2013 10:53 AM To: 'users@kafka.apache.org' Subject: RE: retention size I am using 0.8-beta1. I used only one producer and compression was not turned on. The topic was newly created. The default retention size applied so each of three partitions Is 3Gb. I checked log files for all three partitions and all of them are 1.4G. Regards, Libo -----Original Message----- From: Joel Koshy [mailto:jjkosh...@gmail.com] Sent: Thursday, November 21, 2013 9:50 PM To: users@kafka.apache.org Subject: Re: retention size Are you on 0.8 branch or trunk? Also, what are the sizes for the other partitions? Also how many producers do you have? It could be because the producer sticks to a partition for the metadata refresh period - so if your test run isn't long enough some partitions may be more loaded than the others. On Thu, Nov 21, 2013 at 06:28:39PM +0000, Yu, Libo wrote: > Hi team, > > We have 3 brokers in a cluster. The replication factor is 2. I set the > default retention size to 3G bytes. I published 12G data to a topic, > which is enough to fully load all partitions. I assume on each broker > the partition size should be 3G. However, it is only 1.4G for one partation. > Is this a known issue? Thanks. > > Regards, > > Libo >
