Re: __consumer_offsets rebalance

2016-07-21 Thread Tom Crayford
Hi there, It's enabled with the config log.cleaner.enable Thanks On Wed, Jul 20, 2016 at 5:29 PM, Anderson Goulart < anderson.goul...@boxever.com> wrote: > Hi, > > How can I see if log compaction is enabled? And how can I enable it? I > didn't find it on kafka docs. > > > Thanks, Anderson > > >

Re: __consumer_offsets rebalance

2016-07-20 Thread Anderson Goulart
Hi, How can I see if log compaction is enabled? And how can I enable it? I didn't find it on kafka docs. Thanks, Anderson On 14/07/2016 13:37, Todd Palino wrote: It's safe to move the partitions of the offsets topic around. You'll move the consumer coordinators as you do, however, so the o

Re: __consumer_offsets rebalance

2016-07-14 Thread Anderson Goulart
Hi, Thanks for your help. It is part of our plan to move forward to 0.10, but we need to do it one step at a time. And rebalancing topics, including __consumer_offsets, are on the top as we had incidents associated with it. -- Anderson On 14/07/2016 13:59, Tom Crayford wrote: Also note t

Re: __consumer_offsets rebalance

2016-07-14 Thread Tom Crayford
Also note that there were a number of bugs fixed in the log cleaner thread between 0.8 and the latest release. I wouldn't be comfortable relying on kafka committed offsets on a version under 0.10 for a new production system, and would carefully consider an upgrade all the way to the latest release.

Re: __consumer_offsets rebalance

2016-07-14 Thread Todd Palino
It's safe to move the partitions of the offsets topic around. You'll move the consumer coordinators as you do, however, so the one thing you want to make sure of, especially running an older version, is that log compaction is working on your brokers and those partitions have been compacted. The coo

__consumer_offsets rebalance

2016-07-14 Thread Anderson Goulart
Hi, I am running kafka 0.8.2.1 under aws instances with multiple availability zones. As we want a rack aware partition replication, we have our own partition layout distribution, to make sure all partitions are well balanced between nodes, leaders and availability zones. The problem arises w