Mirror maker higher offset in the mirror.

Am trying to mirror from a production Kafka cluster to a DR cluster. However the offsets between topics (retrieved with GetOffsetShell ) on these two clusters do not always match. While a lesser offset is t

Re: Process to enable log compaction on a cluster

compacted ? On Mon, Jul 4, 2016 at 11:38 AM, Sathyakumar Seshachalam < sathyakumar_seshacha...@trimble.com> wrote: > Ok, Am in situation where all kafka nodes are going to run out of space. > This is because I had been running uncompacted __consumer_offset topic and > everything

Process to enable log compaction on a cluster

Ok, Am in situation where all kafka nodes are going to run out of space. This is because I had been running uncompacted __consumer_offset topic and everything retained topics . Now at a place, where I can afford to compact __consumer_offset topic and also delete certain topics. I would like to kno

Re: Coordinator lost for consumer groups

. Error while executing consumer group command This is not the correct coordinator for this group. org.apache.kafka.common.errors.NotCoordinatorForGroupException: This is not the correct coordinator for this group. On Fri, Jul 1, 2016 at 2:15 PM, Sathyakumar Seshachalam < sathyakumar_sesha

Re: Coordinator lost for consumer groups

And I am willing to suspend log compaction and restart the brokers, but am worried if that will leave the system in a recoverable state or If I just have to wait it out. On Fri, Jul 1, 2016 at 2:06 PM, Sathyakumar Seshachalam < sathyakumar_seshacha...@trimble.com> wrote: > Hi, > >

Coordinator lost for consumer groups

Hi, I have 3 Kafka nodes (running 0.9.0) that all had active consumers and producers. Now all these had uncompacted __consumer_offsets group that grew to 1.8 TB. So I restarted these nodes with a log.cleaner.enabled to save some space. Since then consumers have stalled. When I do a ./kafka-consu

Re: Log retention just for offset topic

Thanks Tom. I think thats good enough for my needs On Thu, Jun 30, 2016 at 4:20 PM, Tom Crayford wrote: > The default cleanup policy is delete, which is the regular time based > retention. > > On Thursday, 30 June 2016, Sathyakumar Seshachalam < > sathyakumar_seshacha...@

Re: Log retention just for offset topic

Or may be am wrong, and Log cleaner only picks up topics with a cleanup.policy. >From the documentation it is not very obvious what the behaviour is. On Thu, Jun 30, 2016 at 10:33 AM, Sathyakumar Seshachalam < sathyakumar_seshacha...@trimble.com> wrote: > Hi, > > Thanks for t

Re: Log retention just for offset topic

; offsets.retention.check.interval.ms > > > Thanks > Manikumar > > On Thu, Jun 30, 2016 at 9:49 AM, Sathyakumar Seshachalam < > sathyakumar_seshacha...@trimble.com> wrote: > > > Am little confused about how log cleaner works. My use case is th

Log retention just for offset topic

Am little confused about how log cleaner works. My use case is that I want to compact just selected topics (or in my case just the internal topic __consumers_offsets and want to leave other topics as is). Whats the right settings/configuration for this to happen. As I understand log cleaner enabl

OffSet checker.

The command line tool that comes with Kafka one seems to check offsets in zookeeper. This does not seem to be related to this issue - As I am sure the offsets are stored in Kafka for burrow seem to retrieve it properly. Is there any easier way to

Re: Kafka console producer stuck

fka-console-producer --broker-list >my-broker-host:9092 --topic my-topic* > >vmstat <http://linux.die.net/man/8/vmstat> and other similar linux >utilities (iostat <http://linux.die.net/man/1/iostat>) are a great way to >get quick and real data for experimentation. >

Re: Kafka console producer stuck

nk and will be included in the next > release. See KAFKA-1711 <https://issues.apache.org/jira/browse/KAFKA-1711> for > more details. > > You can safely ignore the warning and it should not impact the usage of the > console producer. > > Thank you, > Grant &

Re: Kafka console producer stuck

e/src/main/scala/kafka/utils >/VerifiableProperties.scala#L224) >that you are providing some property which kafka does not recognise >Please share the full command that you are trying to use > >Regards, >Prabhjot > >On Wed, Oct 28, 2015 at 2:25 PM, Sathyakumar Seshachalam < >

Kafka console producer stuck

Am trying to get started on Kafka and following instructions on http://kafka.apache.org/documentation.html#quickstart. Am setting up on a single broker. I got as far as creating topics and listing them, but when I try kafka-console-producer.sh to add messages, I ended up in below error. [2015-