MirrorMaker2 metrics not being exported to JMX

Hey all, I'd like to understand why I can't find the metrics for mm2 [1]. By using jconsole, I can find the mbeans for kafka.connect, kafka.consumer and kafka.producer but I can't find anything on kafka.connect.mirror. Attached is the image of the jconsole listing the mbeans. What am I doing wron

Request for Comments/Resolution - KAFKA-10465

Hello, We do appreciate that release 2.7 is keeping us occupied, but this bug (or not) is holding us back from making some design changes/improvements. It'd be awesome if anyone could take a look and perhaps either, rule it out with explanation or acknowledge the changes. Regards,

Mirror Maker 2.0 NOT generating checkpoints for consumers running in assign mode

Hello All, I was using a mirror maker 2.0. I was testing the consumer checkpointing functionality. I found that the RemoteClusterUtils.translateOffsets do not give checkpoints for the consumer which run in assign mode. I am using mirror maker 2.0 of Kafka Version 2.5.0 and Scala version 2.12 My

Re: Amazon MSK Feeback

We evaluated confluent cloud and ended up going with https://aiven.io. It's a completely managed solution, running in a private vpc peered with the rest of our aws infrastructure. At the time we evaluated it was substantially less expensive than confluent cloud (~5x). Aiven implemented their own

Re: Amazon MSK Feeback

We use Confluent Cloud. Make sure you consider the opportunity cost of you labor into your MSK cost. Other than creating topics and managing ACLs you won’t have to lift a finger to manage your Confluent Cloud Kafka cluster. That means you get to spend all your time building apps to run your busines

Transactional Producers/Consumers with LSO and Last Committed Record

Hello, I understand that a consumer with Txn Isolation level set as "read_committed" will read till Last Stable Offset (LSO) and will not be given any messages for aborted Txn. LSO also includes non-transactional messages. We have seen a behaviour where LSO didn't have any data and we had to seek

Re: Amazon MSK Feeback

Hi Himanshu, We looked at MSK when we moved into AWS. Some of the other aspects to consider is that inter-replica network traffic isn't charged for separately under MSK (whereas traffic between EC2 nodes is charged, and at a higher rate when it's inter-AZ) so if you're running a high replica situa

Re: Amazon MSK Feeback

The reasons which I could think are 1. I need not take care of the state of the cluster for scenarios when the broker goes down or the zookeeper goes down. 2. They are by default deploying the cluster across multiple Availability zones. 3. They have their own mirroring policy acros

Re: Amazon MSK Feeback

Himanshu Shukla wrote: We are planning to go for amazon MSK instead of having our own self launched cluster on bare EC2 machines. It would be very helpful, if someone who has used it before or know about it, can share the feedback/observations regarding the same. My main concerns are, Is it

Re: Amazon MSK Feeback

Hi Himanshu, Have you looked at Confluent Cloud? This gives you fully managed Kafka on AWS, with additional benefits. You can read more in this article: https://www.confluent.io/blog/fully-managed-apache-kafka-service/ Disclaimer: I am totally biased, because I work for Confluent :-) thanks, Rob

Amazon MSK Feeback

Hi, We are planning to go for amazon MSK instead of having our own self launched cluster on bare EC2 machines. It would be very helpful, if someone who has used it before or know about it, can share the feedback/observations regarding the same. My main concerns are, Is it worth using Amazon MSK?