Re: Kafka Connect Connector Tasks Uneven Division

2020-05-21 Thread Robin Moffatt
I don't think you're right to assert that this is "expected behaviour": > the tasks are divided in below pattern when they are first time registered Kafka Connect task allocation is non-determanistic. I'm still not clear if you're solving for a theoretical problem or an actual one. If this is a

Problem getting the correct values for records-lag from JMX when using Kafka Connect

2020-05-21 Thread Leon Bam
When monitoring my Kafka Connect, I am getting wrong values for records-lag, records-lag-avg, records-lag-max. I compare the values from *kafka-consumer-groups* with what I see in JConsole. While the LAG is higher then 0, the records-lag, records-lag-avg, records-lag-max shows 0, Nan, NaN respectiv

Re: Avro schema registry error

2020-05-21 Thread AJ Chen
Thanks, Liam. Will run registry along side kafka . -aj On Wed, May 20, 2020 at 8:58 PM Liam Clarke-Hutchinson < liam.cla...@adscale.co.nz> wrote: > Hi AJ, > > No, there is no public schema registry, you will need to deploy and > maintain your own. > > Cheers, > > Liam Clarke-Hutchinson > > On T

Kafka retention policy per topic not working as expected

2020-05-21 Thread Don Ky
Hello Kafka users! I'm posting because we are having an issue with a topic retention that we don't understand and need input on what could potentially be wrong on our end if it's indeed is a configuration issue.¬ First we are running a containerized version of Kafka with the following environment v