[VOTE] 3.1.0 RC1

2022-01-12 Thread David Jacot
Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 3.1.0. * Apache Kafka supports Java 17 * The FetchRequest supports Topic IDs (KIP-516) * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768) * Add broker count metrics (KIP-748) * Dif

Memory leak with kafka clients 3.0.0

2022-01-12 Thread Willian Dallastella
Hi, we are using Spring boot with *spring-kafka* and when updating to Spring Boot 2.6+ we have memory leak in the kafka producer as it comes with *kafka-cients 3.0.0*. After testing, using Spring Boot 2.6 and just *downgrading kafka-clients to 2.7.1*, the *memory leak stopped*. Here are more det

Re: Questions around static membership partition assignment

2022-01-12 Thread jun aoki
Thank you Luke for responding! The unbalance situation we've observed without static membership goes like this; say, we expect 2 pods and k8s statefulset actually starts one by one (PodManagementPolicy=OrderedReady), after the first pod is ready, all partitions are assigned to it, and the very mo

Re: monitoring length of queues

2022-01-12 Thread Florin Andrei
The way I do it now is: For each combination of topic name and partition number, I get LogEndOffset and LogStartOffset from the JMX metrics, and do the difference between them. I believe this will reflect the queue lengths. Does that sound reasonable? Second issue - I've found something call

Re: Enable TLS Authentication + Enable External Access using LoadBalancer

2022-01-12 Thread Wei Yang
Hi Luke, Thanks a lot for the clarifications. Very helpful to me for getting started. As we can import the root CA of all certificates to trust them all, I’d like to understand * Why Kafka needs one LoadBalancer per broker? Thank you very much! Regards, Wei From: Luke Chen Date: Tuesday

Re: Memory leak with kafka clients 3.0.0

2022-01-12 Thread Kirk True
Hi Willian, I don't see any recent Jiras that seem to cover this. I noticed two requests in the linked spring-kafka GitHub thread: 1. File a Jira on the Kafka project with relevant configuration, etc. 2. Provide a stripped-down, reproducible test case using Kafka (sans Spring) Having those tw