Re: What is the node-id tag in kafka producer MetricName ?

Nodes with a negative id refer to the bootstrap servers you configured the client with. There are also metrics that report for nodes with an extremely large node id. These are usually Integer.MAX_VALUE - (coordinator node id) On Tue, Dec 3, 2019 at 1:37 PM Rajkumar Natarajan wrote: > I've have

Re: [DISCUSS] KIP-405: Kafka Tiered Storage

Hi Jun, Thanks for the reply. On Tue, Nov 26, 2019 at 3:46 PM, Jun Rao wrote: > Hi, Satish and Ying, > > Thanks for the reply. > > 40/41. There are two different ways that we can approach this. One is what > you said. We can have an opinionated way of storing and populating the tier > me

Build failed in Jenkins: kafka-trunk-jdk11 #999

See Changes: [john] KAFKA-9231: Streams Threads may die from recoverable errors with EOS [jason] MINOR: Add test to ensure log metrics are removed after deletion (#7750) ---

Re: Broker Interceptors

another note - the current status of the kip 388 reflects the latest stage of the discussion process, which has devolved into a horrible case study in over-engineering (reflected in the latest edits to the kip wiki page) before we gave up. the actual commit represents something much closer to the o

Re: Broker Interceptors

yeah, we tried for this a while back (kip 388 - https://cwiki.apache.org/confluence/display/KAFKA/KIP-388%3A+Add+observer+interface+to+record+request+and+response) its implemented in our kafka repo (linked to above) On Tue, Dec 3, 2019 at 8:59 PM Ignacio Solis wrote: > > At LinkedIn we run a sty

Re: Broker Interceptors

At LinkedIn we run a style of "read-only" interceptor we call an observer. We use this for usage monitoring. https://github.com/linkedin/kafka/commit/a378c8980af16e3c6d3f6550868ac0fd5a58682e There is always a tension between exposing internals, creating stable interfaces and performance. It's und

Jenkins build is back to normal : kafka-2.4-jdk8 #92

See

[jira] [Resolved] (KAFKA-9258) Connect ConnectorStatusMetricsGroup sometimes NPE

[ https://issues.apache.org/jira/browse/KAFKA-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-9258. -- Reviewer: Randall Hauch Resolution: Fixed Merged into `trunk` and backported to the `2.4`

Re: [VOTE] 2.4.0 RC2

Thanks Eric for sharing this test script. It simplifies the process for me. https://github.com/elalonde/kafka/blob/master/bin/verify-kafka-rc.sh I am in the process of setting up my local environment to start running the tests on 2.4.0 RC2 as well and I noticed the following artifacts are being v

Re: [VOTE] 2.4.0 RC2

Hi Manikumar, I've just resolved https://issues.apache.org/jira/browse/KAFKA-9231 . Sorry for the hold-up. Thanks, -John On Tue, Dec 3, 2019, at 10:27, Manikumar wrote: > Hi, > > Thank you all for reporting the issues. > > We'll consider the below JIRAs as blockers for 2.4.0 release. > https:

[jira] [Resolved] (KAFKA-9231) Streams Threads may die from recoverable errors with EOS enabled

[ https://issues.apache.org/jira/browse/KAFKA-9231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-9231. - Resolution: Fixed Fixed in 2.4 by https://github.com/apache/kafka/commit/d7fe494b2a983256092bcc5

Kafka Meetup hosted by Confluent at Mountain View, Thursday 5:30pm, December 5th, 2019

Hello folks, This is a kind reminder of the Bay Area Kafka® meetup this Thursday (Dec. 5th) 5:30pm, at Confluent's new Mountain View HQ office. *RSVP and Register* (if you intend to attend in person): https://www.meetup.com/KafkaBayArea/events/266327152/ *Date* 5:30pm, Thursday, December 5th, 2

Build failed in Jenkins: kafka-trunk-jdk11 #998

See Changes: [github] KAFKA-9203: Revert "MINOR: Remove workarounds for lz4-java bug affecting -- [...truncated 2.75 MB...] org.apache.kafka.streams.internals.KeyValueSt

[jira] [Created] (KAFKA-9265) kafka.log.Log instances are leaking on log delete

Vikas Singh created KAFKA-9265: -- Summary: kafka.log.Log instances are leaking on log delete Key: KAFKA-9265 URL: https://issues.apache.org/jira/browse/KAFKA-9265 Project: Kafka Issue Type: Bug

What is the node-id tag in kafka producer MetricName ?

I've have kafka producer running same code from 3 different linux servers sending messages to same kafka cluster topic. Below is the sample code - KafkaProducer producer = new KafkaProducer(props); Map metrics = producer.metrics(); System.out.println(metrics.keySet()); The sample of

Build failed in Jenkins: kafka-trunk-jdk8 #4081

See Changes: [github] KAFKA-9203: Revert "MINOR: Remove workarounds for lz4-java bug affecting -- [...truncated 2.75 MB...] org.apache.kafka.streams.test.OutputVerifier

Build failed in Jenkins: kafka-trunk-jdk11 #997

See Changes: [rajinisivaram] MINOR: Proactively update producer topic access time. (#7672) -- [...truncated 5.57 MB...] org.apache.kafka.streams.TopologyTestDriverTest >

Re: [VOTE] 2.4.0 RC2

Hi, Thank you all for reporting the issues. We'll consider the below JIRAs as blockers for 2.4.0 release. https://issues.apache.org/jira/browse/KAFKA-9231 https://issues.apache.org/jira/browse/KAFKA-9258 https://issues.apache.org/jira/browse/KAFKA-9156 I will create new release candidate after m

[jira] [Created] (KAFKA-9263) Reocurrence: Transient failure in kafka.api.AdminClientIntegrationTest.testLogStartOffsetCheckpointkafka.api.AdminClientIntegrationTest.testAlterReplicaLogDirs

John Roesler created KAFKA-9263: --- Summary: Reocurrence: Transient failure in kafka.api.AdminClientIntegrationTest.testLogStartOffsetCheckpointkafka.api.AdminClientIntegrationTest.testAlterReplicaLogDirs Key: KAFKA-9263

[jira] [Created] (KAFKA-9264) Reocurrence: Flaky Test DelegationTokenEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

John Roesler created KAFKA-9264: --- Summary: Reocurrence: Flaky Test DelegationTokenEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl Key: KAFKA-9264 URL: https://issues.apache.org/jira/b

[jira] [Reopened] (KAFKA-8937) Flaky Test SaslPlainSslEndToEndAuthorizationTest#testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

[ https://issues.apache.org/jira/browse/KAFKA-8937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reopened KAFKA-8937: - This test has just failed again for me. Apparently, it is not fixed. Unfortunately, Jenkins didn't c

Re: Broker Interceptors

The main challenge is doing this without exposing a bunch of internal classes. I haven't seen a proposal that handles that aspect well so far. Ismael On Tue, Dec 3, 2019 at 7:21 AM Sönke Liebau wrote: > Hi Thomas, > > I think that idea is worth looking at. As you say, if no interceptor is > con

Build failed in Jenkins: kafka-trunk-jdk8 #4080

See Changes: [rajinisivaram] MINOR: Proactively update producer topic access time. (#7672) -- [...truncated 2.75 MB...] org.apache.kafka.streams.test.TestRecordTest > te

Re: Broker Interceptors

Hi Thomas, I think that idea is worth looking at. As you say, if no interceptor is configured then the performance overhead should be negligible. Basically it is then up to the user to decide if he wants tomtake the performance hit. We should make sure to think about monitoring capabilities like t

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

Hi Viktor, The KIP states: "KIP-73 added quotas for replication but it doesn't separate normal replication traffic from reassignment. So a user is able to specify the partition and the throttle rate but it will be appli

Re: [DISCUSS] KIP-542: Partition Reassignment Throttling

Hi Stan, I was about to start a vote on this one but I think I have one more idea to your last point about the total cap. What if we said that the (leader|follower).replication.throttled.rate is the overall limit which we allow for leadership replication (so the total cap) and (leader|follower).re

Re: [VOTE] KIP-373: Allow users to create delegation tokens for other users

Hi Folks, I'd like to bump this once more in the hope of a binding vote or any additional feedback. Thanks, Viktor On Fri, Oct 25, 2019 at 2:24 PM Viktor Somogyi-Vass wrote: > Hi All, > > Would like to bump this in the hope of one binding vote (or any additional > feedback). > > Thanks, > Vikt

[jira] [Created] (KAFKA-9262) kafka broker fails to response fetching request from consumer when quota limit is violated

Junqi Zhang created KAFKA-9262: -- Summary: kafka broker fails to response fetching request from consumer when quota limit is violated Key: KAFKA-9262 URL: https://issues.apache.org/jira/browse/KAFKA-9262

Re: SSLAuthentication Exception JMX

Hi Tyler, We have metrics in Selector for successful and failed authentication. On the broker side, we have mbeans ` kafka.server:type=socket-server-metrics,listener=,networkProcessor=` These have attributes failed-authentication-rate, failed-authentication-total etc. There are similar metrics on

RE: Broker Interceptors

Hi M. Manna, Thank you for your feedback, any and all thoughts on this are appreciated from the community. I think it is important to distinguish that there are two parts to this. One would be a server side interceptor framework and the other would be the interceptor implementations themselves