Re: Kafka wire protocol - Request and response versions.

2024-10-28 Thread Ivan Yurchenko
on, Oct 28, 2024 at 10:25 AM Ivan Yurchenko wrote: > > > Hi, > > > > The broker has to respond with the same API version (the response header > > doesn't even have the version field). > > > > Best, > > Ivan > > > > > > On Sun, Oc

Re: Kafka wire protocol - Request and response versions.

2024-10-27 Thread Ivan Yurchenko
Hi, The broker has to respond with the same API version (the response header doesn't even have the version field). Best, Ivan On Sun, Oct 27, 2024, at 20:22, Chain Head wrote: > Hello, > Suppose a client makes a API Versions request with version 3. Is it legal > for the broker to respond with

Re: CVE-2021-38153: Timing Attack Vulnerability for Apache Kafka Connect and Clients

2021-09-21 Thread Ivan Yurchenko
Hi Randall, Could you please share the JIRA ticket or the fixing commit? It might help to evaluate the impact better. Thank you! Ivan On Tue, 21 Sept 2021 at 19:37, Randall Hauch wrote: > Severity: moderate > > Description: > > Some components in Apache Kafka use `Arrays.equals` to validate a

Re: [idea] Kafka topic metadata

2021-06-15 Thread Ivan Yurchenko
uggestion earlier. I think this will be a great idea > > for a KIP. > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > > You were referring to metadata for the actual topic and not its contents. > > > > Sorry abo

Re: [idea] Kafka topic metadata

2021-06-14 Thread Ivan Yurchenko
Hi, Having metadata for topics seems pretty useful. Currently, one has to use external storage for this (e.g. a database) and the question of keeping topic and metadata in sync exists: A topic is deleted, how to delete its metadata? How to deal with delete-then-recreate scenarios (well, we have to

Re: Question on Kafka Connect

2020-05-17 Thread Ivan Yurchenko
Hi Richard, Connectors [1] and tasks [2] are instantiated inside worker processes. [1] https://github.com/apache/kafka/blob/7746301c2643e0037bff3bcb84c41d512937e4c9/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java#L252 [2] https://github.com/apache/kafka/blob/7746301c264

Re: MirrorMaker2 not mirroring for 5 minutes when adding a topic

2020-03-17 Thread Ivan Yurchenko
Ah, I'm sorry, this is not the setting you need (it's actually the period before "Found ... topic-partitions"). Please, ignore my message :) Ivan On Tue, 17 Mar 2020 at 18:04, Ivan Yurchenko wrote: > Hi, > > You need to set refresh.topics.interval.seconds, whi

Re: MirrorMaker2 not mirroring for 5 minutes when adding a topic

2020-03-17 Thread Ivan Yurchenko
Hi, You need to set refresh.topics.interval.seconds, which is 10 minutes by default. Ivan On Tue, 17 Mar 2020 at 17:45, Péter Sinóros-Szabó wrote: > Hey, > > Running a MM2 cluster to mirror from A->B clusters I noticed that when I > add a new topic to A cluster, MM2 will notice it: > [2020-03

Re: [External] Re: Dead broker in ISR

2020-03-06 Thread Ivan Yurchenko
this topic partition. > > > > > > Thanks > > > Avinash Bhat > > > > > > P.S: I am fairly new to Kafka; it will be great to get a confirmation > on > > > the above from other folks. > > > > > > -Original Message- >

Re: Message prioritization in Kafka

2020-02-15 Thread Ivan Yurchenko
Hi, Kafka does not support message priorities. RabbitMQ may help you. Please evaluate if its clustering capabilities suit you. Best, Ivan On Sat, 15 Feb 2020 at 18:23, Vairavanathan Emalayan < vairavanathan.emala...@move.com> wrote: > Hi There, > > I am looking for a distributed priority queu

Dead broker in ISR

2020-02-13 Thread Ivan Yurchenko
Hi Kafka community, We're running Kafka 2.4 and facing a pretty strange situation. Let's say there were three brokers in the cluster 0, 1, and 2. Then: 1. Broker 3 was added. 2. Partitions were reassigned from broker 0 to broker 3. 3. Broker 0 was shut down (not gracefully) and removed from

Re: Kafka audit logs in Apache Kafka?

2020-02-03 Thread Ivan Yurchenko
Hi, The KIP for this is being discussed now: https://cwiki.apache.org/confluence/display/KAFKA/KIP-567%3A+Kafka+Cluster+Audit Ivan On Fri, 31 Jan 2020 at 17:52, Otis Gospodnetić wrote: > Hi, > > Does Apache Kafka have audit logs equivalent of > https://docs.confluent.io/current/security/audit

Log some types of API requests on broker

2019-10-14 Thread Ivan Yurchenko
Hi, I'm looking for a good way to log some broker API requests which might be considered important. Topic deletion request is quite good an example of such important requests. This piece [1] is the only thing I found that remotely is what I need. However, it logs all the requests and does it on t

Re: (Re-)joining group for a longer time

2019-08-05 Thread Ivan Yurchenko
Hi, Kamesh, does one of worker's logs look like in https://issues.apache.org/jira/browse/KAFKA-7941?focusedCommentId=16899851&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16899851 ? I.e. INFO [Worker clientId=connect-1, groupId=connect] Was selected to perform a

Re: KafkaConnect not consuming from SSL/SASL cluster

2019-06-04 Thread Ivan Yurchenko
Hi Javier, As far as I understand, the reason for this separation is that Connect has separate producers and consumers for its internal mechanics (like Kafka-backed configuration/status/offset storage) and for tasks. It will use some non-prefixed configurations for both tasks' consumers and produc

Thread-safety of KafkaPrincipalBuilder

2019-05-21 Thread Ivan Yurchenko
Hi, I'm implementing a custom `org.apache.kafka.common.security.auth.KafkaPrincipalBuilder` that have state. Should I ensure thread safety of the class (more precisely, of `build` method)? Its thread safety requirements aren't documented, but from the code I can say it's always called in a single