Re: Apache Kafka website "videos" page clarification

I don't think the process here is perfect, but I do think whatever process we have should be self-policing i.e. committers shouldn't need to watch and vet every video that someone chooses to submit to work out if it's good quality, vendor-neutral, and relevant. The Kafka Summit approach is at leas

Re: Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map

Hello Kafka community, I struggled with this issue for a long time and finally figured out the reason. *Issue*: MM2 not emitting checkpoints for my consumer, which is a Apache Flink job with Kafka source connector. Due to this, I’m not able to do the failover. Bi-directional cross-DC replication

How to set consumer.overrides.auto.offset.reset as latest in MM2 ?

Hello. I use kafka 2.4.0 in strimzi 0.17.0 . I have set consumer.overrides.auto.offset.reset in Mirror Maker 2 but Source Connector still reads from the earliest offset . I understand that this seems to be an issue and will be fixed in 2.7.0. https://issues.apache.org/jira/browse/KAFKA-10160

Kafka shuts down abnormally and restarts, always taking more than a few hours to load

- - - - - Kafka shuts down abnormally and restarts, always taking more than a few hours to load. - We used Kafka-2.0.0,we tried to turn it up num.recovery.threads.per.data.dir,but the startup p

How to read-your-writes in a side effect

Imagine I have a Kafka cluster with one producer and one consumer, and behind it an API server. The API allows clients to write (PUT requests). When a client writes, it produces an event into Kafka with that data on a stream. Then there is a consumer listening on that stream that will be respon

Re: [External] How to read-your-writes in a side effect

Your consumer will need to put some sort of response message on a different topic and the API will have to subscribe and wait for that response message. You'll need some sort of key so that it can know which one was for that call. I would avoid this type of API, however. Instead I would create

Kafka shuts down abnormally and restarts, always taking more than a few hours to load

- Kafka shuts down abnormally and restarts, always taking more than a few hours to load. - We used Kafka-2.0.0,we tried to turn it up num.recovery.threads.per.data.dir,but the startup process is always limited by the slow loading of partitions.We observed a partition that too

Re: [External] How to read-your-writes in a side effect

I was hoping there was an API for at least knowing the consumer progress.. — Pedro Teixeira ✉️ i...@pgte.me 💻 https://github.com/pgte 👨🏻‍🎓 https://www.linkedin.com/in/pedroteixeira/ ✍🏻 https://metaduck.com ✍🏻 https://pgte.me --- original message --- On 13 October 2020, 3:42 PM WEST dave.tau

Re: [External] How to read-your-writes in a side effect

When you send a message the returned RecordMetadata has an offset. If you know the name of the Consumer Group that is reading these messages there is an API to get the latest offset of the Consumer Group so you could poll that and wait for the latest Offset to be greater than the offset of you