Re: Patterns for generating ordered streams

2023-05-30 Thread Italo Nesi
Hi Edward, if you use partition number and/or partition offset you are potentially building up a technical debt and tightly coupling the pipeline. What if you want to change the numbers of partitions in the future? Also there is what you mentioned about moving to another Kafka cluster where you hav

Kafka requirement

2023-05-30 Thread Prem Sagar
HI We have a Kafka requirement as per the below scope: *Scope of work for Kafka Cluster and Zookeeper Support:* We would require complete Kafka and zookeeper support to ensure that 100% uptime and no data loss is maintained. Ensuring high performance is assured. On few instances there might be ca

fetch.max.wait.ms Data cannot be pulled for more than one minute

2023-05-30 Thread 1274244774
When I set fetch.max.wait.ms to greater than one minute, I don't pull any data after more than one minute that's my config

Re: [VOTE] 3.4.1 RC3

2023-05-30 Thread Mickael Maison
Hi Luke, I built from source with Java 11 and Scala 2.13 and ran the unit and integration tests. It took a few retries to get some of them to pass. I verified signatures and hashes and also ran the zookeeper quickstart. +1 (binding) Thanks, Mickael On Sat, May 27, 2023 at 12:58 PM Jakub Scholz

Consuming an entire partition with control messages

2023-05-30 Thread Vincent Maurin
Hello ! I am working on an exactly once stream processors in Python, using aiokafka client library. My program stores a state in memory, that is recovered from a changelog topic, like in kafka streams. On each processing loop, I am consuming messages, producing messages to an output topics and to

Re: [VOTE] 3.4.1 RC3

2023-05-30 Thread Chris Egerton
Hi Luke, Many thanks for your continued work on this release! To verify, I: - Built from source using Java 11 with both: - - the 3.4.1-rc3 tag on GitHub - - the kafka-3.4.1-src.tgz artifact from https://home.apache.org/~showuon/kafka-3.4.1-rc3/ - Checked signatures and checksums - Ran the quickst