RE: Random continuous TimeoutException with Topic not present on one KafkaProducer out of many in multithreaded env

2022-06-07 Thread Deepak Jain
Hi Luke, The complete exception is java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic realtimeImport_1 not present in metadata after 250 ms. at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.(KafkaProducer.java:1316)

Re: Newbie how to get key/value pojo out of a stream?

2022-06-07 Thread Matthias J. Sax
`enable.auto.commit` is a Consumer config and does not apply to Kafka Stream. In Kafka Streams, you basically always have auto commit enabled, and you can control how frequently commits happen via `commit.interval.ms`. Also on `close()` Kafka Streams would commit offsets. -Matthias On 5/31

Re: How it is safe to break message ordering but not idempotency after getting an OutOfOrderSequenceException?

2022-06-07 Thread Matthias J. Sax
Yes, the broker de-dupes using the sequence number. But for example, if a sequence number is skipped, you could get this exception: the current batch of messages cannot be appended to the log, as one batch is missing, and the producer would need to re-send the previous/missing batch with lower

Re: How it is safe to break message ordering but not idempotency after getting an OutOfOrderSequenceException?

2022-06-07 Thread Gabriel Giussi
Thanks for the answer Matthias. I still have doubts about the meaning of "risks reordering of sent record". If I understood correctly the example you gave is something like this 1. Producer sends batch with sequence number X 2. That request gets lost in the network 3. Producer sends batch with sequ

Kafka Java clients compatibility

2022-06-07 Thread 刘冲
Hi, are there some information about compatibility between different versions java clients and server? Such as 3.0 java client to 2.6 kafka server? Thanks!

Re: unsubscribed from all topics when adding a KTable

2022-06-07 Thread Guozhang Wang
Hello Meir, >From the code snippet I cannot find where did you add a KTable, it seems you created a KStream from the source topic, and aggregate the stream into a KTable, could you show me the code difference between "adding a KTable" v.s. "adding a KStream"? Anyways, the log line should only hap