Re: Kafka Streams Join does not work

2019-03-15 Thread Ryanne Dolan
If your producer is using transactions, it's possible that the producer was killed in the middle of a transaction, in which case any un-committed records would be logged on the broker but skipped by downstream consumers. Otherwise, it's likely that the leader for the partition crashed before the r

Re: Kafka Streams Join does not work

2019-03-15 Thread Федор Чернилин
Thanks, I understand how consumer works. The main question is related to why the join did not work and how it happened that only one message remained uncommitted. пт, 15 мар. 2019 г. в 16:29, Ryanne Dolan : > Hello! When using exactly-once semantics, uncommitted or aborted records > are skipped b

Re: Kafka Streams Join does not work

2019-03-15 Thread Ryanne Dolan
Hello! When using exactly-once semantics, uncommitted or aborted records are skipped by the consumer as if they don't exist. When inspecting the topic manually, use isolation.level=read_committed to get the same behavior. Ryanne On Fri, Mar 15, 2019, 6:08 AM Федор Чернилин wrote: > I also noti

Re: Kafka Streams Join does not work

2019-03-15 Thread Федор Чернилин
I also noticed another important thing now. Message which used for join is uncommitted. I understood it with the help of consumer's setting isolation level - read_committed. The message got into the topic using the same stream app. Remind that stream app has processing guarantee = exactly_once. How

Kafka Streams Join does not work

2019-03-15 Thread Федор Чернилин
Hello! I encountered the following problem. I have 3 brokers and 1 zookeper. Topics with 10 partitions and replication factor 3. Stream app with 10 threads, exactly_once and commit interval 1000ms. When I run stream app, join of 2 my topics doesn't work for specific message. But for all another mes