Mirrormaker 2.0 - duplicates with idempotence enabled

Hi, I have setup mirrormaker2 (Kafka v.2.6.0) on 2 clusters (CL1,CL2) and the mirroring seems to work properly except with an issue with duplicates in the following scenario: While both clusters are up and running i simulate an incident, stopping one by one the brokers of the CL2 cluster. Stopp

Re: KafkaStreams - InteractiveQuery and num.standby.replicas

yes, exactly it is from spring cloud kafka streams binder, thank you! On 3/9/21 21:59, Matthias J. Sax wrote: Please keep the discussion on the mailing list. Thanks. There is no such thing as `InteractiveQueryService` in Kafka Streams. Seems it's a Spring thing? Not sure how Spring work. Maybe

Re: KafkaStreams - InteractiveQuery and num.standby.replicas

Please keep the discussion on the mailing list. Thanks. There is no such thing as `InteractiveQueryService` in Kafka Streams. Seems it's a Spring thing? Not sure how Spring work. Maybe somebody else can help out? -Matthias On 3/9/21 3:15 AM, Igor Maznitsa wrote: > thanks for advice! > > does s

Slightly Modified Sticky Assignor.

Hi all, I am implementing a custom partition assignor slightly different than the sticky assignor assignor. As known, in the sticky assignor each consumer sends the set of owned partitions as part of its subscription message. This happens in the subscriptionUserData by calling the serializeTop

Kafka SLIs/SLOs

I am looking for defining these Availability, Latency, ErrorRate, Load for Kafka clusters using Prometheus metrics generated by KAfka and jmx exporter. Does anyone have any initial dashboards to start with for defining these indicators? Thank you! Rs Vas

Re: KafkaStreams - InteractiveQuery and num.standby.replicas

thanks for advice! does standby.replicas play role during search of HostInfo through InteractiveQueryService#getHostInfo ? will it return the local host info (instead of remote instance) if it contains local store replica for required key? On 3/9/21 12:49, Matthias J. Sax wrote: You can con

Re: Abort transaction semantics

Hello Thanks for advice, I have looked into examples and spotted yet another place that is not clear to me. It is resetToLastCommittedPositions (in case of abort). I am wondering what will happen when this method crashes, so it will not reset consumer to latest committed offsets. Let's say the pro

Re: KafkaStreams - InteractiveQuery and num.standby.replicas

You can control it: By default only active stores are queried. If you want to query standby stores, you can use `StoreQueryParameters#enableStaleStores()` -Matthias On 3/9/21 1:30 AM, Igor Maznitsa wrote: > Hello > > If I have started two KafkaStream application instances, and they have > para

KafkaStreams - InteractiveQuery and num.standby.replicas

Hello If I have started two KafkaStream application instances, and they have parameter 'num.standby.replicas=1'. It means that each instance has shadow copy of local state stores, so the question: How InteractionQuery behaves in activated stand by replica case? does it aware about replicated

Re: Kafka Streams - Out of Order Handling

In general, Kafka Streams tries to process messages in timestamp order, ie, oldest message first. However, Kafka Streams always need to process messages in offset order per partition, and thus, the timestamp synchronization applied to records from different topic (eg, if you join two topics). Ther