What it means by Committed logs

2017-07-22 Thread nitin sharma
Hi Team, I have couple of questions related to Log commit : a) What Committed Logs means? i understand that Consumer can pick messages when it is committed, i.e. when message reached to all in-sync replicas. But it leaves doubt in my mind. does commit means when all replicas have flushed messa

Re: Joining two topics and emitting each key only once within a sliding window.

2017-07-22 Thread Matthias J. Sax
I am not sure exactly what semantics you want to have. Note, that Kafka Streams provides a sliding window join between two stream. Thus, I am not sure what you mean by >> Track when matches are found so subsequent matches (which within the join >> window would be considered duplicates) aren't reem

Re: Event sourcing with Kafka and Kafka Streams. How to deal with atomicity

2017-07-22 Thread José Antonio Iñigo
Hi Garret, At the moment, to simplify the problem I only have one topic, orders, where I add products and decrement them based on ProductAdded and ProductReserved events. Yeaterday I was reading about EoS but I don't know if it'll solve the problem. Dividing the query-update in two steps means th

Re: Event sourcing with Kafka and Kafka Streams. How to deal with atomicity

2017-07-22 Thread José Antonio Iñigo
My previous mail was in fact addressed to Ben, not Chris, sorry for the mistake. Regards On Sat, 22 Jul 2017 at 00:15, José Antonio Iñigo < joseantonio.in...@gmail.com> wrote: > Hi Chris, > > *"if I understand your problem correctly, the issue is that you need to* > > > *decrement the stock coun

AUTO: Yan Wang is out of the office (returning 07/31/2017)

2017-07-22 Thread Yan Wang
I am out of the office until 07/31/2017. Note: This is an automated response to your message "Re: Tuning up mirror maker for high thruput" sent on 7/22/2017 3:27:24 AM. This is the only notification you will receive while this person is away. ** This email and any attachments may contain i

Re: Tuning up mirror maker for high thruput

2017-07-22 Thread James Cheng
Becket Qin from LinkedIn spoke at a meetup about how to tune the Kafka producer. One scenario that he described was tuning for situations where you had high network latency. See slides at https://www.slideshare.net/mobile/JiangjieQin/producer-performance-tuning-for-apache-kafka-63147600 and vid