> However, Kafka Streams does handle late arriving data. So if you had some > analytics that computes results on a time window or a session window then > Kafka streams will compute on the stream in real time (processing message > B) and then later when message A arrives it will put that message back into > the right temporal context and publish an amended result for the proper > time/session window as if message B were consumed in the timestamp order > before message A.
Can you link me to the javadocs or documentation for where I can read more into how this is done / how to use this? Thanks. On Sat, Mar 18, 2017 at 11:11 PM, Hans Jespersen <h...@confluent.io> wrote: > sorry I mixed up Message A and B wrt the to question but the answer is > still valid. > > -hans > > /** > * Hans Jespersen, Principal Systems Engineer, Confluent Inc. > * h...@confluent.io (650)924-2670 > */ > > On Sat, Mar 18, 2017 at 11:07 AM, Hans Jespersen <h...@confluent.io> > wrote: > > > The only way to make sure A is consumed first would be to delay the > > consumption of message B for at least 15 minutes which would fly in the > > face of the principals of a true streaming platform so the short answer > to > > your question is "no" because that would be batch processing not stream > > processing. > > > > However, Kafka Streams does handle late arriving data. So if you had some > > analytics that computes results on a time window or a session window then > > Kafka streams will compute on the stream in real time (processing message > > B) and then later when message A arrives it will put that message back > into > > the right temporal context and publish an amended result for the proper > > time/session window as if message B were consumed in the timestamp order > > before message A. The end result of this flow is that you eventually get > > the same results you would get in a batch processing system but with the > > added benefit of getting intermediary result at much lower latency. > > > > -hans > > > > /** > > * Hans Jespersen, Principal Systems Engineer, Confluent Inc. > > * h...@confluent.io (650)924-2670 <(650)%20924-2670> > > */ > > > > On Sat, Mar 18, 2017 at 10:29 AM, Ali Akhtar <ali.rac...@gmail.com> > wrote: > > > >> Is it possible to have Kafka Streams order messages correctly by their > >> timestamps, even if they arrived out of order? > >> > >> E.g, say Message A with a timestamp of 5:00 PM and Message B with a > >> timestamp of 5:15 PM, are sent. > >> > >> Message B arrives sooner than Message A, due to network issues. > >> > >> Is it possible to make sure that, across all consumers of Kafka Streams > >> (even if they are across different servers, but have the same consumer > >> group), Message A is consumed first, before Message B? > >> > >> Thanks. > >> > > > > >
