Re: Debugging Kafka Streams Windowing

2017-05-13 Thread Mahendra Kariya
Hi, There is no missing data. But the INFO level logs are infinite and the streams practically stops. For the messages that I posted, we got these INFO logs for around 20 mins. After which we got an alert about no data being produced in the sink topic and we had to restart the streams processes.

Re: Can state stores function as a caching layer for persistent storage

2017-05-13 Thread João Peixoto
Replies in line as well On Sat, May 13, 2017 at 3:25 AM Eno Thereska wrote: > Hi João, > > Some answers inline: > > > On 12 May 2017, at 18:27, João Peixoto wrote: > > > > Thanks for the comments, here are some clarifications: > > > > I did look at interactive queries, if I understood them cor

Re: Kafka-streams process stopped processing messages

2017-05-13 Thread Shimi Kiviti
I tried all these configurations and now like version 0.10.1.1 I see a very slow startup. I decreased the cluster to a single server which was running without any problem for a few hours. Now, each time I restart this process it gets into rebalancing state for several hours. That mean that every ti

Re: Debugging Kafka Streams Windowing

2017-05-13 Thread Matthias J. Sax
Hi, I just dug a little bit. The messages are logged at INFO level and thus should not be a problem if they go away by themselves after some time. Compare: https://groups.google.com/forum/#!topic/confluent-platform/A14dkPlDlv4 Do you still see missing data? -Matthias On 5/11/17 2:39 AM, Mahen

Re: Can state stores function as a caching layer for persistent storage

2017-05-13 Thread Eno Thereska
Hi João, Some answers inline: > On 12 May 2017, at 18:27, João Peixoto wrote: > > Thanks for the comments, here are some clarifications: > > I did look at interactive queries, if I understood them correctly it means > that my state store must hold all the results in order for it to be > querie