Re: "End of Batch" event

2017-01-31 Thread Eric Dain
> Furhtermore, IQ is designed to be use outside of you Streams code, and > thus, you should no use it in SourceTask (not sure if this would even be > possible in the first place). > > However, you might be able to exploit joins... Your CSV input is > KStream, right? > > > -M

Re: "End of Batch" event

2017-01-31 Thread Eric Dain
eloper- > guide.html#interactive-queries > > https://www.confluent.io/blog/unifying-stream-processing- > and-interactive-queries-in-apache-kafka/ > > > > -Matthias > > > On 1/30/17 9:10 PM, Eric Dain wrote: > > Thanks Matthias for your reply. > > > > I&#x

Re: "End of Batch" event

2017-01-30 Thread Eric Dain
> Cf. > https://cwiki.apache.org/confluence/display/KAFKA/ > Kafka+Streams+Data+%28Re%29Processing+Scenarios > > > -Matthias > > > On 1/28/17 1:07 PM, Eric Dain wrote: > > Hi, > > > > I'm pretty new to Kafka Streams. I am using Kafka Streams to ingest l

"End of Batch" event

2017-01-28 Thread Eric Dain
Hi, I'm pretty new to Kafka Streams. I am using Kafka Streams to ingest large csv file. I need to run some clean-up code after all records in the file are processed. Is there a way to send "End of Batch" event that is guaranteed to be processed after all records? If not is there alternative soluti