Re: Signal for End of Stream

2018-05-08 Thread Dhruv Kumar
Fabian, Thanks a lot for your continuous help! Really appreciate it. Sent from Phone. > On May 8, 2018, at 03:06, Fabian Hueske wrote: > > Hi Dhruv, > > The changes look good to me. > > Best, Fabian > > 2018-05-08 5:37 GMT+02:00 Dhruv Kumar : >> Thanks a lot, Fabian for your response. >> >>

Re: Signal for End of Stream

2018-05-08 Thread Fabian Hueske
Hi Dhruv, The changes look good to me. Best, Fabian 2018-05-08 5:37 GMT+02:00 Dhruv Kumar : > Thanks a lot, Fabian for your response. > > What I understand is that if I write my own Sourcefunction such that it > handles the "end of stream” record and make the source exit from run() > method, th

Re: Signal for End of Stream

2018-05-07 Thread Dhruv Kumar
Thanks a lot, Fabian for your response. What I understand is that if I write my own Sourcefunction such that it handles the "end of stream” record and make the source exit from run() method, the flink program will terminate. I have been using SocketTextStreamFunction till now. So, I duplicated

Re: Signal for End of Stream

2018-05-07 Thread Fabian Hueske
Hi, Flink will automatically stop the execution of a DataStream program once all sources have finished to provide data, i.e., when all SourceFunction return from the run() method. The DeserializationSchema.isEndOfStream() method can be used to tell a built-in SourceFunction such as a KafkaConsumer

Re: Signal for End of Stream

2018-05-07 Thread Dhruv Kumar
I notice that there is some DeserializationSchema in org.apache.flink.api.common.serialization which has a function isEndOfStream but I am not sure if I can use it in my use case. -- Dhruv Kumar PhD Candidate Department of Computer Science and Eng

Signal for End of Stream

2018-05-07 Thread Dhruv Kumar
Hi Is there a way I can capture the end of stream signal for streams which are replayed from historical data? I need the end of stream signal to tell the Flink program to finish its execution. Below is the use case in detail: 1. An independent log replayer program sends the records to a socket