Re: Better way to read several stream sources

2017-01-25 Thread Sendoh
Found the reason. I saw using ParallelSourceFunction my override open() is called 4 times, comparing to using sourceFunction open() is called only once, and my override open() constructs the connection to sources, which determines how many source are going to be read. Cheers, Sendoh -- View th

Re: Better way to read several stream sources

2017-01-24 Thread Sendoh
Hi Stephan, Thank you for answering my question. I try option 2 and it gives me correct results reading several sources, while using ParallelSourceFunction it gives 4 times redundancy (same as my number of threads). Can I ask what would be the reason causing the difference? I think I don't under

Re: Better way to read several stream sources

2017-01-23 Thread Stephan Ewen
Why don't you define one source and make it parallel? You can implement "RichParallelSourceFunction" and use that to check which parallel subtask the respective source is during execution. On Mon, Jan 23, 2017 at 6:55 PM, Sendoh wrote: > Hi Flink users, > > Can I ask is what would be the better