Hi Max, if I understood correctly, instead of joining three streams, you actually performed two separate joins, say S1 JOIN S3 and S2 JOIN S3, right? Your plan "(S1 UNION S2) JOIN S3” seems to be identical with “(S1 JOIN S3) UNION (S2 JOIN S3)” and if that’s what you need, your pipeline should be feasible I think.
However, If you want to join three streams, you may first join S1 with S2 to produce S12 with a CoProcessFunction, and then set another CoProcessFunction to join S12 with S3. Hope that helps. Best, Xingcan > On 10 Feb 2018, at 1:06 PM, m@xi <makisnt...@gmail.com> wrote: > > Hello Flinkers, > > I would like to discuss with you about something that bothers me. So, I have > two streams that I want to join along with a third stream which I want to > consult its data from time to time and triggers decisions. > > Essentially, this boils down to coProcessing 3 streams together instead of > 2, which to the best of my knowledge is not possible. > > I thought to append an extra field to the 2 streams I want to join, namely > say S1, S2 are the streams with tuples t1, t2. After the pumping with the > extra field which is the stream id (1 or 2) the tuple would be (1, t1) and > (2, t2) resulting to S1' and S2'. > > Then I will do S1'.union(S2') which gives me a single data stream. Then this > I may join with the 3rd stream and do the processing with a coProcess > function. > > Although, whenever I process and element from the united streams I should > have an if-then-else to check to which stream a tuple belongs and process > and update S1' and S2' state accordingly. > > Do you think this is a good idea? In terms of efficiency compared with > having two functions to do this, namely processElement1() and > processElement2() of the coProcess function in case I only had two streams. > > And if the aforementioned scheme is feasible, then I guess up til now, this > is the only way of joining more than 2 streams. Am I right? > > Thanks in advance for your help. > > Best, > Max > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
