Hi Kant, the non windowed stream-stream join is not equivalent to the full-history join, though they get the same SQL form. The retention times for records must be set to leverage the storage consumption and completeness of the results.
Best, Xingcan > On 7 Mar 2018, at 8:02 PM, kant kodali <kanth...@gmail.com> wrote: > > Hi Cheng, > > The docs here > <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#joins> > states full outer joins are only available for batch (I am not sure if I am > reading that correctly). I am trying to understand how two unbounded streams > can be joined like a batch? If we have to do batch join then it must be > bounded right? If so, how do we bound? I can think Time Window is one way to > bound but other than that if I execute the below join query on the unbounded > stream I am not even sure how that works? A row from one table can join with > a row from another table and that row can come anytime in future right if it > is unbounded. so I am sorry I am failing to understand. > > > SELECT * > FROM Orders o, Shipments s > WHERE o.id <http://o.id/> = s.orderId > > Thanks! > > On Wed, Mar 7, 2018 at 3:49 AM, Hequn Cheng <chenghe...@gmail.com > <mailto:chenghe...@gmail.com>> wrote: > Hi kant, > > It seems that you mean the Time-windowed Join. The Time-windowed Joins are > supported now. You can check more details with the docs given by Xingcan. > As for the non-window join, it is used to join two unbounded stream and the > semantic is very like batch join. > > Time-windowed Join: > SELECT * > FROM Orders o, Shipments s > WHERE o.id <http://o.id/> = s.orderId AND > o.ordertime BETWEEN s.shiptime - INTERVAL '4' HOUR AND s.shiptime > > Non-windowed Join: > SELECT * > FROM Orders o, Shipments s > WHERE o.id <http://o.id/> = s.orderId > > On Wed, Mar 7, 2018 at 7:02 PM, kant kodali <kanth...@gmail.com > <mailto:kanth...@gmail.com>> wrote: > Hi! > > Thanks for all this. and yes I was indeed talking about SQL/Table API so I > will keep track of these tickets! BTW, What is non-windowed Join? I thought > stream-stream-joins by default is a stateful operation so it has to be within > some time window right? Also does the output of stream-stream joins emit > every time so we can see the state of the join at any given time or only when > the watermark elapses and join result fully materializes? > > On a side note, Full outer join seems to be the most useful for my use case. > so the moment its available in master I can start playing and testing it! > > On Tue, Mar 6, 2018 at 10:39 PM, Hequn Cheng <chenghe...@gmail.com > <mailto:chenghe...@gmail.com>> wrote: > Hi Kant, > > The stream-stream outer joins are work in progress now(left/right/full), and > will probably be ready before the end of this month. You can check the > progress from[1]. > > Best, Hequn > > [1] https://issues.apache.org/jira/browse/FLINK-5878 > <https://issues.apache.org/jira/browse/FLINK-5878> > > On Wed, Mar 7, 2018 at 1:01 PM, Xingcan Cui <xingc...@gmail.com > <mailto:xingc...@gmail.com>> wrote: > Hi Kant, > > I suppose you refer to the stream join in SQL/Table API since the outer join > for windowed-streams can always be achieved with the `JoinFunction` in > DataStream API. > > There are two kinds of stream joins, namely, the time-windowed join and the > non-windowed join in Flink SQL/Table API [1, 2]. The time-windowed outer join > has been supported since version 1.5 and the non-windowed outer join is still > work in progress. > > Hope that helps. > > Best, > Xingcan > > [1] > https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#joins > > <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#joins> > [2] > https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql.html#joins > > <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql.html#joins> > > >> On 7 Mar 2018, at 12:45 AM, kant kodali <kanth...@gmail.com >> <mailto:kanth...@gmail.com>> wrote: >> >> Hi All, >> >> Does Flink support stream-stream outer joins in the latest version? >> >> Thanks! > > > > >
