Hi kant, You are right. Batch joins require the inputs are bounded. To join two unbounded streams without window, all data will be stored in join's states, so the late right row will join the previous left row when it is input. As for state retention time, if the input tables of join are both keyed table and key number of the keyed tables are limited, then you don't have to set state retention time, otherwise it is suggested to set the state retention time.
Best, Hequn On Wed, Mar 7, 2018 at 8:02 PM, kant kodali <kanth...@gmail.com> wrote: > Hi Cheng, > > The docs here > <https://ci.apache.org/projects/flink/flink-docs-master/dev/table/tableApi.html#joins> > states > full outer joins are only available for batch (I am not sure if I am > reading that correctly). I am trying to understand how two unbounded > streams can be joined like a batch? If we have to do batch join then it *must > be* bounded right? If so, how do we bound? I can think Time Window is one > way to bound but other than that if I execute the below join query on the > unbounded stream I am not even sure how that works? A row from one table > can join with a row from another table and that row can come anytime in > future right if it is unbounded. so I am sorry I am failing to understand. > > > SELECT * > FROM Orders o, Shipments s > WHERE o.id = s.orderId > > Thanks! > > On Wed, Mar 7, 2018 at 3:49 AM, Hequn Cheng <chenghe...@gmail.com> wrote: > >> Hi kant, >> >> It seems that you mean the Time-windowed Join. The Time-windowed Joins are >> supported >> now. You can check more details with the docs given by Xingcan. >> As for the non-window join, it is used to join two unbounded stream and >> the semantic is very like batch join. >> >> Time-windowed Join: >> >>> SELECT * >>> FROM Orders o, Shipments s >>> WHERE o.id = s.orderId AND >>> o.ordertime BETWEEN s.shiptime - INTERVAL '4' HOUR AND s.shiptime >> >> >> Non-windowed Join: >> >>> SELECT * >>> FROM Orders o, Shipments s >>> WHERE o.id = s.orderId >> >> >> On Wed, Mar 7, 2018 at 7:02 PM, kant kodali <kanth...@gmail.com> wrote: >> >>> Hi! >>> >>> Thanks for all this. and yes I was indeed talking about SQL/Table API so >>> I will keep track of these tickets! BTW, What is non-windowed Join? I >>> thought stream-stream-joins by default is a stateful operation so it has to >>> be within some time window right? Also does the output of stream-stream >>> joins emit every time so we can see the state of the join at any given time >>> or only when the watermark elapses and join result fully materializes? >>> >>> On a side note, Full outer join seems to be the most useful for my use >>> case. so the moment its available in master I can start playing and testing >>> it! >>> >>> On Tue, Mar 6, 2018 at 10:39 PM, Hequn Cheng <chenghe...@gmail.com> >>> wrote: >>> >>>> Hi Kant, >>>> >>>> The stream-stream outer joins are work in progress >>>> now(left/right/full), and will probably be ready before the end of this >>>> month. You can check the progress from[1]. >>>> >>>> Best, Hequn >>>> >>>> [1] https://issues.apache.org/jira/browse/FLINK-5878 >>>> >>>> On Wed, Mar 7, 2018 at 1:01 PM, Xingcan Cui <xingc...@gmail.com> wrote: >>>> >>>>> Hi Kant, >>>>> >>>>> I suppose you refer to the stream join in SQL/Table API since the >>>>> outer join for windowed-streams can always be achieved with the >>>>> `JoinFunction` in DataStream API. >>>>> >>>>> There are two kinds of stream joins, namely, the time-windowed join >>>>> and the non-windowed join in Flink SQL/Table API [1, 2]. The >>>>> time-windowed outer join has been supported since version 1.5 and the >>>>> non-windowed outer join is still work in progress. >>>>> >>>>> Hope that helps. >>>>> >>>>> Best, >>>>> Xingcan >>>>> >>>>> [1] https://ci.apache.org/projects/flink/flink-docs-master/d >>>>> ev/table/tableApi.html#joins >>>>> [2] https://ci.apache.org/projects/flink/flink-docs-master/d >>>>> ev/table/sql.html#joins >>>>> >>>>> >>>>> On 7 Mar 2018, at 12:45 AM, kant kodali <kanth...@gmail.com> wrote: >>>>> >>>>> Hi All, >>>>> >>>>> Does Flink support stream-stream outer joins in the latest version? >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> >>>> >>> >> >
