Thank you Hongshun for your feedback! You are right that restricting the probe-side input to append-only changes deviates from the existing event-time (and disabled proc-time) temporal table joins. I chose this restriction because processing time semantics do not work well with retractions. A probe record of +(k1, p1) could join against a build-side of (k1, v1), while a later arriving probe-side retraction -(k1, p1) would join against (k1, v2). The resulting rectraction record of -(k1, p1, v2) would not match the earlier insertion +(k1, p1, v1). The only changelog format that would work well would be upsert changes (+I, +UA, -D) with key-only deletes (under the assumption that the upsert key is still unique after the join).
I would like to keep the default behavior of only accepting append-only inputs to prevent retraction mismatches. At the same time, the operator implementation only needs to be slightly adjusted to support arbitrary probe-side changes (if at all). The real change would be in the planner / rules. So allowing probe-side retraction input via a config option for power-users who know what they are doing is certainly possible. Do you think this should be part of the proposal or would you be fine to leave this as future work? Best, Fabian Am Mi., 3. Juni 2026 um 08:41 Uhr schrieb Leonard Xu <[email protected]>: > Hi Fabian, > > Thanks for the detailed and thoughtful reply, and especially for agreeing > to add operator metrics — that alone will significantly improve the > debuggability of the join in production. > > (1) On CPU spike / probe-side buffering > > +1 for your reasoning on both points — deferring InputSelectable until > the unaligned-checkpoint limitation is resolved, and leaving micro-batch > transition out of v1. Watermark alignment + probe-side scan-offset is the > principled clean approach, agreed. > > (2) On Jark's backlog idea > > Between the two paths in your reply, I'd lean toward option (a) — having > the source connector emit a special WM (or a dedicated record attribute) at > the end of backlog — as the long-term direction for exact flip-point > semantics. It naturally fits what is almost certainly going to be the > dominant production scenario: CDC sources (mysql-cdc, mongodb-cdc, > postgres-cdc, ...) as the dimension build-side, all of which already have > an explicit "snapshot finished → binlog start" boundary internally. > > My suggestion is to ship FLIP-579 as proposed and collect user feedback > on LATERAL SNAPSHOT Join from real production usage — that will give us > solid evidence to prioritize this follow-up direction. > > (3) On the vote > > Once the metrics section is added and there are no further objections from > others, I'm fine to start the vote. > > > Best, > Leonard > > > > > 2026 6月 3 01:36，Fabian Hueske <[email protected]> 写道： > > > > Thanks for your valuable feedback Jark and Leonard! > > > > You are bringing up three of the tricky challenges that the new join > needs > > to deal with. > > > > (1) Jark: The build-side flip point is not exact > > > > This is correct. However, I would argue that a processing-time join does > > not have exact guarantees anyway and can only produce roughly > time-aligned > > results. WM alignment of build and probe-side input should help to keep > the > > alignment somewhat close. Of course, this does not mean that we shouldn't > > try to give as good guarantees as possible. > > > > The primary mechanism for flipping from LOAD to JOIN phase is the > > build-side watermark crossing a configured point in time. Watermarks are > > used to track progress and completeness in Flink. Using them as a > condition > > to switch from LOAD to JOIN phase, means that the build-side received at > > least all changes up to that point in time. There might have been changes > > with later timestamps as well. These could be buffered on the side to > have > > a stricter FLIP point, but IMO this additional data should be tolerable > > under proc-time semantics. > > > > If the build-side input becomes stale, the processing idle timeout flip > > condition gets applied. The assumption is that the build-side source is > > currently exhausted and all data was consumed but the WM didn't progress > > far enough to exceed the flip point. In this case, we want to flip and > > start the regular JOIN phase. > > > > For the use case of sources with an exact flip point, users would need to > > know the timestamp of the last backlog record (or compute it if they know > > roughly how long it takes to scan the backlog if it is computed > on-demand). > > I agree, this is not very practical. > > I can think of two options > > a) the source connector emits a special WM when it reaches the end of > the > > backlog. This would not require changes to the join operator but to the > > source connectors. > > b) The design of the SNAPSHOT function has the > `load_completed_condition` > > which is an extension point to add logic to determine the flip point. > > > > > > (2) Jark: Buffering probe-side during LOAD phase > > > > I think this is very similar to Leonard's point about "LOAD phase > > backpressure" and also closely related to Leonard's point about > "Flip-point > > CPU spike". > > > > This is indeed a potential problem. If used without care, the probe-side > > state might grow very large. Before talking about possible ways to > address > > this problem, let me explain how I think that the join would be used. > > > > A very common (maybe the most common?) use case should be to initialize > the > > build-side input up to time t_b and then start processing the probe-side > > input from time t_p (with t_p = t_b, or slightly less than t_b) onwards. > WM > > alignment would help to roughly align build-side and probe-side inputs > > (although not being perfectly aligned like the event-time join) > > Initializing the build-side up to t_b and starting consuming the > probe-side > > from t_p with (t_p << t_b) would mean that the first probe-side records > are > > joined with much later versions of the build-side. > > > > The first scenario (t_p = t_b) can be controlled with WM alignment and > > scan-offset table hints on the probe-side input. Since the WMs of the > > build-side input would be less than the WMs of the probe-side input, the > > probe-side input should be throttled until the build-side caught up > (which > > should be close the the flip point). > > Other scenarios (including the t_p << t_b scenario) would benefit from an > > idea [1] that is described in the future work section of the FLIP. That > > mechanism would also be based on WM alignment and would need some > > collaboration from the build-side source operator to indicate > completeness > > of the backlog. > > > > Using the InputSelectable interface is an idea that we also looked into > (as > > Gustavo already pointed out). Unfortunately, it is incompatible with > > unaligned checkpoints and there are no other streaming operators that > > implement the interface. I haven't looked in depth at the current > > limitations, but if some of these would be resolved, it might be possible > > to later extend the join operator. It might even work with relaxed > > guarantees because we don't need to fully block the input but just > throttle > > it such that less probe-side data needs to be buffered. > > > > The idea of limiting the size of the probe-side buffer with a config like > > `max-buffer-size` sounds interesting. However, I'm not sure if applying > > backpressure would really work because we still need to consume the > > build-side to be able to reach the flip point and selective backpressure > is > > not possible without InputSelectible. > > > > An earlier draft of the proposal described eager joining (now moved to > the > > future work section [2]). The idea is that a probe-side record would be > > directly joined when a match was present or received during the LOAD > phase. > > After joining it wouldn't be put into state. This would of course > > significantly reduce the state size and solved the issue of CPU spikes > > during the transition but come at the cost of hard-to-explain semantics > > (the current semantics are rather simple, we collect until the flip point > > and join against that). > > Also, the idea of eager joining was developed when the join operator was > > still restricted to FK-PK joins (single build-side record per join key). > > The design generalized this restriction to arbitrary joins which means > > there might be more build-side matches for a probe-side record such that > > the presence of a single join match (of a possibly earlier version) does > > not guarantee completeness anymore. That's why the idea of eager joining > > was discarded for now. > > > > > > (3) Leonard: Flip-point CPU spikes > > > > This is also a very valid concern. I would argue that the primary > mechanism > > to address this point should be to reduce the amount of buffered > probe-side > > records (see point (2) above). > > > > I also thought about your idea to micro-batch the draining. In the > design, > > the transition join is triggered per-key by event-time timers that also > > emit the "current" probe-side WM downstream. If we want to continue using > > this mechanism, we would need to schedule multiple timers and probably > use > > some clever mechanism to gradually advance probe-side WMs while still > > consuming records. There could be a separate "TRANSITION" phase during > > which we still append to the probe-buffer but use the mechanism for > atomic > > build-side updates. However, this would significantly complicate the > design > > affecting the control flow and recovery. Hence, I would first try to > > address this issue by reducing the probe-side state. > > > > If you have better ideas for how to use micro-batching during flip > > transition, I'm very open to exploring those. > > > > Leonard also brought up a very important point about metrics! > > I will add a section on operator metrics that will help to understand the > > state of the operator. > > > > Please let me know your thoughts! > > > > Best, Fabian > > > > [1] > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421958523#FLIP579:LATERALSNAPSHOTJoin-ReduceBufferingofProbe-SideviaBuild-SideWatermarkSuppression > > [2] > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421958523#FLIP579:LATERALSNAPSHOTJoin-EagerjoinmodeduringLOADphase > > > > > > Am Mo., 1. Juni 2026 um 16:44 Uhr schrieb Leonard Xu <[email protected] > >: > > > >> Hi Fabian > >> > >> Thanks for driving this FLIP and your kind patience. The motivation is > >> spot-on, and the LOAD→JOIN two-phase design is the right structural fix > for > >> the FLINK-19830 initialization problem. Overall direction +1 from my > side. > >> > >> Besides Jark’s idea about backlog and InputSelectable which may need > more > >> prerequisites, I’ve two concerns about current proposal: > >> > >> 1. LOAD phase backpressure. The FLIP assumes "seconds to a few minutes" > >> for build-side init, but nothing enforces it. Large build-side tables > >> (e.g., 50M rows) + fast probe streams → unbuffered probe-side state > >> explosion. Should we add a config like max-buffer-size that applies > >> backpressure when exceeded or some metrics about buffer, rather than > >> silently piling up records? > >> > >> 2. Flip-point CPU spike. Joining all buffered probe records × build-side > >> state in one shot differs fundamentally from event-time join's > incremental > >> watermark-batched emission. In the worst case this could cause a > >> TaskManager CPU spike and downstream shock. Worth considering > micro-batch > >> draining during flip transition? > >> > >> Looking forward to your thoughts. > >> > >> Best, > >> Leonard > >> > >>> 2026 6月 1 16:02，Fabian Hueske <[email protected]> 写道： > >>> > >>> Hi Leonard, > >>> > >>> Sorry, missed your email and already started the vote. > >>> Let me put it on hold for now and continue discussing the proposal. > >>> > >>> Looking forward to your comments, > >>> Fabian > >>> > >>> Am Mo., 1. Juni 2026 um 09:56 Uhr schrieb Leonard Xu < > [email protected] > >>> : > >>> > >>>> @Fabian Thanks for driving this FLIP, sorry for late reply due to my > >>>> personal reason that I shouldn’t miss such an important FLIP. > >>>> > >>>> I’m reviewing the FLIP and will try to finish it today, could you > kindly > >>>> wait one minute to start the vote? > >>>> > >>>> And sorry for interrupt your plan again. > >>>> > >>>> Best, > >>>> Leonard > >>>> > >>>>> 2026 6月 1 15:51，Fabian Hueske <[email protected]> 写道： > >>>>> > >>>>> Thanks everyone for your comments on the FLIP. > >>>>> I will start the vote. > >>>>> > >>>>> Best, Fabian > >>>>> > >>>>> Am Do., 28. Mai 2026 um 20:13 Uhr schrieb David Anderson < > >>>>> [email protected]>: > >>>>> > >>>>>> Fabian, > >>>>>> > >>>>>>> So, I don't think that we should buffer unmatched probe-side > records > >>>>>> beyond > >>>>>> the flip point. > >>>>>> > >>>>>> Thanks for explaining your reasoning. Makes sense to me. > >>>>>> > >>>>>> David > >>>>>> > >>>>>> On Thu, May 28, 2026 at 6:55 PM Fabian Hueske <[email protected]> > >>>> wrote: > >>>>>> > >>>>>>> Hi Xingcan, > >>>>>>> > >>>>>>> Thanks for your comments on the FLIP! > >>>>>>> > >>>>>>> The join's behavior when starting from a savepoint is indeed an > >>>> important > >>>>>>> aspect to consider and the problem of a rapidly advancing dimension > >>>>>>> (build-side) table is of course real. > >>>>>>> > >>>>>>> I would argue that watermark alignment should significantly reduce > >> the > >>>>>>> impact of this. > >>>>>>> If enabled, sources align their consumption based on their current > >>>>>>> watermark such that the (presumably much smaller) build-side source > >>>> would > >>>>>>> be slowed down to the event-time progress of the probe-side. > >>>>>>> While watermark alignment is not an "exact" mechanism, the > semantics > >> of > >>>>>> the > >>>>>>> new processing-time join also do not guarantee "exact" results. > >>>>>>> At the same time, alignment should ensure that build and probe-side > >> are > >>>>>>> roughly aligned in event-time (without the strict guarantees that > the > >>>>>>> event-time temporal table join provides). > >>>>>>> > >>>>>>> However, I really like your idea of starting in event-time mode and > >>>>>>> flipping to processing-time after the initialization duration > passed. > >>>>>>> I'm not sure if it would fully address the problem you described. > As > >>>> you > >>>>>>> said, users would need to be able to reconfigure the flip-point and > >> I'm > >>>>>> not > >>>>>>> sure if there's a good mechanism for this yet. > >>>>>>> But it might have some other properties that would be beneficial, > so > >>>> I'll > >>>>>>> think about that. > >>>>>>> > >>>>>>> Best, > >>>>>>> Fabian > >>>>>>> > >>>>>>> > >>>>>>> Am Do., 28. Mai 2026 um 18:21 Uhr schrieb Fabian Hueske < > >>>>>> [email protected] > >>>>>>>> : > >>>>>>> > >>>>>>>> Thanks for your feedback David! > >>>>>>>> > >>>>>>>>> One question: If I understand correctly, during the JOIN phase of > >> an > >>>>>>>> INNER > >>>>>>>> join, if the desired build-side record is missing, nothing will be > >>>>>>> emitted > >>>>>>>> for the unmatched probe-side record. For an INNER join, I can > >> imagine > >>>>>>>> wanting to buffer unmatched probe-side records, expecting the > build > >>>>>> side > >>>>>>>> will arrive soon. What's your thinking there? > >>>>>>>> > >>>>>>>> Your understanding is correct. If a probe-side record arrives > during > >>>>>> LOAD > >>>>>>>> phase but no matching build-side record is received, > >>>>>>>> the probe-side record would be discarded without being joined > during > >>>>>> the > >>>>>>>> transition from LOAD to JOIN. > >>>>>>>> > >>>>>>>> I would argue that users that want to prevent this, would need to > >>>>>>>> configure a longer initialization time. > >>>>>>>> IMO, dropping unmatched probe records is not a "bad" property of > >> INNER > >>>>>>>> joins but an essential part of their semantics. It might even be > >>>>>> desired > >>>>>>> by > >>>>>>>> some users. > >>>>>>>> If we would buffer probe-side records for INNER joins beyond the > >>>>>>>> transition point, we: > >>>>>>>> * would have different behaviors for INNER and LEFT joins > >>>>>>>> * could not start to emit probe-side watermarks as long as there > are > >>>>>>> still > >>>>>>>> probe-side records buffered (or at least not advance past them > >> without > >>>>>>>> emitting late data at a later point of time) > >>>>>>>> * would either need another config knob to specify when to > "really" > >>>>>> clean > >>>>>>>> up the probe-side state or keep such unmatched records forever in > >>>> state > >>>>>>> (we > >>>>>>>> could also use state TTL...) > >>>>>>>> > >>>>>>>> So, I don't think that we should buffer unmatched probe-side > records > >>>>>>>> beyond the flip point. > >>>>>>>> > >>>>>>>> Best, Fabian > >>>>>>>> > >>>>>>>> Am Do., 28. Mai 2026 um 17:05 Uhr schrieb Xingcan Cui < > >>>>>>> [email protected] > >>>>>>>>> : > >>>>>>>> > >>>>>>>>> Hi Fabian, > >>>>>>>>> > >>>>>>>>> Thanks for this FLIP! The two-phase design is excellent for > >> avoiding > >>>>>>>>> early-joining bugs while maintaining low-latency processing-time > >>>>>>>>> semantics. > >>>>>>>>> > >>>>>>>>> After thinking more about the proposal, I'd like to point out an > >> edge > >>>>>>> case > >>>>>>>>> related to the initialization phase or recovery after prolonged > >>>>>> downtime > >>>>>>>>> (for example, when a job has been down for a day). While a > >>>>>>> processing-time > >>>>>>>>> join works well for live streaming, where results can reasonably > >>>>>> depend > >>>>>>> on > >>>>>>>>> the immediate arrival order of live data, it does not work as > well > >>>> for > >>>>>>>>> catch-up scenarios. > >>>>>>>>> > >>>>>>>>> Currently, if a job initializes or restores from a checkpoint > >> after a > >>>>>>> long > >>>>>>>>> downtime, the operator resumes directly in the processing-time > join > >>>>>>> phase. > >>>>>>>>> During catch-up, however, the natural chronological arrival order > >> of > >>>>>> the > >>>>>>>>> live data is completely lost. As a result, these replayed fact > >>>> records > >>>>>>> are > >>>>>>>>> evaluated against the current machine time and may blindly join > >> with > >>>>>> the > >>>>>>>>> rapidly advancing "current" dimension snapshot, rather than the > >>>>>>> historical > >>>>>>>>> versions they were originally supposed to match. > >>>>>>>>> > >>>>>>>>> To handle this edge case, could we consider: > >>>>>>>>> > >>>>>>>>> 1. changing the first phase into an event-time join phase, and > >>>>>>>>> > >>>>>>>>> 2. allowing the operator to switch back to the first phase after > a > >>>>>>>>> restart? > >>>>>>>>> > >>>>>>>>> For example, users could configure a timestamp threshold. Before > >> the > >>>>>>>>> watermark reaches that point, the operator would run as an > >> event-time > >>>>>>>>> versioned join to safely process the catch-up phase through > >> watermark > >>>>>>>>> alignment. Once the watermark passes the threshold, the operator > >>>> could > >>>>>>>>> purge the old multi-version state and seamlessly transition back > to > >>>>>> the > >>>>>>>>> pure processing-time join phase for live traffic. > >>>>>>>>> > >>>>>>>>> After a job restart, users could either update the target > timestamp > >>>> to > >>>>>>>>> reset the operator back into the event-time phase, or leave it > >>>>>> unchanged > >>>>>>>>> to > >>>>>>>>> continue operating in the processing-time phase. > >>>>>>>>> > >>>>>>>>> I completely understand that this would introduce significant > >>>>>> complexity > >>>>>>>>> to > >>>>>>>>> the operator's state management and lifecycle, so this is only a > >>>>>>> tentative > >>>>>>>>> proposal to explore whether it might be worth considering for the > >>>>>>>>> long-term > >>>>>>>>> robustness of the design. > >>>>>>>>> > >>>>>>>>> Best, > >>>>>>>>> > >>>>>>>>> Xingcan > >>>>>>>>> > >>>>>>>>> On Thu, May 28, 2026 at 8:17 AM David Anderson < > >> [email protected] > >>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I'm quite enthusiastic about this. I want to thank Fabian for > >>>>>> putting > >>>>>>>>>> together such a well-crafted FLIP. And I look forward to > updating > >>>>>> the > >>>>>>>>>> awkward educational content this FLIP will make obsolete. > >>>>>>>>>> > >>>>>>>>>> To my mind, the syntax expresses the semantics of this join > rather > >>>>>>> well. > >>>>>>>>>> > >>>>>>>>>> Until now, developers using event-time temporal joins sometimes > >>>>>>>>> resorted to > >>>>>>>>>> doing weird things with watermarks to handle a build side that's > >>>>>>> mostly > >>>>>>>>>> idle; this lateral snapshot join is clearly better -- not to > >> mention > >>>>>>> the > >>>>>>>>>> added bonus of pre-loading the build table. > >>>>>>>>>> > >>>>>>>>>> One question: If I understand correctly, during the JOIN phase > of > >> an > >>>>>>>>> INNER > >>>>>>>>>> join, if the desired build-side record is missing, nothing will > be > >>>>>>>>> emitted > >>>>>>>>>> for the unmatched probe-side record. For an INNER join, I can > >>>>>> imagine > >>>>>>>>>> wanting to buffer unmatched probe-side records, expecting the > >> build > >>>>>>> side > >>>>>>>>>> will arrive soon. What's your thinking there? > >>>>>>>>>> > >>>>>>>>>> David > >>>>>>>>>> > >>>>>>>>>> On Wed, May 27, 2026 at 12:44 PM Fabian Hueske < > >> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Thanks Gustavo and Timo for the positive feedback! > >>>>>>>>>>> > >>>>>>>>>>> I'd like to bump this thread up to collect more feedback. > >>>>>>>>>>> If there are no more responses, I will start a vote on this > FLIP > >>>>>>> next > >>>>>>>>>>> Monday, June 1st. > >>>>>>>>>>> > >>>>>>>>>>> Best, Fabian > >>>>>>>>>>> > >>>>>>>>>>> Am Do., 21. Mai 2026 um 12:15 Uhr schrieb Timo Walther < > >>>>>>>>>> [email protected] > >>>>>>>>>>>> : > >>>>>>>>>>> > >>>>>>>>>>>> Hi Fabian, > >>>>>>>>>>>> > >>>>>>>>>>>> thanks for proposing this FLIP. I agree that this join is > super > >>>>>>>>> common, > >>>>>>>>>>>> after talking to many people at conferences, I could imagine > it > >>>>>>>>> will be > >>>>>>>>>>>> one of the most used kinds of joins going forward. > >>>>>>>>>>>> > >>>>>>>>>>>> Tightly coupling it with watermarks fits both from a > semantical > >>>>>>>>> point > >>>>>>>>>> of > >>>>>>>>>>>> view but also with other efforts such as FLIP-558 > (Improvements > >>>>>> to > >>>>>>>>>>>> SinkUpsertMaterializer and changelog disorder) [1]. In the > near > >>>>>>>>> future, > >>>>>>>>>>>> we should work on more automated watermarking to power these > >>>>>>>>>>>> watermark-based operators, but this is an orthogonal effort. > >>>>>>>>>>>> > >>>>>>>>>>>> Overall I'm strongly +1 on this. Also +1 on the syntax > >>>>>>> improvements > >>>>>>>>> for > >>>>>>>>>>>> lateral table functions by dropping the TABLE() wrapper. > >>>>>>>>>>>> > >>>>>>>>>>>> Cheers, > >>>>>>>>>>>> Timo > >>>>>>>>>>>> > >>>>>>>>>>>> [1] > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-558%3A+Improvements+to+SinkUpsertMaterializer+and+changelog+disorder > >>>>>>>>>>>> > >>>>>>>>>>>> On 18.05.26 11:47, Gustavo de Morais wrote: > >>>>>>>>>>>>> Hi Fabian, > >>>>>>>>>>>>> > >>>>>>>>>>>>> In general a strong +1 for the feature, without getting into > >>>>>> the > >>>>>>>>>>> details > >>>>>>>>>>>> of > >>>>>>>>>>>>> the FLIP yet. This is a missing feature for years and I'm > >>>>>> happy > >>>>>>>>> that > >>>>>>>>>>>> we're > >>>>>>>>>>>>> putting the time to address this - while also getting rid of > >>>>>>> some > >>>>>>>>> of > >>>>>>>>>>> the > >>>>>>>>>>>>> hard restrictions we had. Thanks! > >>>>>>>>>>>>> > >>>>>>>>>>>>> Kind regards, > >>>>>>>>>>>>> Gustavo > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, 15 May 2026 at 16:39, Fabian Hueske < > >>>>>> [email protected] > >>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi everyone, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I'd like to start a discussion on FLIP-579: LATERAL SNAPSHOT > >>>>>>> Join > >>>>>>>>>> [1]. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Enriching a stream with data from a (slowly changing) > dynamic > >>>>>>>>> table > >>>>>>>>>>> is a > >>>>>>>>>>>>>> super common use case. > >>>>>>>>>>>>>> Flink SQL features Temporal Joins [2] to address these use > >>>>>>> cases. > >>>>>>>>>>>>>> However, SQL users can only use the event-time variant which > >>>>>>> has > >>>>>>>>>> many > >>>>>>>>>>>>>> limitations (heavy dependency on frequent WM updates on both > >>>>>>>>> inputs, > >>>>>>>>>>>>>> build-side table requires a PK, the join predicate must > >>>>>> include > >>>>>>>>> the > >>>>>>>>>>>>>> build-side PK, etc). > >>>>>>>>>>>>>> The processing-time temporal join is disabled (due to > >>>>>>> build-side > >>>>>>>>>>>>>> initialization issues [3]) and temporal table function joins > >>>>>>> are > >>>>>>>>>>>>>> only available in Table API. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> FLIP-579 proposes a new temporal join operator that operates > >>>>>> in > >>>>>>>>>>>>>> processing-time and addresses the limitations of the > existing > >>>>>>>>>>>>>> implementations: > >>>>>>>>>>>>>> * initialization of the build-side before joining > >>>>>>>>>>>>>> * no requirement of continuous, frequent build-side WMs > >>>>>> (after > >>>>>>>>> the > >>>>>>>>>>>>>> initialization completed) > >>>>>>>>>>>>>> * no requirement for a PK on the build-side > >>>>>>>>>>>>>> * table function-based syntax [4] via a built-in SNAPSHOT > >>>>>>>>> function > >>>>>>>>>>>>>> (proposed in FLIP-517 [4]) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Looking forward to your feedback. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>> Fabian > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [1] > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-579%3A+LATERAL+SNAPSHOT+Join > >>>>>>>>>>>>>> [2] > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/joins/#temporal-joins > >>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/FLINK-19830 > >>>>>>>>>>>>>> [4] > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/joins/#temporal-table-function-join > >>>>>>>>>>>>>> [5] > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-517%3A+Better+Handling+of+Dynamic+Table+Primitives+with+PTFs#FLIP517:BetterHandlingofDynamicTablePrimitiveswithPTFs-SNAPSHOTfortemporaljoins > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >>>> > >> > >> > >
