Re: Is there any way to perform left outer join using flink's data stream APIs

2024-08-06 Thread Kirill Ternovsky
Hard to give any specifics without knowing how your data is being streamed in. If both sources are in the same datastore, you can always join at the data source and then emit the joined (left or otherwise) records to your stream processor. On the other hand, if only Stream A or B is coming in,

Re: Is there any way to perform left outer join using flink's data stream APIs

2024-08-06 Thread Sachin Mittal
So, I have a question related to windowing: None of the records from A or B are timestamped. The collections are backed by two master tables (each containing say a few million records) in a database and I want to use flink to basically join them and create an enriched stream C. Hence I won't be a

Re: Is there any way to perform left outer join using flink's data stream APIs

2024-08-06 Thread Kirill Ternovsky
Hi Sachin, You should be able to use `coGroup`: ``` streamA.coGroup(streamB) .where(StreamAFieldFSelector).isEqualTo(StreamBFieldFSelector) .window(/*your windowing spec*/) .apply(/*your cogroup function*/) ``` Your coGroup function will look something like: ``` @Override public void

Is there any way to perform left outer join using flink's data stream APIs

2024-08-06 Thread Sachin Mittal
Hi, I have two streams A and B. Which can be joined or connected using a field f. However, for a given record in A for f = f1, there does not exist any record in B matching this field f = f1. In such cases I want to do a left outer join where the combined record pushed downstream would only have