Forgot to mention, Iam using flink version 1.18.1 On Wed, Apr 23, 2025 at 1:00 AM Yashoda Krishna T < yashoda.kris...@netcoreunbxd.com> wrote:
> Hi, > I have tried using lateral join on two tables in flink sql similar to the > example > > > https://github.com/apache/flink/blob/c724168fad4215626b5596dd63cb66e477948aa0/flink-examples/flink-examples-table/src/main/java/org/apache/flink/table/examples/java/basics/UpdatingTopCityExample.java#L130 > > But I get the the following error > Caused by: org.apache.flink.table.api.TableException: unexpected correlate > variable $cor0 in the plan > > And to confirm that this error is not specific to my use case, I have > tried the flink's UpdatingTopCityExample too with flink deployment on flink > kubernetes operator and I got the same error. > > I have found that there is an unresolved bug that is causing this issue. > https://issues.apache.org/jira/browse/FLINK-31163 > > As a workaround I have tried using a left join, but unlike a lateral join > where I can specify the limit in the correlated subquery, the same cannot > be done in left join, instead I have to assign a rownumber and then filter > on rownumber to reduce the join output. And this is not the optimal way to > do incase of datasets with millions of records. > > Is there an optimal way of doing this in huge datasets? > Or is there a workaround similar to lateral join where the join does not > need to calculate all the matches? > > Any help would be much appreciated. > > Thanks > Yashoda > >
