[Announcement] Planned removal of Spark 2 runner support in 2.46.0

Dear All, The runner for Spark 2 was deprecated quite a while back in August 2022 with the release of Beam 2.41.0 [1]. We’re planning to move ahead with this and finally remove support for Spark 2 (beam-runners-spark) to only maintain support for Spark 3 (beam-runners-spark-3) going forward. N

Re: Beam SQL Alias issue while using With Clause

I am not sure I understand the question, but I do see an issue. Context: "CalcRel" is an optimized relational operation that is somewhat like ParDo, with a small snippet of a single-assignment DSL embedded in it. Calcite will choose to merge all the projects and filters into the node, and then gen

Re: Beam SQL Alias issue while using With Clause

Looking at the code that turns a logical CalcRel into a BeamCalcRel I do not see any obvious cause for this: https://github.com/apache/beam/blob/b3aa2e89489898f8c760294ba4dba2310ac53e70/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/BeamCalcRule.java#L69 I don'