That makes sense. Thanks Alexey!
From: Alexey Romanenko
Date: Tuesday, July 6, 2021 at 10:33 AM
To: Tao Li
Cc: Yuchu Cao , "user@beam.apache.org"
Subject: Re: Beam Calcite SQL SparkRunner Performance
I think it’s quiet expected since Spark may push down the SQL query (or some
pa
I think it’s quiet expected since Spark may push down the SQL query (or some
parts of the query) to IO or/and RDD level and apply different type of
optimisations there, whereas Beam SQL translates an SQL query into the general
Beam pipeline which then is translated by SparkRunner into Spark pipe
