Since the SQL will pass through Spark’s Catalyst SQL optimizer, I suspect that most of the optimizations Calcite has made will be negated.
The ideal thing would be to translate the Calcite plan (RelNodes) into a Spark program, which would then be executed without further optimization. Calcite’s Spark adapter does that (but is old and incomplete); I heard rumors of an improved Spark adapter but it hasn’t landed in open source yet, and I don’t know when/whether that will happen. Julian > On Apr 21, 2017, at 3:50 PM, Khai Tran <[email protected]> wrote: > > Hi all, > I'm trying to figure out some way to execute a given Calcite logical plan > on Spark. One possible way to convert the Calcite plan back to SQL and run > it as Spark SQL. > > Any ideas on that? > > Thanks, > Khai
