The SparkContext is a bit embarrassing. Most adapters are able to create a connection based on purely textual credentials, but the Spark adapter has to start off from an object. So, SparkContext is part of the Calcite instance.
It could perhaps be cleaned up, but on the other hand it is not doing us very much harm. We are not ’thinking to use Spark as a processing engine’. Effectively, we already do - but only if you use the Spark adapter. Julian > On Sep 30, 2020, at 4:00 PM, Bertty Contreras <[email protected]> wrote: > > Hi Guys, > > I'm thinking of using calcite on my project, but I found something that > bothers me about the incongruity from the documentation and some pieces of > the code . > > you said on the documentation "Calcite intentionally stays out of the > business of storing and > processing data"(https://calcite.apache.org/docs/index.html), but when i > was analyzing the code of the core I found one part (links below) where you > use SparkHandler or SparkContext. > > The question is, are you thinking to use spark in the near future as a > processing engine or that code needs to be there because of the adapter of > spark? If the answer is yes to the second question, why is it not on the > Spark adapter instead of the core ? > > Thank you so much for your time, > > Best regards > > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/jdbc/CalcitePrepare.java#L116-L117 > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/jdbc/CalcitePrepare.java#L132-L150 > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/jdbc/CalcitePrepare.java#L152-L233 > > https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/jdbc/CalcitePrepare.java#L173
