From what I read, there is no more Contexts.
"SparkContext, SQLContext, HiveContext merged into SparkSession"
I have not tested it, but I don’t know if it’s true.
Cheers,
Ben
> On Jul 18, 2016, at 8:37 AM, Koert Kuipers <[email protected]> wrote:
>
> in my codebase i would like to gradually transition to SparkSession, so while
> i start using SparkSession i also want a SQLContext to be available as before
> (but with a deprecated warning when i use it). this should be easy since
> SQLContext is now a wrapper for SparkSession.
>
> so basically:
> val session = SparkSession.builder.set(..., ...).getOrCreate()
> val sqlc = new SQLContext(session)
>
> however this doesnt work, the SQLContext constructor i am trying to use is
> private. SparkSession.sqlContext is also private.
>
> am i missing something?
>
> a non-gradual switch is not very realistic in any significant codebase, and i
> do not want to create SparkSession and SQLContext independendly (both from
> same SparkContext) since that can only lead to confusion and inconsistent
> settings.
