[
https://issues.apache.org/jira/browse/HIVE-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brock Noland updated HIVE-7606:
-------------------------------
Description:
In this JIRA we'll design two interfaces:
* SparkSessionState
* SparkSessionPoolManager
and then once that is agreed upon we'll design two implementations:
* SparkSessionStateImpl
* SparkSessionPoolManagerImpl
the form and function of these will be similar to the Tez equivalents. However,
TezSessionState provides some implementation which SparkClient already provides
(refreshLocalResources*). Let's keep SparkSessionState lightweight and not
remove functionality from SparkClient. The scope of this jira is just to create
the shells and basic functionality. The implementations in this jira should be
able to:
* Share a SparkSessionImpl across queries
* Defining when a session can be re-used
* Take ownership of SparkContext objects (Note we can only have a single SC
until SPARK-2243 is resolved)
was:
In this JIRA we'll design two interfaces:
* SparkSession
* SparkSessionPoolManager
and then once that is agreed upon we'll design two implementations:
* SparkSessionImpl
* SparkSessionPoolManagerImpl
the form and function of these will be similar to the Tez equivalents. However,
it appears as though TezSession provides both the SessionState and the "client"
Thus I think SparkSession owns SparkClient but the implementation of what
SparkClient does should stay there and not bleed into SparkSession. The scope
of this jira is just to create the shells and basic functionality. The
implementations in this jira should be able to:
* Share a SparkSessionImpl across queries
* Defining when a session can be re-used
* Take ownership of SparkContext objects (Note we can only have a single SC
until SPARK-2243 is resolved)
> Design SparkSession, SparkSessionManager
> ----------------------------------------
>
> Key: HIVE-7606
> URL: https://issues.apache.org/jira/browse/HIVE-7606
> Project: Hive
> Issue Type: Task
> Reporter: Brock Noland
>
> In this JIRA we'll design two interfaces:
> * SparkSessionState
> * SparkSessionPoolManager
> and then once that is agreed upon we'll design two implementations:
> * SparkSessionStateImpl
> * SparkSessionPoolManagerImpl
> the form and function of these will be similar to the Tez equivalents.
> However, TezSessionState provides some implementation which SparkClient
> already provides (refreshLocalResources*). Let's keep SparkSessionState
> lightweight and not remove functionality from SparkClient. The scope of this
> jira is just to create the shells and basic functionality. The
> implementations in this jira should be able to:
> * Share a SparkSessionImpl across queries
> * Defining when a session can be re-used
> * Take ownership of SparkContext objects (Note we can only have a single SC
> until SPARK-2243 is resolved)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
