[
https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578487#comment-15578487
]
Olivier Armand commented on SPARK-650:
--------------------------------------
Sean, a singleton is not the best option in our case. The Spark Streaming
executors are writing to HBase, we need to initialize the HBase connection. The
singleton seems (or seemed when we tested it for our customer a few months
after this issue was raised) to be created when the first RDD is processed by
the executor, and not when the driver starts. This imposes very high processing
time for the first events.
> Add a "setup hook" API for running initialization code on each executor
> -----------------------------------------------------------------------
>
> Key: SPARK-650
> URL: https://issues.apache.org/jira/browse/SPARK-650
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Reporter: Matei Zaharia
> Priority: Minor
>
> Would be useful to configure things like reporting libraries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
