[
https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581774#comment-15581774
]
Sean Owen commented on SPARK-650:
---------------------------------
BTW I am not suggesting an "empty RDD" for your case. That was specific to the
streaming scenario.
For this, again, why not just access some initialization method during class
init of some class that is referenced wherever you want, including a custom
InputFormat? This can be made to happen once per JVM (class loader), from any
code, at class init time before anything else can happen. It's just a standard
Java mechanism.
If you mean it requires some configuration not available at class-loading time
you can still make such an init take place wherever, as soon as, such
configuration is available. Even in an InputFormat.
Although I can imagine corner cases where this becomes hard, I think it's
over-thinking this to imagine a whole new lifecycle method to accomplish what
basic JVM mechanisms allow.
> Add a "setup hook" API for running initialization code on each executor
> -----------------------------------------------------------------------
>
> Key: SPARK-650
> URL: https://issues.apache.org/jira/browse/SPARK-650
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Reporter: Matei Zaharia
> Priority: Minor
>
> Would be useful to configure things like reporting libraries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
