The CallbackServer is part of Py4j, it's only used in driver, not used in slaves or workers.
On Wed, Feb 11, 2015 at 12:32 AM, Todd Gao <todd.gao.2013+sp...@gmail.com> wrote: > Hi all, > > I am reading the code of PySpark and its Streaming module. > > In PySpark Streaming, when the `compute` method of the instance of > PythonTransformedDStream is invoked, a connection to the CallbackServer > is created internally. > I wonder where is the CallbackServer for each PythonTransformedDStream > instance on the slave nodes in distributed environment. > Is there a CallbackServer running on every slave node? > > thanks > Todd --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
