unsubscribe
Hi all,
Did anyone get a chance to look into it??
Any sort of guidance will be much appreciated.
Thanks,
Amit Rana
On 7 Jul 2016 14:28, "Amit Rana" wrote:
> As mentioned in the documentation:
> PythonRDD objects launch Python subprocesses and communicate with them
> using
guide me along that line??
Thanks,
Amit Rana
On 7 Jul 2016 13:44, "Sun Rui" wrote:
> You can read
> https://cwiki.apache.org/confluence/display/SPARK/PySpark+Internals
> For pySpark data flow on worker nodes, you can read the source code of
> PythonRDD.scala. Python worker
executors and
python interpreter happens using pipes. But I am not able to trace that
data flow.
Please correct me if my understanding is wrong. It would be very helpful
if, someone can help me understand tge code-flow for data transfer between
JVM and python workers.
Thanks,
Amit Rana
development environment
for the same in intellij IDEA in Windows 7.
Thanks,
Amit Rana
