Hi Preeze,
> Is there any designed way that the client connects back to the driver
(still running in YARN) for collecting results at a later stage?
No, there is not support built into Spark for this. For this to happen
seamlessly the driver will have to start a server (pull model) or send the
res
"in yarn-client mode it only controls the environment of the executor
launcher"
So you either use yarn-client mode, and then your app keeps running and
controlling the process
Or you use yarn-cluster mode, and then you send a jar to YARN, and that jar
should have code to report the result back to
