It is possible to start multiple concurrent drivers, Spark dynamically
allocates ports per "spark application" on driver, master, and workers from
a port range. When you collect results back to the driver, they do not go
through the master. The master is mostly there as a coordinator between the
dr
Hi,
as far as I understand you shouldn't send data to driver. Suppose you have
file in hdfs/s3 or cassandra partitioning, you should create your job such
that every executor/worker of spark will handle part of your input,
transform, filter it and at the end write back to cassandra as output(once
ag
