Hi,
I second Luke here: you have to use Spark 1.x or use the PR supporting Spark
2.x.
Regards
JB
On 12/04/2017 08:14 PM, Lukasz Cwik wrote:
It seems like your trying to use Spark 2.1.0. Apache Beam currently relies on
users using Spark 1.6.3. There is an open pull request[1] to migrate to Spa
It seems like your trying to use Spark 2.1.0. Apache Beam currently relies
on users using Spark 1.6.3. There is an open pull request[1] to migrate to
Spark 2.2.0.
1: https://github.com/apache/beam/pull/4208/
On Mon, Dec 4, 2017 at 10:58 AM, Opitz, Daniel A
wrote:
> We are trying to submit a Spa
We are trying to submit a Spark job through YARN with the following command:
spark-submit --conf spark.yarn.stagingDir=/path/to/stage --verbose --class
com.my.class --jars /path/to/jar1,path/to/jar2 /path/to/main/jar/application.jar
The application is being populated in the YARN scheduler howev
