So to answer my own question. It is a bug and there is unmerged PR for that
already.
https://issues.apache.org/jira/browse/SPARK-2624
https://github.com/apache/spark/pull/3238
Jakub
-- Původní zpráva --
Od: spark.dubovsky.ja...@seznam.cz
Komu: spark.dubovsky.ja...@seznam.cz
Dat
Hi,
I had time to try it again. I submited my app by the same command with
these additional options:
--jars lib/datanucleus-api-jdo-3.2.6.jar,lib/datanucleus-core-3.2.10.jar,
lib/datanucleus-rdbms-3.2.9.jar
Now an app successfully creates hive context. So my question remains: Is
"classpa
Next try. I copied whole dist directory created by make-distribution script
to cluster not just assembly jar. Then I used
./bin/spark-submit --num-executors 200 --master yarn-cluster --class org.
apache.spark.mllib.CreateGuidDomainDictionary ../spark/root-0.1.jar ${args}
...to run app again. St
Hi again,
I have tried to recompile and run this again with new assembly created by
./make-distribution.sh -Phadoop-2.3 -Dhadoop.version=2.3.0-cdh5.1.3 -Pyarn -
Phive -DskipTests
It results in exactly the same error. Any other hints?
Bonus question: Should the class org.datanucleus.api.jdo.
JDOP
Hi all,
I have created assembly jar from 1.2 snapshot source by running [1] which
sets correct version of hadoop for our cluster and uses hive profile. I also
have written relatively simple test program which starts by reading data
from parquet using hive context. I compile the code against as
