Hello all, I am attempting to manually build the master branch of Spark against Cloudera’s 5.2.0 deployment. To do this I am running:
mvn -Pyarn -Dhadoop.version=2.5.0-cdh5.2.0 -DskipTests clean package The build completes successfully and then I run: mvn -Pyarn -Phadoop.version=2.5.0-cdh5.2.0 test Both on the cluster and on the local deployment, however, this sequence fails. Locally, I have better luck with the tests but there are still failures. On the cluster I get some passed tests but also numerous failures. Summary is below: [INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM .......................... SUCCESS [ 2.481 s] [INFO] Spark Project Core ................................ FAILURE [11:02 min] [INFO] Spark Project Bagel ............................... SKIPPED [INFO] Spark Project GraphX .............................. SKIPPED [INFO] Spark Project Streaming ........................... SKIPPED [INFO] Spark Project ML Library .......................... SKIPPED [INFO] Spark Project Tools ............................... SKIPPED [INFO] Spark Project Catalyst ............................ SKIPPED [INFO] Spark Project SQL ................................. SKIPPED [INFO] Spark Project Hive ................................ SKIPPED [INFO] Spark Project REPL ................................ SKIPPED [INFO] Spark Project YARN Parent POM ..................... SKIPPED [INFO] Spark Project YARN Stable API ..................... SKIPPED [INFO] Spark Project Assembly ............................ SKIPPED [INFO] Spark Project External Twitter .................... SKIPPED [INFO] Spark Project External Kafka ...................... SKIPPED [INFO] Spark Project External Flume Sink ................. SKIPPED [INFO] Spark Project External Flume ...................... SKIPPED [INFO] Spark Project External ZeroMQ ..................... SKIPPED [INFO] Spark Project External MQTT ....................... SKIPPED [INFO] Spark Project Examples ............................ SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 11:06 min [INFO] Finished at: 2014-10-28T17:21:27-05:00 [INFO] Final Memory: 69M/301M [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "hadoop.version=2.5.0-cdh5.2.0" could not be activated because it does not exist. [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project spark-core_2.10: There are test failures -> [Help 1] What I would like to know is whether spark tests have been successfully run against the Cloudera deployment? Please let me know, thank you all. ________________________________________________________ The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
