Re: Spark Job History Server

Hi Patcharee, It is an alpha feature in HDP distribution, integrating ATS with Spark history server. If you are using upstream, you can configure spark as regular without these configuration. But other related configuration are still mandatory, such as hdp.version related. Thanks. Zhan Zhang

Re: Spark Job History Server

Uh, does that mean HDP shipped Marcelo's uncommitted patch from SPARK-1537 anyway? Given the discussion there, that seems kinda aggressive. On Wed, Mar 18, 2015 at 8:49 AM, Marcelo Vanzin wrote: > Those classes are not part of standard Spark. You may want to contact > Hortonworks directly if they

Re: Spark Job History Server

Those classes are not part of standard Spark. You may want to contact Hortonworks directly if they're suggesting you use those. On Wed, Mar 18, 2015 at 3:30 AM, patcharee wrote: > Hi, > > I am using spark 1.3. I would like to use Spark Job History Server. I added > the following line into conf/sp

Re: Spark Job History Server

Hi, My spark was compiled with yarn profile, I can run spark on yarn without problem. For the spark job history server problem, I checked spark-assembly-1.3.0-hadoop2.4.0.jar and found that the package org.apache.spark.deploy.yarn.history is missing. I don't know why BR, Patcharee On

Re: Spark Job History Server

You are not having yarn package in the classpath. You need to build your spark it with yarn. You can read these docs. Thanks Best Regards On Wed, Mar 18, 2015 at 4:07 PM, patcharee wrote: > I turned it on. But it failed to start. In the

Re: Spark Job History Server

I turned it on. But it failed to start. In the log, Spark assembly has been built with Hive, including Datanucleus jars on classpath Spark Command: /usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin/java -cp :/root/spark-1.3.0-bin-hadoop2.4/sbin/../conf:/root/spark-1.3.0-bin-hadoop2.4/lib/spark-assembl

Re: Spark Job History Server

You can simply turn it on using: ./sbin/start-history-server.sh ​Read more here .​ Thanks Best Regards On Wed, Mar 18, 2015 at 4:00 PM, patcharee wrote: > Hi, > > I am using spark 1.3. I would like to use Spark Job History Server. I > adde