Also, thanks for everyone’s help so far!
> On Jan 13, 2015, at 2:04 PM, Ganon Pierce <ganon.pie...@me.com> wrote:
>
> Yeah upon running the test locally I receive:
>
> "Pi is roughly 3.139948”
>
> So spark is working, it’s just the application ui that is not…
>
>
>> On Jan 13, 2015, at 1:13 PM, Ganon Pierce <ganon.pie...@me.com
>> <mailto:ganon.pie...@me.com>> wrote:
>>
>> My application logs remain stored as .inprogress files, e.g.
>> "app-20150113190025-0004.inprogress” even after completion, could this have
>> something to do with what is going on.
>>
>> @ Ted Yu
>>
>> Where do I find the master log? It’s not very obviously labeled in my /tmp/
>> directory. Sorry if I should know, I’ve read through the documentation many
>> times, but probably missed it. Do I need to set this in my configuration
>> somehow? Could the problem actually be that I’m just not setting a conf
>> property properly somewhere?
>>
>> @Robin East
>>
>> All of these work fine, the only issue I’m having is viewing the application
>> ui after I submit a job to run. Further, the application runs and will even
>> compute some of the outputs I want (they get stored to s3 and I’ve used the
>> outputs in other models I’m making). It’s simply when I access the master
>> web ui, click the application that is currently running, and then click
>> “Application Detail UI” that I receive this error. When I an application has
>> completed and I click the same link I receive:
>>
>> “Application history not found (app-201501131190025-004)"
>>
>>> On Jan 13, 2015, at 8:07 AM, Ted Yu <yuzhih...@gmail.com
>>> <mailto:yuzhih...@gmail.com>> wrote:
>>>
>>> Gabon:
>>> Can you check the master log to see if there is some clue ?
>>>
>>> Cheers
>>>
>>>
>>>
>>> On Jan 13, 2015, at 2:03 AM, Robin East <robin.e...@xense.co.uk
>>> <mailto:robin.e...@xense.co.uk>> wrote:
>>>
>>>> I’ve just pulled down the latest commits from github, and done the
>>>> following:
>>>>
>>>> 1)
>>>> mvn clean package -DskipTests
>>>>
>>>> builds fine
>>>>
>>>> 2)
>>>> ./bin/spark-shell works
>>>>
>>>> 3)
>>>> run SparkPi example with no problems:
>>>>
>>>> ./bin/run-example SparkPi 10
>>>>
>>>> 4)
>>>> Started a master
>>>>
>>>> ./sbin/start-master.sh
>>>>
>>>> grabbed the MasterWebUI from the master log - Started MasterWebUI at
>>>> http://x.x.x.x:8080 <http://x.x.x.x:8080/>
>>>>
>>>> Can view the MasterWebUI from local browser
>>>>
>>>> 5)
>>>> grabbed the spark url from the master log and started a local slave:
>>>>
>>>> ./bin/spark-class org.apache.spark.deploy.worker.Worker
>>>> spark://<hostname>:7077 &
>>>>
>>>> 6)
>>>> Ran jps to confirm both Master and Worker processes are present.
>>>>
>>>> 7)
>>>> Ran SparkPi on the mini-cluster:
>>>>
>>>> MASTER=spark://<host>:7077 ./bin/run-example SparkPi 10
>>>>
>>>> All worked fine, can see information in the MasterWebUI
>>>>
>>>> Which of these stops doesn’t work for you? I presume you’ve tried
>>>> re-pulling from git and a clean build again.
>>>>
>>>> Robin
>>>> On 13 Jan 2015, at 08:07, Ganon Pierce <ganon.pie...@me.com
>>>> <mailto:ganon.pie...@me.com>> wrote:
>>>>
>>>>> After clean build still receiving the same error.
>>>>>
>>>>>
>>>>>
>>>>> On Jan 6, 2015, at 3:59 PM, Sean Owen <so...@cloudera.com
>>>>> <mailto:so...@cloudera.com>> wrote:
>>>>>
>>>>>> FWIW I do not see any such error, after a "mvn -DskipTests clean
>>>>>> package" and "./bin/spark-shell" from master. Maybe double-check you
>>>>>> have done a full clean build.
>>>>>>
>>>>>> On Tue, Jan 6, 2015 at 9:09 PM, Ganon Pierce <ganon.pie...@me.com
>>>>>> <mailto:ganon.pie...@me.com>> wrote:
>>>>>> I’m attempting to build from the latest commit on git and receive the
>>>>>> following error upon attempting to access the application web ui:
>>>>>>
>>>>>> HTTP ERROR: 500
>>>>>>
>>>>>> Problem accessing /jobs/. Reason:
>>>>>>
>>>>>> Server Error
>>>>>> Powered by Jetty://
>>>>>>
>>>>>> My driver also prints this error:
>>>>>>
>>>>>> java.lang.UnsupportedOperationException: empty.max
>>>>>> at scala.collection.TraversableOnce$class.max(TraversableOnce.scala:216)
>>>>>> at scala.collection.AbstractTraversable.max(Traversable.scala:105)
>>>>>> at org.apache.spark.ui.jobs.AllJobsPage.org
>>>>>> <http://org.apache.spark.ui.jobs.alljobspage.org/>$apache$spark$ui$jobs$AllJobsPage$$makeRow$1(AllJobsPage.scala:46)
>>>>>> at
>>>>>> org.apache.spark.ui.jobs.AllJobsPage$$anonfun$jobsTable$1.apply(AllJobsPage.scala:91)
>>>>>> at
>>>>>> org.apache.spark.ui.jobs.AllJobsPage$$anonfun$jobsTable$1.apply(AllJobsPage.scala:91)
>>>>>> at
>>>>>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>>>>>> at
>>>>>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>>>>>> at scala.collection.immutable.List.foreach(List.scala:318)
>>>>>> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>>>>>> at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>>>>>> at org.apache.spark.ui.jobs.AllJobsPage.jobsTable(AllJobsPage.scala:91)
>>>>>> at org.apache.spark.ui.jobs.AllJobsPage.render(AllJobsPage.scala:106)
>>>>>> at org.apache.spark.ui.WebUI$$anonfun$attachPage$1.apply(WebUI.scala:68)
>>>>>> at org.apache.spark.ui.WebUI$$anonfun$attachPage$1.apply(WebUI.scala:68)
>>>>>> at org.apache.spark.ui.JettyUtils$$anon$1.doGet(JettyUtils.scala:68)
>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>>>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>>>>>> at
>>>>>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
>>>>>> at
>>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501)
>>>>>> at
>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
>>>>>> at
>>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
>>>>>> at
>>>>>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
>>>>>> at
>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>>>>>> at
>>>>>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>>>>>> at
>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>>>>>> at org.eclipse.jetty.server.Server.handle(Server.java:370)
>>>>>> at
>>>>>> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
>>>>>> at
>>>>>> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971)
>>>>>> at
>>>>>> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033)
>>>>>> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
>>>>>> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
>>>>>> at
>>>>>> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
>>>>>> at
>>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:667)
>>>>>> at
>>>>>> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
>>>>>> at
>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>>>>>> at
>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>
>>>>>>
>>>>>> Has the ui been disabled intentionally for development purposes, have I
>>>>>> not set something up correctly, or is this a bug?
>>>>>>
>>>>
>>
>
