Can you check the Spark worker logs on that machine. Either from the web ui, or directly. Should be /test/spark-XXX/logs/ See if that has any error. If there is not permission issue, I am not why stdout and stderr is not being generated.
TD On Mon, May 5, 2014 at 7:13 PM, Francis.Hu <francis...@reachjunction.com>wrote: > The file does not exist in fact and no permission issue. > > > > francis@ubuntu-4:/test/spark-0.9.1$ ll work/app-20140505053550-0000/ > > total 24 > > drwxrwxr-x 6 francis francis 4096 May 5 05:35 ./ > > drwxrwxr-x 11 francis francis 4096 May 5 06:18 ../ > > drwxrwxr-x 2 francis francis 4096 May 5 05:35 2/ > > drwxrwxr-x 2 francis francis 4096 May 5 05:35 4/ > > drwxrwxr-x 2 francis francis 4096 May 5 05:35 7/ > > drwxrwxr-x 2 francis francis 4096 May 5 05:35 9/ > > > > Francis > > > > *发件人:* Tathagata Das [mailto:tathagata.das1...@gmail.com] > *发送时间:* Tuesday, May 06, 2014 3:45 > *收件人:* user@spark.apache.org > *主题:* Re: java.io.FileNotFoundException: > /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or > directory) > > > > Do those file actually exist? Those stdout/stderr should have the output > of the spark's executors running in the workers, and its weird that they > dont exist. Could be permission issue - maybe the directories/files are not > being generated because it cannot? > > > > TD > > > > On Mon, May 5, 2014 at 3:06 AM, Francis.Hu <francis...@reachjunction.com> > wrote: > > Hi,All > > > > > > We run a spark cluster with three workers. > > created a spark streaming application, > > then run the spark project using below command: > > > > shell> sbt run spark://192.168.219.129:7077 tcp://192.168.20.118:5556 foo > > > > we looked at the webui of workers, jobs failed without any error or info, > but FileNotFoundException occurred in workers' log file as below: > > Is this an existent issue of spark? > > > > > > -----------------------------in workers' > logs/spark-francis-org.apache.spark.deploy.worker.Worker-1-ubuntu-4.out---------------------------- > > > > 14/05/05 02:39:39 WARN AbstractHttpConnection: > /logPage/?appId=app-20140505053550-0000&executorId=2&logType=stdout > > java.io.FileNotFoundException: > /test/spark-0.9.1/work/app-20140505053550-0000/2/stdout (No such file or > directory) > > at java.io.FileInputStream.open(Native Method) > > at java.io.FileInputStream.<init>(FileInputStream.java:138) > > at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687) > > at > org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119) > > at > org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52) > > at > org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52) > > at > org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61) > > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040) > > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976) > > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > > at > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) > > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > > at org.eclipse.jetty.server.Server.handle(Server.java:363) > > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483) > > at > org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920) > > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982) > > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635) > > at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) > > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628) > > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) > > at java.lang.Thread.run(Thread.java:722) > > 14/05/05 02:39:41 WARN AbstractHttpConnection: > /logPage/?appId=app-20140505053550-0000&executorId=9&logType=stderr > > java.io.FileNotFoundException: > /test/spark-0.9.1/work/app-20140505053550-0000/9/stderr (No such file or > directory) > > at java.io.FileInputStream.open(Native Method) > > at java.io.FileInputStream.<init>(FileInputStream.java:138) > > at org.apache.spark.util.Utils$.offsetBytes(Utils.scala:687) > > at > org.apache.spark.deploy.worker.ui.WorkerWebUI.logPage(WorkerWebUI.scala:119) > > at > org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52) > > at > org.apache.spark.deploy.worker.ui.WorkerWebUI$$anonfun$6.apply(WorkerWebUI.scala:52) > > at > org.apache.spark.ui.JettyUtils$$anon$1.handle(JettyUtils.scala:61) > > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1040) > > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:976) > > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > > at > org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) > > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > > at org.eclipse.jetty.server.Server.handle(Server.java:363) > > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:483) > > at > org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:920) > > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:982) > > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635) > > at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) > > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628) > > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) > > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > Thanks, > > Francis.Hu > > >
