I think this is SPARK-1949 again: https://github.com/apache/spark/pull/906
I think this change fixed this issue for a few people using the SBT
build, worth committing?
On Thu, Jun 5, 2014 at 6:40 AM, Shrikar archak <shrika...@gmail.com> wrote:
> Hi All,
> Now that the Spark Version 1.0.0 is release there should not be any problem
> with the local jars.
> Shrikars-MacBook-Pro:SimpleJob shrikar$ cat simple.sbt
> name := "Simple Project"
>
> version := "1.0"
>
> scalaVersion := "2.10.4"
>
> libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" % "1.0.0",
> "org.apache.spark" %% "spark-streaming" %
> "1.0.0")
>
> resolvers += "Akka Repository" at "http://repo.akka.io/releases/";
>
> I am still having this issue
> [error] (run-main) java.lang.NoClassDefFoundError:
> javax/servlet/http/HttpServletResponse
> java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletResponse
> at org.apache.spark.HttpServer.start(HttpServer.scala:54)
> at
> org.apache.spark.broadcast.HttpBroadcast$.createServer(HttpBroadcast.scala:156)
> at
> org.apache.spark.broadcast.HttpBroadcast$.initialize(HttpBroadcast.scala:127)
> at
> org.apache.spark.broadcast.HttpBroadcastFactory.initialize(HttpBroadcastFactory.scala:31)
> at
> org.apache.spark.broadcast.BroadcastManager.initialize(BroadcastManager.scala:48)
> at
> org.apache.spark.broadcast.BroadcastManager.<init>(BroadcastManager.scala:35)
> at org.apache.spark.SparkEnv$.create(SparkEnv.scala:218)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:202)
>
> Any help would be greatly appreciated.
>
> Thanks,
> Shrikar
>
>
> On Fri, May 23, 2014 at 3:58 PM, Shrikar archak <shrika...@gmail.com> wrote:
>>
>> Still the same error no change
>>
>> Thanks,
>> Shrikar
>>
>>
>> On Fri, May 23, 2014 at 2:38 PM, Jacek Laskowski <ja...@japila.pl> wrote:
>>>
>>> Hi Shrikar,
>>>
>>> How did you build Spark 1.0.0-SNAPSHOT on your machine? My
>>> understanding is that `sbt publishLocal` is not enough and you really
>>> need `sbt assembly` instead. Give it a try and report back.
>>>
>>> As to your build.sbt, upgrade Scala to 2.10.4 and "org.apache.spark"
>>> %% "spark-streaming" % "1.0.0-SNAPSHOT" only that will pull down
>>> spark-core as a transitive dep. The resolver for Akka Repository is
>>> not needed. Your build.sbt should really look as follows:
>>>
>>> name := "Simple Project"
>>>
>>> version := "1.0"
>>>
>>> scalaVersion := "2.10.4"
>>>
>>> libraryDependencies += "org.apache.spark" %% "spark-streaming" %
>>> "1.0.0-SNAPSHOT"
>>>
>>> Jacek
>>>
>>> On Thu, May 22, 2014 at 11:27 PM, Shrikar archak <shrika...@gmail.com>
>>> wrote:
>>> > Hi All,
>>> >
>>> > I am trying to run the network count example as a seperate standalone
>>> > job
>>> > and running into some issues.
>>> >
>>> > Environment:
>>> > 1) Mac Mavericks
>>> > 2) Latest spark repo from Github.
>>> >
>>> >
>>> > I have a structure like this
>>> >
>>> > Shrikars-MacBook-Pro:SimpleJob shrikar$ find .
>>> > .
>>> > ./simple.sbt
>>> > ./src
>>> > ./src/main
>>> > ./src/main/scala
>>> > ./src/main/scala/NetworkWordCount.scala
>>> > ./src/main/scala/SimpleApp.scala.bk
>>> >
>>> >
>>> > simple.sbt
>>> > name := "Simple Project"
>>> >
>>> > version := "1.0"
>>> >
>>> > scalaVersion := "2.10.3"
>>> >
>>> > libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" %
>>> > "1.0.0-SNAPSHOT",
>>> > "org.apache.spark" %% "spark-streaming" %
>>> > "1.0.0-SNAPSHOT")
>>> >
>>> > resolvers += "Akka Repository" at "http://repo.akka.io/releases/";
>>> >
>>> >
>>> > I am able to run the SimpleApp which is mentioned in the doc but when I
>>> > try
>>> > to run the NetworkWordCount app I get error like this am I missing
>>> > something?
>>> >
>>> > [info] Running com.shrikar.sparkapps.NetworkWordCount
>>> > 14/05/22 14:26:47 INFO spark.SecurityManager: Changing view acls to:
>>> > shrikar
>>> > 14/05/22 14:26:47 INFO spark.SecurityManager: SecurityManager:
>>> > authentication disabled; ui acls disabled; users with view permissions:
>>> > Set(shrikar)
>>> > 14/05/22 14:26:48 INFO slf4j.Slf4jLogger: Slf4jLogger started
>>> > 14/05/22 14:26:48 INFO Remoting: Starting remoting
>>> > 14/05/22 14:26:48 INFO Remoting: Remoting started; listening on
>>> > addresses
>>> > :[akka.tcp://spark@192.168.10.88:49963]
>>> > 14/05/22 14:26:48 INFO Remoting: Remoting now listens on addresses:
>>> > [akka.tcp://spark@192.168.10.88:49963]
>>> > 14/05/22 14:26:48 INFO spark.SparkEnv: Registering MapOutputTracker
>>> > 14/05/22 14:26:48 INFO spark.SparkEnv: Registering BlockManagerMaster
>>> > 14/05/22 14:26:48 INFO storage.DiskBlockManager: Created local
>>> > directory at
>>> >
>>> > /var/folders/r2/mbj08pb55n5d_9p8588xk5b00000gn/T/spark-local-20140522142648-0a14
>>> > 14/05/22 14:26:48 INFO storage.MemoryStore: MemoryStore started with
>>> > capacity 911.6 MB.
>>> > 14/05/22 14:26:48 INFO network.ConnectionManager: Bound socket to port
>>> > 49964
>>> > with id = ConnectionManagerId(192.168.10.88,49964)
>>> > 14/05/22 14:26:48 INFO storage.BlockManagerMaster: Trying to register
>>> > BlockManager
>>> > 14/05/22 14:26:48 INFO storage.BlockManagerInfo: Registering block
>>> > manager
>>> > 192.168.10.88:49964 with 911.6 MB RAM
>>> > 14/05/22 14:26:48 INFO storage.BlockManagerMaster: Registered
>>> > BlockManager
>>> > 14/05/22 14:26:48 INFO spark.HttpServer: Starting HTTP Server
>>> > [error] (run-main) java.lang.NoClassDefFoundError:
>>> > javax/servlet/http/HttpServletResponse
>>> > java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletResponse
>>> > at org.apache.spark.HttpServer.start(HttpServer.scala:54)
>>> > at
>>> >
>>> > org.apache.spark.broadcast.HttpBroadcast$.createServer(HttpBroadcast.scala:156)
>>> > at
>>> >
>>> > org.apache.spark.broadcast.HttpBroadcast$.initialize(HttpBroadcast.scala:127)
>>> > at
>>> >
>>> > org.apache.spark.broadcast.HttpBroadcastFactory.initialize(HttpBroadcastFactory.scala:31)
>>> > at
>>> >
>>> > org.apache.spark.broadcast.BroadcastManager.initialize(BroadcastManager.scala:48)
>>> > at
>>> >
>>> > org.apache.spark.broadcast.BroadcastManager.<init>(BroadcastManager.scala:35)
>>> > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:218)
>>> > at org.apache.spark.SparkContext.<init>(SparkContext.scala:202)
>>> > at
>>> >
>>> > org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:549)
>>> > at
>>> >
>>> > org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:561)
>>> > at
>>> >
>>> > org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:91)
>>> > at
>>> > com.shrikar.sparkapps.NetworkWordCount$.main(NetworkWordCount.scala:39)
>>> > at com.shrikar.sparkapps.NetworkWordCount.main(NetworkWordCount.scala)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > at
>>> >
>>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> >
>>> >
>>> > Thanks,
>>> > Shrikar
>>> >
>>>
>>>
>>>
>>> --
>>> Jacek Laskowski | http://blog.japila.pl
>>> "Never discourage anyone who continually makes progress, no matter how
>>> slow." Plato
>>
>>
>
