Re: Building spark 1.2 from source requires more dependencies

2015-03-30 Thread yash datta
Hi all, When selecting large data in sparksql (Select * query) , I see Buffer overflow exception from kryo : 15/03/27 10:32:19 WARN scheduler.TaskSetManager: Lost task 6.0 in stage 3.0 (TID 30, machine159): com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 1, required: 2 Seri

Re: Building spark 1.2 from source requires more dependencies

2015-03-29 Thread Sean Owen
Given that's it's an internal error from scalac, I think it may be something to take up with the Scala folks to really fix. We can just look for workarounds. Try blowing away your .m2 and .ivy cache for example. FWIW I was running on Linux with Java 8u31, latest scala 2.11 AFAIK. On Sun, Mar 29, 2

Re: Building spark 1.2 from source requires more dependencies

2015-03-29 Thread Pala M Muthaia
Sean, I did a mvn clean and then build, it produces the same error. I also did a fresh git clone of spark and invoked the same build command and it resulted in identical error (I also had a colleague do a same thing, lest there was some machine specific issue, and saw the same error). Unless i mis

Re: Building spark 1.2 from source requires more dependencies

2015-03-27 Thread Sean Owen
This is not a compile error, but an error from the scalac compiler. That is, the code and build are fine, but scalac is not compiling it. Usually when this happens, a clean build fixes it. On Fri, Mar 27, 2015 at 7:09 PM, Pala M Muthaia wrote: > No, i am running from the root directory, parent of

Re: Building spark 1.2 from source requires more dependencies

2015-03-27 Thread Pala M Muthaia
No, i am running from the root directory, parent of core. Here is the first set of errors that i see when i compile from source (sorry the error message is very long, but adding it in case it helps in diagnosis). After i manually add javax.servlet dependency for version 3.0, these set of errors g

Re: Building spark 1.2 from source requires more dependencies

2015-03-27 Thread Sean Owen
I built from the head of branch-1.2 and spark-core compiled correctly with your exact command. You have something wrong with how you are building. For example, you're not trying to run this from the core subdirectory are you? On Thu, Mar 26, 2015 at 10:36 PM, Pala M Muthaia wrote: > Hi, > > We ar

Re: Building spark 1.2 from source requires more dependencies

2015-03-26 Thread Pala M Muthaia
+spark-dev Yes, the dependencies are there. I guess my question is how come the build is succeeding in the mainline then, without adding these dependencies? On Thu, Mar 26, 2015 at 3:44 PM, Ted Yu wrote: > Looking at output from dependency:tree, servlet-api is brought in by the > following: > >