Re: More productive compilation of spark code base

2014-08-11 Thread Sean Owen
Try setting it to handle incremental compilation of Scala by itself (IntelliJ) and to run its own compile server. This is in global settings, under the Scala settings. It seems to compile incrementally for me when I change a file or two. On Mon, Aug 11, 2014 at 8:57 PM, Ron's Yahoo! wrote: > Hi,

Re: More productive compilation of spark code base

2014-08-11 Thread Stephen Boesch
Hi Ron, A possible recommendation is to use maven for the entire process (avoiding the sbt artifacts/processing). IJ is pretty solid in its maven support. a) mvn -DskipTests -Pyarn -Phive -Phadoop-2.3 compile package b) Inside IJ: Open the parent/root pom.xml as a new maven project c) Ins

More productive compilation of spark code base

2014-08-11 Thread Ron's Yahoo!
Hi, I’ve been able to get things compiled on my environment, but I’m noticing that it’s been quite difficult in IntelliJ. It always recompiles everything when I try to run one test like BroadcastTest, for example, despite having compiled make-distribution previously. In eclipse, I have no such