Re: Problems running examples in IDEA

2014-08-24 Thread Ron Gonzalez
indeed made a run profile that also included Spark on the classpath. You would never modify the .iml files anyway. You can change the Maven pom.xml files if you were to need to modify a dependency scope. On Mon, Aug 25, 2014 at 12:12 AM, Ron Gonzalez wrote: Hi, After getting the code base to c

Problems running examples in IDEA

2014-08-24 Thread Ron Gonzalez
Hi, After getting the code base to compile, I tried running some of the scala examples. They all fail since it can't find classes like SparkConf. If I change the iml file to convert provided scope from PROVIDED to COMPILE, I am able to run them. It's simple by doing the following in the

Re: Spark Avro Generation

2014-08-11 Thread Ron Gonzalez
If you don't want to build the entire thing, you can also do mvn generate-sources in externals/flume-sink Thanks, Ron Sent from my iPhone > On Aug 11, 2014, at 8:32 AM, Hari Shreedharan > wrote: > > Jay running sbt compile or assembly should generate the sources. > >> On Monday, August 11,

Re: 1.1.0-SNAPSHOT possible regression

2014-08-08 Thread Ron Gonzalez
ala/org/apache/spark/broadcast/TorrentBroadcast.scala#L232 > > >And TorrentBroadcast is turned on by default for 1.1 right now. Do you want to >submit a pull request to fix that? This would be a critical fix for 1.1 that's >worth doing. > > > > > >On Fri, Aug 8, 20

Re: 1.1.0-SNAPSHOT possible regression

2014-08-08 Thread Ron Gonzalez
.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62) > at org.apache.spark.SparkContext.broadcast(SparkContext.scala:809) > > > On Fri, Aug 8, 2014 at 10:12 AM, Ron Gonzalez < > zlgonza...@yahoo.com.invalid> wrote: > >> Hi, >> I hav

1.1.0-SNAPSHOT possible regression

2014-08-08 Thread Ron Gonzalez
Hi, I have a running spark app against the released version of 1.0.1. I recently decided to try and upgrade to the trunk version. Interestingly enough, after building the 1.1.0-SNAPSHOT assembly, replacing it as my assembly in my app caused errors. In particular, it seems Kryo serialization isn'

Re: Buidling spark in Eclipse Kepler

2014-08-07 Thread Ron Gonzalez
as a Maven project in IntelliJ.) On Thu, Aug 7, 2014 at 4:53 AM, Ron Gonzalez wrote: > So I downloaded community edition of IntelliJ, and ran sbt/sbt gen-idea. > I then imported the pom.xml file. > I'm still getting all sorts of errors from IntelliJ about unresolved > dependenc

Re: Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
> jar. This is an open issue, and u can find the detail in JIRA. > > Sent from my Google Nexus 5 > >> On Aug 6, 2014 8:54 PM, "Ron Gonzalez" wrote: >> So I downloaded community edition of IntelliJ, and ran sbt/sbt gen-idea. >> I then imported the pom.xml fi

Re: Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
So I downloaded community edition of IntelliJ, and ran sbt/sbt gen-idea. I then imported the pom.xml file. I'm still getting all sorts of errors from IntelliJ about unresolved dependencies. Any suggestions? Thanks, Ron On Wednesday, August 6, 2014 12:29 PM, Ron Gonzalez wrote: Hi,

Re: Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
ve a > choice, I would strongly recommend IntelliJ over Eclipse for working > with Maven and Scala. > > On Wed, Aug 6, 2014 at 8:29 PM, Ron Gonzalez > wrote: >> Hi, >> I'm trying to get the apache spark trunk compiling in my Eclipse, but I >> can't seem

Buidling spark in Eclipse Kepler

2014-08-06 Thread Ron Gonzalez
Hi,   I'm trying to get the apache spark trunk compiling in my Eclipse, but I can't seem to get it going. In particular, I've tried sbt/sbt eclipse, but it doesn't seem to create the eclipse pieces for yarn and other projects. Doing mvn eclipse:eclipse on yarn seems to fail as well as sbt/sbt ec