Re: Apache Spark 0.9.0 Build Error

2014-03-18 Thread Robin Cjc
hi, if you run that under windows, you should use "\" to replace "/". sbt/sbt means the sbt file under the sbt folder. On Mar 18, 2014 8:42 PM, "wapisani" wrote: > I tried that command on Fedora and I got a lot of random downloads (around > 250 downloads) and it appeared that something was trying

Re: SPARK_JAVA_OPTS not picked up by the application

2014-03-12 Thread Robin Cjc
Some properties can be set in System.setProperty(). Like the -Dfile.encoding, can be set as ("file.encoding", "utf-8"). but some cannot, like the heap size, as it is too late to set the heap size. And in the new 0.9 version, I think you can use sparkconf.set() instead of System.setProperty(). It se

Re: SPARK_JAVA_OPTS not picked up by the application

2014-03-11 Thread Robin Cjc
I haven't tried it, but I think you still can use the system.setproperty to set the property. or if you run the application with sbt, I think you also can set the javaOptions in sbt. is that working for you? Thanks Best Regards, Chen Jingci On Tue, Mar 11, 2014 at 1:15 PM, Linlin wrote: > Tha

Re: SPARK_JAVA_OPTS not picked up by the application

2014-03-10 Thread Robin Cjc
The properties in spark-env.sh are machine-specific. so need to specify in you worker as well. I guess you ask is the System.setproperty(). you can call it before you initialize your sparkcontext. Best Regards, Chen Jingci On Tue, Mar 11, 2014 at 6:47 AM, Linlin wrote: > > Hi, > > I have a jav

Re: how to use the log4j for the standalone app

2014-03-10 Thread Robin Cjc
Hi lihu, you can extends the org.apache.spark.logging class. Then use the function like logInfo(). Then will log according to the config in your log4j.properties. Best Regards, Chen Jingci On Tue, Mar 11, 2014 at 11:36 AM, lihu wrote: > Hi, >I use the spark0.9, and when i run the spark-sh