Re:Re: Spark SQL 1.3.1　"saveAsParquetFile"　will output tachyon file with different block size

HiActually I did not use Tachyon 0.6.3,just compiled it with 0.5.0 by make-distribution.sh. When I pulled the spark code from github,the Tachyon version was still 0.5.0 in pom,xml. Regards Zhang At 2015-04-29 04:19:20, "sara mustafa" wrote: >Hi Zhang, > >How did you compile Spark 1.3.1 with

Re: Spark SQL 1.3.1 "saveAsParquetFile" will output tachyon file with different block size

Hi, You can apply this patch and recompile. Hope this helps, Calvin On Tue, Apr 28, 2015 at 1:19 PM, sara mustafa wrote: > Hi Zhang, > > How did you compile Spark 1.3.1 with Tachyon? when i changed Tachyon > version > to 0.6.3 in core/pom.xml, make-d

Re: Spark SQL 1.3.1　"saveAsParquetFile"　will output tachyon file with different block size

Hi Zhang, How did you compile Spark 1.3.1 with Tachyon? when i changed Tachyon version to 0.6.3 in core/pom.xml, make-distribution.sh and try to compile again, many compilation errors raised. Thanks, -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/S

RE: [SQL][Feature] Access row by column name instead of index

Hi, I add a few helper method on this, to make java developer life easier, because we can’t benefit from the generic feature on getAs[T]. Please let me know if I should not do that. Regards, Shuai From: Michael Armbrust [mailto:mich...@databricks.com] Sent: Friday, April 24,

SPARK_SUBMIT_CLASSPATH Windows PYSPARK

Hi, So I was trying to get the Kafka Streaming working in a standalone python application. I needed to add the dependencies for this to work. The suggested way for this is doing --jars using spark-submit, which is not practical considering I wanted to launch and debug an application. Also you sti

How to deploy self-build spark source code on EC2

Hi all, I have an issue. I added some timestamps in Spark source code and built it using: mvn package -DskipTests I checked the new version in my own computer and it works. However, when I ran spark on EC2, the spark code EC2 machines ran is the original version. Anyone knows how to deploy th

Re:Re: java.lang.StackOverflowError when recovery from checkpoint in Streaming

Hi Akhil Das, Thank you for your reply. It is very similar to my problem, I will focus on it. Thanks Best Regards At 2015-04-28 18:08:32,"Akhil Das" wrote: >There's a similar issue reported over here >https://issues.apache.org/jira/browse/SPARK-6847 > >Thanks >Best Regards > >On Tue, Apr 28, 2015

Re: java.lang.StackOverflowError when recovery from checkpoint in Streaming

There's a similar issue reported over here https://issues.apache.org/jira/browse/SPARK-6847 Thanks Best Regards On Tue, Apr 28, 2015 at 7:35 AM, wyphao.2007 wrote: > Hi everyone, I am using val messages = > KafkaUtils.createDirectStream[String, String, StringDecoder, > StringDecoder](ssc, kafk

Re: creating hive packages for spark

Yash, This is exactly what I wanted! Thanks a bunch. On 27 April 2015 at 15:39, yash datta wrote: > Hi, > > you can build spark-project hive from here : > > https://github.com/pwendell/hive/tree/0.13.1-shaded-protobuf > > Hope this helps. > > > On Mon, Apr 27, 2015 at 3:23 PM, Manku Timma > wro