You can give Spark-Avro a try. It works great for our project.
https://github.com/databricks/spark-avro
> From: deepuj...@gmail.com
> Date: Thu, 5 Mar 2015 10:27:04 +0530
> Subject: Fwd: Unable to Read/Write Avro RDD on cluster.
> To: dev@spark.apache.org
>
> I am trying to read RDD avro, transfo
You have to change most of dependences in the spark-example model from
"provided" to "compile", so you can run the example in Intellij.
Yong
> Date: Fri, 3 Apr 2015 09:22:13 -0700
> From: eng.sara.must...@gmail.com
> To: dev@spark.apache.org
> Subject: IntelliJ Runtime error
>
> Hi,
>
> I have
Hi, This email is sent to both dev and user list, just want to see if someone
familiar with Spark/Maven build procedure can provide any help.
I am building Spark 1.2.2 with the following command:
mvn -Phadoop-2.2 -Dhadoop.version=2.2.0 -Phive -Phive-0.12.0
The spark-assembly-1.2.2-hadoop2.2.0.jar
I am comparing the log of Spark line by line between the hanging case (big
dataset) and not hanging case (small dataset).
In the hanging case, the Spark's log looks identical with not hanging case for
reading the first block data from the HDFS.
But after that, starting from line 438 in the spark
