There was a avro-mapred version conflict described in
https://issues.apache.org/jira/browse/SPARK-3039.
Fixed by https://github.com/apache/spark/pull/4315 for Spark 1.3.
Here is a link that describes how to fix Spark 1.2.1 for avro-mapred
hadoop2:
https://github.com/medale/spark-mail/blob/mast
On 02/04/2015 02:04 PM, Josh Rosen wrote:
It looks like you replied just to me; mind CC’ing the mailing list, too?
On February 4, 2015 at 11:02:34 AM, M. Dale (medal...@yahoo.com
<mailto:medal...@yahoo.com>) wrote:
Josh,
That was a bug that was present earlier. It was marked as fi
SPARK-3039 "Spark assembly for new hadoop API (hadoop 2) contains
avro-mapred for hadoop 1 API" was reopened
and prevents v.1.2.1-rc3 from using Avro Input format for Hadoop 2
API/instances (it includes the hadoop1 avro-mapred library files).
What are the chances of getting the fix outlined her
SPARK-3039 "Spark assembly for new hadoop API (hadoop 2) contains
avro-mapred for hadoop 1 API"
was marked resolved with Spark 1.2.0 release. However, when I download the
pre-built Spark distro for Hadoop 2.4 and later
(spark-1.2.0-bin-hadoop2.4.tgz) and run it
against Avro code compiled agains
