Re: AVRO specific records

2014-11-07 Thread Simone Franzini
classOf[NullWritable], conf) (From >> https://github.com/julianpeeters/avro-scala-macro-annotation-examples/blob/master/spark/src/main/scala/AvroSparkScala.scala) >> Keep in mind you'll need to use the kryo serializer as well. >> >> From: Frank Austin Nothaft >&g

Re: AVRO specific records

2014-11-06 Thread Simone Franzini
ion-examples/blob/master/spark/src/main/scala/AvroSparkScala.scala) > Keep in mind you'll need to use the kryo serializer as well. > > From: Frank Austin Nothaft > Date: Wednesday, November 5, 2014 at 5:06 PM > To: Simone Franzini > Cc: "user@spark.apache.org" >

Re: AVRO specific records

2014-11-05 Thread Anand Iyer
scala/AvroSparkScala.scala) > Keep in mind you'll need to use the kryo serializer as well. > > From: Frank Austin Nothaft > Date: Wednesday, November 5, 2014 at 5:06 PM > To: Simone Franzini > Cc: "user@spark.apache.org" > Subject: Re: AVRO specific records &

Re: AVRO specific records

2014-11-05 Thread Laird, Benjamin
gmail.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: AVRO specific records Hi Simone, Matt Massie put together a good tutorial on his blog<http://zenfractal.com/2013/08/21/a-powerful-big-data-trio/>.

Re: AVRO specific records

2014-11-05 Thread Frank Austin Nothaft
Hi Simone, Matt Massie put together a good tutorial on his blog. If you’re looking for more code using Avro, we use it pretty extensively in our genomics project. Our Avro schemas are here, and we have serialization code here. We use Parquet for storing the Avro records, but there is also an Av