Thanks will take a look... Sent from my iPad
> On Apr 3, 2014, at 7:49 AM, FRANK AUSTIN NOTHAFT <fnoth...@berkeley.edu> > wrote: > > We use avro objects in our project, and have a Kryo serializer for generic > Avro SpecificRecords. Take a look at: > > https://github.com/bigdatagenomics/adam/blob/master/adam-core/src/main/scala/edu/berkeley/cs/amplab/adam/serialization/ADAMKryoRegistrator.scala > > Also, Matt Massie has a good blog post about this at > http://zenfractal.com/2013/08/21/a-powerful-big-data-trio/. > > Frank Austin Nothaft > fnoth...@berkeley.edu > fnoth...@eecs.berkeley.edu > 202-340-0466 > > >> On Thu, Apr 3, 2014 at 7:16 AM, Ian O'Connell <i...@ianoconnell.com> wrote: >> Objects been transformed need to be one of these in flight. Source data can >> just use the mapreduce input formats, so anything you can do with mapred. >> doing an avro one for this you probably want one of : >> https://github.com/kevinweil/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapreduce/input/*ProtoBuf* >> >> or just whatever your using at the moment to open them in a MR job probably >> could be re-purposed >> >> >>> On Thu, Apr 3, 2014 at 7:11 AM, Ron Gonzalez <zlgonza...@yahoo.com> wrote: >>> >>> Hi, >>> I know that sources need to either be java serializable or use kryo >>> serialization. >>> Does anyone have sample code that reads, transforms and writes avro files >>> in spark? >>> >>> Thanks, >>> Ron >
