Re: Spark support for Hadoop Formats (Avro)

2015-04-13 Thread Michael Armbrust
The problem is likely that the underlying avro library is reusing objects for speed. You probably need to explicitly copy the values out of the reused record before the collect. On Sat, Apr 11, 2015 at 9:23 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) wrote: > The read seem to be successfully as the values for each field

Re: Spark support for Hadoop Formats (Avro)

2015-04-11 Thread ๏̯͡๏
The read seem to be successfully as the values for each field in record are different and correct. The problem is when i collect it or trigger next processing (join with other table) , each of this probably triggers serialization and thats when all the fields in the record get the value of first fi