RE: NullWritable not serializable

2014-09-16 Thread Yan Zhou.sc
amp;& finalStage.parents.isEmpty && partitions.length == 1 I'm wondering whether by default "running locally" is disabled. Yan From: Du Li [mailto:l...@yahoo-inc.com.INVALID] Sent: Tuesday, September 16, 2014 12:26 PM To: Matei Zaharia Cc: u...@spark.apache.org; dev@spark.apache.or

Re: NullWritable not serializable

2014-09-16 Thread Du Li
ptember 15, 2014 at 10:52 PM To: Du Li mailto:l...@yahoo-inc.com>> Cc: "u...@spark.apache.org<mailto:u...@spark.apache.org>" mailto:u...@spark.apache.org>>, "dev@spark.apache.org<mailto:dev@spark.apache.org>" mailto:dev@spark.apache.org>> Subject:

Re: NullWritable not serializable

2014-09-15 Thread Matei Zaharia
To: Du Li , "u...@spark.apache.org" , "dev@spark.apache.org" Subject: Re: NullWritable not serializable Hi Du, I don't think NullWritable has ever been serializable, so you must be doing something differently from your previous program. In this case though, just use a map(

Re: NullWritable not serializable

2014-09-12 Thread Matei Zaharia
Hi Du, I don't think NullWritable has ever been serializable, so you must be doing something differently from your previous program. In this case though, just use a map() to turn your Writables to serializable types (e.g. null and String). Matie On September 12, 2014 at 8:48:36 PM, Du Li (l...