Re: What is correct behavior for spark.task.maxFailures?

2017-04-24 Thread Dongjin Lee
Sumit, I think the post below is describing the very case of you. https://blog.cloudera.com/blog/2017/04/blacklisting-in-apache-spark/ Regards, Dongjin -- Dongjin Lee Software developer in Line+. So interested in massive-scale machine learning. facebook: http://www.facebook.com

Re: Issues: Generate JSON with null values in Spark 2.0.x

2017-03-21 Thread Dongjin Lee
, 2017 at 4:44 PM, Chetan Khatri wrote: > Exactly. > > On Sat, Mar 11, 2017 at 1:35 PM, Dongjin Lee wrote: > >> Hello Chetan, >> >> Could you post some code? If I understood correctly, you are trying to >> save JSON like: >> >> { >> "firs

Re: Issues: Generate JSON with null values in Spark 2.0.x

2017-03-11 Thread Dongjin Lee
; we generate json will null values as a Dataframe ? > > Thanks. > -- *Dongjin Lee* *Software developer in Line+.So interested in massive-scale machine learning.facebook: www.facebook.com/dongjin.lee.kr <http://www.facebook.com/dongjin.lee.kr>linkedin: kr.linkedin.com/in/dongj