Re: Strange behavior with 'not' and filter pushdown

2017-02-13 Thread Takeshi Yamamuro
Oh, Thanks for checking! On Tue, Feb 14, 2017 at 12:32 PM, Xiao Li wrote: > https://github.com/apache/spark/pull/16894 > > Already backported to Spark 2.0 > > Thanks! > > Xiao > > 2017-02-13 17:41 GMT-08:00 Takeshi Yamamuro : > >> cc: xiao >> >> IIUC a xiao's commit below fixed this issue in mas

Re: Strange behavior with 'not' and filter pushdown

2017-02-13 Thread Xiao Li
https://github.com/apache/spark/pull/16894 Already backported to Spark 2.0 Thanks! Xiao 2017-02-13 17:41 GMT-08:00 Takeshi Yamamuro : > cc: xiao > > IIUC a xiao's commit below fixed this issue in master. > https://github.com/apache/spark/commit/2eb093decb5e87a1ea71bbaa280928 > 76a8c84996 > > I

Re: Strange behavior with 'not' and filter pushdown

2017-02-13 Thread Takeshi Yamamuro
cc: xiao IIUC a xiao's commit below fixed this issue in master. https://github.com/apache/spark/commit/2eb093decb5e87a1ea71bbaa28092876a8c84996 Is this fix worth backporting to the v2.0 branch? I checked I could reproduce there: --- scala> Seq((1, "a"), (2, "b"), (3, null)).toDF("c0", "c1").wri