Whole-stage codegen and SparkPlan.newPredicate

2017-12-30 Thread Jacek Laskowski
Hi, While working on an issue with Whole-stage codegen as reported @ https://stackoverflow.com/q/48026060/1305344 I found out that spark.sql.codegen.wholeStage=false does *not* turn whole-stage codegen off completely. It looks like SparkPlan.newPredicate [1] gets called regardless of the value of

https://issues.apache.org/jira/browse/SPARK-19609

2017-12-30 Thread Lalwani, Jayesh
Is there any interest in fixing this issue https://issues.apache.org/jira/browse/SPARK-19609 It would help us solve couple of problems. It would help a lot of applications that use Structured Streaming I am interested in making a contribution, but would need some guidance. __