How to change the attributes order in Apache SparkSQL `Project` operator ?

2018-02-04 Thread parana
See below my queryExecution.optimizedPlan before apply my Rule. 01 Project [x#9, p#10, q#11, if (isnull(q#11)) null else UDF(q#11) AS udfB_10#28, if (isnull(p#10)) null else UDF(p#10) AS udfA_99#93]02 +- InMemoryRelation [x#9, p#10, q#11], true, 1, StorageLevel(disk, memory, deserialized, 1 rep

How to change the attributes order in Apache SparkSQL `Project` operator ?

2018-02-04 Thread parana
See below my queryExecution.optimizedPlan before apply my Rule.01 Project [x#9, p#10, q#11, if (isnull(q#11)) null else UDF(q#11) AS udfB_10#28, if (isnull(p#10)) null else UDF(p#10) AS udfA_99#93]02 +- InMemoryRelation [x#9, p#10, q#11], true, 1, StorageLevel(disk, memory, deserialized, 1 repl

Re: [VOTE] Spark 2.3.0 (RC2)

2018-02-04 Thread Xingbo Jiang
I filed another NPE problem in WebUI, I believe this is regression in 2.3: https://issues.apache.org/jira/browse/SPARK-23330 2018-02-01 10:38 GMT-08:00 Tom Graves : > I filed a jira [SPARK-23304] Spark SQL coalesce() against hive not > working - ASF JIRA