Is CBO broken?

2018-09-11 Thread emlyn
. Does CBO work with partitioned parquet data, or is this a known issue? Is there a workaround? Thanks, Emlyn -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr

AnalysisException in first/last during aggregation since 2.0.1

2016-11-03 Thread emlyn
The following simple (pyspark) code fails in Spark 2.0.1: It only fails with all three arguments to .agg, removing any of them prevents the failure. Similar code in Java also fails in the same way, so it isn't specific to the Python API. It runs without error in Spark 2.0.0, so I suspect it migh