This is an automated email from the ASF dual-hosted git repository.
xxyu pushed a change to branch kylin-on-parquet-v2
in repository https://gitbox.apache.org/repos/asf/kylin.git.
from dfd012f KYLIN-4945 repartition encoded dataset to avoid data skew
caused by single column
add 6153889 KYLIN-4967 Forbid to set 'spark.sql.adaptive.enabled' to true
when building cube with Spark 2.X
No new revisions were added by this update.
Summary of changes:
.../org/apache/spark/utils/SparkVersionUtils.scala | 74 ++++++++++++++++++++++
.../engine/spark/application/SparkApplication.java | 12 ++++
.../kylin/engine/spark/job/NSparkExecutable.java | 13 ++++
.../spark/builder/CubeDictionaryBuilder.scala | 16 ++---
.../engine/spark/builder/CubeTableEncoder.scala | 5 ++
.../kylin/engine/spark/utils/Repartitioner.java | 4 --
.../engine/spark/builder/TestGlobalDictBuild.scala | 3 -
7 files changed, 108 insertions(+), 19 deletions(-)
create mode 100644
kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/utils/SparkVersionUtils.scala
