This is an automated email from the ASF dual-hosted git repository.
xxyu pushed a change to branch kylin-on-parquet-v2
in repository https://gitbox.apache.org/repos/asf/kylin.git.
from 32f8d66 fix KYLIN-4850, merge Cube's override kylin_properties into
building job's configuration
new a6f78cd KYLIN-4818 Calculate cuboid rowcount via HLL
new 3d6db49 KYLIN-4818 Support Cube Planner Phase One in Kylin 4
new 9e33d93 KYLIN-4818 Persist metadata in SparkExecutable
new d8e5db8 KYLIN-4818 Performance profile for CuboidStatisticsJob
new 2e9597a KYLIN-4818 Reduce toString method call
new af8c8fe KYLIN-4818 Refine CuboidStatisticsJob to improve performance
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../kylin/engine/mr/common/CubeStatsReader.java | 26 ++--
.../engine/mr/common/CuboidRecommenderUtil.java | 6 +-
.../engine/mr/common/StatisticsDecisionUtil.java | 8 +-
.../org/apache/kylin/common/KylinConfigBase.java | 27 +++-
.../kylin/common/annotation/Clarification.java | 6 +-
.../java/org/apache/kylin/cube/CubeSegment.java | 19 ++-
.../cube/cuboid/algorithm/CuboidRecommender.java | 3 +-
.../cuboid/algorithm/greedy/GreedyAlgorithm.java | 16 +-
.../org/apache/kylin/cube/kv/CubeDimEncMap.java | 2 +
.../org/apache/kylin/measure/hllc/HLLCounter.java | 4 +-
.../engine/spark/application/SparkApplication.java | 2 +-
.../engine/spark/builder/NBuildSourceInfo.java | 16 +-
.../kylin/engine/spark/job/NSparkExecutable.java | 7 +-
.../engine/spark/utils/UpdateMetadataUtil.java | 41 +++--
.../kylin/engine/spark/job/CubeBuildJob.java | 133 ++++++++++++++--
.../kylin/engine/spark/job/CuboidAggregator.scala | 1 +
.../engine/spark/job/CuboidStatisticsJob.scala | 167 +++++++++++++++++++++
.../engine/spark/job/ParentSourceChooser.scala | 36 ++++-
.../kylin/engine/spark/metadata/MetaData.scala | 21 ++-
.../engine/spark/metadata/MetadataConverter.scala | 14 +-
.../kylin/rest/controller/CubeController.java | 51 ++++---
webapp/app/partials/cubes/cube_detail.html | 2 +-
22 files changed, 492 insertions(+), 116 deletions(-)
create mode 100644
kylin-spark-project/kylin-spark-engine/src/main/scala/org/apache/kylin/engine/spark/job/CuboidStatisticsJob.scala
