[kylin] branch kylin-on-parquet-v2 updated: KYLIN-4887 Segment pruner support string type partition col in spark query engine

[xxyu]

This is an automated email from the ASF dual-hosted git repository.

xxyu pushed a commit to branch kylin-on-parquet-v2
in repository https://gitbox.apache.org/repos/asf/kylin.git


The following commit(s) were added to refs/heads/kylin-on-parquet-v2 by this 
push:
     new 0cf9e3d  KYLIN-4887 Segment pruner support string type partition col 
in spark query engine
0cf9e3d is described below

commit 0cf9e3d5951e880f04033efa599059237deba766
Author: zhengshengjun <shengjun_zh...@sina.com>
AuthorDate: Wed Jan 27 17:12:57 2021 +0800

    KYLIN-4887 Segment pruner support string type partition col in spark query 
engine
---
 .../scala/org/apache/spark/sql/execution/datasource/FilePruner.scala | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git 
a/kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/sql/execution/datasource/FilePruner.scala
 
b/kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/sql/execution/datasource/FilePruner.scala
index 0910d86..9f62f52 100644
--- 
a/kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/sql/execution/datasource/FilePruner.scala
+++ 
b/kylin-spark-project/kylin-spark-common/src/main/scala/org/apache/spark/sql/execution/datasource/FilePruner.scala
@@ -132,11 +132,8 @@ class FilePruner(
         // we can only get col ID in layout cuz data schema is all ids.
         val id = layoutEntity.getOrderedDimensions.asScala.values.find(column 
=> column.columnName.equals(ref.getName))
         if (id.isDefined && (ref.getType.isDateTimeFamily || 
ref.getType.isStringFamily)) {
-          if (ref.getType.isDateTimeFamily) {
-            pattern = desc.getPartitionDateFormat
-          }
+          pattern = desc.getPartitionDateFormat
           dataSchema.filter(_.name == String.valueOf(id.get.id))
-
         } else {
           Seq.empty
         }