This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
from 447af5a8dbc7 fix(spark): Ignore duplicate fields when merging schema
in IncrementalRelation (#17776)
add 180592a0ad71 feat(spark): implement column pruning for incremental
queries (#17514)
No new revisions were added by this update.
Summary of changes:
.../org/apache/hudi/IncrementalRelationV1.scala | 33 +--
.../org/apache/hudi/IncrementalRelationV2.scala | 33 +--
.../apache/hudi/util/IncrementalRelationUtil.scala | 110 ++++++++++
.../sql/hudi/streaming/HoodieStreamSourceV1.scala | 4 +-
.../sql/hudi/streaming/HoodieStreamSourceV2.scala | 4 +-
.../TestIncrementalQueryColumnPruning.scala | 242 +++++++++++++++++++++
6 files changed, 394 insertions(+), 32 deletions(-)
create mode 100644
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/util/IncrementalRelationUtil.scala
create mode 100644
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestIncrementalQueryColumnPruning.scala
