Re: [PR] Support Schema Evolution in iceberg [datafusion-comet]

2025-05-08 Thread via GitHub
andygrove commented on code in PR #1723: URL: https://github.com/apache/datafusion-comet/pull/1723#discussion_r2079832390 ## spark/src/test/scala/org/apache/comet/parquet/ParquetReadSuite.scala: ## @@ -1217,34 +1217,6 @@ abstract class ParquetReadSuite extends CometTestBase {

Re: [PR] Support Schema Evolution in iceberg [datafusion-comet]

2025-05-08 Thread via GitHub
hsiang-c commented on code in PR #1723: URL: https://github.com/apache/datafusion-comet/pull/1723#discussion_r2080426070 ## spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala: ## @@ -33,7 +33,7 @@ import org.apache.spark.sql.execution.datasources.v2.parquet.Parquet

Re: [PR] Support Schema Evolution in iceberg [datafusion-comet]

2025-05-08 Thread via GitHub
hsiang-c commented on code in PR #1723: URL: https://github.com/apache/datafusion-comet/pull/1723#discussion_r2080419849 ## common/src/main/java/org/apache/comet/parquet/Utils.java: ## @@ -28,33 +28,33 @@ public class Utils { - /** This method is called from Apache Iceberg

Re: [PR] Support Schema Evolution in iceberg [datafusion-comet]

2025-05-08 Thread via GitHub
huaxingao commented on code in PR #1723: URL: https://github.com/apache/datafusion-comet/pull/1723#discussion_r2079865065 ## spark/src/test/scala/org/apache/comet/parquet/ParquetReadSuite.scala: ## @@ -1217,34 +1217,6 @@ abstract class ParquetReadSuite extends CometTestBase {

Re: [PR] Support Schema Evolution in iceberg [datafusion-comet]

2025-05-08 Thread via GitHub
andygrove commented on PR #1723: URL: https://github.com/apache/datafusion-comet/pull/1723#issuecomment-2863268668 Thanks @huaxingao. The implementation changes LGTM, but I would like to understand how this will be tested. -- This is an automated message from the Apache Git Service. To r

[PR] Support Schema Evolution in iceberg [datafusion-comet]

2025-05-07 Thread via GitHub
huaxingao opened a new pull request, #1723: URL: https://github.com/apache/datafusion-comet/pull/1723 ## Which issue does this PR close? We original have CometConf.COMET_SCHEMA_EVOLUTION_ENABLED to set schema evolution to true in Scan rule if the scan is Iceberg table scan. However, i