yihua opened a new issue, #20067: URL: https://github.com/apache/hudi/issues/20067
CDC incremental queries on MERGE_ON_READ tables of table version 6 are broken: - Spark and Flink fail on the driver with `HoodieException: Fail to parse the base file and log files from DeltaCommit` (caused by `InvalidAvroMagicException`) when inferring changes from log files. The delta commit metadata is JSON in timeline layout 1 but is always parsed as Avro. - Flink bounds the log blocks it reads by the instant in the log file name. Before table version 8 that is the file slice's base instant, so changes of later delta commits are dropped and the deleted images of a replace commit are stale. part of #20064 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
