Gabriel39 opened a new pull request, #68142: URL: https://github.com/apache/doris/pull/68142
### What problem does this PR solve? An Iceberg snapshot can reference a data file that has been removed from storage. With the default `ignore_not_found_file_in_external_table=true`, both file scanners can skip the missing file and return a successful but incomplete result. Exclude Iceberg ranges from the missing-file skip policy, regardless of that setting, while preserving the existing behavior for other table formats. Apply the shared policy at reader initialization and read boundaries. This also prevents a missing delete file reported as `NOT_FOUND` from being swallowed by the scanner. Add unit coverage for Iceberg Parquet/ORC/JNI and ordinary external ranges with both setting values. Add an external regression that creates multiple data files, deletes one actual object, and expects row scans to fail with V1/V2 and both setting values. The regression disables file/result caches and restores the BE configuration after testing. ### Release note Iceberg scans now fail when a required snapshot file is missing instead of silently skipping it, including when `ignore_not_found_file_in_external_table` is enabled. ### Check List (For Author) - Test: The shared-policy GoogleTests passed in a standalone build with actual generated Thrift types; the Iceberg assertion failed with the extracted original policy. Groovy compilation, clang-format 16, and whitespace checks passed. The full BE UT runner was blocked during local clucene/Zstd dependency configuration. The external regression has not been executed locally. - Behavior changed: Yes; missing Iceberg files are no longer ignored. Other table formats retain their existing behavior. - Does this need documentation: No new configuration or API; the corrected behavior is described above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
