[
https://issues.apache.org/jira/browse/HUDI-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sagar Sumit closed HUDI-7954.
-----------------------------
Resolution: Fixed
> Fix data skipping with secondary index when there are no log files
> ------------------------------------------------------------------
>
> Key: HUDI-7954
> URL: https://issues.apache.org/jira/browse/HUDI-7954
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Sagar Sumit
> Assignee: Sagar Sumit
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.0.0-beta2, 1.0.0
>
>
> When there are no log files in index, then the lookup returns no secondary
> keys or candidate files, because of a bug - `logRecordsMap` is empty in this
> code and base file records are ignored -
> [https://github.com/apache/hudi/blob/70f44efe298771fcef9d029820a9b431e1ff165c/hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java#L970]
> Even though current tests for pruning asserts the filtered files count <
> total data files count. It is weak in the sense that it does not filtered
> files count > 0, and hence the assertion passed even when filtered files
> count = 0. Ultimately, all files were getting scanned. We should fix this
> behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
