vinishjail97 opened a new pull request, #20245: URL: https://github.com/apache/hudi/pull/20245
### Describe the issue this Pull Request addresses The commit that creates the metadata table with column stats and a secondary index drops the secondary index definition from `.hoodie/.index_defs/index.json`. The metadata writer registers the secondary index definition through its own meta client, and after the commit `HoodieMetadataWriteUtils.updateColsToIndex` updates the column stats definition through the data table's meta client. That meta client loaded the definitions before the commit, and `reloadTableConfig()` does not reload them, so `buildIndexDefinition` writes its stale view and the secondary index definition is lost. Every later commit then updates the secondary index through its definition and fails with `Expression Index definition is not present for index: secondary_index_<column>`, unless its write config names the same secondary index column. Found in Apache XTable while building several secondary indexes on one table. ### Summary and Changelog `HoodieTableMetaClient.buildIndexDefinition` and `deleteIndexDefinition` reload the index definitions from storage before they change them. The definitions are written as one file, so applying a change to the latest definitions keeps the definitions other meta clients wrote. - Test: `TestHoodieTableMetaClient#testIndexDefinitionChangesKeepDefinitionsOfOtherMetaClients` builds and deletes definitions through two meta clients that loaded them at the same time. Without the fix, the second meta client's write drops the first one's secondary index definition. ### Impact Secondary index definitions survive the commit that creates them. Each definition change reads the definitions file once more. No API or config change. ### Risk Level low. The definitions file is small, and the change only adds a read before an existing write. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
