codope commented on issue #6259: URL: https://github.com/apache/hudi/issues/6259#issuecomment-1210421580
Interesting! If I understand correctly, @fengjian428 's suggestion is to go back to a previous snapshot if the latest one is corrupt. Hudi already supports time travel. However, @melin 's suggestion is to not even publish the snapshot if it is found to be corrupt. Shouldn't savepoint/restore along with a staging area be sufficient to support this feature? It's about how to control the **visibility of a snapshot**. Hudi metadata can be first written to a staging area before being published. Audit tool runs ETL and validations with the staging metadata. If all is well, then the staging metadata changes are applied to production. @melin Hudi has all the necessary abstractions to support this feature. Can you please explain your use case and functional requirements in more detail? Are you also proposing abstractions for an audit tool? I think this could be useful to other users as well and worthy of RFC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
