fengjian428 commented on issue #6259:
URL: https://github.com/apache/hudi/issues/6259#issuecomment-1210479536
> Interesting! If I understand correctly, @fengjian428 's suggestion is to
go back to a previous snapshot if the latest one is corrupt. Hudi already
supports time travel. However, @melin 's suggestion is to not even publish the
snapshot if it is found to be corrupt.
yeah, it is like implementing the same function in different ways
>
> Shouldn't savepoint/restore along with a staging area be sufficient to
support this feature? It's about how to control the **visibility of a
snapshot**. Hudi metadata can be first written to a staging area before being
published. Audit tool runs ETL and validations with the staging metadata. If
all is well, then the staging metadata changes are applied to production.
I think so, Like I said in the last monthly sync call, I've implemented
Snapshot view in our company based on the savepoint feature, although it is for
a different scenario @melin FYI
https://docs.google.com/presentation/d/1xypcr9onk0ogpj1lrPFQ3ERiXQpTw9NZGW4nok5i80I/edit#slide=id.g13ec3137431_2_195.
>
> @melin Hudi has all the necessary abstractions to support this feature.
Can you please explain your use case and functional requirements in more
detail? Are you also proposing abstractions for an audit tool? I think this
could be useful to other users as well and worthy of RFC. cc
@prasannarajaperumal @vinothchandar @nsivabalan @xushiyan
I can be a co-author for this RFC if you want to create one. basically, we
need to point the Hudi table to a specific savepoint， but I think we need to do
some enhancement on mor's savepoint， since the savepoint seems only tags the
base file for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
