Hi Team,

During the discussion around the Flink Table Maintenance [1], [2], I have
highlighted that one of the main decision points is the way we prevent
concurrent Maintenance Tasks from happening concurrently.

At that time we did not find better solution than providing an interface
for locking, and provide a basic implementation which is based on Iceberg
tags [3]:

*"For convenience an Iceberg tag based solution is provided, so no external
dependencies are needed. On lock creation a tag named '__flink_maitenance'
is created for the current snapshot, and on lock removal this tag is
removed. This solution is not ideal, as it creates multiple new versions of
the table metadata, and mixes infrastructural metadata with user metadata.
If this is not acceptable then other implementations could be created which
could use external components like JDBC/Kafka/ZooKeeper."*


We are in the implementation phase, and we agreed with Steven to take one
final round with the community, to see if anyone has a better suggestion,
or we could proceed with the originally agreed one.

So if you have a better idea, please share.

Thanks,
Peter

[1] - https://lists.apache.org/thread/qjf83v2xj6lxb9sr8z0v9p0979f8wsmf
[2] - https://lists.apache.org/thread/vjf8m5wg840o58yz4y3q35k2mfhbm49l
[3] -
https://docs.google.com/document/d/16g3vR18mVBy8jbFaLjf2JwAANuYOmIwr15yDDxovdnA/edit#heading=h.lt9eaimi6zyz

Reply via email to