Timothy Brown created HUDI-9384:
-----------------------------------
Summary: Unify Creation of Timestamps and Persistence to Timeline
Key: HUDI-9384
URL: https://issues.apache.org/jira/browse/HUDI-9384
Project: Apache Hudi
Issue Type: Epic
Reporter: Timothy Brown
Currently we take a lock to generate timestamps which ensures that the
timestamps are generated in a strictly increasing order. This helps us reason
about the order of the commits but the timeline is not necessarily updated when
these timestamps when they are created. This leads to inconsistencies when
dealing with multiple writers since the other writer may not have the knowledge
of this new timestamp. For the other writer to have this knowledge, we need to
persist the instant to the timeline.
On top of this, we want to minimize the amount of time where the table is
locked while still ensuring correctness and preventing any opportunities for
corruption of the table or its metadata. This will help us better support
multiple writers to the same table going forward.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
