[
https://issues.apache.org/jira/browse/HUDI-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900033#comment-17900033
]
Vinoth Chandar edited comment on HUDI-8464 at 11/21/24 3:35 PM:
----------------------------------------------------------------
`TimeGenerator` implementations already achieve this using a
WaitBasedTimeGenerator (default, with a max skew of 200 ms), to generate any
times for requesting and completing actions.
Looked at all calls to createNewInstantTime(false) that skips locks...
Its skipped for
* rollbacks of pending writes in upgrade flow.
* MT writes that already happen within a lock
However, there are places where the requesting on the timeline is not done in
the same lock. This is not problematic for multi writers (OCC or NBCC) due to
completion time file slicing. But we need to ensure table services scheduling
and time generation happens in a single step. We will take this up in this
JIRA.
was (Author: vc):
`TimeGenerator` implementations already achieve this using a
WaitBasedTimeGenerator (default, with a max skew of 200 ms), to generate any
times for requesting and completing actions.
However, there are places where the requesting on the timeline is not done in
the same lock. This is not problematic for multi writers (OCC or NBCC) due to
completion time file slicing. But we need to ensure table services scheduling
and time generation happens in a single step. We will take this up in this
JIRA.
> Strengthen TrueTime API and locking
> -----------------------------------
>
> Key: HUDI-8464
> URL: https://issues.apache.org/jira/browse/HUDI-8464
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Y Ethan Guo
> Assignee: Vinoth Chandar
> Priority: Blocker
> Fix For: 1.0.0
>
> Attachments: image-2024-11-21-06-05-26-888.png,
> image-2024-11-21-07-00-33-053.png
>
> Original Estimate: 6h
> Remaining Estimate: 6h
>
> For serializability and strict time ordering, we should provide Spanner's
> TrueTime guarantee, i.e., lock and wait for a time period considering the
> practical largest clock skew (e.g., 500ms) before generating a new time, so
> that the time generated is always monotonically increasing for both instant
> time and completion time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
