sivabalan narayanan created HUDI-4631:
-----------------------------------------
Summary: Enhance retries for failed writes w/ write conflicts in a
multi writer scenarios
Key: HUDI-4631
URL: https://issues.apache.org/jira/browse/HUDI-4631
Project: Apache Hudi
Issue Type: Improvement
Components: multi-writer
Reporter: sivabalan narayanan
lets say there are two writers from t0 to t5. so hudi fails w2 and succeeds w1.
and user restarts w2 and for next 5 mins, lets say there are no other
overlapping writers. So the same write from w2 will now succeed. so, whenever
there is a write conflict and pipeline fails, all user needs to do is, just
restart the pipeline or retry to ingest the same batch.
Ask: can we add retries within hudi during such failures. Anyways, in most
cases, users just restart the pipeline in such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
