K1taIkuyo0421 opened a new issue, #20038:
URL: https://github.com/apache/hudi/issues/20038

   ### Bug Description
   
   **What happened:**
   
   When a Flink writer recovers from a checkpoint, it sends a bootstrap event 
containing the restored metadata on the first attempt. On subsequent attempts, 
it sends only an empty event to clean up write events that may remain in the 
Coordinator.
   
   If only some subtasks have sent their bootstrap events before another 
failure occurs, the next attempt sends only empty events instead of the 
restored bootstrap metadata.
   
   As a result, the Coordinator cannot collect the complete bootstrap metadata, 
and some restored data may not be committed.
   
   **What you expected:**
   
   The writer should send the actual bootstrap event on every recovery attempt, 
together with the current `attemptId`.
   
   The Coordinator should use the `attemptId` to clean up bootstrap and write 
events left by previous attempts. It should process and commit the restored 
write metadata only after receiving bootstrap events with the same `attemptId` 
from all subtasks.
   
   **Steps to reproduce:**
   
   1. Start a Flink Hudi write job with writer parallelism greater than 1 and 
trigger a checkpoint.
   2. Restore the job from the checkpoint and allow only some writer subtasks 
to send their bootstrap events.
   3. Trigger another failure before the remaining subtasks send their 
bootstrap events.
   4. Restore the job from the checkpoint again.
   5. The subsequent attempt sends only empty bootstrap events, so the 
Coordinator cannot collect the complete bootstrap metadata.
   
   
   
   ### Environment
   
   **Hudi version:*1.1.0 and later*
   **Query engine:** Flink
   
   
   ### Logs and Stack Trace
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to