ingridymartinss opened a new issue, #9518: URL: https://github.com/apache/hudi/issues/9518
**Describe the problem you faced** Currently, we have a pipeline with approximately 2 billion records and 95 columns that runs every day. Yesterday, at the time of execution, there was an intermittence in the EKS that we have to execute the pipeline (we don't know in which part of the execution, if it was in the inflight part or already in the commit). With that, when we tried to perform a next execution, we had the error below:  We tried everything: running low volume, high volume, changing spark resources, and we always got the same error. We also tried to make a savepoint, but when performing the restore we also had errors. In that case, how to proceed? **To Reproduce** Steps to reproduce the behavior: 1. Start a pipeline with Hudi 0.12.0 running on EKS 2. Stop the pipeline abruptly 3. Error **Expected behavior** Despite the interruption, I expected the pipeline to start working again. **Environment Description** * EMR on EKS * Hudi version : 0.12.2 * Spark version : 3.2.1 * Hive version : - * Hadoop version : - * Storage (HDFS/S3/GCS..) : S3 * Running on Docker? (yes/no) : Running on EKS -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
