andykrk commented on issue #4604: URL: https://github.com/apache/hudi/issues/4604#issuecomment-1024091806
@nsivabalan I was able to reproduce the issue on the other table. Let me add some additional information to it that may help. We start with our current table that was created with hudi 0.7.0. Since the beginning it had following settings apllied: 'hoodie.keep.min.commits': 999, 'hoodie.keep.max.commits': 1000, 'hoodie.cleaner.commits.retained': 998, therefore in our case there was no archival happening for that table. Currently since we want to enable archival process we are modifying the parameters to: 'hoodie.keep.min.commits': 29, 'hoodie.keep.max.commits': 30, 'hoodie.cleaner.commits.retained': 28, however for my testing I run the first processing with hudi 0.9.0 and: 'hoodie.keep.min.commits': 999, 'hoodie.keep.max.commits': 1000, 'hoodie.cleaner.commits.retained': 998, to make sure that if anything has to upgrade or update in the background it happens before we run with archival option. This run works fine and completes successfully. Then the settings go to 29, 30, 28 and I run again (possibly with no data change as I do this right away). That's when I see the errors presented here. Wanted to add this here as I am not sure if the facts that table was created with older hudi version and that the run without archiving works fine may matter for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
