nsivabalan commented on issue #4656: URL: https://github.com/apache/hudi/issues/4656#issuecomment-1030886715
I guess the replaced file groups(15201 file groups, where as actual valid file groups are only 17 or 18) are causing a lot of impact. We can probably trigger archival and see whats happening. I am assuming you are using default configs for cleaning and archiving. especially below configs. ``` hoodie.cleaner.commits.retained hoodie.keep.min.commits hoodie.keep.max.commits ``` default values are 10, 20 and 30. Can you make the 2nd and 3rd configs to 11 and 12. This should trim your active timeline and likely replaced file groups as well. And I assume you have metadata disabled (hoodie.metadata.enable) in both. bcoz, in your description, I see the config value is different in table1 to table2. CC @manojpec for perf issues reported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
