veenaypatil commented on issue #6014: URL: https://github.com/apache/hudi/issues/6014#issuecomment-1423646531
@nsivabalan sorry for late response on this issue, I am not seeing this issue as of now, we were only seeing this issue when we killed the job and restarted it. > I see you are using non partitioned key gen. So, index look up is going to relative to the number of file groups you have in total. do you know whats total file groups you have. can you enable clustering to batch smaller file groups into larger ones. How to get the number of file groups ? is there a command for it ? also what should be the limit on the number of file groups -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
