danny0405 commented on issue #10988: URL: https://github.com/apache/hudi/issues/10988#issuecomment-2055029717
A solution to mediate the issue is to increase the retained commits for historical data, so that the reader has enough buffer time for consumption. Another solution is we add some progress signal on the reader side so that cleaner can be aware of the data that been consumed, but generally I don't think we should do this(reader should not block writer), so the task is suspended for a long time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
