kazdy commented on issue #3724: URL: https://github.com/apache/hudi/issues/3724#issuecomment-991237684
Glad to hear that @alex-shchetkov :) There is one option that I want to try - [hoodie.datasource.read.incr.path.glob](hoodie.datasource.read.incr.path.glob) I don't know how it works exactly yet, but maybe you'll find it useful :) Knowing how it works now, I'd keep commits for the last x days, and when there's a need for reprocessing I'd run it in batch mode, only for some partitions if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
