keerthiskating commented on issue #11712: URL: https://github.com/apache/hudi/issues/11712#issuecomment-2264043227
@ad1happy2go I do not have access to EMR cluster and so unable to run the spark-submit. However, I explicitly set following parameters to allow for aggressive cleaning. Even then, first 10-12 writes to the table are running fine and then the write job next to it hangs on `Preparing compaction metadata: gft_fact_consol_hudi_metadata` 'hoodie.keep.max.commits':'2', 'hoodie.keep.min.commits':'1', How do I ensure compaction happens after each write? Also, is this log file compaction? Is there a way I can set parallelism for compaction? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
