bvaradar commented on issue #2345: URL: https://github.com/apache/hudi/issues/2345#issuecomment-748340463
@kingkongpoon : Actually, preCombine is used to merge duplicate rows in a single batch that are to be ingested together. In your case, if both a.csv and 2.csv were a single file, you will see the preCombine come into effect. For the scenario you have the API that matters is combineAndGetUpdateValue. The default record payload always overwrites the existing row in the dataset with the incoming record. Hence, the behavior you are seeing. I have opened the jira to clarify this in Javadocs https://issues.apache.org/jira/browse/HUDI-1475 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
