nsivabalan commented on a change in pull request #4180:
URL: https://github.com/apache/hudi/pull/4180#discussion_r782690404
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
##########
@@ -241,6 +246,26 @@ private void deleteInstantFile(HoodieInstant instant) {
return readDataFromPath(detailPath);
}
+ /**
+ * Get the last instant with data written, and convert this to
HoodieCommitMetadata
+ */
+ public Option<Pair<HoodieInstant, HoodieCommitMetadata>>
getLastCommitMetadataToWrite() {
Review comment:
nit: getLastCommitMetadataWithValidData
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java
##########
@@ -114,7 +113,7 @@ private MessageType getTableParquetSchemaFromDataFile()
throws Exception {
if (lastDeltaCommit.isPresent()) {
HoodieInstant lastDeltaInstant = lastDeltaCommit.get();
// read from the log file wrote
- commitMetadata =
HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(lastDeltaInstant).get(),
+ HoodieCommitMetadata commitMetadata =
HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(lastDeltaInstant).get(),
Review comment:
there is a chance that last delta commit is empty and does not have
valid schema. so, the fix we have done for COW above, might also be required
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
