codope commented on code in PR #6266:
URL: https://github.com/apache/hudi/pull/6266#discussion_r939540008
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/MarkerBasedRollbackUtils.java:
##########
@@ -54,25 +64,35 @@ public static List<String> getAllMarkerPaths(HoodieTable
table, HoodieEngineCont
String instant, int
parallelism) throws IOException {
String markerDir = table.getMetaClient().getMarkerFolderPath(instant);
FileSystem fileSystem = table.getMetaClient().getFs();
- Option<MarkerType> markerTypeOption =
MarkerUtils.readMarkerType(fileSystem, markerDir);
+ Option<MarkerType> markerTypeOption = readMarkerType(fileSystem,
markerDir);
- // If there is no marker type file "MARKERS.type", we assume "DIRECT"
markers are used
+ // If there is no marker type file "MARKERS.type", first assume "DIRECT"
markers are used.
+ // If not, then fallback to "TIMELINE_SERVER_BASED" markers.
Review Comment:
That's possible. However, it was reported in #5591 that the rollback failed.
The other thing is that the creation of marker type file is contingent on
existence of marker dir. What if the first time marker dir got created but
marker type file was not created, then on retry
`MarkerDirState#writeMarkerTypeToFile` sees that the marker dr already exists
so it skips marker type file creation and sets `isMarkerTypeWritten` to true
and goes ahead with writing marker files. For such cases, we need to explicitly
check whether marker type file exists or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
