nsivabalan commented on code in PR #13399:
URL: https://github.com/apache/hudi/pull/13399#discussion_r2138238721
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/DirectWriteMarkers.java:
##########
@@ -189,26 +188,17 @@ public Option<StoragePath>
createWithEarlyConflictDetection(String partitionPath
private Option<StoragePath> create(StoragePath markerPath, boolean
checkIfExists) {
HoodieTimer timer = HoodieTimer.start();
- StoragePath dirPath = markerPath.getParent();
- try {
- if (!storage.exists(dirPath)) {
- storage.createDirectory(dirPath); // create a new partition as needed.
Review Comment:
do we really need this fix though?
I was looking at the write handles.
looks like we create the partition path if not exists, before calling into
creation of markers.
screenshots attached:
<img width="1326" alt="image"
src="https://github.com/user-attachments/assets/898d2a0c-3f06-4ebe-89d4-f82da22d6c5b";
/>
<img width="1325" alt="image"
src="https://github.com/user-attachments/assets/5d983102-29d7-41ff-a1c5-8c589ae7a908";
/>
ref code links:
https://github.com/apache/hudi/blob/f9555910e5dcb0380569591e6b20f02c1433f24f/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java#L96
https://github.com/apache/hudi/blob/f9555910e5dcb0380569591e6b20f02c1433f24f/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java#L118
So, looks like we don't even need to account for partition dir creation in
direct style markers.
this is also keeping the ownership tight. bcoz, if timeline server based
markers is being used, the write handles anyway has to take care of this which
is what is happening anyways.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
