TheR1sing3un commented on code in PR #12389:
URL: https://github.com/apache/hudi/pull/12389#discussion_r1884979611
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/bucket/ConsistentBucketIndexUtils.java:
##########
@@ -184,12 +184,31 @@ public static boolean saveMetadata(HoodieTable table,
HoodieConsistentHashingMet
StoragePath dir = FSUtils.constructAbsolutePath(
table.getMetaClient().getHashingMetadataPath(),
metadata.getPartitionPath());
StoragePath fullPath = new StoragePath(dir, metadata.getFilename());
+ byte[] bytes;
try {
- storage.createImmutableFileInPath(fullPath,
Option.of(metadata.toBytes()));
- return true;
+ bytes = metadata.toBytes();
} catch (IOException e) {
+ LOG.error("Error when converting hashing metadata: {} to bytes",
metadata.getPartitionPath(), e);
+ throw new HoodieIndexException("Error while converting hashing metadata
to bytes", e);
+ }
+ try {
+ storage.createImmutableFileInPath(fullPath, Option.of(bytes));
+ return true;
+ } catch (HoodieIOException e) {
LOG.warn("Failed to update bucket metadata: " + metadata, e);
- return false;
+ boolean exist;
+ try {
+ exist = storage.exists(fullPath);
Review Comment:
> can you elaborate why the test case fails without changing the parallelism?
For example, if the parallelism level is 2, two tasks load the index mapping
relationship respectively. task1 finds that there is no valid metadata when
loading metadata, and creates an initial metadata file. When task1 creates the
file but has not finished writing the content, task2 loads the mapping and
reads the empty file created by task1, causing an exception
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
