liangyu-1 commented on code in PR #25226:
URL: https://github.com/apache/flink/pull/25226#discussion_r1764495979
##########
flink-filesystems/flink-hadoop-fs/src/test/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableWriterTest.java:
##########
@@ -83,6 +86,18 @@ static void destroyHDFS() throws Exception {
}
}
+ private RecoverableWriter getNoLocalWriteFileSystemWriter() throws
Exception {
+ return fileSystem.createRecoverableWriter(true);
+ }
+
+ @Test
+ void testNoLocalWrite() throws Exception {
+ final HadoopRecoverableWriter writer =
+ (HadoopRecoverableWriter) getNoLocalWriteFileSystemWriter();
+
+ Assertions.assertTrue(writer.noLocalWrite);
+ }
Review Comment:
I have added a new Unit Test by modeling it after a Hadoop Unit Test
`org.apache.hadoop.hdfs.TestDFSOutputStream#testNoLocalWriteFlag` and removed
the former UT as it cannot prove that the data is not write to the local path.
The difference is that by setting replicas numbers to 3 or by disable
no_local_write, you cannot pass this test.
This unit test must use `Mockito` because when Hadoop attempts to determine
which DataNode is the local node, it looks for it by IP address. In this case,
we are using `MiniCluster`, where all DataNodes share the same IP address
"127.0.0.1," but differ by port. Therefore, we need to use `Mockito` to mock a
method that consistently returns a single DataNode as the local node each time
`getDatanodeByHost("127.0.0.1")` is called.
Additionally, class `DatanodeManager` does not have a public constructor,
which prevents us from creating a subclass. As a result, we cannot implement a
new method,` getDatanodeByHost` in this context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
