Zhe Zhang created HDFS-10854:
--------------------------------
Summary: Remove createStripedFile and addBlockToFile by creating
real EC files
Key: HDFS-10854
URL: https://issues.apache.org/jira/browse/HDFS-10854
Project: Hadoop HDFS
Issue Type: Bug
Components: erasure-coding, test
Affects Versions: 3.0.0-alpha2
Reporter: Zhe Zhang
{{DFSTestUtil#createStripedFile}} and {{addBlockToFile}} were developed before
we completed EC client. They were used to test the {{NameNode}} EC logic when
the client was unable to really create/read/write EC files.
They are causing confusions in other issues about {{NameNode}}. For example, in
one of the patches under {{HDFS-10301}},
{{testProcessOverReplicatedAndMissingStripedBlock}} fails because in the test
we fake a block report from a DN, with a randomly generated storage ID. The DN
itself is never aware of that storage. This is not possible in a real
production environment.
{code}
DatanodeStorage storage = new
DatanodeStorage(UUID.randomUUID().toString());
StorageReceivedDeletedBlocks[] reports = DFSTestUtil
.makeReportForReceivedBlock(block,
ReceivedDeletedBlockInfo.BlockStatus.RECEIVING_BLOCK, storage);
for (StorageReceivedDeletedBlocks report : reports) {
ns.processIncrementalBlockReport(dn.getDatanodeId(), report);
}
{code}
Now that we have a fully functional EC client, we should remove the old testing
logic and use similar logic as non-EC tests (creating real files and emulate
blocks missing / being corrupt).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
