SequenceFile writer does not properly flush stream with external
DataOutputStream
---------------------------------------------------------------------------------
Key: HADOOP-6339
URL: https://issues.apache.org/jira/browse/HADOOP-6339
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 0.20.1
Reporter: Jonathan Hsieh
When using the SequenceFile.createWriter(..,FSDataOutputStream, ...) method to
create a Writer, data is not flushed when the encapsulating SequenceFile is
closed.
Example test case skeleton:
{code}
public void testWhyFail() throws IOException {
// There a was a failure case using :
Configuration conf = ... ;
Path path = new Path("file:///tmp/testfile");
FileSystem hdfs = path.getFileSystem(conf);
// writing
FSDataOutputStream dos = hdfs.create(path);
hdfs.deleteOnExit(path);
// it is specifically with this writer.
Writer writer = SequenceFile.createWriter(conf, dos,
WriteableEventKey.class, WriteableEvent.class,
SequenceFile.CompressionType.NONE, new DefaultCodec());
Writable value = ...;
Writable key = ...;
writer.append(key, value);
writer.sync();
writer.close();
// Test fails unless I close the underlying FSDataOutputStream handle with
the line below.
// dos.close();
// WTF: nothing written by this writer!
FileStatus stats = hdfs.getFileStatus(path);
assertTrue(stats.getLen() > 0);
// it should have written something but it failed.
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
