steveloughran opened a new issue, #3203:
URL: https://github.com/apache/parquet-java/issues/3203
### Describe the bug, including details regarding any error messages,
version, and platform.
HadoopPositionOutputStream calls FSDataOutputStream.hflush() in close()
#1309 added an `hflush()` when closing a stream. This is needless and
somewhat troublesome
* hdfs: extra overhead of a blocking write to all datanodes participating in
the write (it doesn't know that close() is about to follow, after all)
* abfs: overhead of another write
* s3a: prints a message warning user that the Syncable api is unsupported.
(once per process...we don't want log spam)
If anyone has a deployment with the setting
`fs.s3a.downgrade.syncable.exceptions=false` then s3a will actually fail. This
is the "fail fast if anyone tries to run algorithms expecting Syncable to
persist data". it's a great way to find code which does exactly this.
And it won't be the cause of the problem. A simple flush() should suffice,
if that is the underlying problem anyway.
### Component(s)
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
