[
https://issues.apache.org/jira/browse/ARROW-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rok Mihevc updated ARROW-4053:
------------------------------
External issue URL: https://github.com/apache/arrow/issues/20651
> [Python/Integration] HDFS Tests failing with I/O operation on closed file
> -------------------------------------------------------------------------
>
> Key: ARROW-4053
> URL: https://issues.apache.org/jira/browse/ARROW-4053
> Project: Apache Arrow
> Issue Type: Bug
> Components: Integration, Python
> Reporter: Krisztian Szucs
> Assignee: Wes McKinney
> Priority: Major
> Labels: hdfs
> Fix For: 0.12.0
>
>
> To reporoduce run `docker-compose run hdfs-integration`
> {code:python}
> self = <pyarrow.tests.test_hdfs.TestLibHdfs testMethod=test_cat>
> def test_cat(self):
> path = pjoin(self.tmp_path, 'cat-test')
> data = b'foobarbaz'
> with self.hdfs.open(path, 'wb') as f:
> f.write(data)
> > contents = self.hdfs.cat(path)
> opt/conda/lib/python3.6/site-packages/pyarrow/tests/test_hdfs.py:99:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
> opt/conda/lib/python3.6/site-packages/pyarrow/filesystem.py:42: in cat
> return f.read()
> pyarrow/io.pxi:298: in pyarrow.lib.NativeFile.read
> ???
> pyarrow/io.pxi:180: in pyarrow.lib.NativeFile.size
> ???
> pyarrow/io.pxi:142: in pyarrow.lib.NativeFile.get_random_access_file
> ???
> pyarrow/io.pxi:159: in pyarrow.lib.NativeFile._assert_readable
> ???
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
> > ???
> E ValueError: I/O operation on closed file
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
