zianliu66-me opened a new issue, #51674:
URL: https://github.com/apache/arrow/issues/51674
### Describe the bug, including details regarding any error messages,
version, and platform.
The following bug happens specifically on a Python `venv` I created on an
HPC. Python version is 3.13.5 with pyarrow version 25.0.1. This is a
bioinformatics environment aimed to use single-cell packages (such as
scanpy/anndata), so possibly other contributing packages also include
`numpy==2.5.3`, `pandas==3.0.6`, `anndata==0.13.4`.
The following error reproducibly occurs whenever I run
`pyarrow.parquet.ParquetFile(filename)`. However, this error no longer occurs
if I 1) run `pyarrow.parquet.ParquetFile(filename)` _directly_ from the
interpreter, or 2) if I specifically do a try/except loop and re-run
`ParquetFile()` on the except (but it doesn't work with a while loop).
Error trace:
```
Traceback (most recent call last):
File "/globalscratch/<example>/debug_parquetread.py", line 92, in <module>
main()
~~~~^^
File "/globalscratch/<example>/debug_parquetread.py", line 86, in main
pq_example(PQ_SUBSET)
~~~~~~~~~~^^^^^^^^^^^
File "/globalscratch/<example>/debug_parquetread.py", line 51, in
pq_example
parquet_file = pq.ParquetFile(filename)
File
"/globalscratch/<example>/venv/lib/python3.13/site-packages/pyarrow/parquet/core.py",
line 331, in __init__
self.reader.open(
~~~~~~~~~~~~~~~~^
source, use_memory_map=memory_map,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<8 lines>...
arrow_extensions_enabled=arrow_extensions_enabled,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "pyarrow/_parquet.pyx", line 1676, in
pyarrow._parquet.ParquetReader.open
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
OSError: [Errno 14] Error reading bytes from file. Detail: [errno 14] Bad
address
```
See the following script for reproducing this issue. This is guaranteed to
fail on my setup; if I comment out `pq_example()`, the other two functions
behave exactly as described.
```
import os
import pyarrow.parquet as pq
def pq_example(filename: str):
# This always fails.
parquet_file = pq.ParquetFile(filename)
print("This import worked.")
return None
def pq_example_2(filename: str):
# This also always fails, but won't return error due to the try clause.
i = 0
while i < 10:
try:
parquet_file = pq.ParquetFile(filename)
print("Loaded successfully.")
break
except:
print(f"Read failed on loop {i}. Retrying...")
i += 1
pass
return None
def pq_example_working(filename: str):
# This one consistently works on the 'except' read.
try:
parquet_file = pq.ParquetFile(filename)
print("The first import worked.")
except:
print("The first read failed. Retrying:")
parquet_file = pq.ParquetFile(filename)
print("The second read succeeded.")
return None
def main():
pq_example(filename)
pq_example_working(filename)
pq_example_2(filename)
if __name__ == '__main__':
main()
```
The exact OS information:
```
NAME="Rocky Linux"
VERSION="8.10 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.10"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.10 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/";
BUG_REPORT_URL="https://bugs.rockylinux.org/";
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.10"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.10"
```
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
