Greetings,
Apologies in advance if this is the wrong forum to raise this issue. I would be
happy to file a bug in Jira If more appropriate. I am experiencing issues
accessing files over NFS from Big Sure with pyarrow 5.0.0. I am running an arm
chip, in case that matters. Pyarrow 4.0.1 does work, but when I update to 5.0.0
I immediately receive an alert from the crowdstrike falcon sensor noting that
the arrow_iterator.cypthon-38-darwin.so file cannot be opened because the
developer cannot be verified. This issue does not manifest when run on a host
that is not enrolled in CrowdStrike, nor with pyarrow 4.0.1.
[Graphical user interface Description automatically generated]
Perhaps unrelated, but executing pyarrow.parquet.read_table against an NFS
source in pyarrow 5.0.0 yields the following exception: (pyarrow 4.0.1 succeeds
with the same sample)
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/parquet.py in
read(self, columns, use_threads, use_pandas_metadata)
1742 use_threads = False
1743
-> 1744 table = self._dataset.to_table(
1745 columns=columns, filter=self._filter_expression,
1746 use_threads=use_threads
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/_dataset.pyx in
pyarrow._dataset.Dataset.to_table()
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/_dataset.pyx in
pyarrow._dataset.Scanner.to_table()
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/error.pxi in
pyarrow.lib.pyarrow_internal_check_status()
~/gitlab/cache-access/pyenv/lib/python3.8/site-packages/pyarrow/error.pxi in
pyarrow.lib.check_status()
OSError: [Errno 25] fcntl(fd, F_RDADVISE, ...) failed. Detail: [errno 25]
Inappropriate ioctl for device
A sample call is probably not important; but just in case, a minimalist example
is below. Removing filters makes no difference in my experience.
import pandas as pd
import pyarrow.parquet as pq
from datetime import date
cols = 'RefIssuerID DateActive DateInactive Name CountryOfRisk
PrimaryRefIdentifierID PricingSourceID'.split()
df = pq.read_table('/mnt/dataCacheA/phoenix/v2/reference/issuer/',
filters=[('CountryOfRisk', 'in', ['RU', 'CN']),
('RefIssuerID', 'not in', ['2477', '2771', '116214',
'55955']),
('DateActive', '>=', date(2021, 2, 1)),
('DateInactive', '==', date(9999, 12, 31))
],
columns=cols)
df = df.to_pandas()
Thanks,
Jay
________________________________
This e-Mail and any attachments may contain privileged and confidential
information. If you are not the intended recipient or have received this e-mail
in error, please notify the sender immediately and destroy/delete this e-mail.
You are hereby notified that any unauthorized copying, disclosure or
distribution of the material in this e-mail is strictly prohibited.
Confidentiality and privilege are not lost by this transmission having been
sent or passed on to you in error. Acadian is not liable for any damage that
may be caused by viruses or transmission errors.
This communication is for informational purposes only. It is not intended as an
offer or solicitation for the purchase or sale of any security or an offer to
establish an account. All information contained in this communication is not
warranted as to completeness or accuracy and is subject to change without
notice.
Acadian Asset Management LLC is registered as an investment adviser with the
U.S. Securities and Exchange Commission. Registered Office: 260 Franklin
Street, Boston, Massachusetts 02110. Our Data Protection Notice can be found at
www.acadian-asset.com. Should you no longer wish to receive marketing-related
materials from Acadian please email [email protected].
________________________________
