N Gautam Animesh created ARROW-18313:
----------------------------------------
Summary: Issues with open_dataset()
Key: ARROW-18313
URL: https://issues.apache.org/jira/browse/ARROW-18313
Project: Apache Arrow
Issue Type: Bug
Reporter: N Gautam Animesh
Attachments: image-2022-11-11-09-19-16-065.png
On using open_dataset, it creates a connection due to which the files in the
directory get blocked and we cannot perform other operations on the file like
replace!
Actual issue:
# We are running an atomic operation on a bunch of files, which replaces the
temp file names to the target file names.
# But while this is happening, if we try to run open_dataset() on that
particular directory, the atomic operation is failing and there are both target
files and temp files in the directory.
# It is blocking the files that have been read through open_dataset().
# Please, provide me with more about how we can handle such problems.
# Snapshot: !image-2022-11-11-09-19-16-065.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
