[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
New submission from Hans Lawrenz:
Inside a virtualbox vm, calling tempfile.TemporaryFile(dir=foo) where foo is a
directory which resides on a volume mounted from the host OS, a
FileNotFoundError exception is thrown.
In the following code sample, the second block will print "Path 2: ERROR" on
Python 3.5 but not on 3.4 or 2.7 (assuming the vagrant environment provided
below):
import tempfile
try:
with tempfile.TemporaryFile() as tf:
tf.write("testing testing testing\n".encode('utf-8'))
print("Path 1: worked")
except FileNotFoundError as e:
print("Path 1: ERROR")
try:
with tempfile.TemporaryFile(dir="/vagrant") as tf:
tf.write("testing testing testing\n".encode('utf-8'))
print("Path 2: worked")
except FileNotFoundError as e:
print("Path 2: ERROR")
A runnable test case can be found here:
https://github.com/hlawrenz/py35tempfiletest
--
components: Library (Lib)
messages: 255246
nosy: Hans Lawrenz
priority: normal
severity: normal
status: open
title: tempfile.TemporaryFile fails when dir option set to directory residing
on host OS mount
type: behavior
versions: Python 3.5
___
Python tracker
<http://bugs.python.org/issue25717>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment:
Host OS: Mac OS 10.11.1
Guest OS: Ubuntu Trusty 64
Virtualbox: 4.3.30
Vagrant: 1.7.4
Example with trace thrown:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ cat tt.py
import tempfile
with tempfile.TemporaryFile(dir="/vagrant") as tf:
tf.write("testing testing testing\n".encode('utf-8'))
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python3.5 tt.py
Traceback (most recent call last):
File "tt.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python3.5/tempfile.py", line 753, in TemporaryFile
newline=newline, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory
I think the underlying system call that's causing the error (I attached the
full strace in case that's helpful):
open("/vagrant", O_RDWR|O_EXCL|O_DIRECTORY|O_NOFOLLOW|O_CLOEXEC|0x40) = -1
EOPNOTSUPP (Operation not supported)
Finally, the mount point looks like this:
vagrant on /vagrant type vboxsf (uid=1000,gid=1000,rw)
--
Added file: http://bugs.python.org/file41145/tempfile-strace.txt
___
Python tracker
<http://bugs.python.org/issue25717>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: Unfortunately changing the tempfile call isn't an easy option for me. The situation in which I'm encountering the error is running tox on our project which is mounted in the /vagrant directory in the VM (this is standard for vagrant). Tox makes its own temp directory--presumably for test isolation--in the .tox directory in the project root. The actual call to tempfile.TemporaryFile() which is triggering the error is in a third party library that is called during a test run. I was able to work around the issue by changing the tox workdir setting to outside the mount. However, I'd like to mention that developing in vagrant in this fashion isn't uncommon and once 3.5 gains adoption I would guess this issue may affect a good number of people. I'm happy to take a stab at writing a patch but looking at the code it's somewhat out of my comfort zone and I worry I'd make a hash of it. -- ___ Python tracker <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment:
Serhiy and Emanuel, I'll paste below the surrounding code and attach the exact
tempfile.py. It is the version distributed with the 3.5.0 release. If you take
a look at the github repo I linked in the first comment you can also try it out
for yourself if you've got vagrant and virtualbox installed. I was able to
recreate the error with the provisioning compiling python and with it
installing from a ppa. You can see the details in the Vagrantfile. I also had a
coworker test in a nearly identical environment and he had the same result.
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
try:
_os.unlink(name)
return _io.open(fd, mode, buffering=buffering,
newline=newline, encoding=encoding)
except:
_os.close(fd)
raise
One final note. I decided to try this out with a windows host and the only
clarity this test adds is that the problem doesn't show up there but it fails
for its own reasons on all the python versions I tried. See below for reference:
Last login: Tue Nov 24 17:16:12 2015 from 10.0.2.2
vagrant@vagrant-ubuntu-trusty-64:~$ cat /vagrant/foo.py
import tempfile
with tempfile.TemporaryFile(dir="/vagrant") as tf:
tf.write("testing testing testing\n".encode('utf-8'))
print("Path 2: worked")
vagrant@vagrant-ubuntu-trusty-64:~$ python /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python2.7/tempfile.py", line 495, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmpvx7Mie'
vagrant@vagrant-ubuntu-trusty-64:~$ python2.7 /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python2.7/tempfile.py", line 495, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmpNXQ6Cf'
vagrant@vagrant-ubuntu-trusty-64:~$ python3.4 /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python3.4/tempfile.py", line 638, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmpfwhj7ip4'
vagrant@vagrant-ubuntu-trusty-64:~$ python3.5 /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python3.5/tempfile.py", line 751, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmp02s19r_a'
vagrant@vagrant-ubuntu-trusty-64:~$ python2.7 --version
Python 2.7.6
vagrant@vagrant-ubuntu-trusty-64:~$ python3.4 --version
Python 3.4.3
vagrant@vagrant-ubuntu-trusty-64:~$ python3.5 --version
Python 3.5.0
--
Added file: http://bugs.python.org/file41153/tempfile.py
___
Python tracker
<http://bugs.python.org/issue25717>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: The file system causing the problem is of type vboxsf which is the Virtualbox shared folder file system type. -- ___ Python tracker <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: Emanuel, sorry, I missed the request for sys.version earlier. The tempfile.py I attached earlier is from the python 3.5 pulled from a ppa. I wouldn't be surprised if it has some patches applied by debian/ubuntu. To be clear though the problem also presents itself with a python 3.5 I compiled. The Vagrantfile in the github repository shows exactly how I compiled it. Here it is from the version from the ppa: Python 3.5.0 (default, Sep 17 2015, 00:00:00) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.version '3.5.0 (default, Sep 17 2015, 00:00:00) \n[GCC 4.8.4]' >>> sys.implementation namespace(_multiarch='x86_64-linux-gnu', cache_tag='cpython-35', hexversion=50659568, name='cpython', version=sys.version_info(major=3, minor=5, micro=0, releaselevel='final', serial=0)) Here are the same for the version I compiled: vagrant@vagrant-ubuntu-trusty-64:~$ python3.5 Python 3.5.0 (default, Nov 24 2015, 19:50:42) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.version '3.5.0 (default, Nov 24 2015, 19:50:42) \n[GCC 4.8.4]' >>> sys.implementation namespace(cache_tag='cpython-35', hexversion=50659568, name='cpython', version=sys.version_info(major=3, minor=5, micro=0, releaselevel='final', serial=0)) -- ___ Python tracker <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: Martin, I applied your patch and it proved your hypothesis. See below for how I tested. I also updated the github repo for others to reproduce if they wish. cd wget https://www.python.org/ftp/python/3.5.0/Python-3.5.0.tar.xz mkdir ~/dist cd ~/dist tar xJf ~/Python-3.5.0.tar.xz cd Python-3.5.0 ./configure --prefix=/home/vagrant/py35/dist && \ make && make install mkdir ~/patch cd ~/patch tar xJf ~/Python-3.5.0.tar.xz cd Python-3.5.0 patch -p1 < /vagrant/fstat-failure.patch ./configure --prefix=/home/vagrant/py35/patch && \ make && make install vagrant@vagrant-ubuntu-trusty-64:~$ ./py35/dist/bin/python3.5 /vagrant/temptest.py Path 1: worked Path 2: ERROR vagrant@vagrant-ubuntu-trusty-64:~$ ./py35/patch/bin/python3.5 /vagrant/temptest.py Path 1: worked Path 2: worked -- ___ Python tracker <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26796] BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
New submission from Hans Lawrenz: In issue 21527 <http://bugs.python.org/issue21527> the concurrent.futures.ThreadPoolExecutor was changed to have a default value for max_workers. When asyncio.base_events.BaseEventLoop.run_in_executor creates a default ThreadPoolExecutor it specifies a value of 5 for max_workers, presumably because at the time it was written ThreadPoolExecutor didn't have a default for max_workers. This is confusing because on reading the documentation for ThreadPoolExecutor one might assume that the default specified there is what will be used if a default executor isn't supplied via BaseEventLoop.set_default_executor. I propose that BaseEventLoop.run_in_executor be changed to not supply a default for max_workers. If this isn't acceptable, a note ought to be put in the run_in_executor documentation. -- components: asyncio files: run_in_executor_max_workers.patch keywords: patch messages: 263669 nosy: Hans Lawrenz, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file42506/run_in_executor_max_workers.patch ___ Python tracker <http://bugs.python.org/issue26796> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26796] BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
Hans Lawrenz added the comment: Thanks, that makes sense. I've attached a patch with a version check. -- Added file: http://bugs.python.org/file42507/run_in_executor_max_workers_vcheck.patch ___ Python tracker <http://bugs.python.org/issue26796> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26796] BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
Hans Lawrenz added the comment: New patch attached. Includes comments and a note in the documentation. The documentation note is inside a versionchanged:: 3.5 block. Should this be more specific about the version it changed in? It could be confusing for someone using a version of 3.5 that doesn't have the change. Contributor form is signed. Thanks! -- Added file: http://bugs.python.org/file42509/run_in_executor_max_workers_with_docs.patch ___ Python tracker <http://bugs.python.org/issue26796> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
