Undocumented issue: Open system call blocks on named pipes (and a feature request)

[Daniel Ojalvo via Python-list]

Hello,

I've been working on a python3 project and I came across an issue with the open 
system call that, at the very least, isn't documented. In my humble opinion, 
the documentation should 
be updated because folks wouldn't expect open to be a blocking operation and 
simply error out. Worse yet, open doesn't have an option to make itself 
non-blocking. You have to use the os system calls to kludge a solution.

Here is how I reproduced the issue:

root@beefy:~/sandbox# mkfifo this_is_a_pipe
root@beefy:~/sandbox# ls -l this_is_a_pipe
prw-r--r-- 1 root root 0 Dec 27 14:28 this_is_a_pipe
root@beefy:~/sandbox# python3 --version
Python 3.6.7
root@beefy:~/sandbox# python3
Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> open("this_is_a_pipe")


The mitigation is to use the os calls and specify to be nonblocking when 
opening the file.

I'm doing this to get a fileobject and make it error out if we do have a 
blocking special file:
with os.fdopen(os.open(, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') as 
file_obj:

I think this is mostly a documentation bug because this wouldn't be expected 
behavior to someone reading the docs, but open is behaving as the fifo man 
page is documented. The 
feature request would be to add a non-blocking option to the default open 
system call.

I've also found this with some named special character files, but I don't have 
a reproduction at the moment.

Thank you and have a good day!
Dan
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Undocumented issue: Open system call blocks on named pipes (and a feature request)

[Daniel Ojalvo via Python-list]

Thank you for the advice!

I haven't used the opener argument before, but I'll keep it for future 
reference. I think it's still a little kludge-y, but it works.

I agree that previous behavior shouldn't be changed, but I would suggest 
updating the documentation to point it out as a footnote. The current behavior 
is correct just unclear. Most people just learning about the open command 
wouldn't have this expectation. I came across the issue when I had a program 
that would open up all the files in a directory to read a few bytes from the 
beginning. My concern would be someone just making a named pipe over a file 
that a program would open. Arguably, anyone affected by that would be shooting 
themselves in the foot to begin with, but I think there are "security" concerns 
because someone could cause a bit of mischief that would be difficult to 
diagnose.

That all being said, I think I would like to put in a feature request for a 
non-blocking option. How should I go about doing so?

Thanks again,
Dan

-Original Message-
From: Chris Angelico  
Sent: Thursday, December 27, 2018 7:10 PM
To: python-list@python.org
Subject: Re: Undocumented issue: Open system call blocks on named pipes (and a 
feature request)

On Fri, Dec 28, 2018 at 1:38 PM Daniel Ojalvo via Python-list 
 wrote:
>
> Hello,
>
> I've been working on a python3 project and I came across an issue with the 
> open system call that, at the very least, isn't documented. In my humble 
> opinion, the 
> documentation<https://docs.python.org/3/library/functions.html#open> should 
> be updated because folks wouldn't expect open to be a blocking operation and 
> simply error out. Worse yet, open doesn't have an option to make itself 
> non-blocking. You have to use the os system calls to kludge a solution.
>

Hmm. I disagree that the docs are deceptive here; I would normally expect 
open() to block if it needs to. But looking at this as a feature request, it 
seems reasonable. Actually, it's not even that hard to do, since open() is 
already pluggable:

rosuav@sikorsky:~/tmp$ rm rene_magritte
rosuav@sikorsky:~/tmp$ mkfifo rene_magritte rosuav@sikorsky:~/tmp$ ls -l 
rene_magritte
prw-r--r-- 1 rosuav rosuav 0 Dec 28 14:05 rene_magritte rosuav@sikorsky:~/tmp$ 
python3 Python 3.8.0a0 (heads/master:8b9c33ea9c, Nov 20 2018, 02:18:50) [GCC 
6.3.0 20170516] on linux Type "help", "copyright", "credits" or "license" for 
more information.
>>> import os
>>> def nonblock(fn, mode): return os.open(fn, mode | os.O_NONBLOCK)
...
>>> open("rene_magritte", opener=nonblock)
<_io.TextIOWrapper name='rene_magritte' mode='r' encoding='UTF-8'>
>>> _.read(1)
''

> Here is how I reproduced the issue:
>
> root@beefy:~/sandbox# mkfifo this_is_a_pipe

(my example file name is a more subtle reference...)

> I'm doing this to get a fileobject and make it error out if we do have a 
> blocking special file:
> with os.fdopen(os.open(, os.O_RDONLY| os.O_NONBLOCK) , mode='rb') 
> as file_obj:
>
> I think this is mostly a documentation bug because this wouldn't be expected 
> behavior to someone reading the docs, but open is behaving as the fifo man 
> page<http://man7.org/linux/man-pages/man7/fifo.7.html> is documented. The 
> feature request would be to add a non-blocking option to the default open 
> system call.
>

Honestly, I don't think there's a problem with it blocking by default.
Most of Python works that way. But it would be pretty straight-forward to add 
"nonblocking=False" as another keyword-only parameter, and for compatibility 
with existing versions (back as far as 3.3), the opener should work just fine.

ChrisA

-- 
https://mail.python.org/mailman/listinfo/python-list