Charles-François Natali added the comment:
It's a duplicate of issue #10806, fixed in 2.7, 3.1 and 3.2.
Closing.
--
nosy: +charles-francois.natali
resolution: -> duplicate
stage: -> committed/rejected
status: open -> closed
superseder: -> Subprocess error if fds
Charles-François Natali added the comment:
Actually, it's part of a more general problem with EINTR being returned by many
posix/socket module functions, see for example issue #7978.
On the one hand, having to retry manually on EINTR is cumbersome, on the other
hand, some code might re
Charles-François Natali added the comment:
>> That's because of the _PyIO_ConvertSsize_t converter, which silently
>> converts None to -1.
>> There's probably a good reason for doing this in the _io module
>
> I'm not sure about the original reason, but
Charles-François Natali added the comment:
Hmm, thinking about it, I don't see any reason to make the flags argument
optional.
Here's a patch changing that (also, pipe2 is now declared as METH_O instead of
METH_VARARGS).
--
Added file: http://bugs.python.org/file22265/pipe
Charles-François Natali added the comment:
@nvetoshkin
Could you update your patch against py3k?
I've got a couple comments (can't login to Rietveld, it's probably due to the
change of my tracker account name):
if(flags & ~(SOCK_NONBLOCK | SOCK_CLOEXEC)) {
Charles-François Natali added the comment:
> Ok, the dependencies are now committed. Here is a new patch addressing
> Charles-François' comments: select() is now called before each call to read()
> when sentinels are given, to avoid race conditions.
The patch looks fine to me
Charles-François Natali added the comment:
Patch committed.
Thanks for the report and the patch!
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
New submission from Charles-François Natali :
ossaudiodev's writeall method doesn't check that the FD is less than FD_SETSIZE
when passing it to FD_SET: since FD_SET typically doesn't do bound check, it
will write to a random location in memory (in this case on the stack).
I
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file22285/test_oss.py
___
Python tracker
<http://bugs.python.org/issue12287>
___
___
Python-bug
Charles-François Natali added the comment:
> For subprocess.wait(), we can do something with signals (SIGCHLD and/or
> SIGCLD).
There's just one problem: SIGCHLD is ignored by default, which means that
sigwait and friends won't return when a child exits.
Well, it actually
Charles-François Natali added the comment:
> Antoine is right: we don't have to be portable.
We don't have to, but writing one POSIX-conformant solution is better than
writing N OS-specific solutions, no? That's what POSIX is about.
> Should we block the signal?
Yes.
Charles-François Natali added the comment:
Trying to revive this issue.
I'm +1 on this change: duping the file descriptor is definitely unexpected and
has some side effects.
I think we should remove the call to dup and close, it's really the user's
responsibility to avoid clo
Charles-François Natali added the comment:
I just noticed something "funny": signal_sigwait doesn't release the
GIL, which means that it's pretty much useless :-)
Patch attached.
Also, test_sigwait doesn't block the signal before calling sigwait: it
happens to work
Charles-François Natali added the comment:
> You don't check that 0 <= fd (e.g. oss.close()).
>
Actually, none of ossaudiodev's method does...
This makes it even easier to trigger a segfault (at least on RHEL4):
"""
import ossaudiodev
o = ossaudi
Charles-François Natali added the comment:
>> I just noticed something "funny": signal_sigwait doesn't release
>> the GIL, which means that it's pretty much useless :-)
>
> sigwait() is not useless: the signal is not necessary send by another thread.
>
Charles-François Natali added the comment:
> I don't think the order should be defined.
I disagree.
The C standard explicitely states that the functions are called in LIFO order
(which is the natural order in this case).
The current implementation guarantees LIFO order, I think it s
Charles-François Natali added the comment:
Two patches attached:
- a patch checking that the ossaudiodev object isn't closed
- a patch adding _PyIsSelectable_fd()
--
Added file: http://bugs.python.org/file22324/is_selectable.diff
Added file: http://bugs.python.org/file
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22284/oss_select.diff
___
Python tracker
<http://bugs.python.org/issue12287>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22301/oss_check_closed.diff
___
Python tracker
<http://bugs.python.org/issue12287>
___
___
New submission from Charles-François Natali :
http://www.python.org/dev/buildbot/all/builders/x86 FreeBSD 6.4
3.x/builds/1570/steps/test/logs/stdio
"""
[ 29/356] test_signal
Fatal error 'mutex is on list' at line 540 in file
/usr/src/lib/libpthread/thread/thr_
Charles-François Natali added the comment:
> In oss_check_closed.diff, you might want to add tests for the other
> methods as well.
I've added checks for other methods (not all of them, there are quite a few).
I've also added a check for the mixer object.
> You may add par
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file22327/oss_check_closed.diff
___
Python tracker
<http://bugs.python.org/issue12287>
___
___
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22324/is_selectable.diff
___
Python tracker
<http://bugs.python.org/issue12287>
___
___
Pytho
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22325/oss_check_closed.diff
___
Python tracker
<http://bugs.python.org/issue12287>
___
___
Charles-François Natali added the comment:
I think it might be related to Issue #6721.
Using a mutex/condition variable after fork (from the child process) is unsafe:
it can lead to deadlocks, and on OS-X, it seems like it can lead to segfaults.
Normally, Queue's synchronization primi
Charles-François Natali added the comment:
No one seems to object, and since this approach has been suggested by
Martin and is consistent with the posix module's policy (i.e. a thin
wrapper around the underlying syscall), I guess you can go
Charles-François Natali added the comment:
For oss_check_closed.diff, should I apply it to default only or to every branch?
--
___
Python tracker
<http://bugs.python.org/issue12
Charles-François Natali added the comment:
> Perhaps by adding some new argument to the mmap constructor? (dup_fd = True)
I don't really like the idea of exposing the FD duplication to the
user, because:
- it's an implementation detail
- it doesn't reflect any argument
Charles-François Natali added the comment:
Yes, I also tried this.
This fixed the test_multiprocessing failure, but I think it triggered a failure
in test_concurrent_futures (didin't look in more detail).
Would it be possible to try this on the buildbot to see if it fixes the
segf
Charles-François Natali added the comment:
The patches look good to me, except that instead of passing
(addrlen > buflen) ? buflen : addrlen
as addrlen argument every time makesockaddr is called, I'd prefer if this min
was done inside makesockaddr itself, i.e. perform min(addrlen
Charles-François Natali added the comment:
> If there are only two versions of the structure on all operating
> systems, we can add a check in configure
Isn't it a bit overkill?
OpenBSD is the only OS to define struct kevent that way, adding a
#ifdef __OpenBSD__
check would
Charles-François Natali added the comment:
Patch attached:
- a new process is spawned (using assert_python_ok()) to avoid
undefined behavior (and crash on FreeBSD)
- I've kept SIGUSR1 default handler, because, as noted in
http://bugs.python.org/issue8407#msg138066 , there can be s
New submission from Charles-François Natali :
Linus recently decided that the next Linux kernel version would 3.0 [1].
As a consequence, sys.platform (which is actually MACHDEP) will be 'linux3' on
machines running Linux 3 kernels, and tests checking explicitely against
'linu
Charles-François Natali added the comment:
> I would expect changing sys.platform will also break a lot of third-
> party code.
Maybe, but this would be an application bug.
Here's sys.platform current implementation:
const char *
Py_GetPlatform(void)
{
return PLATFORM;
}
And
Changes by Charles-François Natali :
--
nosy: +lemburg, loewis
___
Python tracker
<http://bugs.python.org/issue12326>
___
___
Python-bugs-list mailing list
Unsub
Charles-François Natali added the comment:
> I'm sure Linus Torvalds is fully aware of the possible
> consequences of the version change, and just accepted the breakage
> that this would cause.
Any application relying on sys.platform == 'linux2' is already broken.
It&
Charles-François Natali added the comment:
This is normal.
You're binding twice to the same port thanks to SO_REUSEADDR, but in case of
multiple binds to the same UDP port, all packets received are duplicated by the
kernel to every socket.
Closing as invalid.
--
nosy: +neo
Charles-François Natali added the comment:
Note that the duplication is mandatory for multicast, but with unicast the
kernel is free to do whatever he wants, most of the time only one socket will
receive it.
--
___
Python tracker
<h
Charles-François Natali added the comment:
> However, it appears to me that vast majority of them is broken anyway,
> because what they really mean to check is
That's exactly my point.
Code checking sys.platform against 'linux2' is already broken, there's
no poin
Charles-François Natali added the comment:
Thanks for reporting this.
There's indeed a bug which can lead to this deadlock.
Relevant code in Lib/multiprocessing/heap.py
- the BufferWrapper class uses a single Heap() shared among instances,
protected by a mutex (threading.Lock), from
Charles-François Natali added the comment:
The patch looks good to me.
In the subprocess, why not use the standard 0 exit code in case of success?
Also, points 1 and 3 could be handled simply by having the parent
process send a signal to the child (but this wouldn't address the
timeout
Charles-François Natali added the comment:
> The obvious solution is to use a recursive lock instead.
Note that it's not really a solution, just a workaround to avoid
deadlocks, become this might lead to a corruption if free is called
while the heap is in an inconsistent state. I have
Charles-François Natali added the comment:
> Looking closely to the gdb stack, there is that frame:
Yeah, but it calls _free, which runs unlocked. That's not the problem.
> I am still unable to reproduce the bug in a simple script.
Try with this one:
"""
import
Charles-François Natali added the comment:
The test still fails on FreeBSD 7.2, Tiger and Debian parallel:
"""
==
FAIL: test_siginterrupt_on (test.test_signal.S
Charles-François Natali added the comment:
Duh, don't know what I was thinking: the syscall is not restarted
(even though ERESTARTSYS is displayed by strace): the real problem is
that the 3s timeout to communicate is not enough, because spawning a
new interpreter can take a long time (An
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file22421/test_siginterrupt.diff
___
Python tracker
<http://bugs.python.org/issue12363>
___
___
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22420/test_siginterrupt.diff
___
Python tracker
<http://bugs.python.org/issue12363>
___
___
Changes by Charles-François Natali :
--
status: closed -> open
___
Python tracker
<http://bugs.python.org/issue12363>
___
___
Python-bugs-list mailing list
Un
Charles-François Natali added the comment:
Here's a patch.
--
keywords: +needs review, patch
stage: -> patch review
Added file: http://bugs.python.org/file22423/kevent_openbsd.diff
___
Python tracker
<http://bugs.python.org
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22423/kevent_openbsd.diff
___
Python tracker
<http://bugs.python.org/issue12181>
___
___
Charles-François Natali added the comment:
Patch attached.
I've used a wildcard so that it won't break when AIX 8 goes out (and it's
POSIX).
--
keywords: +needs review, patch
nosy: +haypo, neologix
stage: -> patch review
Added file: http://bugs.py
Charles-François Natali added the comment:
> Nobody is able to get more information, or maybe nobody wants to dig this
> issue. It would be better to get the real cause of the issue,
Well, it looks like a libc or kernel bug, and we probably can't do
much about it. Just falling ba
Charles-François Natali added the comment:
> Attached patch implements the suggested fix.
The patch looks good to me.
> It would be possible to fix the test to fail instead of blocking
I think this issue deserves a specific test, since:
- test_pending tests something completely dif
Charles-François Natali added the comment:
I'm closing.
Feel free to re-open if this fails with 2.7 or 3.2.
--
resolution: -> duplicate
status: open -> closed
superseder: -> Throw away more radioactive locks that could be held across a
fork i
Charles-François Natali added the comment:
test_2_join_in_forked_process fails on FreeBSD 6.4 buildbot.
http://www.python.org/dev/buildbot/all/builders/x86 FreeBSD 6.4
3.x/builds/1606/steps/test/logs/stdio
"""
Charles-François Natali added the comment:
It's a little bit more complicated than that:
- signals and threads don't mix well together
- this will make syscalls fail with EINTR
- the old SIGALRM handler is lost
- etc
In short, don't use signals.
I'm not sure there'
Charles-François Natali added the comment:
Patch (with test) attached. It disables the garbage collector inside
critical sections.
Of course, if another thread re-enables the gc while the current
thread is inside a critical section, things can break (the probability
is really low, but who knows
Changes by Charles-François Natali :
--
keywords: +patch
Added file: http://bugs.python.org/file22476/heap_gc_deadlock.diff
___
Python tracker
<http://bugs.python.org/issue12
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22476/heap_gc_deadlock.diff
___
Python tracker
<http://bugs.python.org/issue12352>
___
___
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file22477/heap_gc_deadlock.diff
___
Python tracker
<http://bugs.python.org/issue12352>
___
___
Charles-François Natali added the comment:
[...]
> def free(self, block):
> if self._lock.acquire(False):
> self._exec_pending_free()
> self._free(block)
> else:
> # malloc holds the lock
> self._pending_free.append(block)
>
_pending_free uses a lo
Charles-François Natali added the comment:
> This skip gives very few information, and it is duplicated for each
> function. I would prefer a constant of the "broken OSes" with your
> following comment attached to the constant:
Ok, I'll try to write something alon
New submission from Charles-François Natali :
As noted in issue #11870, making faulthandler capable of dumping child
processes' tracebacks could be a great aid in debugging tricky deadlocks
involving for example multiprocessing and subprocess.
Since there's no portable way to find
Charles-François Natali added the comment:
[...]
> I don't like touching such global "variable", but you are right.
>
Well, I don't like it either, but I can't really think of any other solution.
Antoine, any thought on that?
--
___
Changes by Charles-François Natali :
--
resolution: -> out of date
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.o
Charles-François Natali added the comment:
> You are probably right. Can't we use a lock-less list? list.append is
> atomic thanks to the GIL, isn't it? But I don't know how to implement
> the lock-less list consumer. It would be nice to have a function to
> remove an
Charles-François Natali added the comment:
> You mean that the tracebacks of children should be dumped on a timeout of the
> parent? Or do you also want them on a segfault of the parent? In my
> experience, the most common problem with the multiprocessing and subprocess
> modules
Charles-François Natali added the comment:
> So people who say sys.platform shouldn't be used: what do you propose
> to do with Lib/plat-linux2 (or, more generally, Lib/plat-*)?
I can't speak, as I've never used those.
But can't those directories be renamed
Charles-François Natali added the comment:
Patch attached.
--
Added file: http://bugs.python.org/file22489/test_threading_fork.diff
___
Python tracker
<http://bugs.python.org/issue11870>
___diff -r fca745
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22468/test_threading_fork.diff
___
Python tracker
<http://bugs.python.org/issue11
Charles-François Natali added the comment:
Here's a patch based on the second approach.
--
Added file: http://bugs.python.org/file22490/heap_gc_deadlock_lockless.diff
___
Python tracker
<http://bugs.python.org/is
Charles-François Natali added the comment:
> That would be incorrect for some systems. For example, FreeBSD does
> change sets of symbolic constants across system releases (mostly
> additions, but sometimes also removals). Back then, SunOS 4 and SunOS
> 5 were completely unrel
Charles-François Natali added the comment:
> Nice work! I also think heap_gc_deadlock_lockless.diff is good, except for
> Victor's reservation: is it deliberate that you reversed the following two
> statements in _free_pending_blocks(), compared to the code in free()?
>
&g
Charles-François Natali added the comment:
> Oh, the problem is that sigwait() behaviour changes after a fork: it is
> interrupted if an unexpected signal is received, but the signal handler is
> not called. It behaves correctly (the signal handler is called) without the
> fork
Charles-François Natali added the comment:
> The way I see it is that Charles-François' patch trades a possibility of a
> deadlock for a possibility of a child process running with inconsistent
> states due to forcibly reinitialized locks.
>
Yeah, that's why I let thi
Charles-François Natali added the comment:
> Well, I ping my view that we should:
>
Could you please detail the following points:
- what would be the API of this atfork() mechanism (with an example of
how it would be used in the library)?
- how do you find the correct order to acquire lo
Charles-François Natali added the comment:
> The initial problem was test_3_join_in_forked_from_thread() and the hangs
> does still happen:
>
Yes, the patch was there to fix test_2_join_in_forked_from_thread.
> [324/356] test_threading
> Timeout (1:00:00)!
> Thread 0x40424
Charles-François Natali added the comment:
>> - how do you find the correct order to acquire locks in the parent process?
>
> One option is to use the import graph to determine call order of atfork
> handlers.
> If a current std library does not fit into this scheme we c
Charles-François Natali added the comment:
longjmp() is used in only two places:
./Modules/fpectlmodule.c:longjmp(PyFPE_jbuf, 1);
./Modules/readline.c:longjmp(jbuf, 1);
Both use it to jump out of a signal handler, which can lead to undefined
behaviour (see
https
Charles-François Natali added the comment:
That's because struct timeval is not defined by on IRIX, or it
doesn't get included in Modules/signalmodule.c.
Can you try to compile the following code with the same compiler/options?
"""
#include
int main(int argc, c
Charles-François Natali added the comment:
Ok, what happens if you change, in Modules/signalmodule.c
20 #ifdef HAVE_SYS_TIME_H
21 #include
22 #endif
to
21 #include
and rebuild Python?
--
___
Python tracker
<http://bugs.python.
Charles-François Natali added the comment:
What happens if you create the thread after having registered the SIGBUS
handler?
--
___
Python tracker
<http://bugs.python.org/issue12
Charles-François Natali added the comment:
You can't include both and on IRIX...
Nice one!
A couple suggestions (in this order):
1) try putting "#include " before "#include "Python.h""
2) try this:
20 #undef select
21 #include
3) fix the header yourse
Charles-François Natali added the comment:
>> On FreeBSD 6, os.kill(os.getpid(), signum) calls immediatly
>> the signal handler before the creation of the first thread (...),
>> whereas the signal handler is called "later" (when exactly?) after
>> the creation
Charles-François Natali added the comment:
Updated patch.
--
Added file: http://bugs.python.org/file22545/heap_gc_deadlock_lockless.diff
___
Python tracker
<http://bugs.python.org/issue12352>
___d
Changes by Charles-François Natali :
Added file: http://bugs.python.org/file22546/heap_gc_deadlock_lockless.diff
___
Python tracker
<http://bugs.python.org/issue12
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22477/heap_gc_deadlock.diff
___
Python tracker
<http://bugs.python.org/issue12352>
___
___
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22490/heap_gc_deadlock_lockless.diff
___
Python tracker
<http://bugs.python.org/issue12
Changes by Charles-François Natali :
Removed file: http://bugs.python.org/file22545/heap_gc_deadlock_lockless.diff
___
Python tracker
<http://bugs.python.org/issue12
Charles-François Natali added the comment:
Digging a little deeper:
- in ./Modules/fpectlmodule.c, the longjmp() is actually not used at all (dead
code)
- in Modules/readline.c, the jmp_buf is correctly initialized (well, there's a
tiny race condition because SIGINT handler is inst
Charles-François Natali added the comment:
And the backtrace leaves no doubt:
=== Backtrace: =
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f2415de61d7]
/lib/x86_64-linux-gnu/libc.so.6(+0xfe169)[0x7f2415de6169]
/lib/x86_64-linux-gnu/libc.so.6(__longjmp_chk+0x33
Charles-François Natali added the comment:
> Revert the patch doesn't change anything for the test suite.
I know, but at least it doesn't change the default - be it broken - behaviour
on FreeBSD 6.
> I just fear that it is a little bit overkill just to support an "old&
Charles-François Natali added the comment:
The test passes on all the buildbots, closing.
greg, thanks for reporting this!
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bug
Charles-François Natali added the comment:
> I am pretty sure this is another instance of issue 6721.
Indeed.
Closing as duplicate.
--
status: open -> closed
superseder: -> Locks in python standard library should be sanitized on fork
_
Changes by Charles-François Natali :
--
keywords: +needs review
stage: needs patch -> patch review
___
Python tracker
<http://bugs.python.org/issue8035>
___
_
Charles-François Natali added the comment:
That's because the other end of the pipe (p1) is open in the child process (FDs
are inherited on fork()).
Just add
p1.close()
at the beginning of fn() and you'll get EOF.
Closing as invalid.
--
nosy: +neologix
resolution: ->
Charles-François Natali added the comment:
The patch looks good to me.
A couple remarks:
- could you please post it as a Mercurial diff? (it makes it easier to
review and apply)
@@ -1151,6 +1151,25 @@ makesockaddr(int sockfd, struct sockaddr
}
+ return Py_BuildValue(&qu
Charles-François Natali added the comment:
Well, in this regard it behaves like a Unix pipe/socket (in the duplex case
it's implemented with a Unix domain socket), so I find it quite natural (of
course, you have to know about FD inheritance upon fork()).
I'm not convinced it&#
Charles-François Natali added the comment:
> subprocess doesn't use a shell by default, and I don't think that
> multiprocessing uses a shell to start Python.
>
No, but we precisely want subprocess/multiprocessing-created processes
to be in the same process group.
Charles-François Natali added the comment:
Alright.
Luke, if you're motivated, feel free to provide a patch.
The relevant file is Doc/library/multiprocessing.rst.
--
___
Python tracker
<http://bugs.python.org/is
501 - 600 of 2227 matches
Mail list logo
