lesha added the comment:
On a moment's reflection, a lot of my prior comment is wrong. Sorry about that.
- glog does not, that I know of, sanitize locks on fork. You just shouldn't log
after fork but before exec.
- Using `pthread_atfork` to clean up the `logging` lock might be enou
lesha added the comment:
I work on a multi-million-line C++ codebase that uses fork() from multithreaded
programs all over the place. We use `glog` ubiquitously.
This bug here that spans 6 years and has affected dozens of people
(conservatively) simply does not exist for us. That is because
lesha added the comment:
1) I'm totally in favor of making the standard library safe. For that purpose,
I think we should do a combination of:
a) Use file locks in logging, whenever possible.
b) Introduce LockUnsafelyReinitializedAtFork, using a generation counter, or
whatever else,
lesha added the comment:
Actually, we might be able to automatically spawn a safe fork server _only_
when people start mixing threading and subprocess.
I'm not totally sure if this would allow us to salvage multiprocessing as
well...
The tricky bit is that we'd need to proxy int
lesha added the comment:
A slightly more ambitious solution than crashing / deadlocking always is to
have Python automatically spawn a "fork server" whenever you start using
threads.
Then, you would be able to have "subprocess" work cleanly, and not worry about
any of
lesha added the comment:
I feel like I'm failing to get my thesis across. I'll write it out fully:
== Thesis start ==
Basic fact: It is an error to use threading locks in _any_ way after a
fork. I think we mostly agree on this. The programs we discussing are
**inherently buggy**.
W
lesha added the comment:
Re "threading locks cannot be used to protect things outside of a single
process":
The Python standard library already violates this, in that the "logging" module
uses such a lock to protect the file/socket/whatever, to which it is writing.
I
lesha added the comment:
Deadlocks are dandy, but corruption is cruel.
--
___
Python tracker
<http://bugs.python.org/issue6721>
___
___
Python-bugs-list mailin
lesha added the comment:
> I think forkall() on Solaris acts like that, but the normal fork()
> function does not. Only the thread which performs fork() will survive
> in the child process.
Sorry, brain fail. A slightly more contrived failure case is this:
subproc
lesha added the comment:
I am really alarmed by the reinit_locks patches.
I scanned the comment history, and looked at the patch. I may have missed
something, but it looks to me like the basic behavior is this:
"After fork(), all locks are replaced by brand-new lock objects that ar
lesha added the comment:
> So what are you suggesting? That a lock of the default type should
> raise an error if you try to acquire it when it has been acquired in a
> previous process?
I was suggesting a way to make 'logging' fork-safe. No more, no less.
Does what my pr
lesha added the comment:
This is a reply to: http://bugs.python.org/issue6721#msg151266
Charles-François raises two problems:
1) LinuxThreads have different PIDs per thread. He then points out that
LinuxThreads have long been deprecated.
2) you cannot release locks on fork() because that
lesha added the comment:
Just wanted to say that I spent something like 8 hours debugging a subprocess +
threading + logging deadlock on a real production system.
I suspected one of my locks at first, but I couldn't find any. The post-fork
code was very simple, and I didn't su
lesha added the comment:
Actually, I think it does not matter which thread owns the lock, it is still
invalid to try to acquire a lock that was grabbed by the fork() parent. Why?
Because the fork() parent cannot free the child's copy of the lock anyway, and
it's guaranteed to be
New submission from lesha :
Here is a great description of the issue:
http://docs.oracle.com/cd/E19683-01/806-6867/gen-1/index.html
This enhancement proposes a way to make Python more resistant to this kind of
deadlock.
Consider this program:
import threading
import subprocess
import
15 matches
Mail list logo
