Re: Lock acquisition by the same thread - deadlock protection

2020-03-11 Thread Yonatan
That code I'm talking about didn't require a reentrant lock - the
algorithm really wasn't reentrant.

Let me clarify my point: I'm wondering why the non-reentrant lock
doesn't raise an exception immediately on this
erroneous situation.
I thought it could be altered, or at least we could add an option to
let a `threading.Lock` behave like a pthread
mutex in mode `PTHREAD_MUTEX_ERRORCHECK`: Disallow double locking by
same thread, disallow unlocking
by another thread.
However, after searching a bit more, I found a reference mentioning
current behavior in a docstring defined
in `_threadmodule.c`: "A lock is not owned by the thread that locked
it; another thread may unlock it.".
It was added in 75e9fc31d3a18068, a commit from 1998...

Since it's a well-documented behavior I guess it's here to stay. At
least the "unlock by another thread" part.
But I question the double locking.


On Tue, Mar 10, 2020 at 5:07 PM Barry Scott  wrote:
>
>
>
> > On 9 Mar 2020, at 22:53, Yonatan Goldschmidt  
> > wrote:
> >
> > I recently debugged a program hang, eventually finding out it's a deadlock 
> > of a single thread,
> > resulting from my usage of 2 libraries. One of them - call it library A - 
> > is reentrant & runs code in
> > GC finalizers, while the other - library B - is not reentrant at all.
> > Library B held one of its `threading.Lock` locks, and during this period, 
> > GC was invoked, running
> > finalizers of library A which call back into library B, now attempting to 
> > take the lock again,
> > locking the thread forever.
> >
> > Considering how relatively common this scenario might be (Python, by 
> > design, can preempt any user code
> > to run some other user code, due to GC finalizers), I was surprised Python 
> > code is not protected
> > from this simple type of deadlock. It makes sense that while 
> > `threading.RLock` allows for recursive
> > locking, `threading.Lock` will prevent it - raising an exception if you 
> > attempt it.
> >
> > I might be missing something, but why isn't it the status? Why taking a 
> > `threading.Lock` twice from
> > the same thread just hangs, instead of raising a friendly exception?
> > I ran a quick search in bpo but found nothing about this topic. I also 
> > tried to
> > search this mailing list but couldn't find how to, so I grepped a few 
> > random archives
> > but found nothing about it.
> >
> > Would be happy if anyone could shed some light on it...
>
> threading.Lock is not reentrant and its implementation does not allow 
> detection of the problem
> from what I recall.
>
> In this case the code might want to use the threading.RLock that is reentrant.
> Of course there may be other issues in the code that prevent the finalizers 
> working if it holds the lock.
>
> Barry
>
>
>
-- 
https://mail.python.org/mailman/listinfo/python-list


new feature in Python.

2020-09-30 Thread yonatan
Hi, My name is Jonatan and i am programming in Python for about 4 years, 
I have a great idea, there are __iX__` methods, such as `__ior__`, `__iadd__`, 
`__iand__` etc.., which implements the |=, +=, &= behavior, 
it would be nice if you could implement also `__igetattr__` or something, which 
means:

instead of
con = "some text here"
con  = con.replace("here", "there")

we could do

con = "some text here"
con  .= replace("here", "there")


Please let me know what do you think about it, Jonatan.
-- 
https://mail.python.org/mailman/listinfo/python-list


Lock acquisition by the same thread - deadlock protection

2020-03-09 Thread Yonatan Goldschmidt
I recently debugged a program hang, eventually finding out it's a deadlock of a 
single thread,
resulting from my usage of 2 libraries. One of them - call it library A - is 
reentrant & runs code in
GC finalizers, while the other - library B - is not reentrant at all.
Library B held one of its `threading.Lock` locks, and during this period, GC 
was invoked, running
finalizers of library A which call back into library B, now attempting to take 
the lock again,
locking the thread forever.

Considering how relatively common this scenario might be (Python, by design, 
can preempt any user code
to run some other user code, due to GC finalizers), I was surprised Python code 
is not protected
from this simple type of deadlock. It makes sense that while `threading.RLock` 
allows for recursive
locking, `threading.Lock` will prevent it - raising an exception if you attempt 
it.

I might be missing something, but why isn't it the status? Why taking a 
`threading.Lock` twice from
the same thread just hangs, instead of raising a friendly exception?
I ran a quick search in bpo but found nothing about this topic. I also tried to
search this mailing list but couldn't find how to, so I grepped a few random 
archives
but found nothing about it.

Would be happy if anyone could shed some light on it...

Thanks,
Yonatan
-- 
https://mail.python.org/mailman/listinfo/python-list