[issue33713] memoryview can set an exception in tp_clear

Change by Irit Katriel : -- versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.6, Python 3.7, Python 3.8 ___ Python tracker ___

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: Okay that makes sense. :) I looked a bit at the gc code. A consumer object always has one reference to a memoryview with an export, which isn't visited. So it looks to me that the gc_refs of that memoryview cannot fall to 0. So memory_clear() isn't called in tha

[issue33713] memoryview can set an exception in tp_clear

Antoine Pitrou added the comment: Yes, tp_clear can be called with refcount > 0. It's exactly why it's separate from tp_dealloc, actually :-) -- ___ Python tracker ___ __

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: Well yes, I still want to understand tp_clear(). :) The docs are a bit vague. -- ___ Python tracker ___ ___

[issue33713] memoryview can set an exception in tp_clear

STINNER Victor added the comment: If the bug cannot occur, just add "assert(!PyErr_Occurred());" no? -- nosy: +vstinner ___ Python tracker ___

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: Well, the example would need exports: >>> a = [bytes()] >>> a.append(memoryview(a[0])) >>> a.append(memoryview(a[1])) >>> a.append(a) >>> a [b'', , , [...]] The first memoryview has one export, so its refcount > 0. Do I fundamentally misunderstand tp_clear() a

[issue33713] memoryview can set an exception in tp_clear

Antoine Pitrou added the comment: Serhiy is right about the theoretical concern here. However, it's probably quite difficult to find a concrete situation where this occurs, because we're talking about mbuf_clear and the managerbuffer object can't really get involved in a reference cycle by

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: The point is that *no garbage collection is triggered* if self->exports > 0. It would be a major bug if it were and I suspect it would be reported within a week. Fortunately, no such bug has been reported in 6 years. -- ___

[issue33713] memoryview can set an exception in tp_clear

Serhiy Storchaka added the comment: The GC calls tp_clear() if the memoryview is a part of the reference loop. a = [memoryview(...)] a.append(a) del a The GC will call tp_clear() of the list or the memoryview. What will be called first is not specified. -- _

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: Yes, but who calls tp_clear() if the memoryview is not being deallocated? -- ___ Python tracker ___

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: This looks the same as #25525. I think it cannot happen, and no one has ever reported an actual issue for 6 years now. You *really* need to show a reproducer if you assert that something can crash. -- ___ Python tra

[issue33713] memoryview can set an exception in tp_clear

Serhiy Storchaka added the comment: See the delete_garbage() function line 770 in Modules/gcmodule.c for changes in the master branch relevant to this issue. See Py_FatalError() in the collect() function at line 974 for a crash. -- ___ Python trac

[issue33713] memoryview can set an exception in tp_clear

Serhiy Storchaka added the comment: I don't know how to reproduce a failure in tp_clear(). I just can't prove that it never fails. Maybe it is needed a bug in the implementation of the buffer protocol in third-party extension. If it should not happen then we can just add assert(!PyErr_O

[issue33713] memoryview can set an exception in tp_clear

Stefan Krah added the comment: Could you please show how tp_clear() can be called when self->exports > 0? It should not happen. #33622 is a big issue with many commits. Would it be possible to extract the relevant part? -- nosy: +pitrou ___ Pyth

[issue33713] memoryview can set an exception in tp_clear

New submission from Serhiy Storchaka : The tp_clear handler of memoryview can set an exception when fail to release the buffer. An exception in tp_clear is not expected and caused a crash in the garbage collector. In the master branch it will cause just writing a traceback to stderr (see issu