[issue12029] Catching virtual subclasses in except clauses

Changes by Berker Peksag : -- nosy: +berker.peksag ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: This question came up again recently over in #27814, in the context of a proposal to add an "unless" parameter to contextlib.suppress(). I declined the RFE mainly on the basis of API complexity, but I also noted you can get something comparable in the current AP

[issue12029] Catching virtual subclasses in except clauses

Martin Panter added the comment: In the current (v4) patch, it looks like there is an unneeded “allow_new_exception” parameter that is always set to zero. So maybe the patch needs more careful thought. Also I agree it needs documentation, what’s new, “changed in version 3.6”, etc. --

[issue12029] Catching virtual subclasses in except clauses

R. David Murray added the comment: Note from discussion on duplicate issue 25448: when this is fixed, the try/except documentation should be updated to indicate that the except clause test is equivalent to 'issubclass', so that the handling of virtual subclasses are implied by the doc. (The c

[issue12029] Catching virtual subclasses in except clauses

Josh Rosenberg added the comment: On rereading #22540, maybe that won't be an issue (in the common case where no custom metaclasses are used for the exceptions to be caught, it looks like maybe there is no slow path to traverse?). Still worth double checking. -- __

[issue12029] Catching virtual subclasses in except clauses

Josh Rosenberg added the comment: Does this introduce a slowdown when the type doesn't match? That is, clearly George's example: try: {}["a"] except KeyError: pass won't be slowed because the fast path will get an immediate hit. But what about: try: {}["a"]

[issue12029] Catching virtual subclasses in except clauses

Changes by Ethan Furman : -- nosy: -ethan.furman ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.

[issue12029] Catching virtual subclasses in except clauses

Changes by Berker Peksag : -- stage: needs patch -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue12029] Catching virtual subclasses in except clauses

Yuriy Taraday added the comment: Can we move forward and land this patch? It seems to be working and for some reason it even makes that microbenchmark work faster. -- versions: +Python 3.5 -Python 3.3 ___ Python tracker

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: IsSubclass speedup patch now tracked in #22540. -- dependencies: +speed up isinstance and issubclass for the usual cases ___ Python tracker ___ __

[issue12029] Catching virtual subclasses in except clauses

Changes by Georg Brandl : Added file: http://bugs.python.org/file36780/exception_proper_subclass_matching_v4.patch ___ Python tracker ___ ___

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: OK, with these two patches (speedup and v4) I can't see a significant slowdown anymore. -- Added file: http://bugs.python.org/file36779/pyobject_issubclass_isinstance_speedup.patch ___ Python tracker

[issue12029] Catching virtual subclasses in except clauses

Antoine Pitrou added the comment: > Since type has __subclasscheck__ (why I don't know) Ouch, really? That means the PyType_IsSubtype path in PyObject_IsSubclass() is never taken? I think we should add a tp_subclasscheck slot to minimize the general cost of this. Also, try to see if there are

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: Quick microbenchmark: try: {}["a"] except KeyError: pass original tip: 1.35 usec with patch v3: 1.55 usec so it's about 15% slowdown for catching a simple exception on my machine. -- _

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: Agreed. Since type has __subclasscheck__ (why I don't know) this might result in a slowdown since all checks have to go through calling it in PyObject_IsSubclass. Just noticed that given_exception_matches_inner can be simplified a bit since PyObject_IsSubclass

[issue12029] Catching virtual subclasses in except clauses

Antoine Pitrou added the comment: I'm worried about the runtime cost of this. Code like this is an extremely common idiom and should remain fast: try: x = d[key] except KeyError: # do something else -- ___ Python tracker

[issue12029] Catching virtual subclasses in except clauses

Changes by Georg Brandl : Removed file: http://bugs.python.org/file36776/exception_proper_subclass_matching.patch ___ Python tracker ___ ___

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: Clarifying some comments in this one. -- Added file: http://bugs.python.org/file36778/exception_proper_subclass_matching_v3.patch ___ Python tracker

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: New version including (I think) correct refcount handling. -- Added file: http://bugs.python.org/file36777/exception_proper_subclass_matching_v2.patch ___ Python tracker ___

[issue12029] Catching virtual subclasses in except clauses

Georg Brandl added the comment: I'm attaching a patch that works without changing the recursion limit, and adds some tests for pathological cases. Instead, PyErr_GivenExceptionMatches is changed so that if an exception was previously set, it is replaced by an exception that PyObject_IsSubclass

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: Right, I had a specific concern related to the way the C level code works. On closer inspection, it turned out all the Python level code execution is complete by the time we reach the point I was worried about. -- ___ P

[issue12029] Catching virtual subclasses in except clauses

Guido van Rossum added the comment: ISTM Nick meant that the exception that was raised can't cause arbitrary code execution. On Wednesday, October 1, 2014, Antony Lee wrote: > > Antony Lee added the comment: > > "it looks like all the avenues for arbitrary code execution while checking > if an

[issue12029] Catching virtual subclasses in except clauses

Antony Lee added the comment: "it looks like all the avenues for arbitrary code execution while checking if an exception handler matches a thrown an exception are closed off." This seems to be directly contradicted by your previous sentence: "the except clause accepts any expressions producing

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: Ah, you're right - I found the example I was thinking of (Richard Jones's "Don't do this!" talk), and it was just demonstrating that the except clause accepts any expressions producing a tuple or BaseException instance, not that we call __iter__ at that point.

[issue12029] Catching virtual subclasses in except clauses

Guido van Rossum added the comment: "I remembered we already run arbitrary code at roughly this point in the eval loop, as we have to invoke __iter__ to get the exceptions to check when an iterable is used in except clause." Are you sure? IIRC the except clause only accept exceptions and tupl

[issue12029] Catching virtual subclasses in except clauses

Changes by Jakub Wilk : -- nosy: +jwilk ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: A point on the safety/correctness front: I remembered we already run arbitrary code at roughly this point in the eval loop, as we have to invoke __iter__ to get the exceptions to check when an iterable is used in except clause. That means allowing the subclass c

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: Because context managers are closer to try/finally blocks than they are to exception handling, the class-based implementation for the contextlib.suppress API uses issubclass rather than emulating the CPython exception handling semantics: http://hg.python.org/cpy

[issue12029] Catching virtual subclasses in except clauses

Yuriy Taraday added the comment: Can someone please point out why do we have to do that dance with recursion limit? I've came upon this problem as well. I had some (bad) API I had to work with. It always raised the same exception with the only difference in the message. So I thought I could d

[issue12029] Catching virtual subclasses in except clauses

Changes by Ethan Furman : -- nosy: +ethan.furman ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue12029] Catching virtual subclasses in except clauses

Changes by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue12029] Catching virtual subclasses in except clauses

Antoine Pitrou added the comment: > Actually, I take back the performance comment - Jim's right that in > the normal case, the slowdown should be minimal (it's just some > additional checks that certain slots aren't populated on each listed > exception class), and types can already influence that

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: Actually, I take back the performance comment - Jim's right that in the normal case, the slowdown should be minimal (it's just some additional checks that certain slots aren't populated on each listed exception class), and types can already influence that by mes

[issue12029] Catching virtual subclasses in except clauses

Nick Coghlan added the comment: The performance hit is that such a change would potentially make it more expensive to figure out that a raised exception *doesn't* match a given "except" clause, along with the complexity of introducing execution of arbitrary code while still unwinding the stack

[issue12029] Catching virtual subclasses in except clauses

Changes by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue12029] Catching virtual subclasses in except clauses

Changes by Andreas Stührk : -- nosy: +Trundle ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue12029] Catching virtual subclasses in except clauses

Jim Jewett added the comment: When does the performance hit occur? If it is only when an exception has been raised, and its own class is not listed by the except clause, then I personally wouldn't worry about it; tracing the MRO *could* get arbitrarily long already; it just doesn't in practic

[issue12029] Catching virtual subclasses in except clauses

George-Cristian Bîrzan added the comment: As promissed the patch. It doesn't break any tests, and it passes the ones I added. I have a pybench one as well, which even though trivial, does point to the fact that there is a degradation in performance, but not sure it's worth posting here.

[issue12029] Catching virtual subclasses in except clauses

Changes by James Henstridge : -- type: behavior -> enhancement ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue12029] Catching virtual subclasses in except clauses

James Henstridge added the comment: Benjamin: if you are after a use case for this feature, see https://code.djangoproject.com/ticket/15901 In Django, there are multiple database backends, each of which currently catch the adapter's DatabaseError and reraise it as Django's DatabaseError so t

[issue12029] Catching virtual subclasses in except clauses

Changes by Yury Selivanov : -- nosy: +Yury.Selivanov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue12029] Catching virtual subclasses in except clauses

George-Cristian Bîrzan added the comment: I have a patch, with tests, but no Internet on my computer so going out, will post it when I get back/my Internet comes back -- ___ Python tracker ___

[issue12029] Catching virtual subclasses in except clauses

George-Cristian Bîrzan added the comment: I posted on python dev that this would slow exception checking considerably so that is a concern. As for possible bugs, this has been working in the 2 branch for a while now, so I don't think that is the biggest issue. As for possible use cases, writi

[issue12029] Catching virtual subclasses in except clauses

Benjamin Peterson added the comment: Basically, someone needs to produce a patch and we can go from there. -- ___ Python tracker ___

[issue12029] Catching virtual subclasses in except clauses

Antoine Pitrou added the comment: > perhaps it could just work in a simple, consistent way? That would be best obviously. But as Benjamin explained it's quite delicate to make it work while avoiding pitfalls where code involved in exception checking may itself fail with arbitrary errors - say

[issue12029] Catching virtual subclasses in except clauses

Changes by andrew cooke : -- nosy: -acooke ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue12029] Catching virtual subclasses in except clauses

andrew cooke added the comment: perhaps it could just work in a simple, consistent way? in my original report i wondered whether there was a significant performance hit. but so far the objections against fixing this seem to be (1) a lawyer could be convinced the current behaviour is consiste

[issue12029] Catching virtual subclasses in except clauses

Antoine Pitrou added the comment: Perhaps ABCMeta could raise a UserWarning when creating an Exception subclass? -- nosy: +pitrou ___ Python tracker ___

[issue12029] Catching virtual subclasses in except clauses

Benjamin Peterson added the comment: I think being able to catch exception with ABCs is esssentially useless. The originally stated "usecase" can be simply solved by putting classes into a tuple and putting that in the except clause. In general, the whole abc machinary causes lots of code whi

[issue12029] Catching virtual subclasses in except clauses

Changes by Antoine Pitrou : -- nosy: +benjamin.peterson type: behavior -> enhancement versions: -Python 3.2 ___ Python tracker ___ __

[issue12029] Catching virtual subclasses in except clauses

Changes by Éric Araujo : -- stage: -> needs patch versions: +Python 3.2 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsu

[issue12029] Catching virtual subclasses in except clauses

Guido van Rossum added the comment: I agree it's a bug and should be fixed. It's too confusing that there would be two slightly different interpretations of isinstance/issubclass where the isinstance() and issubclass() would be using the extended interpretation but the except clause would us

[issue12029] Catching virtual subclasses in except clauses

Changes by Éric Araujo : -- title: ABC registration of Exceptions -> Catching virtual subclasses in except clauses versions: +Python 3.3 -Python 3.2 ___ Python tracker ___ _