[issue46166] Get "self" args or non-null co_varnames from frame object with C-API
New submission from Aaron Gokaslan : Hello, I am a maintainer with the PyBind11 project. We have been following the 3.11 development branch and have noticed an issue we are encountering with changes to the C-API. Particularly, we have an edge case in our overloading dispatch mechanism that we used to solve by inspecting the "self" argument in the co_varnames member of the python frame object: (https://github.com/pybind/pybind11/blob/a224d0cca5f1752acfcdad8e37369e4cda42259e/include/pybind11/pybind11.h#L2380). However, in the new struct, the co_varnames object can now be null. There also doesn't appear to be any public API to populate it on the C-API side. Accessing it via the "inspect" module still works, but that requires us to run a Python code snippit in a potentially very hot code path: (https://github.com/pybind/pybind11/blob/a224d0cca5f1752acfcdad8e37369e4cda42259e/include/pybind11/pybind11.h#L2408). As such, we were hoping that either there is some new API change we have missed, or if there is some way other modern (and hopefully somewhat stable way to access the API) so we can emulate the old behavior with the C-API. -- components: C API messages: 409100 nosy: Skylion007 priority: normal severity: normal status: open title: Get "self" args or non-null co_varnames from frame object with C-API type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue46166> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46166] Get "self" args or non-null co_varnames from frame object with C-API
Aaron Gokaslan added the comment: We didn't want to read colocalsplus directly because we were worried about the stability of that approach and the code complexity / readability. Also, I wasn't aware that colocalsplus would work or if that was lazily populated as well. The functions used in CPython to extract the args from colocalsplus do not seem to be public and would need to be reimplemented by PyBind11, right? That seems very brittle as try to support future Python versions and may break in the future. Having a somewhat stable C-API to query this information seems like it would be the best solution, but I am open to suggestions on how to best proceed. How would you all recommend PyBind11 proceed with supporting 3.11 if not a C-API addition? The PyBind11 authors want to resolve this before the API becomes too locked down for 3.11. -- ___ Python tracker <https://bugs.python.org/issue46166> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46166] Get "self" args or non-null co_varnames from frame object with C-API
Aaron Gokaslan added the comment: `PyCodeObject_GetVariableName()` and `PyCodeObject_GetVariableKind()` work? - Some public-gettters such as these functions would be ideal. OOI, how do you cope with non-local self? - We only care about checking self to prevent an infinite recursion in our method dispatch code so I am not sure a non-local self would be applicable in this case? Correct me if I am wrong. -- ___ Python tracker <https://bugs.python.org/issue46166> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46166] Get "self" args or non-null co_varnames from frame object with C-API
Aaron Gokaslan added the comment: I saw the latest Python 3.11 5A release notes on the frame API changes. Do the notes mean the only officially supported way of accessing co_varnames is now through the Python interface and the inspect module? By using PyObject_GetAttrString? Also, the documentation in the WhatsNew is a bit unclear as PyObject_GetAttrString(frame, "f_locals") doesn't work for PyFrameObject*, only PyObject* and it doesn't describe how to get the PyObject* version of FrameObject. The same problem also happens when trying to access the co_varnames field of the PyCodeObject*. -- ___ Python tracker <https://bugs.python.org/issue46166> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46166] Get "self" args or non-null co_varnames from frame object with C-API
Aaron Gokaslan added the comment: The frame object I am referring to was: PyFrameObject *frame = PyThreadState_GetFrame(PyThreadState_Get()); This frame can not be used with PyObject_GetAttrString. Is there anyway to get the PyObject* associated with a PyFrameObject*? It seems weird that some functionality is just not accessible using the Stable ABI of PyThreadState_GetFrame . To elabroate: I was referring to the migration guide in the changelog btw: f_code: removed, use PyFrame_GetCode() instead. Warning: the function returns a strong reference, need to call Py_DECREF(). f_back: changed (see below), use PyFrame_GetBack(). f_builtins: removed, use PyObject_GetAttrString(frame, "f_builtins"). // this frame object actually has to be a PyObject*, the old one was a PyFrameObject* . Dropping this in does not work. f_globals: removed, use PyObject_GetAttrString(frame, "f_globals"). f_locals: removed, use PyObject_GetAttrString(frame, "f_locals"). f_lasti: removed, use PyObject_GetAttrString(frame, "f_lasti"). I tried importing sys._getframe(), but that gave an attribute error interestingly enough. Run a full code snippit here works: https://github.com/pybind/pybind11/blob/96b943be1d39958661047eadac506745ba92b2bc/include/pybind11/pybind11.h#L2429, but is really slow and we would like avoid having to rely on it. Not to mention relying on a function that is an starts with an underscore seems like it really should be avoided. -- ___ Python tracker <https://bugs.python.org/issue46166> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
