keeping local state in an C extension module
Hi all.
For a project I implemented a extension module in C. Given the
excellent documentation everything was straightforward and works fine
so far.
Up until now I kept local state in global static variables, but would
like to change this to pass a pointer to a state structure around. The
Py_InitModule4() [1] function seems to allow this by giving the user
the option to specify the 'self' object passed to the module's
functions. In this case, the 'self' argument should be a raw C pointer
malloc'ed elsewhere that should be rolled into a PyObject and be made
accessible later in the extension functions. Example code:
static PyObject* module_func(PyObject *self, PyObject *args) {
state_t *state = (state_t*) PyRawPointerFromPyObject(self);
/* use state */
Py_RETURN_NONE;
}
static PyMethodDef bms_methods[] = {
{ "func", module_func, METH_VARARGS, "" },
{ NULL, NULL, NULL, NULL }
};
void my_module_init(state_t *state) {
Py_InitModule4("_mymodule", module_methods,
"My Extension Module",
PyObjectFromRawPointer(state),
PYTHON_API_VERSION);
}
Here, of course, the functions PyObjectFromRawPointer(void*) and void*
PyRawPointerFromPyObject(PyObject*) are missing. Is there anything
like this in the Python C-API? If not, how could it be implemented? Or
is this approach a stupid idea to begin with? Pointers would be highly
appreciated.
Thanks
Daniel
[1]
http://docs.python.org/c-api/allocation.html?highlight=py_initmodule4#Py_InitModule4
--
http://mail.python.org/mailman/listinfo/python-list
Re: keeping local state in an C extension module
Hi Stefan. 2011/6/30 Stefan Behnel : > Don't miss out taking a look at Cython, just in case it's going to be a > non-trivial project. I think this work still qualifies as trivial enough in this regard. But thanks for the hint! > There's a PEP for Py3 that enables this through a general framework. In > general, to achieve this, you may want to allocate the module content > (including types etc.) on the heap rather than statically. > > http://www.python.org/dev/peps/pep-3121/ Perfect! Unfortunately, it seems not to be available in 2.x, so unusable for now. For the foreseeable future the project can not depend on 3.x as many (internal) target systems are "stable", i.e. are not updated besides security stuff. > Take a look at PyCapsule, it may (or may not) be enough for your use case. Also a perfectly good solution - unfortunately the same comment about 3.0 applies here. > However, note the comment about Py_InitModule4() in the docs, where it says > that the 'self' passing feature isn't used very often. You may or may not > want to (or need to) use it. If there are too many hoops to jump through to get a reasonable solution working for 2.[567], I might as well go back to Py_InitModule() and static variables. It's somewhat ugly, but it works. Thanks for the info! Daniel -- http://mail.python.org/mailman/listinfo/python-list
Re: keeping local state in an C extension module
2011/6/30 Stefan Behnel : >> If there are too many hoops to jump through to get a reasonable >> solution working for 2.[567], I might as well go back to >> Py_InitModule() and static variables. It's somewhat ugly, but it >> works. > > Py2.x has a PyCObject, basically the predecessor to PyCapsule. Writing your > own little type is another option. For reference: PyCObject has PyCObject_FromVoidPtr() and PyCObject_AsVoidPtr() -- exactly what I was looking for (no idea why I missed it for so long?!). Ok, and it also has the note: "The CObject API is deprecated as of Python 2.7. Please switch to the new Capsules API." which is of no concern here :) Thanks heaps! Daniel -- http://mail.python.org/mailman/listinfo/python-list
defining class and subclass in C
Hi all.
I spent some days and nights on this already and my google-fu is running out.
I'd like to implement the equivalent of this Python code in a C-extension:
>>> class A(object):
... pass
>>> class B(A):
... pass
>>> A
>>> B
>>> B.__bases__
(,)
However, loading my C-code (quoted below) I get:
>>> import ca
>>> ca
>>> ca.ca
Here I'd expect "" instead?! And I never managed a proper
subclass :|
The point of the excercise: with "ca" and "cb" properly implemented, I'd like
to subclass "cb" not from "ca", but from the Python "class A" - if possible?!
Could somepne kindly point out my mistake(s) and set me back on track?
Thanks
Daniel
--
#include
typedef struct {
PyObject_HEAD
} ca;
static PyTypeObject ca_Type = {
PyObject_HEAD_INIT(NULL)
};
PyMODINIT_FUNC initca(void) {
PyObject *ca;
ca_Type.tp_name = "ca.ca";
ca_Type.tp_basicsize = sizeof(ca);
ca_Type.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
if (PyType_Ready(&ca_Type) < 0)
return;
ca = Py_InitModule3("ca", NULL, "ca module");
if (ca == NULL)
return;
Py_INCREF(&ca_Type);
PyModule_AddObject(ca, "ca", (PyObject*)&ca_Type);
}
$ gcc -Wall -Wextra -g -fPIC -I/usr/include/python2.7 ca.c \
-shared -Wl,-soname,ca.so -o ca.so -lpython2.7
--
http://mail.python.org/mailman/listinfo/python-list
Re: defining class and subclass in C
On Saturday 14 January 2012 22:15:36 Daniel Franke wrote:
> Here I'd expect "" instead?! And I never managed a proper
> subclass :|
Found an explanation on type/class at [1]: "he difference between the two is
whether the C-level type instance structure is flagged as having been
allocated on the heap or not" - hu?
With this info, I went ahead and tried the code quoted below. Now I get:
>>> import cmod
>>> cmod.ca
>>> cmod.cb
>>> cmod.ca.__bases__
(,)
>>> cmod.cb.__bases__
(,)
which seems to make sense besides the "type" instead of "class"?!
What threw me is that I expected that I'd need to explicitly subclass the
"object" type, from which I assumed for some reason I'd also inherit the
.tp_new member. This is obviously not the case.
Now the last question is, how do I get the a_type of Python class to use it as
base for "cb"?
On lives and learns.
Daniel
[1] http://utcc.utoronto.ca/~cks/space/blog/python/ClassesAndTypes
--
#include
typedef struct {
PyObject_HEAD
} ca;
static PyTypeObject ca_Type = {
PyObject_HEAD_INIT(NULL)
};
typedef struct {
PyObject_HEAD
} cb;
static PyTypeObject cb_Type = {
PyObject_HEAD_INIT(NULL)
};
PyMODINIT_FUNC initcmod(void) {
PyObject *cmod;
ca_Type.tp_name = "cmod.ca";
ca_Type.tp_basicsize = sizeof(ca);
ca_Type.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
ca_Type.tp_new = PyType_GenericNew;
ca_Type.tp_base = &PyBaseObject_Type;
if (PyType_Ready(&ca_Type) < 0)
return;
cb_Type.tp_name = "cmod.cb";
cb_Type.tp_basicsize = sizeof(cb);
cb_Type.tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE;
cb_Type.tp_base = &ca_Type;
if (PyType_Ready(&cb_Type) < 0)
return;
cmod = Py_InitModule3("cmod", NULL, "c module");
if (cmod == NULL)
return;
Py_INCREF(&ca_Type);
PyModule_AddObject(cmod, "ca", (PyObject*)&ca_Type);
Py_INCREF(&cb_Type);
PyModule_AddObject(cmod, "cb", (PyObject*)&cb_Type);
}
--
http://mail.python.org/mailman/listinfo/python-list
building foo.pyd, _initfoo vs. initfoo
Hi all.
Usually I work on Linux and all my cmake-built Python extensions working there
without trouble. Now these things need to work on Windows as well. While the
code itself compiles fine, linking and loading makes trouble.
First of, to successfully link everything with mingw (g++-4.4.0, somewhat
oldish) I had to define:
-DPy_ENABLE_SHARED -DPy_BUILD_CORE
Not sure if this makes sense, but if it works, fine by me. Now, loading the
module is different.
First attempts of loading the module failed miserably until I found out that
Python expects files with a .pyd suffix. Very innovative to throw off the
layman. Took that hurdle.
Finally, on load of foo.pyd, I get:
"Import Error: dynamic module does not define init function (initfoo)"
As this module loads fine in Linux, there is an initfoo() defined. Checking
with `nm`, I find two things:
1. The function name is mangled (__Z12initfoov)
-> the module is compiled with g++, but I believed that PyMODINIT_FUNC
includes the 'extern "C"' apparatus?
2. Manually enclosing the init function with 'extern "C" {}" still yields the
same error
-> the function name also comes with a leading underscore '_', i.e.
"_initfoo", not "initfoo"
Could someone knowledge with Windows and MinGW help me to sort this out? Tools
used: cmake-2.8.3, mingw-? with gcc-4.4 and Python-2.7.2 from python.org.
Thanks
Daniel
--
http://mail.python.org/mailman/listinfo/python-list
