Wai Yip <auror...@gmail.com> wrote:
> I started with ctypes because it is the battery included with the
> Python standard library. My code is very fluid and I'm looking for
> easy opportunity to optimize it. One example is to find the longest
> common prefix among two strings. Right now I am comparing it character
> by character with pure Python. It seems like an ideal low hanging
> fruit. With ctype I can rewriting a few lines of Python into a few
> lines of C. All the tools are available and no third party library is
> needed.
Yes ctypes is very easy for this and I've used it like this in the
past too. It makes interfacing C code very easy.
You might consider just writing a plain bit of C code. If you make a
setup.py too (not hard) then it is very easy to distribute and use on
other platforms.
The Python C API is very easy to use - documentation is extensive.
You need to be very careful with references and the error checking is
boring and tedious!
> It turned out the performance fail to match pure Python. This function
> is called million of times in an inner loop. The overhead overwhelm
> any performance gain with C. Eventually I have found success grouping
> the data into larger chunk for each API call. This is what I
> originally planned to do anyway. I am only sharing my experience here
> that doing fine grained ctype function call has its limitation.
Interesting but not too unexpected - that translation of types is
non-trivial.
> I have looked into Pyrex before and I was quite confused by the fusion
> of C and Python langauage. Perhaps it is time for me to give it a
> second look. I just heard of Cython now and it also look interesting.
> I think it is helpful for both project to articulate more clearly what
> they are, how they achieve the performance gain and to give more
> guidance on good use cases. On the other hand, perhaps it is just me
> are confused because I don't know enough of the Python internal.
Pyrex evolved/was forked into Cython.
Cython allows you to write a subset of python code that is compiled
into C. You can also optionally provide some type annotations to get
it to produce better C code at the cost of making your code look less
like python.
I find Cython alternatively brilliant and frustrating! Sometimes it
works really well, and sometimes I spend hours working out why my
program doesn't compile, or to find out the exact syntax I need to
interface with such and such a library.
I did a C library interfacing project with both ctypes and cython
recently as a comparison. It came out to be about the same number of
lines of code for both. I decided to run with the ctypes version as
it was part python.
Here is a short Cython example which interfaces with opendir /
closedir / readdir which python doesn't do natively. Note the way you
interface with C structures (the real ones are used in the C code -
the annotations just tell cython how to use them). Note also that
Directory is a class defined in C which I don't think you can't do
with ctypes without a wrapper class.
This compiles into 1320 lines of C, which in turn compile into 11380
bytes of shared object (when stripped).
import cython
cdef extern from "dirent.h":
struct dirent:
char d_name[0]
ctypedef struct DIR
DIR *opendir(char *name)
int closedir(DIR *dirp)
dirent *readdir(DIR *dirp)
cdef extern from "errno.h":
int errno
cdef extern from "string.h":
char *strerror(int errnum)
cdef class Directory:
"""Represents an open directory"""
cdef DIR *handle
def __init__(self, path):
self.handle = opendir(path)
if self.handle is NULL:
raise OSError(errno, "Failed to open directory: %s" %
strerror(errno))
def readdir(self):
"""Read the next name in the directory"""
cdef dirent *p
p = readdir(self.handle)
if p is NULL:
return None
return p.d_name
def close(self):
"""Close the directory"""
if self.handle is not NULL:
closedir(self.handle)
self.handle = NULL
def __dealloc__(self):
self.close()
--
Nick Craig-Wood <n...@craig-wood.com> -- http://www.craig-wood.com/nick
--
http://mail.python.org/mailman/listinfo/python-list
