snorble, 23.11.2011 06:19:
Sometimes I want to prototype a program in Python, with the idea of
optimizing it later by rewriting parts of it in C or Cython. But I
usually find that in order to rewrite the slow parts, I end up writing
those parts very much like C or C++ anyway, and I end up wondering
what is the point of using Python in such a project.
I liked the idea of Cython, mainly the idea of being able to write in
pure Python, then create a file that modifies the pure Python file
(with type declarations and such), but I ran into the same problems.
In order to be able to modify the code, I can't make a lot of use of
the really helpful things in Python like dicts and such. I have to
dumb down Python to where I'm basically writing C code in Python, so
again I ask myself what is the point?
Is it reasonable to prototype an application in Python that will
require performance? Are there any recommendations on how to write
code in such a way that it can later be optimized or replaced with a
module written in C or Cython?
I think the usual approach is to write it in Python and make sure it works
by writing "enough" tests, then benchmark it, then decide if a non-Python
level optimisation is required.
If it's required, Cython compile the entire module and add static types to
your hot spots, either in Python notation ("pure mode") or in an external
.pxd file. If that doesn't yield enough performance, copy code into a
separate Cython module and optimise it there using C paradigms instead of
Python paradigms, i.e. apply algorithmic optimisations by dropping data
structures into C. Then use an import to use the code from your so-called
accelerator module in your original module. Try to provide both a pure
Python implementation and a Cython implementation, as that will allow you
to run your code in other Python implementations directly and to compare
your two implementations for equivalence and performance.
The advantage of stepping down into C-ish Cython code incrementally is that
you will have a working implementation at each step and can decide to stop
optimising because it is "good enough". If you started 'optimising' your
code while still writing it, it will take longer to write it and you can
never be sure that the complexity you add to the long-term maintenance by
writing C-ish code is truly worth the performance gain (or if there even is
any gain).
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
