On Thu, Apr 10, 2008 at 9:27 AM, Michael.Abshoff
<[EMAIL PROTECTED]> wrote:
>
> Brian Granger wrote:
> > Hi,
>
> Hi Brian,
>
>
> > (dual posted to sage and cython)
> >
> > A few of us (ipython and mpi4py devs) are wondering what the
> > best/safest way of allocating dynamic memory in a local scope
> > (method/function) is when using cython. An example would be if you
> > need an array of c ints that is locally scoped.
> >
> > The big question is how to make sure that the memory gets freed - even
> > if something goes wrong in the function/method. That is, you want to
> > prevent memory leaks. It looks like in sage, the
> > sage_malloc/sage_free functions are used for this purpose:
>
> They generally aren't used in most of the code.
They should be.
> The idea for those
> functions is that in the future we can wrap other allocators like slab
> allocators.
>
>
> > from sage/graphs/graph_isom.pyx:
> >
> > 176 def incorporate_permutation(self, gamma):
> > 202 cdef int *_gamma = <int *> sage_malloc( n * sizeof(int) )
> > 203 if not _gamma:
> > 204 raise MemoryError("Error allocating memory.")
> > 205 for k from 0 <= k < n:
> > 206 _gamma[k] = gamma[k]
> > 207 self._incorporate_permutation(_gamma, n)
> > 208 sage_free(_gamma)
> >
> > Because sage_malloc is #defined to malloc in stdsage.h, I think there
> > is a significant potential for memory leaks in code like this. Are we
> > thinking correctly on this issue? Isn't this a huge problem?
>
> Well, I don't see an advantage in using Python's allocator there. It is
> likely slower for large allocations and make debugging memory issues
> much more complicated since issues like pointer corruption is
> significantly harder to debug.
Since our answers differ, I should clarify. The point is that by giving
bad input a user could cause a memory leak. It's something our doctests
probably wouldn't notice.
>
>
> > Lisandro Dalcin (author of mpi4py) came up with the following trick
> > that, while more complicated, prevents memory leaks:
> >
> > cdef extern from "Python.h":
> > object PyString_FromStringAndSize(char*,Py_ssize_t)
> > char* PyString_AS_STRING(object)
> >
> > cdef inline object pyalloc_i(int size, int **i):
> > if size < 0: size = 0
> > cdef Py_ssize_t n = size * sizeof(int)
> > cdef object ob = PyString_FromStringAndSize(NULL, n)
> > i[0] = <int*> PyString_AS_STRING(ob)
> > return ob
> >
> > and now
> >
> > def foo(sequence):
> > cdef int size = len(sequence),
> > cdef int *buf = NULL
> > cdef object tmp = pyalloc_i(size, &buf)
> >
> > This could probably be adapted into a malloc-like function. What do
> > people think?
>
> We valgrind the complete test suite at least weekly and with ever
> increasing doctest coverage I don't see that we have a problem. Python's
> memory management has also some serious issues and I doubt it will offer
> any advantage for anything but loads of small allocs. And by switching
> to a slab allocator on our end will fix that problem.
I think the point Brian is making is that if you think about a certain class
of code that is common in Sage/Numpy/Scipy, you can construct situations
where exceptions are raised and code isn't freed. We're almost surely
not actually testing these situations since they are "exceptional". But
they do exist, and we've been planning to address them "some day".
--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
