On Thu, Apr 10, 2008 at 10:27 AM, Michael.Abshoff
<[EMAIL PROTECTED]> wrote:
> Brian Granger wrote:
>
> > Hi,
> >
>
> Hi Brian,
>
>
>
> > (dual posted to sage and cython)
> >
> > A few of us (ipython and mpi4py devs) are wondering what the
> > best/safest way of allocating dynamic memory in a local scope
> > (method/function) is when using cython. An example would be if you
> > need an array of c ints that is locally scoped.
> >
> > The big question is how to make sure that the memory gets freed - even
> > if something goes wrong in the function/method. That is, you want to
> > prevent memory leaks. It looks like in sage, the
> > sage_malloc/sage_free functions are used for this purpose:
> >
>
> They generally aren't used in most of the code. The idea for those
> functions is that in the future we can wrap other allocators like slab
> allocators.
Maybe so, but I am am not very familiar with the sage codebase and I
quickly found numerous examples of sage_malloc :) Also, from Williams
reponse it sounds like the idea is that sage code _would_ use these
functions. Also, how else would you allocate dynamic memory?
>
>
> > from sage/graphs/graph_isom.pyx:
> >
> > 176 def incorporate_permutation(self, gamma):
> > 202 cdef int *_gamma = <int *> sage_malloc( n * sizeof(int) )
> > 203 if not _gamma:
> > 204 raise MemoryError("Error allocating memory.")
> > 205 for k from 0 <= k < n:
> > 206 _gamma[k] = gamma[k]
> > 207 self._incorporate_permutation(_gamma, n)
> > 208 sage_free(_gamma)
> >
> > Because sage_malloc is #defined to malloc in stdsage.h, I think there
> > is a significant potential for memory leaks in code like this. Are we
> > thinking correctly on this issue? Isn't this a huge problem?
> >
>
> Well, I don't see an advantage in using Python's allocator there. It is
> likely slower for large allocations and make debugging memory issues much
> more complicated since issues like pointer corruption is significantly
> harder to debug.
I am not concerned about performance here, but rather memory leaks. I
don't think it is a good idea to trade memory leaks for performance.
>
>
>
> > Lisandro Dalcin (author of mpi4py) came up with the following trick
> > that, while more complicated, prevents memory leaks:
> >
> > cdef extern from "Python.h":
> > object PyString_FromStringAndSize(char*,Py_ssize_t)
> > char* PyString_AS_STRING(object)
> >
> > cdef inline object pyalloc_i(int size, int **i):
> > if size < 0: size = 0
> > cdef Py_ssize_t n = size * sizeof(int)
> > cdef object ob = PyString_FromStringAndSize(NULL, n)
> > i[0] = <int*> PyString_AS_STRING(ob)
> > return ob
> >
> > and now
> >
> > def foo(sequence):
> > cdef int size = len(sequence),
> > cdef int *buf = NULL
> > cdef object tmp = pyalloc_i(size, &buf)
> >
> > This could probably be adapted into a malloc-like function. What do
> > people think?
> >
> We valgrind the complete test suite at least weekly and with ever
> increasing doctest coverage I don't see that we have a problem. Python's
> memory management has also some serious issues and I doubt it will offer any
> advantage for anything but loads of small allocs. And by switching to a slab
> allocator on our end will fix that problem.
But, test test suite doesn't test for all of the odd input that users
will feed to sage. These are the cases that will leak memory and
there is not possible way to test for all of them. Also debugging
memory leaks is super nasty. Compared to that pain, having a slightly
slower memory allocator is not a big deal.
Brian
>
> > Thanks,
> >
> > Brian
> >
>
> Cheers,
>
> Michael
>
>
> > _______________________________________________
> > Cython-dev mailing list
> > [EMAIL PROTECTED]
> > http://codespeak.net/mailman/listinfo/cython-dev
> >
> >
>
>
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send email to sage-devel@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
