On Aug 5, 10:14 pm, Albert Hopkins <mar...@letterboxes.org> wrote: > ... Or perhaps mod_wsgi is not the way to go. > > Basically I have this situation. I have a Django app that also has an > xmlrpc interface via the wiki recipe[1]. All works fine until I enabled > the ability to upload files. Since they're binary files I use the > xmlrpc binary type. > > Well this loads the files entirely in memory on the server side, which > ordinarily is ok since the files are only typically a few MB. However, > python/mod_wsgi/apache are not releasing the memory. I've tried > manually del'ing the objects and calling the garbage collector, but have > only had minimal success. If I continue to upload files en-mass via > xmlrpc eventually Apache processes max out the available memory. > > I'm hoping someone can point me to a coding or configuration solution. > Or perhaps mod_wsgi is not the best solution for this and I should go > FastCGI?
It isn't going to be a mod_wsgi issue but your Python web application. That or how you have configured Apache/mod_wsgi. Do realize though that the way memory allocations work, if there is a large transient memory usage for a request which increases overall process memory usage, then once that request finishes the size of the process isn't then going to magically shrink. This is because the working memory size of a process will normally remain at that level. The memory will however go back into the free memory list for that process and can be used for subsequent requests. If you don't see memory usage plateau based on largest transient memory requirement for how ever many concurrent requests can execute that handler at a time, then you likely have a memory leak, or more likely what I call a resource leakage. That is where Python can't actually free objects because something is still referencing them or cycles exist withing object dependencies and del methods exist which mean the cycle can't be broken and the objects reclaimed. As far as Apache/mod_wsgi configuration goes, ensure you are not using embedded mode, especially if using prefork MPM with Apache as that can be a recipe for disaster. Do a Google search for: site:blog.dscpl.com.au transient memory And read my blog post about memory spikes and load issues for mod_python. Same issue applies to mod_wsgi embedded mode. So, make sure you are using daemon mode and fixed number of processes. Otherwise use a memory profiler for Python such as Dozer to work out where memory use is occurring for a request and why objects aren't being released. Graham > [1]http://code.djangoproject.com/wiki/XML-RPC -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-us...@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
