On Thursday, March 31, 2011 4:57:20 PM UTC+11, bird sky wrote: > > hi Graham: > i'm using the nginx+fcgi . And my startup command is below > > python manage.py runfcgi host=127.0.0.1 port=3033 method=prefork > pidfile=/var/run/fcgi.pid minspare=5 maxspare=30 maxchildren=60 > maxrequests=200 >
Same questions I posed before apply here. Why are you using prefork? Why do you need to run so many processes? Why can't you use a small number of persistent multithreaded processes? Your memory size for a single process isn't out of the ordinary, it is not using multithreading and so that you need so many processes which is going to kill you on memory usage. As with Apache, be wary of starting minimal number of processes and then allowing process manager to dynamically increase and decrease number of active processes. You will suffer the same load spike problems that you can get with Apache as documented in the blog post I linked to. For fat Python web applications with large start up cost you are better off having small fixed number of multithreaded processes. That way memory usage is more predictable and you avoid load spikes from spinning up more processes. You just need to work out how many fixed processes/threads to use to handle realistic level of requests. You can't base it on unrealistic loads from benchmarking as in practice you will probably never see such levels of requests unless you become very popular very quickly or get Slashdotted. > p.s I've invite you on gtalk, hope your response. > Talk sessions are almost as bad as IRC for discussing such stuff. Keep it on the mailing list. Graham > On Thu, Mar 31, 2011 at 6:25 AM, Graham Dumpleton > <graham.d...@gmail.com>wrote: > >> Why are you using prefork MPM and running Django embedded that way. >> >> Prefork MPM may be fine for PHP, but it is a poor solution for fat Python >> web applications unless you are prepared to give it the necessary memory >> resources and configure Apache properly specifically for that single Python >> web applications requirements. >> >> If you are going to use embedded mode of Apache, then you should at least >> use worker MPM and then you still need to be mindful of how Apache MPM >> settings are configured so as to try and keep processes persistent all the >> time and not let Apache kill off/restart processes when it feels it needs to >> as the startup load will only make performance worse when you need it, ie., >> when load spikes. >> >> For a discussion on why embedded mode is not always the best, especially >> with prefork MPM, go read: >> >> >> http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html >> >> If using Apache/mod_wsgi the preferred setup would be daemon mode where >> you can better control processes/threads independent of Apache MPM, meaning >> you can still use prefork MPM if you are stuck with support a PHP >> application on the same Apache. >> >> Graham >> >> >> On Wednesday, March 30, 2011 11:03:37 PM UTC+11, bird sky wrote: >>> >>> Hello Everybody: >>> I encounter a problem that my Django project, it has more than 60 >>> app modules, and some models are very large, more than 30 fields . And >>> when I startup my project, regardless of in development server, fast >>> cgi(flup),or mod_wsgi. i found it cost at least 60M memory per >>> instance. I guess this has some relation with that "django will load >>> all models on startup" . but how can I deal with my problem? Because >>> if an instance cost 60M memory, when I deploy my project in prefork >>> web server with 100 instance, i will cost 6GB memory. I don't think >>> this is a normal state. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To post to this group, send email to django...@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
