Graham, thanks for your input, which I found enlightening, especially the information regarding extension modules and sub interpreters. I can see how some of the things I said previously may have been unfairly biased against apache, when the real issues were inherent to how WSGI works and so might very well have manifested no matter the web server involved (is that a fair way to state it?).
Regarding the topic of this thread, I am still inclined to think that the lighter servers are a better choice than apache for django devs in general (i.e. people who are not sysadmins or apache experts) simply because fewer options and fewer unneeded features makes for an easier/better installation, easier learning curve, and the like. I also note that everyone who advocates for apache concedes that the default installation/configuration is really not appropriate for django apps. I intend to look more deeply into the suitability of the default configurations of nginx / lighttpd / cherokee as delivered by various linux distros. Finally, your point about mod_wsgi's superior configurability is a good one, and something I had not considered. I especially appreciate and echo your points about the importance of knowing what your production system is doing and the key role of monitoring. I'm hoping the work you are doing with New Relic will go a long way towards solving those problems. --Stuart On Saturday, January 14, 2012 9:46:00 PM UTC-6, Graham Dumpleton wrote: > > > > On Jan 15, 9:40 am, Stuart Laughlin <stu...@bistrotech.net> wrote: > > Graham turned up on a thread in another forum where I was asking about > how > > to fix a strange error I had been seeing intermittently for months. He > > instructed me on how to put mod-wsgi into daemon mode, among other > things, > > and finally solved my problem. > > > > Here's a link to the thread, if you're interested. > > > > http://groups.google.com/group/satchmo-users/browse_thread/thread/6a6... > > > And the problem was actually due to a likely bug in psycopg2 and not > mod_wsgi. > You would have had the same problems with knowing what the cause was > if you had hit this bug when using uWSGI in one of the ways that it > uses sub interpreters. > > So, that the issue could arise isn't unique to mod_wsgi, but an issue > for any system using sub interpreters, which for WSGI hosting means > both mod_wsgi and uWSGI. > > Ultimately, if an issue like this had occurred with gunicorn you would > have been equally stuffed. This is because the problem was in a C > extension module and problems in C extension modules like that are > hard to debug and invariably are fixed through thinking about the > problem and looking at source code and not through trying to run pdb > or even gdb on a running process. You are lucky when someone else > knows what the issue is already. > > That all said, that mod_wsgi and uWSGI in some modes use sub > interpreters does cause more than it share of problems. This though > pretty well always derives from the fact that people writing C > extension modules take short cuts and don't even contemplate that the > extension module may be used in sub interpreters or even in multiple > sub interpreters at the same time. > > There have even been examples where in the Python interpreter itself > they have broken things for sub interpreters. For example, fork() is > broken in Python 2.7.2 in sub interpreters. So, even the core > developers don't always get it quite right and they would generally > know more than most people who write C extension modules. > > One of the benefits at least is that mod_wsgi is configurable enough > that one can configure around broken third party software when these > sorts of issues arise, other systems may not provide you as much > flexibility. The config you used was also not a 'perfect config' in > the general sense, but was one that just solved your specific problem. > What configuration is best is going to be specific to a particular > application and the requirements. > > Now being part of the Apache eco system, mod_wsgi is a bit ham strung > by the default configs forced on people by PHP and Linux distros. So > the default isn't necessarily a good config for sites. Even so, it > still serves adequately for a lot people because the server generally > isn't the issue anyway. The only big problem is usually memory usage > because Apache/mod_wsgi for a Linux distro is embedded and prefork > MPM, which is bad for fat Python web applications. > > As to ease of debugging, if you are debugging your code and need to > run pdb or other IDE debugger, although it can be done under mod_wsgi > with the right knowledge, you are much better off doing it in a proper > development system. Debugging is not the missing thing here. What most > people really need is decent monitoring of their production systems. > What use is the ability to debug code if you have no idea what the > problems are that are occurring in your production system to start > with. > > So monitoring is going to be more important going forward. The WSGI > hosting issue is reasonably solved and debugging is something one > could always do. > > Think about it. Do you really have any idea what your production > system is doing? How much time is spent in your application vs > database, vs time spent calling out to external web services, memcache > etc etc. Do you know how long the application is even taking to do a > connect to your database for every Django request and how much time > that is as a percentage of overall response time. If you don't know, > how do you expect to be able to improve the performance of your > application. This is why monitoring is going to be key and where > Python web applications have been really lacking in the past. > > Graham > > > --Stuart > > On Jan 14, 2012 8:50 AM, "Sells, Fred" <fred....@adventistcare.org> > wrote: > > > > > > > > > > > > > > > > > You referenced Graham’s “The Perfect Config”. I’ve been googling > without > > > success to find that. Do you have a link?**** > > > > > ** ** > > > > > *From:* django...@googlegroups.com [mailto: > > > django...@googlegroups.com] *On Behalf Of *Stuart Laughlin > > > *Sent:* Friday, January 13, 2012 12:52 PM > > > *To:* django...@googlegroups.com > > > *Subject:* Re: Web Servers for Django Projects [WAS: Does anyone know > any > > > blogs...]**** > > > > > ** ** > > > > > Snip**** > > > > > After all, in spite of Graham coming through with The Perfect Config, > I > > > personally decided to not put myself through that experience again and > > > moved away from apache on subsequent projects. I've been pleased with > that > > > decision, and I know many others share my experience. > > > > > **** > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Django users" group. > > > To post to this group, send email to django...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > django-users...@googlegroups.com. > > > For more options, visit this group at > > >http://groups.google.com/group/django-users?hl=en. On Saturday, January 14, 2012 9:46:00 PM UTC-6, Graham Dumpleton wrote: > > > > On Jan 15, 9:40 am, Stuart Laughlin <stu...@bistrotech.net> wrote: > > Graham turned up on a thread in another forum where I was asking about > how > > to fix a strange error I had been seeing intermittently for months. He > > instructed me on how to put mod-wsgi into daemon mode, among other > things, > > and finally solved my problem. > > > > Here's a link to the thread, if you're interested. > > > > http://groups.google.com/group/satchmo-users/browse_thread/thread/6a6... > > > And the problem was actually due to a likely bug in psycopg2 and not > mod_wsgi. > > You would have had the same problems with knowing what the cause was > if you had hit this bug when using uWSGI in one of the ways that it > uses sub interpreters. > > So, that the issue could arise isn't unique to mod_wsgi, but an issue > for any system using sub interpreters, which for WSGI hosting means > both mod_wsgi and uWSGI. > > Ultimately, if an issue like this had occurred with gunicorn you would > have been equally stuffed. This is because the problem was in a C > extension module and problems in C extension modules like that are > hard to debug and invariably are fixed through thinking about the > problem and looking at source code and not through trying to run pdb > or even gdb on a running process. You are lucky when someone else > knows what the issue is already. > > That all said, that mod_wsgi and uWSGI in some modes use sub > interpreters does cause more than it share of problems. This though > pretty well always derives from the fact that people writing C > extension modules take short cuts and don't even contemplate that the > extension module may be used in sub interpreters or even in multiple > sub interpreters at the same time. > > There have even been examples where in the Python interpreter itself > they have broken things for sub interpreters. For example, fork() is > broken in Python 2.7.2 in sub interpreters. So, even the core > developers don't always get it quite right and they would generally > know more than most people who write C extension modules. > > One of the benefits at least is that mod_wsgi is configurable enough > that one can configure around broken third party software when these > sorts of issues arise, other systems may not provide you as much > flexibility. The config you used was also not a 'perfect config' in > the general sense, but was one that just solved your specific problem. > What configuration is best is going to be specific to a particular > application and the requirements. > > Now being part of the Apache eco system, mod_wsgi is a bit ham strung > by the default configs forced on people by PHP and Linux distros. So > the default isn't necessarily a good config for sites. Even so, it > still serves adequately for a lot people because the server generally > isn't the issue anyway. The only big problem is usually memory usage > because Apache/mod_wsgi for a Linux distro is embedded and prefork > MPM, which is bad for fat Python web applications. > > As to ease of debugging, if you are debugging your code and need to > run pdb or other IDE debugger, although it can be done under mod_wsgi > with the right knowledge, you are much better off doing it in a proper > development system. Debugging is not the missing thing here. What most > people really need is decent monitoring of their production systems. > What use is the ability to debug code if you have no idea what the > problems are that are occurring in your production system to start > with. > > So monitoring is going to be more important going forward. The WSGI > hosting issue is reasonably solved and debugging is something one > could always do. > > Think about it. Do you really have any idea what your production > system is doing? How much time is spent in your application vs > database, vs time spent calling out to external web services, memcache > etc etc. Do you know how long the application is even taking to do a > connect to your database for every Django request and how much time > that is as a percentage of overall response time. If you don't know, > how do you expect to be able to improve the performance of your > application. This is why monitoring is going to be key and where > Python web applications have been really lacking in the past. > > Graham > > > --Stuart > > On Jan 14, 2012 8:50 AM, "Sells, Fred" <fred....@adventistcare.org> > wrote: > > > > > > > > > > > > > > > > > You referenced Graham’s “The Perfect Config”. I’ve been googling > without > > > success to find that. Do you have a link?**** > > > > > ** ** > > > > > *From:* django...@googlegroups.com [mailto: > > > django...@googlegroups.com] *On Behalf Of *Stuart Laughlin > > > *Sent:* Friday, January 13, 2012 12:52 PM > > > *To:* django...@googlegroups.com > > > *Subject:* Re: Web Servers for Django Projects [WAS: Does anyone know > any > > > blogs...]**** > > > > > ** ** > > > > > Snip**** > > > > > After all, in spite of Graham coming through with The Perfect Config, > I > > > personally decided to not put myself through that experience again and > > > moved away from apache on subsequent projects. I've been pleased with > that > > > decision, and I know many others share my experience. > > > > > **** > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Django users" group. > > > To post to this group, send email to django...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > django-users...@googlegroups.com. > > > For more options, visit this group at > > >http://groups.google.com/group/django-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Django users" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-users/-/XkUVnWU9-LwJ. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
