For 3+ years, everything worked. Linode + nginx + uwsgi + web2py. Took
weeks of effort to config because of the poor articulation between the
layers and the profusion of slightly incompatible approaches to config
(open source + Linux = sysadmin hell).
Today, I upgraded ubuntu from 12.04 to 12.8. That seemed to work. I
stopped uwsgi (should have been stopped by the reinstall) and nginx. I
started both services.
Went to the site: "Internal Server Error"
Whose message is this: nginx, uwsgi, web2py? No way to know. Great
diagnostics.
I really want sysadmin to be a black box. Unless you are in the devops
tooling business then one is not advancing the world by debugging config.
It is not the work we want to do. It is not producing results or anything
new, It is fixing yesterday's bugs. It is non-essential except--oh,
yeah--in the obvious sense that it determines if our sites work at all.
But, the perpetual debugging effort across tens of thousands of people and
tens of hours per person per year is just a deadweight loss in productivity
and innovation. Isn't this what Docker, Fabric, Ansible, etc, etc are
meant to solve--but don't? Clearly the world needs some "contained"
solution that constrains our options (many of the variations are spurious
as in 5 ways to do the same exact thing), but simply works. That is for
the devops innovators to solve.
Back to the simple problem. Below are what I think (could be VERY wrong)
are the relevant config files and status information for nginx and uwsgi.
These are unchanged from when the site worked (pre upgrade to ubuntu 12.8).
Subset of processes running:
www-data 5153 1 0 18:08 ? 00:00:00 /usr/bin/uwsgi --ini
/usr/share/
www-data 5161 5153 0 18:08 ? 00:00:00 /usr/bin/uwsgi --ini
/usr/share/
www-data 5163 5153 0 18:08 ? 00:00:00 /usr/bin/uwsgi --ini
/usr/share/
www-data 5164 5153 0 18:08 ? 00:00:00 /usr/bin/uwsgi --ini
/usr/share/
www-data 5165 5153 0 18:08 ? 00:00:00 /usr/bin/uwsgi --ini
/usr/share/
root 5210 2 0 18:13 ? 00:00:00 [kworker/0:0]
root 5287 2799 0 18:15 ? 00:00:00 sshd: root@pts/0
root 5306 5287 0 18:15 pts/0 00:00:00 -bash
root 5368 1 0 18:16 ? 00:00:00 nginx: master process
/usr/sbin/
www-data 5369 5368 0 18:16 ? 00:00:00 nginx: worker process
www-data 5370 5368 0 18:16 ? 00:00:00 nginx: worker process
www-data 5371 5368 0 18:16 ? 00:00:00 nginx: worker process
www-data 5372 5368 0 18:16 ? 00:00:00 nginx: worker process
nginx config file in sites-enabled, called web2py:
server {
listen 80;
server_name $hostname;
location ~* /(\w+)/static/ {
root /var/web2py/applications/;
}
location / {
#uwsgi_pass 127.0.0.1:9001;
uwsgi_pass unix:///run/uwsgi/app/web2py/web2py.socket;
include uwsgi_params;
uwsgi_param UWSGI_SCHEME $scheme;
uwsgi_param SERVER_SOFTWARE nginx/$nginx_version;
}
}
server {
listen 443;
server_name $hostname;
ssl on;
ssl_certificate /etc/nginx/ssl/web2py.crt;
ssl_certificate_key /etc/nginx/ssl/web2py.key;
location / {
#uwsgi_pass 127.0.0.1:9001;
uwsgi_pass unix:///run/uwsgi/app/web2py/web2py.socket;
include uwsgi_params;
uwsgi_param UWSGI_SCHEME $scheme;
uwsgi_param SERVER_SOFTWARE nginx/$nginx_version;
}
}
uwsgi config file called web2py.xml in apps-enabled:
<uwsgi>
<plugin>python</plugin>
<socket>/run/uwsgi/app/web2py/web2py.socket</socket>
<pythonpath>/var/web2py/</pythonpath>
<app mountpoint="/">
<script>wsgihandler</script>
</app>
<master/>
<processes>4</processes>
<harakiri>60</harakiri>
<reload-mercy>8</reload-mercy>
<cpu-affinity>1</cpu-affinity>
<stats>/tmp/stats.socket</stats>
<max-requests>2000</max-requests>
<limit-as>512</limit-as>
<reload-on-as>256</reload-on-as>
<reload-on-rss>192</reload-on-rss>
<no-orphans/>
<vacuum/>
</uwsgi>
web2py wsgi handler:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
This file is part of the web2py Web Framework
Copyrighted by Massimo Di Pierro <mdipie...@cs.depaul.edu>
License: LGPLv3 (http://www.gnu.org/licenses/lgpl.html)
This is a WSGI handler for Apache
Requires apache+mod_wsgi.
In httpd.conf put something like:
LoadModule wsgi_module modules/mod_wsgi.so
WSGIScriptAlias / /path/to/wsgihandler.py
"""
# change these parameters as required
LOGGING = False
SOFTCRON = False
import sys
import os
path = os.path.dirname(os.path.abspath(__file__))
os.chdir(path)
if not os.path.isdir('applications'):
raise RuntimeError('Running from the wrong folder')
sys.path = [path] + [p for p in sys.path if not p == path]
sys.stdout = sys.stderr
import gluon.main
if LOGGING:
application = gluon.main.appfactory(wsgiapp=gluon.main.wsgibase,
logfilename='httpserver.log',
profiler_dir=None)
else:
application = gluon.main.wsgibase
if SOFTCRON:
from gluon.settings import global_settings
global_settings.web2py_crontype = 'soft'
Maybe I should also show uwsgi_params, but I have no idea where to find it
in the directory structure. In any case, I did not change it at all.
So, how can I further diagnose this?
Is there any obvious problem that you can suggest I fix?
What is the future solution to avoid this time wasting?
Sorry for the rant, but every time this happens, and it has happened to
everyone, many hours are wasted and nothing is accomplished (oh, yes--our
sites go from non-working to working). Collectively we should purge
unnecessary options and get to DETERMINISTIC, PREDICTABLE, DURABLE,
RELIABLE solutions and get back to meeting user needs and innovating. (Who
will take this on? Probably nothing but a fundamental rethinking of Linux
can solve it and no one is going to do that...)
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
