On Apr 20, 3:51 am, yish <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I am having a problem with Apache (prefork),mod_python, Django (trunk-
> version 6410) and python 2.5.1. I am currently testing my site with 5
> apache processes and haven't yet set up memcached (on the list but had
> some issue with some of my cache keys not being valid for memcached).
>
> Every couple days I will find all my apache processes have locked up.
> I finally investigated a bit deeper this morning and after gdb-ing
> into each process, they were all stuck with an identical python stack
> trace. Now I am assuming that each process has arrived at the same
> state independent of each other (given prefork MPM) which would seem
> to indicate that this problem is quite rampant. Has anyone seen this
> before? Separately does anyone know if there is a way in apache/mod_pythonto
> detect and kill these processes stuck in this way?
There is no way to auto detect and kill off these processes if using
mod_python and Apache.
If however you use mod_wsgi in daemon mode, its ability to detect
deadlocked Python processes may work depending on the original cause.
If its deadlock detection mechanism doesn't help, then you could use
the inactivity-timeout option for the daemon process to force a
process restart when there is no detected input/output by active
requests, or no new requests. So, even if the deadlock isn't due to a
dodgy Python C extension where multithreading is present, the
inactivity timeout mechanism should help to kill of a stuck process.
Graham
> I am only using the cache for some model level pre-calculations and
> not for page or template caching. Also I took a quick look at the
> locmem cache code and was a little concerned about locking within
> set->_cull->delete where both set and delete try to acquire the same lock.
>
> Haven't put too much thought into it yet since threading issues always
> drive me a little batty.
>
> Here is a sample of the relevant python corresponing gdb stack traces
>
> Thanks, Yishai
>
> /usr/local/lib/python2.5/threading.py (216): wait
> /usr/local/lib/python2.5/threading.py (216): wait
> /usr/local/lib/python2.5/threading.py (292): acquire
> /usr/local/lib/python2.5/threading.py (292): acquire
> /home/yish/lib/python2.5/django/utils/synch.py (69): writer_enters
> /home/yish/lib/python2.5/django/core/cache/backends/locmem.py (69):
> delete
> /home/yish/lib/python2.5/django/core/cache/backends/simple.py (55):
> _cull
> /home/yish/lib/python2.5/django/core/cache/backends/simple.py (64):
> set
> /home/yish/lib/python2.5/django/core/cache/backends/simple.py (39):
> set
> /home/yish/lib/python2.5/django/core/cache/backends/locmem.py (48):
> set
> /home/yish/lib/python2.5/django/core/cache/backends/locmem.py (48):
> set
> /home/yish/webapps/django/branches/main/djangotest/goodrec/models.py
> (857): get_entity_brief
>
> #0 0x0013d7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
> #1 0x00442c1f in [EMAIL PROTECTED] () from /lib/tls/libpthread.so.0
> #2 0x00d3f483 in PyThread_acquire_lock (lock=0x966e1a0, waitflag=1)
> at Python/thread_pthread.h:334
> #3 0x00d43dbb in lock_PyThread_acquire_lock (self=0xfffffffc,
> args=0xb7ee802c) at ./Modules/threadmodule.c:46
> #4 0x00ccbb7a in PyCFunction_Call (func=0xb7679dec, arg=0xb7ee802c,
> kw=0x0) at Objects/methodobject.c:108
> #5 0x00d14ce0 in PyEval_EvalFrameEx (f=0x95a0324, throwflag=0) at
> Python/ceval.c:3564
> #6 0x00d15ead in PyEval_EvalCodeEx (co=0xb7c5f698,
> globals=0xb7be5604, locals=0x0, args=0x9425cf8, argcount=1,
> kws=0x9425cfc, kwcount=0, defs=0xb7bec9d8,
> defcount=1, closure=0x0) at Python/ceval.c:2831
> #7 0x00d13ec4 in PyEval_EvalFrameEx (f=0x9425bb4, throwflag=0) at
> Python/ceval.c:3659
> #8 0x00d15ead in PyEval_EvalCodeEx (co=0xb7c5f890,
> globals=0xb7be5604, locals=0x0, args=0x9426428, argcount=1,
> kws=0x942642c, kwcount=0, defs=0xb7beca78,
> defcount=1, closure=0x0) at Python/ceval.c:2831
> #9 0x00d13ec4 in PyEval_EvalFrameEx (f=0x94262ec, throwflag=0) at
> Python/ceval.c:3659
> #10 0x00d157c8 in PyEval_EvalFrameEx (f=0x94b6514, throwflag=0) at
> Python/ceval.c:3650
> #11 0x00d157c8 in PyEval_EvalFrameEx (f=0x963fbec, throwflag=0) at
> Python/ceval.c:3650
> #12 0x00d157c8 in PyEval_EvalFrameEx (f=0x9426454, throwflag=0) at
> Python/ceval.c:3650
> #13 0x00d15ead in PyEval_EvalCodeEx (co=0xb7aa4ec0,
> globals=0xb7aaaf0c, locals=0x0, args=0x94262ac, argcount=4,
> kws=0x94262bc, kwcount=0, defs=0xb7ab0598,
> defcount=1, closure=0x0) at Python/ceval.c:2831
> #14 0x00d13ec4 in PyEval_EvalFrameEx (f=0x9426164, throwflag=0) at
> Python/ceval.c:3659
> #15 0x00d15ead in PyEval_EvalCodeEx (co=0xb7aa4020,
> globals=0xb7aaad74, locals=0x0, args=0x94c1b0c, argcount=3,
> kws=0x94c1b18, kwcount=0, defs=0xb7ab06d8,
> defcount=1, closure=0x0) at Python/ceval.c:2831
> #16 0x00d13ec4 in PyEval_EvalFrameEx (f=0x94c19b4, throwflag=0) at
> Python/ceval.c:3659
> #17 0x00d157c8 in PyEval_EvalFrameEx (f=0x94c0c14, throwflag=0) at
> Python/ceval.c:3650
> #18 0x00d15ead in PyEval_EvalCodeEx (co=0xb78a47b8,
> globals=0xb78891c4, locals=0x0, args=0xb75490e0, argcount=7,
> kws=0x966b280, kwcount=5, defs=0xb78392a8,
> defcount=6, closure=0x0) at Python/ceval.c:2831
> #19 0x00cb9cf4 in function_call (func=0xb784a684, arg=0xb75490d4,
> kw=0xb746424c) at Objects/funcobject.c:517
> #20 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb75490d4,
> kw=0xb746424c) at Objects/abstract.c:1860
> #21 0x00ca5264 in instance_call (func=0xb784c7ec, arg=0xb75490d4,
> kw=0xb746424c) at Objects/classobject.c:2053
> #22 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb75490d4,
> kw=0xb746424c) at Objects/abstract.c:1860
> #23 0x00d0fc94 in PyEval_EvalFrameEx (f=0x94c109c, throwflag=0) at
> Python/ceval.c:3775
> #24 0x00d15ead in PyEval_EvalCodeEx (co=0xb78a4770,
> globals=0xb78891c4, locals=0x0, args=0xb7546ec0, argcount=3,
> kws=0x95d3e98, kwcount=1, defs=0xb782f740,
> defcount=3, closure=0x0) at Python/ceval.c:2831
> #25 0x00cb9cf4 in function_call (func=0xb784a64c, arg=0xb7546eb4,
> kw=0xb7464604) at Objects/funcobject.c:517
> #26 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb7546eb4,
> kw=0xb7464604) at Objects/abstract.c:1860
> #27 0x00ca5264 in instance_call (func=0xb784c7cc, arg=0xb7546eb4,
> kw=0xb7464604) at Objects/classobject.c:2053
> #28 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb7546eb4,
> kw=0xb7464604) at Objects/abstract.c:1860
> ---Type <return> to continue, or q <return> to quit---
> #29 0x00d0fc94 in PyEval_EvalFrameEx (f=0x94ca66c, throwflag=0) at
> Python/ceval.c:3775
> #30 0x00d15ead in PyEval_EvalCodeEx (co=0xb78309f8,
> globals=0xb7827acc, locals=0x0, args=0xb74e5038, argcount=1,
> kws=0x95ac388, kwcount=1, defs=0xb77ca6b8,
> defcount=1, closure=0x0) at Python/ceval.c:2831
> #31 0x00cb9cf4 in function_call (func=0xb77cd2cc, arg=0xb74e502c,
> kw=0xb746d24c) at Objects/funcobject.c:517
> #32 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb74e502c,
> kw=0xb746d24c) at Objects/abstract.c:1860
> #33 0x00d129b1 in PyEval_EvalFrameEx (f=0x959d7ac, throwflag=0) at
> Python/ceval.c:3844
> #34 0x00d15ead in PyEval_EvalCodeEx (co=0xb796e260,
> globals=0xb796624c, locals=0x0, args=0xb74e8e98, argcount=1,
> kws=0x95590d8, kwcount=1, defs=0x0,
> defcount=0, closure=0xb77c9f2c) at Python/ceval.c:2831
> #35 0x00cb9cf4 in function_call (func=0xb77cd374, arg=0xb74e8e8c,
> kw=0xb74622d4) at Objects/funcobject.c:517
> #36 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb74e8e8c,
> kw=0xb74622d4) at Objects/abstract.c:1860
> #37 0x00d129b1 in PyEval_EvalFrameEx (f=0x94083cc, throwflag=0) at
> Python/ceval.c:3844
> #38 0x00d15ead in PyEval_EvalCodeEx (co=0xb78f7770,
> globals=0xb7882934, locals=0x0, args=0xb7669758, argcount=1,
> kws=0x9645ba0, kwcount=1, defs=0x0,
> defcount=0, closure=0xb77ca64c) at Python/ceval.c:2831
> #39 0x00cb9cf4 in function_call (func=0xb77cd33c, arg=0xb766974c,
> kw=0xb7a6d2d4) at Objects/funcobject.c:517
> #40 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb766974c,
> kw=0xb7a6d2d4) at Objects/abstract.c:1860
> #41 0x00d129b1 in PyEval_EvalFrameEx (f=0x91a4d3c, throwflag=0) at
> Python/ceval.c:3844
> #42 0x00d157c8 in PyEval_EvalFrameEx (f=0x91b3d4c, throwflag=0) at
> Python/ceval.c:3650
> #43 0x00d157c8 in PyEval_EvalFrameEx (f=0x915f934, throwflag=0) at
> Python/ceval.c:3650
> #44 0x00d15ead in PyEval_EvalCodeEx (co=0xb7b66458,
> globals=0xb7b771c4, locals=0x0, args=0xb76eb218, argcount=2, kws=0x0,
> kwcount=0, defs=0x0, defcount=0,
> closure=0x0) at Python/ceval.c:2831
> #45 0x00cb9cf4 in function_call (func=0xb7b1b80c, arg=0xb76eb20c,
> kw=0x0) at Objects/funcobject.c:517
> #46 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb76eb20c, kw=0x0) at
> Objects/abstract.c:1860
> #47 0x00ca5ad5 in instancemethod_call (func=0xb76b657c,
> arg=0xb76eb20c, kw=0x0) at Objects/classobject.c:2497
> #48 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb754068c, kw=0x0) at
> Objects/abstract.c:1860
> #49 0x00ce5447 in slot_tp_call (self=0xb767b24c, args=0xb754068c,
> kwds=0x0) at Objects/typeobject.c:4633
> #50 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb754068c, kw=0x0) at
> Objects/abstract.c:1860
> #51 0x00d0fc94 in PyEval_EvalFrameEx (f=0x9133ec4, throwflag=0) at
> Python/ceval.c:3775
> #52 0x00d157c8 in PyEval_EvalFrameEx (f=0x914c91c, throwflag=0) at
> Python/ceval.c:3650
> #53 0x00d157c8 in PyEval_EvalFrameEx (f=0x90c6344, throwflag=0) at
> Python/ceval.c:3650
> #54 0x00d15ead in PyEval_EvalCodeEx (co=0xb7bf5530,
> globals=0xb7bf08ac, locals=0x0, args=0x90c62d4, argcount=0,
> kws=0x90c62d4, kwcount=7, defs=0x0,
> defcount=0, closure=0x0) at Python/ceval.c:2831
> #55 0x00d13ec4 in PyEval_EvalFrameEx (f=0x90c614c, throwflag=0) at
> Python/ceval.c:3659
> #56 0x00d15ead in PyEval_EvalCodeEx (co=0xb7bf5608,
> globals=0xb7bf08ac, locals=0x0, args=0xb76eb198, argcount=2, kws=0x0,
> kwcount=0, defs=0x0, defcount=0,
> closure=0x0) at Python/ceval.c:2831
> #57 0x00cb9cf4 in function_call (func=0xb7b73b8c, arg=0xb76eb18c,
> kw=0x0) at Objects/funcobject.c:517
> ---Type <return> to continue, or q <return> to quit---
> #58 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb76eb18c, kw=0x0) at
> Objects/abstract.c:1860
> #59 0x00ca5ad5 in instancemethod_call (func=0xb7b70d74,
> arg=0xb76eb18c, kw=0x0) at Objects/classobject.c:2497
> #60 0x00c9b31c in PyObject_Call (func=0x0, arg=0xb76eb1cc, kw=0x0) at
> Objects/abstract.c:1860
> #61 0x00c9b661 in PyObject_CallMethod (o=0xb7edeb0c, name=0x620e6d
> "HandlerDispatch", format=0x620e98 "O") at Objects/abstract.c:1891
> #62 0x0061cc32 in python_handler (req=0x90a7570, phase=0x6200d5
> "PythonHandler") atmod_python.c:1652
> #63 0x004c09d7 in ap_run_handler () from /home/goodrec/webapps/django/
> apache2/bin/httpd
> #64 0x004c0e43 in ap_invoke_handler () from /home/goodrec/webapps/
> django/apache2/bin/httpd
> #65 0x004bd8c5 in ap_process_request () from /home/goodrec/webapps/
> django/apache2/bin/httpd
> #66 0x004b863f in _start () from /home/goodrec/webapps/django/apache2/
> bin/httpd
> #67 0x090a7570 in ?? ()
> #68 0x00000004 in ?? ()
> #69 0x090a7570 in ?? ()
> #70 0x090a1630 in ?? ()
> #71 0x090a1644 in ?? ()
> #72 0x090a1558 in ?? ()
> #73 0x00000001 in ?? ()
> #74 0x004e8314 in ?? () from /home/goodrec/webapps/django/apache2/bin/
> httpd
> #75 0x0900de1c in ?? ()
> #76 0x00000001 in ?? ()
> #77 0xbff29388 in ?? ()
> #78 0x004cb369 in ap_run_process_connection () from /home/goodrec/
> webapps/django/apache2/bin/httpd
> #79 0x004cb369 in ap_run_process_connection () from /home/goodrec/
> webapps/django/apache2/bin/httpd
> #80 0x004cb6d4 in ap_process_connection () from /home/goodrec/webapps/
> django/apache2/bin/httpd
> #81 0x004be919 in ap_graceful_stop_signalled () from /home/goodrec/
> webapps/django/apache2/bin/httpd
> #82 0x004beb5c in ap_graceful_stop_signalled () from /home/goodrec/
> webapps/django/apache2/bin/httpd
> #83 0x004bec2f in ap_graceful_stop_signalled () from /home/goodrec/
> webapps/django/apache2/bin/httpd
> #84 0x004bf4f4 in ap_mpm_run () from /home/goodrec/webapps/django/
> apache2/bin/httpd
> #85 0x004c636a in main () from /home/goodrec/webapps/django/apache2/
> bin/httpd
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
