Re: [Twisted-Python] Asynchronous context in Twisted
On Mar 3, 2011, at 2:39 PM, Glyph Lefkowitz wrote:
> On Mar 3, 2011, at 7:31 AM, Fantix King wrote:
>
> > Hi,
> >
> > I tried to make python.context work in asynchronous code between main
loops. Anyone has similar experience to share please?
> >
> > Not sure if I am rebuilding a wheel :P
> >
> >
http://code.google.com/p/little-site/source/browse/littlesite/custom_reactor.py
>
> This is something I've often thought about doing in Twisted itself,
actually :). But I wasn't sure that chaining context would actually do
anything practically useful most of the time. Have you found that it's
actually useful? Have you managed to leverage this to, for example, get
more informative error messages out of Deferred failures?
>
> Doing it as a subclass like this is not optimal, as it limits you to one
reactor (and the Select reactor is not really the best one). A wrapper
would be slightly more tricky (you'd have to deal with the places that the
reactor passes itself through to things like Process and Port, so you'd have
to create wrappers for those as well) but much more general.
Thanks for replying! :)
Yes! That's a wonderful idea to use this context for asynchronous traceback!
I made
some small changes to the code and wrote a patch for Twisted (as addReader
and
addWriter is quite different from one impl to another, I changed
SelectReactor only.
I haven't got a better idea for this, please advise), please see attachment.
With a simple example of raising exception in deferLater-ed function
(a-b-c-deferLater-d-e-f-g):
from twisted.internet import reactor
from twisted.internet.task import deferLater
reactor.usingAsyncTraceback = True
def g():
raise Exception('Something happened inside.')
def f():
return g()
def e():
return f()
def d():
return e()
def c():
deferred = deferLater(reactor, 1, lambda: None)
deferred.addCallback(lambda x: d())
return deferred
def b():
return c()
def a():
return b()
if __name__ == '__main__':
deferred = a()
def errback(failure):
failure.printTraceback()
deferred.addErrback(errback)
deferred.addBoth(lambda x: reactor.stop())
reactor.run()
I could get this:
Traceback (most recent call last):
File "test.py", line 31, in
deferred = a()
File "test.py", line 28, in a
return b()
File "test.py", line 25, in b
return c()
File "test.py", line 20, in c
deferred = deferLater(reactor, 1, lambda: None)
File "/home/fantix/ac/twisted/internet/task.py", line 751, in deferLater
delayedCall = clock.callLater(delay, d.callback, None)
File "/home/fantix/ac/twisted/internet/base.py", line 701, in callLater
_f, args, kw = self._chainContext(_f, args, kw)
*--- ---*
File "/home/fantix/ac/twisted/python/context.py", line 59, in
callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/home/fantix/ac/twisted/python/context.py", line 37, in
callWithContext
return func(*args,**kw)
File "/home/fantix/ac/twisted/internet/defer.py", line 361, in callback
self._startRunCallbacks(result)
File "/home/fantix/ac/twisted/internet/defer.py", line 455, in
_startRunCallbacks
self._runCallbacks()
--- ---
File "/home/fantix/ac/twisted/internet/defer.py", line 542, in
_runCallbacks
current.result = callback(current.result, *args, **kw)
File "test.py", line 21, in
deferred.addCallback(lambda x: d())
File "test.py", line 17, in d
return e()
File "test.py", line 14, in e
return f()
File "test.py", line 11, in f
return g()
File "test.py", line 8, in g
raise Exception('Something happened inside.')
Additionally, in my scenario of a 5 years old asynchronous Twisted web
application, we
need the "request" object available throughout all code between asynchronous
network
accesses and database accesses because our global configuration system
needs the
request object. It would greatly reduce our manual work to pass through the
request
object here and there to have a context working in the asynchronous way.
BR,
Fantix.
Index: twisted/python/failure.py
===
--- twisted/python/failure.py (revision 31014)
+++ twisted/python/failure.py (working copy)
@@ -18,7 +18,7 @@
import opcode
from cStringIO import StringIO
-from twisted.python import reflect
+from twisted.python import reflect, context
count = 0
traceupLength = 4
@@ -286,6 +286,8 @@
else:
self.parents = [self.type]
+self.async_stacks = context.get('__stacks__', [])[:]
+
def trap(self, *errorTypes):
"""Trap this failure if its type is in a predetermined list.
@@ -520,6 +522,11 @@
else:
w( 'Traceback (most recent call last):\n')
+# Asynchronous stacks
+for stack in self.async_stacks:
+format_frames(stack, w, detail)
+w('--- ---\n')
+
# Frames, formatted in appropriate style
if self.fra
[Twisted-Python] AMP cred stuff
Hey Glyph, Jp, and twisted sprinters, Sorry I couldn't stay longer -- I stretched my "travel day" as far as possible, but have to get back to my real job now (on which I'm under the gun atm). Hope I can sprint longer next time -- I had a lot of fun, and the energy in the room is amazing! I'll attach the code to a bug tonight (I'll create one if there isn't one). I hope I can carve out time to add Kerberos (in another issue, of course) soon -- selfishly, because I really need it for my app at work. Gotta get back to work now -- you guys all rock! Cheers, Steve ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] TestCase methods names
Hello! The TestCase class, historically, provided several ways to do the same (I guess because we inherited the semantics from Java). For example, to check equality, In Python2 we have: * assertEquals * assertEqual * failUnlessEqual However, note that "assertEquals" was never documented (not in Python's TestCase, neither in Trial's one). In Python 3, the "There should be one-- and preferably only one --obvious way to do it" rule was applied, and now the preferred way to do that is: * assertEqual Furthermore, assertEquals and failUnlessEqual are *deprecated* in Python 3: >>> unittest.TestCase.assertEquals >>> unittest.TestCase.failUnlessEqual (I'm showing examples here using assertEqual, but the same happens for all the functionalities there) So, I propose to stick to the same method names than Python; this way we'll be more consistent and easy to learn than keep providing all variants. In concrete, I say that we should: - Deprecate those names that are deprecated in Python 3 (I mean, still provide the functionality, but with a DeprecationWarning) - Stop using them in internal code. - Fix documentation to explain all this and show the chosen methods. What do you think? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] AMP cred stuff
On 02:25 pm, water...@pangalactic.us wrote: >Hey Glyph, Jp, and twisted sprinters, > >Sorry I couldn't stay longer -- I stretched my "travel day" >as far as possible, but have to get back to my real job now >(on which I'm under the gun atm). Hope I can sprint longer >next time -- I had a lot of fun, and the energy in the room >is amazing! > >I'll attach the code to a bug tonight (I'll create one if there >isn't one). I hope I can carve out time to add Kerberos (in >another issue, of course) soon -- selfishly, because I really >need it for my app at work. > >Gotta get back to work now -- you guys all rock! Great having you at the sprint, Steve! If only PyCon could be more than once a year. ;) I'm really looking forward to having proper authentication for AMP in Twisted. Thanks for working on it! Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] LoopingCalls and unclean reactor
Hi
I have a series of MultiService objects, with child Services. Some of these
services are TCPServers, for example, and others are my own objects
(extending from Service).
In the instance that I have a Service which controls a LoopingCall, I am
getting intermittent 'unclean reactor' errors during tests. I feel I might
be missing some handling of deferreds, perhaps.
(using 8.2.0 - looking to migrate to 10.2.0 soon)
Here is an example of the approach being used:
class AdapterQueue(service.MultiService):
def startService(self):
service.Service.startService(self)
self._looping_controller = LoopingCall(self._action)
d = self._looping_controller.start(self.delay, False)
d.addErrback(self._errorInScheduler)
def stopService(self):
service.Service.stopService(self)
d = self._looping_controller.deferred
self._looping_controller.stop()
return d
And an example error (happens about 1/3 of the time):
DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
(AdapterQueue._action, *(), **{})()
traceback at creation:
File "C:\Python26\lib\threading.py", line 497, in __bootstrap
self.__bootstrap_inner()
File "C:\Python26\lib\threading.py", line 525, in __bootstrap_inner
self.run()
File "C:\Python26\lib\threading.py", line 477, in run
self.__target(*self.__args, **self.__kwargs)
File "D:\dev\eggs\nose-0.11.3-py2.6.egg\nose\twistedtools.py", line
57, in
installSignalHandlers=False))
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\base.py", line
1128, in run
self.mainLoop()
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\base.py", line
1137, in mainLoop
self.runUntilCurrent()
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\base.py", line
757, in runUntilCurrent
call.func(*call.args, **call.kw)
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\task.py", line
115, in __call__
d.addCallback(cb)
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\defer.py", line
195, in addCallback
callbackKeywords=kw)
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\defer.py", line
186, in addCallbacks
self._runCallbacks()
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\defer.py", line
328, in _runCallbacks
self.result = callback(self.result, *args, **kw)
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\task.py", line
103, in cb
self._reschedule()
File
"D:\dev\eggs\twisted-8.2.0-py2.6-win32.egg\twisted\internet\task.py", line
139, in _reschedule
self.call = self.clock.callLater(nextTime - currentTime, self)
>
Many thanks
Brad
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
