On Mon, Jan 10, 2011 at 11:57:14PM +0200, Alex Grönholm wrote:
> The main focus of the discussion is to figure out how to best integrate
> support for this new API to Twisted. If possible, existing protocols
> should remain compatible through the use of adapters or some other
> mechanism. If not, a way to port them over with a minimal amount of work
> would be the next target.
I think I've read PEP 3148 before, but that was a while ago. Looking
over it again now, here are my initial thoughts as just a user of
Twisted.
- Executor.submit() seems to be an analogue to Twisted's
reactor.callInThread(), that should be easy enough to add to Twisted's
implementation.
- Executor.map() doesn't havve a Twisted analogue, as far as I know, but
seems a useful addition.
- Executor.shutdown() seems like a Bad Idea, at least for implementation
in Twisted's reactor - I don't want to call some third-party library
and have it shut down my entire process when I'm done! Perhaps, rather
than making Twisted's reactor implement the Executor interface, there
should be .makeExecutor() method that returns an Executor instance
that can support the .shutdown() semantics without actually shutting
down the Twisted event loop.
- Futures are obviously fairly similar to Deferreds, but there are some
differences of approach:
- Futures support cancellation; Twisted finally managed to get rid
of cancellation support in Deferreds.
- Futures pass themselves to a series of callbacks, and have
.result() and .exception() methods so that the callbacks can
discover what happened. Deferreds pass results or Failure objects
through a double list of callbacks and errbacks.
- Futures store the results of the function call that was run in
a thread, and pass the same information to each calback. Deferreds
allow each callback to transform the result and even pause the
callback-chain to wait for more asynchronous results.
- Futures allow code to wait indefinitely for a result or exception
to appear, which makes sense if the result is being calculated in
a thread, but which would cause a deadlock in an event-based
system like Twisted.
It should be pretty simple to create a Deferred that wraps a Future:
from twisted.internet import defer
def deferredFromFuture(future):
d = defer.Deferred()
def callback(future):
e = future.exception()
if e:
d.fail(e)
return
d.succeed(future.result())
future.add_done_callback(callback)
return d
I guess a creating a future that wraps a Deferred wouldn't be hard
either:
from concurrent.futures import Future
def futureFromDeferred(deferred):
f = Future()
f.set_running_or_notify_cancel()
def callback(result):
f.set_result(result)
return result
d.addcallback(callback)
def errback(failure):
f.set_exception(failure.value)
return failure
d.adderrback(errback)
return f
Of course, if somebody added another callback after the ones added by
futureFromDeferred(), the new values wouldn't be passed to the Future,
and if somebody called future.result() or future.exception() without
a timeout, that could cause a deadlock as mentioned above. I'm not sure
it's possible to work around either of those problems, so perhaps
futureFromDeferred() is a thing that should not be included in
a library, at least not without big warnings all over it like Twisted's
integration with the stdlib "logging" module has.
I also can't immediately see a way to make an object that functions as
both a Deferred and a Future, or build Deferred's functionality on top
of Futures. Perhaps I just haven't thought it through sufficiently - I'd
be interested to hear if anyone else has any ideas.
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
