Re: [Twisted-Python] Could Service.startService return a Deferred?
On Wed, Nov 27, 2013, at 19:20, exar...@twistedmatrix.com wrote:
> On 06:00 pm, peter.westl...@pobox.com wrote:
...
> >class Runner(service.Service):
> >def __init__(self, baton):
> >self.baton = baton
> >
> >def startService(self):
> >print 'startService', self.name, reactor.running
> >self.baton.addCallback(lambda ignore:deferLater(reactor, 1.0,
> >self.realStartService))
> >self.baton.addErrback(report)
>
> I'd be concerned with the state of `Runner` at this point in the
> process.
>
> What happens if the application gets shut down while that `deferLater`
> is still pending?
>
> `stopService` has a harder job because it might need to deal with a
> service that is partially initialized or a service that is completely
> initialized (and "partially initialized" may cover a multitude of
> different states depending on the complexity of your service).
> >
...
>
> What about something like this instead?
>
> @implementer(IService)
> class Runner(object):
> ...
> @classmethod
> def loadFromWhatever(cls, name):
> return deferLater(reactor, Runner, name)
>
> def __init__(self, name):
> self.name = name
>
> def startService(self):
> self.running = True
> print 'realStartService', self.name, reactor.running
>
> def parent(service):
> application.setServiceParent(service)
>
> loading = Runner.initializeFromWhatever("foo")
> loading.addCallback(parent)
> loading.addCallback(lambda ignored:
> Runner.initializeFromWhatever("bar"))
> loading.addCallback(parent)
> loading.addErrback(stopTheReactorOrWhatever)
>
> The advantage I see of this approach is that a `Runner` never exists in
> the service hierarchy until it is fully initialized, started, and
> running.
>
> If a `Runner` is only partially ready and the process shuts down then
> its `stopService` method isn't called because it's not part of the
> service hierarchy.
I'll do that, thank you!
Could the documentation for Service say something about which methods
can be called when? For instance, it would never have occurred to me
that
setServiceParent could be called after control had passed out of the
.tac
file and the reactor had started running. I see from the source that it
calls
startService, but this is definitely the sort of non-obvious tip that it
would
be helpful to have written down. Likewise the fact that startService
normally runs before the reactor starts.
Out of interest, was there a reason for not making Runner a subclass
of Service? There are some methods of IService that this version
doesn't implement.
> I could definitely imagine a library to help with this kind of thing.
> For example, perhaps you want the above encapsulated as:
>
> asynchronouslySetUpServices(application, [
> lambda: Runner.initializeFromWhatever("foo"),
> lambda: Runner.initialifrFromWhatever("bar")])
>
> And maybe then you want to add in some logic so that if the application
> gets shut down while some things are still being initialized then you
> cancel their Deferred. Then you have good cleanup support for the
> uninitialized case - without complicating `stopService` (the cleanup
> logic is isolated in the implementation of Deferred cancellation where
> it belongs - eg, with this `deferLater`-based asynchronousness it's
> alreay present since deferLater implements cancellation already).
I'll add it to my list of things to do one of these years :-)
Would you like to put in a ticket with a spec?
Thanks for the help,
Peter.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Could Service.startService return a Deferred?
This is kind of an ongoing bugaboo for me, too. Buildbot uses services *heavily*, and at runtime (to support reconfigs, etc.). We've had to work around issues like stop being called before asynchronous starts complete using DeferredLocks and other such fun. I filed a bug with some of my issues - https://twistedmatrix.com/trac/ticket/6813 It'd be great to fix some of this in the existing implementation, but #4366 seems to demonstrate that this is impossible without breaking compatibility. So, maybe we need to introduce a new service hierarchy -- perhaps with some ability to use the existing Service classes as child services. I'd be interested to hear thoughts on the matter. Dustin ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Could Service.startService return a Deferred?
On Wed, Nov 27, 2013, at 19:20, exar...@twistedmatrix.com wrote:
...
> What about something like this instead?
>
> @implementer(IService)
> class Runner(object):
> ...
> @classmethod
> def loadFromWhatever(cls, name):
> return deferLater(reactor, Runner, name)
>
> def __init__(self, name):
> self.name = name
>
> def startService(self):
> self.running = True
> print 'realStartService', self.name, reactor.running
>
> def parent(service):
> application.setServiceParent(service)
A small detail: should this be service.setServiceParent(application)?
Peter.
> loading = Runner.initializeFromWhatever("foo")
> loading.addCallback(parent)
> loading.addCallback(lambda ignored:
> Runner.initializeFromWhatever("bar"))
> loading.addCallback(parent)
> loading.addErrback(stopTheReactorOrWhatever)
>
> The advantage I see of this approach is that a `Runner` never exists in
> the service hierarchy until it is fully initialized, started, and
> running.
>
> If a `Runner` is only partially ready and the process shuts down then
> its `stopService` method isn't called because it's not part of the
> service hierarchy.
>
> I could definitely imagine a library to help with this kind of thing.
> For example, perhaps you want the above encapsulated as:
>
> asynchronouslySetUpServices(application, [
> lambda: Runner.initializeFromWhatever("foo"),
> lambda: Runner.initialifrFromWhatever("bar")])
>
> And maybe then you want to add in some logic so that if the application
> gets shut down while some things are still being initialized then you
> cancel their Deferred. Then you have good cleanup support for the
> uninitialized case - without complicating `stopService` (the cleanup
> logic is isolated in the implementation of Deferred cancellation where
> it belongs - eg, with this `deferLater`-based asynchronousness it's
> alreay present since deferLater implements cancellation already).
>
> Jean-Paul
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] IMAP4 Proxy to add headers and modify body of an incoming message
I will do that. I'm struggling with doing this myself so far because of my noobness but making small progress every day. Sent from my BatPhone > On Nov 27, 2013, at 23:34, Glyph wrote: > > > On Nov 27, 2013, at 6:39 PM, Krzysztof Oblucki wrote: Am I on the right track? Is imap4 lib the right tool for this job? If so, is modifying IMAP4Server to handle Deferreds the right thing to do? If not, what would you recommend? >>> >>> Sounds like you diagnosed the problem correctly. IMAP4Server needs to >>> handle Deferreds here and doesn't currently. >> Awesome, that's what I've been trying to do for the past couple of days. >> Definitely learning a lot of new things :) > > Perhaps you should file a ticket for this? (Searching for one first, of > course, to see if it already exists). > > Thanks, > > -glyph > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] IMAP4 Proxy to add headers and modify body of an incoming message
On Nov 28, 2013, at 12:10 PM, Krzysztof Oblucki wrote: > I will do that. I'm struggling with doing this myself so far because of my > noobness but making small progress every day. Feel free to ask here for any help you might need :). -glyph ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] IMAP4 Proxy to add headers and modify body of an incoming message
It would be great if I could get some pointers as to the correct way of doing it. If I were to start with search_DELETED what would be your recommendation? Sent from my BatPhone > On Nov 28, 2013, at 20:07, Glyph wrote: > >> On Nov 28, 2013, at 12:10 PM, Krzysztof Oblucki wrote: >> >> I will do that. I'm struggling with doing this myself so far because of my >> noobness but making small progress every day. > > Feel free to ask here for any help you might need :). > > -glyph > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
