[Twisted-Python] Is there a recommended way for a Service to request Application stop?

[Daniel Sutcliffe]

I have a hierarchy of Services some of which is MultiService and other
parts are my own implementations of IServiceCollection - in some
situations the a child Service may want to 'suggest' that the
Application's job is done (error, or simply task completed) and I'm
looking for some sort of standardized way to pass this info upstream.
The idea being that I may using my implemented Services in a variety
of Applications.

In this type of situation, is it the general intention a child Service
would use the Application directly, such that potential StopService()s
could bubble down? Or is there a normal pattern here to have messages
bubble up through the Services hierarchy? I'm not seeing anything like
this in the examples I've found or looking through the sources, but
I'm probably missing something.

Cheers
/dan
-- 
Daniel Sutcliffe 

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Is there a recommended way for a Service to request Application stop?

[Glyph Lefkowitz]

> On Oct 10, 2016, at 9:41 AM, Daniel Sutcliffe  wrote:
> 
> I have a hierarchy of Services some of which is MultiService and other
> parts are my own implementations of IServiceCollection - in some
> situations the a child Service may want to 'suggest' that the
> Application's job is done (error, or simply task completed) and I'm
> looking for some sort of standardized way to pass this info upstream.
> The idea being that I may using my implemented Services in a variety
> of Applications.
> 
> In this type of situation, is it the general intention a child Service
> would use the Application directly, such that potential StopService()s
> could bubble down? Or is there a normal pattern here to have messages
> bubble up through the Services hierarchy? I'm not seeing anything like
> this in the examples I've found or looking through the sources, but
> I'm probably missing something.


Services are just things that can be started and stopped.  Application is just 
a top-level object that associates a thing-to-start with a few bits of global 
process-level state, like logging and pidfile settings.

Therefore, the Service hierarchy abstraction is a poor fit for some code that 
needs to do some work and then exit; it's designed for long-running tools which 
can be started and stopped on demand.  For example, what happens if two Service 
objects think that the Application's job is "done"?

If you want to exit a process, calling `stop` on the reactor is generally the 
right way to go.

But: talking about this in such vague, abstract terms is unlikely to be 
helpful.  What, concretely, are you actually trying to do with the "Services in 
a variety of Applications"?

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Is there a recommended way for a Service to request Application stop?

[Daniel Sutcliffe]

Thanks for your feedback Glyph, responses embedded below.

On Oct 10, 2016, at 9:41 AM, Daniel Sutcliffe  wrote:
> I have a hierarchy of Services some of which is MultiService and other
> parts are my own implementations of IServiceCollection - in some
> situations the child Service may want to 'suggest' that the
> Application's job is done (error, or simply task completed) and I'm
> looking for some sort of standardized way to pass this info upstream.
> The idea being that I may using my implemented Services in a variety
> of Applications.
>
> In this type of situation, is it the general intention a child Service
> would use the Application directly, such that potential StopService()s
> could bubble down? Or is there a normal pattern here to have messages
> bubble up through the Services hierarchy? I'm not seeing anything like
> this in the examples I've found or looking through the sources, but
> I'm probably missing something.

On Mon, Oct 10, 2016 at 5:51 PM, Glyph Lefkowitz
 wrote:
> Services are just things that can be started and stopped.  Application is
> just a top-level object that associates a thing-to-start with a few bits of
> global process-level state, like logging and pidfile settings.
>
> Therefore, the Service hierarchy abstraction is a poor fit for some code
> that needs to do some work and then exit; it's designed for long-running
> tools which can be started and stopped on demand.  For example, what happens
> if two Service objects think that the Application's job is "done"?

This is perhap the type of thing I'm trying to code for; my goal is
that the Services themselves do not have the final say in when the
Application (or parent Service) is done. The event a Service feels is
a problem may cause its parent to stop it, or it could simply call a
method on it to remediate - depending on situation.

> If you want to exit a process, calling `stop` on the reactor is generally
> the right way to go.

ie. specifically I would not want a Service to stop the reactor, as
what it feels is a problem might only be a minor inconvenience for the
Application as a whole; but in another Application, or Application
state, it might be 'game over'.

> But: talking about this in such vague, abstract terms is unlikely to be
> helpful.  What, concretely, are you actually trying to do with the "Services
> in a variety of Applications"?

I'll try to give one fairly concrete example of where I'd like to use
this kind of pattern.
The Services are long running polling ModBus clients whose configs are
read from a DB by the parent, the child Service has no knowledge of
where its config came from. Occasionally the child Services config may
become totally unworkable (for a variety of reasons) and so they want
to tell their parent the situation to give it a chance to reconfig,
call child.stopService(), or...The parent will have many such
Services, appropriate action may include a child reconfig, or if all
children showing issues telling its parent the situation to ask for
'advice'.

It seems to me in Twisted's Applications the job of calling
stopService() (or similar) should always be the responsibility of the
parent Service, but as a child how should I give my parent a clue I
need attention?

I can see implementing this with my own Interfaces so the child
Services know more about their parent's Interfaces/attributes, or
related objects, and can bubble information up through these, but I'd
prefer my child Services to know as little about their parents as
possible so they can be re-used in other simple Twisted apps.
Basically, I didn't want to go implementing stuff when there were
already tried and tested Twisted patterns for dealing with this kind
of thing.

Generically, can I somehow bubble up events through the Service
hierarchy, or should I communicate with external objects to the
hierarchy that can bubble down actions from higher up?

Any suggestions, or pointers to similar examples will be most
appreciated - I'm not stuck on this quandary, just wanting to write
code which fits best with the Twisted way of doing things.
Cheers
/dan
-- 
Daniel Sutcliffe 

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Is there a recommended way for a Service to request Application stop?

[Glyph Lefkowitz]

> On Oct 10, 2016, at 4:38 PM, Daniel Sutcliffe  wrote:
> 
> Generically, can I somehow bubble up events through the Service
> hierarchy, or should I communicate with external objects to the
> hierarchy that can bubble down actions from higher up?


Following the >, the service 
hierarchy's job is just to make sure everything gets started up and shut down 
together.

It sounds to me like you have a pretty well-defined hierarchy which seems like 
it fits into the service hierarchy because it's roughly parallel in terms of 
which objects participate; however, you have very application-specific 
semantics for this parallel hierarchy.  For example, it's pretty unusual to 
have a super-service reconfigure a subordinate service in order to recover from 
an error condition, in my experience, unless you're talking about stuff like 
erlang supervision hierarchies, but that requires runtime support like the code 
being recovered running in a subprocess that doesn't share state.

It often feels like abstractions are expensive so you should have as few of 
them as possible; but, in reality, *simple* abstractions are cheap, and what 
makes abstraction expensive is when you overload them.  Make a new, simple 
abstraction that contains exactly the semantics you just described, and use 
composition to point at the appropriate point in the MultiService hierarchy.  
When it's time to "stop" a service, do setServiceParent(None); when it's time 
to "start" it, do setServiceParent(appropriateServiceParent).  This should take 
care of keeping your services in the appropriate state.

BTW, if you have stateful long-running services that have to self-modify based 
on changing circumstances, you might want to also check out 
https://github.com/glyph/automat  to see if 
it can help you ensure that everything's in a consistent state.

Good luck!

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python