[Twisted-Python] Graceful shutdown

[Jonas Lindmark]

Hi!

I'm trying to implement graceful shutdown of a HTTP server and I am unsure
of the preferred way of implementing it.

I'm attempting to do this by adding a system event trigger for "before"
"shutdown" that should stop accepting new requests and wait for current
ones to finish.

My problem is that though my requests has fired their notifyFinish()
deferreds the data has not been written to the client. Calling
reactor.getWriters() shows me that there are active writers who are
probably writing/flushing data to the client.

What is the preferred way of waiting for the requests to finish and finish
writing the response to the client?

Here is example code for my own attempt:

from twisted.internet import reactor, defer
from twisted.application import internet, service
from twisted.web.server import Site, NOT_DONE_YET
from twisted.web.resource import Resource

class SlowResource(Resource):
isLeaf = True
waiting_requests = []


def notify_no_more_waiting(self):
if not self.waiting_requests:
return defer.succeed(None)
return defer.gatherResults(self.waiting_requests, consumeErrors=True) \
.addBoth(lambda ign: None)

def write_result(self, request):
request.write('{}')
request.finish()

def render_GET(self, request):
reactor.callLater(5, self.write_result, request)

d = request.notifyFinish()
self.waiting_requests.append(d)
d.addBoth(lambda ign: self.waiting_requests.remove(d))

return NOT_DONE_YET


slow_resource = SlowResource()
site = Site(slow_resource)
application = service.Application("MyApp")
server = internet.TCPServer(8080, site)
server.setServiceParent(application)

def wait_for_writers():
d = defer.Deferred()

def check_writers():
if len(reactor.getWriters()) > 0:
reactor.callLater(0.1, check_writers)
else:
d.callback(None)

check_writers()

return d

@defer.inlineCallbacks
def graceful_shutdown():
yield server.stopService()
yield slow_resource.notify_no_more_waiting()
#yield wait_for_writers()


reactor.addSystemEventTrigger('before', 'shutdown', graceful_shutdown)


Uncommenting the yield in graceful_shutdown gives me my intended behaviour
but I don't like the solution since I have no idea if the writers there
have anything to do with writing the results. They could be any writer
writing anything I assume.

Running this code with twistd -ny graceful.tac and curling
http://localhost:8080 and then issuing a SIGINT to the twistd process
results in an curl: (52) Empty reply from server.

-- 
/Jonas
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Graceful shutdown

[Jonas Lindmark]

Thanks for the link. Looking at your code I found the magic lines I was
missing.

Implementing my own HTTPChannel was the key to success. Now I don't have to
poll reactor.getWriters() to see if there are unfinished writers.

I ended up with something like this:
class ApiHTTPChannel(HTTPChannel):
_connection_lost = None

def notifyConnectionLost(self):
if self._connection_lost is not None:
return self._connection_lost
return defer.succeed(None)

def connectionMade(self):
HTTPChannel.connectionMade(self)
self._connection_lost = defer.Deferred()

def connectionLost(self, reason):
HTTPChannel.connectionLost(self, reason)
self._connection_lost.callback(None)

Thanks


On 28 August 2013 18:09, Aaron Gallagher <_...@habnab.it> wrote:

> I didn't read over your code, but this is something that I've
> independently implemented. You can see my implementation at
> . The Site
> subclass does some other monitoring-related things, but you should be
> able to tease out just the graceful shutdown parts if that's all you
> want.
>
> Polecat needs some work and a lot of documentation, but for now, the way
> to do a graceful shutdown is to make a PolecatSite and then
> addSystemEventTrigger('before', 'shutdown',
> yourPolecatSite.gracefullyStopActiveClients).
>



-- 
/Jonas
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python