While a socket is open and receiving data, recv() will either give you a non-zero number of bytes if bytes are ready, or an EWOULDBLOCK (AKA EAGAIN) if no bytes are ready. A result of zero bytes (the empty string) means "end of file" - the other end has closed the socket.
So what's happening here is your client is timing out or otherwise canceling its request by closing the socket, and this is the correct, intentional response to that scenario. -g > On Jan 24, 2021, at 11:57 AM, Robert DiFalco <robert.difa...@gmail.com> wrote: > > You're absolutely right, I meant "cancel the deferred". I don't grok server > sockets very well so maybe someone can help. But apparently, klein does a > .doRead from our server socket (getting the request from the client?). This > returns a "why" of "connection done" so that closes the connection before we > have written our response to the client, and that cancels the deferred SQS > write. > > https://github.com/racker/python-twisted-core/blob/master/twisted/internet/selectreactor.py#L148-L155 > > <https://github.com/racker/python-twisted-core/blob/master/twisted/internet/selectreactor.py#L148-L155> > > The method above is "doRead". Which calls this: > https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/tcp.py#L239 > > <https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/tcp.py#L239> > > I guess if If socket.rcv() returns an empty string it simply closes the > connection. > https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/tcp.py#L249-L250 > > <https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/tcp.py#L249-L250> > > Is that normal? I mean I guess it must be but then why is the read getting an > empty string and closing the connection? I can't really account for it? Some > kind of back pressure due to load? > > Thanks for any thoughts. > > > > On Sun, Jan 24, 2021 at 11:32 AM Colin Dunklau <colin.dunk...@gmail.com > <mailto:colin.dunk...@gmail.com>> wrote: > > > On Sun, Jan 24, 2021 at 11:45 AM Robert DiFalco <robert.difa...@gmail.com > <mailto:robert.difa...@gmail.com>> wrote: > Hi, I apologize this question is a little vague. I'm looking for pointers. I > have a klein route that makes an underlying deferToThread call with a simple > single thread (an IO based sync call I can't change, a boto3 sqs write). The > thread pool is simple, just a couple of threads, nothing fancy. > > VERY rarely it appears that Klein cancels the thread. What techniques can I > use to figure out why my thread is being Canceled? There's nothing in the > failure to tell me "who, why, or where" it was canceled. Also, I cannot get > this down to a reproducible case, but here's the boto3 sqs wrapper, this fall > back works fine, but it's a band-aide for an error I can't track down.: > def write(self, payload): > """ > Write message to SQS async from thread pool. If twisted cancels the > thread, instead write synchronously. > > def _retrySynchronously(error): > if error.type != CancelledError: > return error > > log.warn("Async SQS write cancelled. Calling synchronously.") > return defer.succeed(self._writeSyncFallback(payload)) > > deferredCall = self._deferToThread(self.sqs.write, payload) > deferredCall.addErrback(_retrySynchronously) > return deferredCall > > def _writeSyncFallback(self, payload): > return self.sqs.write(payload) > > The _deferToThread call just uses my own thread pool with 2 threads, but is > otherwise stock. > > Is there a level of logging I'm missing or some other thing that would tell > me why the thread is being canceled? The retry works great and Klein does not > return an error from the route. > > Thanks in advance. > > > I think we'll need to see more code for this, specifically the caller of that > `write` method, and its callers, etc. Note that the thread itself isn't being > cancelled, the Deferred you get from _deferToThread is... so you'll most > likely need to find out what code interacts with that object to progress in > isolating this. > > In my quick skim of the deferToThread and ThreadPool source, I can't find any > explicit cancellations. While that certainly doesn't rule it out, it does > make me think you're more likely to find the issue by inspecting the callers > involved. > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com <mailto:Twisted-Python@twistedmatrix.com> > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > <https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python> > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
