Jason Madden added the comment: gevent has another simple reproducer for this. I do believe it's not gevent's fault, the fault is in the standard library; SimpleHandler._write needs to loop until `sent += self.stdeout.write(data)` is `len(data)`.
I have written up more on this at https://github.com/gevent/gevent/issues/778#issuecomment-205046001 For convenience I'll reproduce the bulk of that comment here: The user supplied a django application that produced a very large response that was getting truncated when using gevent under Python 3.4. (I believe gevent's non-blocking sockets are simply running into a different buffering behaviour, making it more likely to be hit under those conditions simply because they are faster). This looks like a bug in the standard library's `wsgiref` implementation. I tracked this down to a call to `socket.send(data)`. This method only sends whatever portion of the data it is possible to send at the time, and it returns the count of the data that was sent. The caller of `socket.send()` is responsible for looping to make sure the full `len` of the data is sent. This [is clearly documented](https://docs.python.org/3/library/socket.html#socket.socket.send). In this case, there is a call to `send` trying to send the full response, but only a portion of it is able to be immediately written. Here's a transcript of the first request (I modified gevent's `socket.send` method to print how much data is actually sent each time): ``` Django version 1.9.5, using settings 'gdc.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C. SENDING 17 SENT 17 OF 17 SENDING 37 SENT 37 OF 37 SENDING 38 SENT 38 OF 38 SENDING 71 SENT 71 OF 71 SENDING 1757905 SENT 555444 OF 1757905 [03/Apr/2016 19:48:31] "GET / HTTP/1.1" 200 1757905 ``` Note that there's no retry on the short send. Here's the stack trace for that short send; we can clearly see that there is no retry loop in place: ``` //3.4/lib/python3.4/wsgiref/handlers.py(138)run() 136 self.setup_environ() 137 self.result = application(self.environ, self.start_response) --> 138 self.finish_response() 139 except: 140 try: //3.4/lib/python3.4/wsgiref/handlers.py(180)finish_response() 178 if not self.result_is_file() or not self.sendfile(): 179 for data in self.result: --> 180 self.write(data) 181 self.finish_content() 182 finally: //3.4/lib/python3.4/wsgiref/handlers.py(279)write() 277 278 # XXX check Content-Length and truncate if too many bytes written? --> 279 self._write(data) 280 self._flush() 281 //3.4/lib/python3.4/wsgiref/handlers.py(453)_write() 451 452 def _write(self,data): --> 453 self.stdout.write(data) 454 455 def _flush(self): //3.4/lib/python3.4/socket.py(398)write() 396 self._checkWritable() 397 try: --> 398 return self._sock.send(b) 399 except error as e: 400 # XXX what about EINTR? > //gevent/_socket3.py(384)send() 382 from IPython.core.debugger import Tracer; Tracer()() ## DEBUG ## 383 --> 384 return count ``` `self.stdout` is an instance of `socket.SocketIO` (which is returned from `socket.makefile`). This is not documented on the web, but the [docstring also clearly documents](https://github.com/python/cpython/blob/3.4/Lib/socket.py#L389) that callers of `write` should loop to make sure all data gets sent. ---------- nosy: +jmadden versions: +Python 3.4 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24291> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
