[issue7322] Socket timeout can cause file-like readline() method to lose data

Martin Panter added the comment: IMO it might make sense in some cases to disallow subsequent reading from a buffered socket reader (or probably any BufferedReader) that raises an exception (not just a timeout). But the restriction seems unnecessary for unbuffered raw readers, and it also seem

[issue7322] Socket timeout can cause file-like readline() method to lose data

David Beazley added the comment: This bug is still present in Python 3.5, but it occurs if you attempt to do a readline() on a socket that's in non-blocking mode. In that case, you probably DO want to retry at a later time (unlike the timeout case). -- ___

[issue7322] Socket timeout can cause file-like readline() method to lose data

R. David Murray added the comment: Your suggestion sounds good to me. I still think that it is a common failure mode in a client server transaction for the server to fail to send a (complete) line that the client is expecting, and vice versa, requiring a timeout, but not necessarily a "restar

[issue7322] Socket timeout can cause file-like readline() method to lose data

Gregory P. Smith added the comment: If the server failed to close a transaction the protocol stream is over unless you mime relying on hope and luck. Poplib has a nasty set of server implementation bugs to work around here. Readline as defined today no longer suits its needs but I still strongl

[issue7322] Socket timeout can cause file-like readline() method to lose data

Changes by Florent Xicluna : -- nosy: +flox ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue7322] Socket timeout can cause file-like readline() method to lose data

R. David Murray added the comment: I don't think it is optimistic. The poplib transaction pattern is: send a command, get a response. If the response is not properly terminated, throw it away. Send a new command, get a response. There's no ambiguity there. In addition, this is a common t

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: > One could then continue processing, sending new transactions to the POP > server and getting responses. That's optimistic. You don't know how much data has been lost in readline(). Sure, again your test server, it happens to work :) But again other kinds o

[issue7322] Socket timeout can cause file-like readline() method to lose data

R. David Murray added the comment: This patch has caused a non-trivial regression between 3.2 and 3.2.1. The scenario in which I observed it is poplib. I create a POP3 connection with a timeout. At one point in its processing, poplib is reading lines until it gets a line '.\r\n', at which

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: Committed in r88622 (3.3) and r88623 (3.2). The 2.7 implementation is too different for the patch to apply, so if you want to fix it too, feel free to upload a patch. Thank you! -- resolution: -> fixed stage: patch review -> committed/rejected status

[issue7322] Socket timeout can cause file-like readline() method to lose data

David Beazley added the comment: Just wanted to say that I agree it's nonsense to continue reading on a socket that timed out (I'm not even sure what I might have been thinking when I first submitted this bug other than just experimenting with edge cases of the socket interface).It's stil

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: Looks good and simple enough. I would probably shift the timeout test after the closed test, but that's almost a detail. -- ___ Python tracker

[issue7322] Socket timeout can cause file-like readline() method to lose data

Ross Lagerwall added the comment: Attached patch disallows further reads after a timeout. -- Added file: http://bugs.python.org/file20398/i7322.patch ___ Python tracker ___ _

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: > > That's my opinion too. So, instead, of doing the above surgery inside > > the IO stack, the SocketIO layer could detect the timeout and disallow > > further access. What do you think? > > So after a timeout occurs the file-object basically becomes worthless

[issue7322] Socket timeout can cause file-like readline() method to lose data

Ross Lagerwall added the comment: > That complicates things quite a bit, > especially given that it has to be grafted on at least two layers of the > IO stack (the raw IO layer, and the buffered IO layer). Also the TextIO layer I think. > That's my opinion too. So, instead, of doing the above

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: > """Generally there is no guarantee that a buffered object works > "properly" when the raw IO object raises some exception > intermittently""" > > I disagree. EINTR is a classic case of this and is something that > buffering IO layers deal with all the time.

[issue7322] Socket timeout can cause file-like readline() method to lose data

Gregory P. Smith added the comment: """Generally there is no guarantee that a buffered object works "properly" when the raw IO object raises some exception intermittently""" I disagree. EINTR is a classic case of this and is something that buffering IO layers deal with all the time. (readli

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: By the way, I recently fixed the makefile() documentation: “The socket must be in blocking mode; it can have a timeout, but the file object’s internal buffer may end up in a inconsistent state if a timeout occurs.” (in http://docs.python.org/dev/library/socke

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: This is an interesting approach. The problem is that AFAICT the issue is not limited to readline. If you call e.g. read(1) and the socket times out after having returned the first 5000 bytes, then those 5000 bytes might get lost as well (depending on spec

[issue7322] Socket timeout can cause file-like readline() method to lose data

David Beazley added the comment: Have any other programming environments ever had a feature where a socket timeout returns an exception containing partial data?I'm not aware of one offhand and speaking as a systems programmer, something like this might be somewhat unexpected. My concern

[issue7322] Socket timeout can cause file-like readline() method to lose data

Ross Lagerwall added the comment: How about this? Instead of just losing the data that's been read so far in readline(), this patch adds the data as a new field to the exception that is thrown - this way the semantics remain exactly the same but the data is not discarded when a timeout occur

[issue7322] Socket timeout can cause file-like readline() method to lose data

Antoine Pitrou added the comment: While this patch looks conformant to the documentation, it is very likely to break code in the wild. Even in the stdlib, there are uses of makefile() + socket timeouts (e.g. in http.client and urllib). It would be better to find a way to make readline() funct

[issue7322] Socket timeout can cause file-like readline() method to lose data

Changes by Ross Lagerwall : -- nosy: +loewis, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue7322] Socket timeout can cause file-like readline() method to lose data

Ross Lagerwall added the comment: Attached is a patch which fixes the issue. Instead of allowing the readline method to lose data, it adds a check to SocketIO.readinto() to ensure that the socket does not have a timeout and throws an IOError if it does. Also does the same for SocketIO.write()

[issue7322] Socket timeout can cause file-like readline() method to lose data

Ned Deily added the comment: I see Issue7995 also addresses the issue of accept sockets inheriting nonblocking status and provides a suggested patch. -- ___ Python tracker ___ _

[issue7322] Socket timeout can cause file-like readline() method to lose data

Roy Smith added the comment: This is kind of ugly. On the one hand, I'm all for adding a check in makefile() to catch it being called on a non-blocking socket. On the other hand, you are correct that a user could change the mode leter. Even if we added checks for this in socket.setblocking(

[issue7322] Socket timeout can cause file-like readline() method to lose data

Ned Deily added the comment: This would seem to be an invalid test case. It is specifically documented that socket.makefile does not support this: "The socket must be in blocking mode (it can not have a timeout)". http://docs.python.org/py3k/library/socket.html#socket.socket.makefile I supp

[issue7322] Socket timeout can cause file-like readline() method to lose data

Roy Smith added the comment: Ataching a test case which demonstrates the bug. -- Added file: http://bugs.python.org/file19711/test-issue7322.py ___ Python tracker ___ ___

[issue7322] Socket timeout can cause file-like readline() method to lose data

Roy Smith added the comment: I'm looking into this now. In the meantime, I've opened a marginally-related bug, issue10473 -- nosy: +roysmith ___ Python tracker ___

[issue7322] Socket timeout can cause file-like readline() method to lose data

Changes by Mark Lawrence : -- stage: -> needs patch versions: +Python 2.7, Python 3.1, Python 3.2 -Python 2.6 ___ Python tracker ___ _

[issue7322] Socket timeout can cause file-like readline() method to lose data

Changes by Gregory P. Smith : -- assignee: -> gregory.p.smith nosy: +gregory.p.smith priority: -> normal ___ Python tracker ___ ___ P

[issue7322] Socket timeout can cause file-like readline() method to lose data

New submission from David M. Beazley : Consider a socket that has had a file-like wrapper placed around it using makefile() # s is a socket created previously f = s.makefile() Now, suppose that this socket has had a timeout placed on it. s.settimeout(15) If you try to read data from f, but n