On Mon, 2015-09-21, Chris Angelico wrote: > On Mon, Sep 21, 2015 at 6:38 PM, Marko Rauhamaa <ma...@pacujo.net> wrote: >> Chris Angelico <ros...@gmail.com>: >> >>> On Mon, Sep 21, 2015 at 5:59 PM, Marko Rauhamaa <ma...@pacujo.net> wrote: >>>> You can read a full buffer even if you have a variable-length length >>>> encoding. >>> >>> Not sure what you mean there. Unless you can absolutely guarantee that >>> you didn't read too much, or can absolutely guarantee that your >>> buffering function will be the ONLY way anything reads from the >>> socket, buffering is a problem. >> >> Only one reader can read a socket safely at any given time so mutual >> exclusion is needed. >> >> If you read "too much," the excess can be put in the application's read >> buffer where it is available for whoever wants to process the next >> message. > > Oops, premature send - sorry! Trying again. > > Which works only if you have a single concept of "application's read > buffer". That means that you have only one place that can ever read > data. Imagine a protocol that mainly consists of lines of text > terminated by CRLF, but allows binary data to be transmitted by > sending "DATA N\r\n" followed by N arbitrary bytes. The simplest and > most obvious way to handle the base protocol is to buffer your reads > as much as possible, but that means potentially reading the beginning > of the data stream along with its header. You therefore cannot use the > basic read() method to read that data - you have to use something from > your line-based wrapper, even though you are decidedly NOT using a > line-based protocol at that point. > > That's what I mean by guaranteeing that your buffering function is the > only way data gets read from the socket. Either that, or you need an > underlying facility for un-reading a bunch of data - de-buffering and > making it readable again.
The way it seems to me, reading a TCP socket always ends up as: - keep an application buffer - do one socket read and append to the buffer - consume 0--more complete "entries" from the beginning of the buffer; keep the incomplete one which may exist at the end - go back and read some more when there's a chance more data has arrived So the buffer is a circular buffer of octets, which you chop up by parsing it so you can see it as a circular buffer of complete and incomplete entries or messages. At that level, yes, the line-oriented data and the binary data would coexist in the same application buffer. /Jorgen -- // Jorgen Grahn <grahn@ Oo o. . . \X/ snipabacken.se> O o . -- https://mail.python.org/mailman/listinfo/python-list
