On Sunday 19 July 2009 02:12:32 John Machin wrote: > > Apologies in advance for my ignorance -- the last time I dipped my toe > in that kind of water, protocols like zmodem and Kermit were all the > rage -- but I would have thought there would have been an off-the- > shelf library for peer-to-peer file transfer over a socket > interface ... not so?
*Grins at the references to Kermit and zmodem, and remembers Laplink and PC Anywhere* If there is such a transfer beast in Python, I have not found it. (There is an FTP module but that is not quite the same thing) I think it is because the network stuff is all done in the OS or NFS and SAMBA now - with drag and drop support and other nice goodies. I have ended up writing a netstring thingy, that addresses the string transfer problem by having a start sentinel, a four byte ASCII length (so you can see it with a packet sniffer/displayer) and the rest of the data escaped to take out the start sentinel and the escape character. It works, but the four byte ASCII limits the size of what can be sent and received. It guarantees to deliver either the whole string, or fail, or timeout. If anybody is interested I will attach the code here. It is not a big module. This question seems to come up periodically in different guises. To the OP: There are really very few valid ways of solving the string transfer problem, given a featureless stream of bytes like a socket. The first thing that must be addressed is to sync up - you have to somehow find the start of the thing as it comes past. And the second is to find the end of the slug of data that you are transferring. So the simplest way is to designate a byte as a start and end sentinel, and to make sure that such a byte does not occur in the data stream, other than as a start and end marker. This process is called escaping, and the reverse is called unescaping. (SDLC/HDLC does this at a bit pattern level) Another way is to use time, namely to rely on there being some minimum time between slugs of data. This does not work well on TCP/IP sockets, as retries at the lower protocol levels can give you false breaks in the stream. It works well on direct connections like RS-232 or RS-485/422 lines. Classic netstrings send length, then data. They rely on the lower level protocols and the length sent for demarcation of the slug, and work well if you connect, send a slug or two, and disconnect. They are not so hot for long running processes, where processors can drop out while sending - there is no reliable way for a stable receiver to sync up again if it is waiting for a slug that will not finish. Adapting the netstring by adding a sync character and time out is a compromise that I have found works well in practice. - Hendrik -- http://mail.python.org/mailman/listinfo/python-list
