I have a pair of programs which trade python data back and forth by pickling up lists of objects on one side (using pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket connection to the receiver, who unpickles the data and uses it.
So far this has been working fine, but I now need a way of separating multiple chunks of pickled binary data in the stream being sent back and forth. Questions: Is it safe to do what I'm doing? I didn't think there was anything fundamentally wrong with sending binary pickled data, especially in the closed, safe environment these programs operate under...but maybe I'm making a poor assumption? I was going to separate the chunks of pickled data with some well-formed string, but couldn't that string potentially randomly appear in the pickled data? Do I just pick an extremely unlikely-to-be-randomly-generated string as the separator? Is there some string that will definitely NEVER show up in pickled binary data? I thought about base64 encoding the data, and then decoding on the opposite side (like what xmlrpclib does), but that turns out to be a very expensive operation, which I want to avoid, speed is of the essence in this situation. Is there a reliable way to determine the byte count of some pickled binary data? Can I rely on len(<pickled data>) == bytes? Thanks for all responses, -David -- Presenting: mediocre nebula. -- http://mail.python.org/mailman/listinfo/python-list
