On Mon, 12 Dec 2005, Laszlo Zsolt Nagy wrote: > I think to be effective, I need to use TCP_NODELAY, and manually > buffered transfers.
Why? > I would like to create a general messaging object that has methods like > > sendinteger > recvinteger > sendstring > recvstring Okay. So you're really developing a marshalling layer, somewhere between the transport and application layers - fair enough, there are a lot of protocols that do that. > To be more secure, Do you really mean secure? I don't think using pickle will give you security. If you want security, run your protocol over an TLS/SSL connection. If, however, you mean robustness, then this is a reasonable thing to do - it reduces the amount of code you have to write, and so reduces the number of bugs you'll write! One thing to watch out for, though, is the compatibility of the pickling at each end - i have no idea what the backwards- and forwards-compatibility of the pickle protocols is like, but you might find that if they're on different python versions, the ends won't understand each other. Defining your own protocol down to the bits-on-the-socket level would preclude that possibility. > I think I can use this loads function to transfer more elaborate python > stuctures: > > def loads(s): > """Loads an object from a string. > @param s: The string to load the object from. > @return: The object loaded from the string. This function will not > unpickle globals and instances. > """ > f = cStringIO.StringIO(s) > p = cPickle.Unpickler(f) > p.find_global = None > return p.load() I don't know the pickle module, so i can't comment on the code. > Am I on the right way to develop a new protocol? Aside from the versioning issue i mention above, you should bear in mind that using pickle will make it insanely hard to implement this protocol in any language other than python (unless someone's implemented a python pickle library in it - is there such a beast for any other language?). Personally, i'd steer clear of doing it like this, and try to use an existing, language-neutral generic marshalling layer. XML and ASN.1 would be the obvious ones, but i wouldn't advise using either of them, as they're abominations. JSON would be a good choice: http://www.json.org/ If it's expressive enough for your objects. This is a stunningly simple format, and there are libraries for working with it for a wide range of languages. > Are there any common mistakes that programmers do? The key one, i'd say, is not thinking about the future. Make sure your protocol is able to grow - use a version number, so peers can figure out what language they're talking, and perhaps an option negotiation mechanism, if you're doing anything complex enough to warrant it (hey, you could always start without it and add it in a later version!). Try to allow for addition of new commands, message types or whatever, and for extension of existing ones (within reason). > Is there a howto where I can read more about this? Not really - protocol design is a bit of a black art. Someone asked about this on comp.protocols.tcp-ip a while ago: http://groups.google.co.uk/group/comp.protocols.tcp-ip/browse_thread/thread/39f810b43a6008e6/72ca111d67768b83 And didn't get much in the way of answers. Someone did point to this, though: http://www.internet2.edu/~shalunov/writing/protocol-design.html Although i don't agree with much of what that says. tom -- limited to concepts that are meta, generic, abstract and philosophical -- IEEE SUO WG -- http://mail.python.org/mailman/listinfo/python-list
