[Twisted-Python] Supporting a two-part client protocol.
Hi there, I'm planning to use Twisted to write a client for the following protocol: Each messages is composed of two separate messages: 1. A header, which is a serialized C struct, containing multiple fields, among them a `length` field. 2. A Protocol Buffer payload, which length is specified by the aforementioned `length` field on the header. While the initial implementation is focused on TCP, I do hope to support this same protocol over UDP eventually. What's the best way for me to implement such a client with Twisted? Specifically, to implement support for sending/receiving messages in the above format to/from a server? Thanks, Go ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Supporting a two-part client protocol.
On Mon, Feb 3, 2020 at 8:06 PM Go Luhng wrote: > > Hi there, > > I'm planning to use Twisted to write a client for the following protocol: > > Each messages is composed of two separate messages: > > 1. A header, which is a serialized C struct, containing multiple > fields, among them a `length` field. > 2. A Protocol Buffer payload, which length is specified by the > aforementioned `length` field on the header. > > While the initial implementation is focused on TCP, I do hope to > support this same protocol over UDP eventually. > > What's the best way for me to implement such a client with Twisted? > Specifically, to implement support for sending/receiving messages in > the above format to/from a server? > > Thanks, Go Assuming the header has a fixed length, https://twistedmatrix.com/documents/current/api/twisted.protocols.basic.IntNStringReceiver.html and its more-concrete subclasses are a decent source of inspiration. OTOH, that's for stream protocols, so if you want to eventually handle UDP, it's probably nicer to do the full sans-io thing (https://sans-io.readthedocs.io/) and wire it up with a more-basic Twisted protocol. Well, that's probably the better approach in any case. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Supporting a two-part client protocol.
> Colin Dunklau wrote: > > Assuming the header has a fixed length, It does. The header is just a serialized C struct, so it's fully-specified for length and offset of each field. > OTOH, that's for stream protocols, so if you want to eventually handle > UDP, it's probably nicer to do the full sans-io thing > (https://sans-io.readthedocs.io/) and wire it up with a more-basic > Twisted protocol. Could you elaborate on this? I'm new to Twisted, and also unfamiliar with sans-io. Specifically, I'm wondering what type of "more-basic" Twisted protocol you mean. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Supporting a two-part client protocol.
On Tue, Feb 4, 2020 at 1:18 AM Go Luhng wrote: > > > Colin Dunklau wrote: > > > > Assuming the header has a fixed length, > > It does. The header is just a serialized C struct, so it's > fully-specified for length and offset of each field. > > > OTOH, that's for stream protocols, so if you want to eventually handle > > UDP, it's probably nicer to do the full sans-io thing > > (https://sans-io.readthedocs.io/) and wire it up with a more-basic > > Twisted protocol. > > Could you elaborate on this? I'm new to Twisted, and also unfamiliar > with sans-io. > > Specifically, I'm wondering what type of "more-basic" Twisted protocol you > mean. The sans-io pattern is described well at that site, including a link to Cory Benfield's great talk (https://www.youtube.com/watch?v=7cC3_jGwl_U). The idea is to keep your protocol logic strictly separate from anything that does IO (like a Twisted Protocol's `dataReceived` and its transport's `write` method, or an asyncio thing, or blocking socket operations, etc), to make it easier to test and reuse. By "more-basic" I mean twisted.internet.protocol.Protocol and twisted.internet.protocol.DatagramProtocol. If you don't go full sans-io, I'd still strongly recommend splitting up you protocol implementation into distinct pieces. Twisted protocols can become hard to reason about when they become implicit state machines... avoid it by making a separate, explicit state machine and use that in the Protocol, instead of dumping the bits on the protocol instance itself. This way you at least still get the testability. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
