On 09:49 pm, [email protected] wrote:
>for a multiprocess server, I'd like to set SO_REUSEPORT on a TCP
>listening socket.
You might find that https://stackoverflow.com/questions/10077745
/twistedweb-on-multicore-multiprocessor obviates the need for this
option (and
thus your follow-up features as well).
Jean-Paul,
yes, I am aware of this (awesome) answer;) This works, but has 2
slightly
undesirable aspects:
1) it requires to have a master process that creates the socket and
starts workers
which "adopt" the port. I'd like to be able to fire up workers
independently .. with
no master at all.
There are ways around this but they're a bit more work (eg, workers
listen on a unix socket and hand out the port to new workers that start
up and ask for it).
2) the distribution of incoming connections to workers isn't completely
uniformly
distributed across workers.
This problem is explained here (they report up to 3:1 ratios of
connections per worker):
https://lwn.net/Articles/542629/
https://lwn.net/Articles/542718/
and I could verify this during some experiments (though not such
extreme non-uniformity)
Huh. I didn't know about that. That's too bad.
Not that Twisted shouldn't offer some way to gain more control over
this kind
of platform-specific option. But, until it does...
I guess that means there is no "recommended" way currently;)
Correct.
Would you mind giving a tip on how to make use of a CustomPort deriving
of Port? How to "plug" that into Twisted?
I have two suggestions, one of which I hope you'll like and the other of
which you might not. :)
First, endpoints are the intended extension point for this sort of thing
now. You can write a plugin for the parser so that `serverFromString`
will give out server endpoints for your port type (giving an appropriate
string description). Applications won't be able to tell what's going on
because the server endpoint interface is just `.listen(factory)ยด.
Second, please don't subclass `Port`. It's true there are no
underscores anywhere in its name (`twisted.internet.tcp.Port`) making it
part of Twisted's public API. Nevertheless, it's very much a reactor
implementation detail. It's a mistake that it's public. To compounded
this, the exactly interface between a class and its subclasses is hazy
and gross at best. I'd discourage you (and everyone else) from
subclassing *most* things in Twisted these days (lots of our APIs are
still subclassing-oriented so it's not always possible to avoid, of
course).
In this case, I think you might actually be able to re-use all of the
important parts of `Port` without subclassing it. You can create a
bare-bones implementation of `IListeningPort` that creates a socket and
sets the flags you want on it. Then, use `reactor.adoptStreamPort` to
get the reactor to create and initialize a new `Port` with your socket.
This leaves you with a little code to duplicate (basically
`createInternetSocket`) but a pretty small amount - and the upside is
that you're totally isolated from the internals of `Port`, from the
accidentally-public implementation details of `Port`, and even from the
implementation detail of whether the reactor even *uses* `Port` or not.
All you rely on is `reactor.adoptStreamPort` which is a nice,
documented, tested, intentionally-public interface. :)
(For SSL, then you can wrap your own twisted.protocols.tls wrapper
around the factory - which is all the reactor's listenSSL does these
days, anyway).
Hope this helps,
Jean-Paul
_______________________________________________
Twisted-Python mailing list
[email protected]
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
