Hi,
By this e-mail I'd like to start a discussion about current zmq driver
internal design problems I've found out.
I wish to collect here all proposals and known issues. I hope this
discussion will be continued on Liberty design summit.
And hope it will drive our further zmq driver development efforts.
ZMQ Driver issues list (I address all issues with # and references are
in []):
1. ZMQContext per socket (blocker is neutron improper usage of messaging
via fork) [3]
2. Too many different contexts.
We have InternalContext used for ZmqProxy, RPCContext used in
ZmqReactor, and ZmqListener.
There is also zmq.Context which is zmq API entity. We need to
consider a possibility to unify their usage over inheritance (maybe
stick to RPCContext)
or to hide them as internal entities in their modules (see
refactoring #6)
3. Topic related code everywhere. We have no topic entity. It is all
string operations.
We need some topics management entity and topic itself as an entity
(not a string).
It causes issues like [4], [5]. (I'm already working on it).
There was a spec related [7].
4. Manual implementation of messaging patterns.
Now we can observe poor usage of zmq features in zmq driver. Almost
everything is implemented over PUSH/PULL.
4.1 Manual polling - use zmq.Poller (listening and replying for
multiple sockets)
4.2 Manual request/reply implementation for call [1].
Using of REQ/REP (ROUTER/DEALER) socket solves many issues. A
lot of code may be reduced.
4.3 Timeouts waiting
5. Add possibility to work without eventlet [2]. #4.1 is also related
here, we can reuse many of the implemented solutions
like zmq.Poller over asynchronous sockets in one separate thread
(instead of spawning on each new socket).
I will update the spec [2] on that.
6. Put all zmq driver related stuff (matchmakers, most classes from
zmq_impl) into a separate package.
Don't keep all classes (ZmqClient, ZmqProxy, Topics management,
ZmqListener, ZmqSocket, ZmqReactor)
in one impl_zmq.py module.
_drivers (package)
+-- impl_rabbit.py
+-- impl_zmq.py <- leave only ZmqDriver class here
+-- zmq_driver (package)
| +--- matchmaker.py
| +--- matchmaker_ring.py
| +--- matchmaker_redis.py
| +--- matchmaker_.py
...
| +--- client.py
| +--- reactor.py
| +--- proxy.py
| +--- topic.py
...
7. Need more technical documentation on the driver like [6].
I'm willing to prepare a current driver architecture overview with
some graphics UML charts, and to continue discuss the driver architecture.
Please feel free to add or to argue about any issue, I'd like to have
your feedback on these issues.
Thanks.
Oleksii Zamiatin
References:
[1] https://review.openstack.org/#/c/154094/
[2] https://review.openstack.org/#/c/151185/
[3] https://review.openstack.org/#/c/150735/
[4] https://bugs.launchpad.net/oslo.messaging/+bug/1282297
[5] https://bugs.launchpad.net/oslo.messaging/+bug/1381972
[6] https://review.openstack.org/#/c/130943/8
[7] https://review.openstack.org/#/c/144149/1
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
