Hi, Cinder is lacking ZeroMQ messaging support in multibackend configurations. This is due to the fact that we're abusing ``Target.server`` property by appending "@backend" suffix to the hostname. This works just fine in RabbitMQ, as it routes messages using queue names, but fails in ZeroMQ when 'server' property stops matching hostnames.
There is a workaround for this in oslo.messaging [1], but it breaks multibackend configrations, as backend A on a host can get message addressed for the backend B on the same host. We've seen tries [2], [3], [4] to fix it, but these implementations were based on doing special operations when ZeroMQ is configured. This invalidates concept of oslo.messaging being an abstraction layer. My current idea to fix this [5] is to switch whole multibackend message routing from being based on ``Target.server`` to ``Target.topic``. Implementation is based on keeping the old RPC server that listens on "cinder-volume" topic for fanout messages and creating a new RPC server which listens on "cinder-volume.host@backend" topic and "host" server. This is fine with rolling upgrades, as '$topic.$server' is how server queue names are built in oslo.messaging, so we'll still listen on old queues. Please note that only the messaging layer is being changed. Internally volumes will still be seen as existing on "host@backend" servers. I've added an experimental gate-tempest-dsvm-zeromq-multibackend job, which passes fine on [5], but fails on master, which proves that the fix works. Is anyone seeing dangers in this approach? If not - I'm asking for reviews on [5]. Thanks, Michal [1] https://github.com/openstack/oslo.messaging/blob/7b5bec3133b6d74c4144fb8a33b9c9d2803e8858/oslo_messaging/_drivers/zmq_driver/zmq_address.py#L36-L39 [2] https://review.openstack.org/#/c/271848/ [3] https://review.openstack.org/#/c/277113/ [4] https://review.openstack.org/#/c/351862 [5] https://review.openstack.org/#/c/398452 __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
