Hello,
Returning to performance issues of oslo.messaging.
I've found 2 biggest issue in existing implementation of rabbit (kombu)
driver:
1) For almost every message sent/received a new object of
Consumer/Publisher class is created. And each object of this class tries
to declare it's queue even if it's already declared.
https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_rabbit.py#L159
This causes a huge slowdown.
2) with issue #1 is fixed (I've applied a small hack to fix it in my
repo) the next big issue araise. For every rpc message received a reply
is sent when processing is done (it seems that reply is sent even for
"cast" calls which it really strange to me). Reply sent is using
connection pool to "speed up" replies. Due to bad implementation of
custom connection pool for every message sent underlying connection
channel is closed and reopened:
https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_rabbit.py#L689
Cause of this major issues oslo.messaging performance is at least 10
times slower than it could be.
My opinion is that there is no simple and elegant fix for this issues in
current implementation of oslo.messaging (most because of bad
architecture of the library and processing flow). My proposal is that we
should start working on new major release of messaging library with
architectural issues fixed. This will allow us to avoid mentioned issues
and provide much more performance and flexibility for end users.
Main goal that we should achieve is separate rpc code from messaging
code this will allow us to implement both parts in much simpler and
cleaner way and in the same time it would be much faster.
Please share your thoughts on this topic.
Regards,
Alexei Kornienko
On 06/16/2014 02:47 PM, Gordon Sim wrote:
On 06/13/2014 02:06 PM, Ihar Hrachyshka wrote:
On 10/06/14 15:40, Alexei Kornienko wrote:
On 06/10/2014 03:59 PM, Gordon Sim wrote:
>>>
I think there could be a lot of work required to significantly
improve that driver, and I wonder if that would be better spent
on e.g. the AMQP 1.0 driver which I believe will perform much
better and will offer more choice in deployment.
>>
I agree with you on this. However I'm not sure that we can do such
a decision. If we focus on amqp driver only we should mention it
explicitly and deprecate qpid driver completely. There is no point
in keeping driver that is not really functional.
The driver is functional. It may be not that efficient as
alternatives, but that's not a valid reason to deprecate it.
The question in my view is what the plan is for ongoing development.
Will the driver get better over time, or is it likely to remain as is
at best (or even deteriorate)?
Choice is good, but every choice adds to the maintenance burden, in
testing against regressions if nothing else.
I think an explicit decision about the future is beneficial, whatever
the decision may be.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
