-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 12/07/14 03:17, Mike Bayer wrote: > > On 7/11/14, 7:26 PM, Carl Baldwin wrote: >> >> >> On Jul 11, 2014 5:32 PM, "Vishvananda Ishaya" >> <vishvana...@gmail.com > <mailto:vishvana...@gmail.com>> wrote: >>> >>> I have tried using pymysql in place of mysqldb and in real >>> world > concurrency >>> tests against cinder and nova it performs slower. I was >>> inspired by > the mention >>> of mysql-connector so I just tried that option instead. > Mysql-connector seems >>> to be slightly slower as well, which leads me to believe that >>> the > blocking inside of >> >> Do you have some numbers? "Seems to be slightly slower" doesn't > really stand up as an argument against the numbers that have been > posted in this thread. >> >>> sqlalchemy is not the main bottleneck across projects. >>> >>> Vish >>> >>> P.S. The performanace in all cases was abysmal, so performance >>> work > definitely >>> needs to be done, but just the guess that replacing our mysql > library is going to >>> solve all of our performance problems appears to be incorrect >>> at > first blush. >> >> The motivation is still mostly deadlock relief but more >> performance > work should be done. I agree with you there. I'm still hopeful > for some improvement from this. > > > To identify performance that's alleviated by async you have to > establish up front that IO blocking is the issue, which would > entail having code that's blazing fast until you start running it > against concurrent connections, at which point you can identify via > profiling that IO operations are being serialized. This is a very > specific issue. > > In contrast, to identify why some arbitrary openstack app is slow, > my bet is that async is often not the big issue. Every day I look > at openstack code and talk to people working on things, I see > many performance issues that have nothing to do with concurrency, > and as I detailed in my wiki page at > https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy there is a > long road to cleaning up all the excessive queries, hundreds of > unnecessary rows and columns being pulled over the network, > unindexed lookups, subquery joins, hammering of Python-intensive > operations (often due to the nature of OS apps as lots and lots of > tiny API calls) that can be cached. There's a clear path to tons > better performance documented there and most of it is not about > async - which means that successful async isn't going to solve all > those issues. >
Of course there is a long road to decent performance, and switching a library won't magically fix all out issues. But if it will fix deadlocks, and give 30% to 150% performance boost for different operations, and since the switch is almost smooth, this is something worth doing. > > > > _______________________________________________ OpenStack-dev > mailing list OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCgAGBQJTwrP+AAoJEC5aWaUY1u57gRUH+wTAUcl1kujT5iVUwcZUEEfx P9nC0JPduXxYlobiFYyQKVVQm6pTaPgbgBoq4M/vKD0PbxBYMSTFA+igmewX6cHN RlsfvQgTsB/FU+dxK3gfRBQU3OnHLUSKWwZydp0YjmrCCHP3wiPj/HqWdD7vl1H8 Cxl+R2Zr7dR1ZMQBHDbAtu6FrGEa5SBnAwvAcsIr6mKymkFzQ4DU2DKOc8mm1i1f GhwCG8IvhKYo/w3yt0CUjPJoPSDaAoIGv984NDv7sjHeSZCRxNmV8jXlRUcztFcG lBAWguZtdyOiX+qHNmRZHV1Mm0SybcGIZN0Zw+u+1hpoiR0ZjAbLoofBwkCHxDE= =OXs7 -----END PGP SIGNATURE----- _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
