Hi, This is the fix for this image listing performance issue of glance v2 api: https://review.openstack.org/#/c/105381/ , welcome to review/input.
IFY, by this change, the execution time of image listing is about four times as fast as old logic (16.606s .vs. 4.151s). zhiyan On Thu, Jul 3, 2014 at 9:34 PM, Zhi Yan Liu <lzy....@gmail.com> wrote: > Hi, > > I have done some tests [0] to try to figure out what's perf problem > there. As comparing, I did it by two separated cases a) sending > request by glanceclient cli normally, just like what you did before. > b) to around the glanceclient cli to send those equal api calls to > glance server directly. The results as below [1][2]. > > From the result of my test, it seems we have more overhead in v2 > handling stuff on glanceclient side but service side, IMO. > > In the result [1], you can see glanceclient handles v1 requests is > faster then v2 stuff, 0m6.958s .vs. 0m9.624s (user's real time), this > is close to your result, but as the comparing, from [2] you can see v2 > api handling on service side is not bad than v1, even ~2 times faster, > 0m6.189s .vs. 0m3.446s, and based on the api spec requirement, v2 > needs send two more requests for the schema validation of image and > member resource. > > It will be great if you can try this on your env as a > verification/double-check for my test result, and I'd like to know > more test details of your result if follow comparison test needed. > > Next step I will look into glanceclient. > > [0] > * I did it in my local dev env, it's not devstack but a setup based on > master sroucecode manually (all services). > * I prepared 1000 images for the test which under queue status. (so > there's no data) > * I didn't test v2 with registry yet. > > [1] > // v1 api with glanceclient cli > > $ glance --os-image-api-version 1 image-list | grep 'test-' | wc -l > 1000 > > $ time glance --os-image-api-version 1 image-list >/dev/null > real 0m6.958s > user 0m0.812s > sys 0m0.100s > > // v2 api with glanceclient cli > > $ glance --os-image-api-version 2 image-list | grep 'test-' | wc -l > 1000 > > $time glance --os-image-api-version 2 image-list >/dev/null > real 0m9.624s > user 0m6.268s > sys 0m0.080s > > [2] > // v1 api with curl > > $ head -5 img1000-curl-v1.sh > #!/bin/bash > > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v1/images/detail?sort_key=name\&sort_dir=asc\&limit=20 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v1/images/detail?sort_key=name\&sort_dir=asc\&limit=20\&marker=1696392e-0d04-4a72-a23a-317405c7ef01 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v1/images/detail?sort_key=name\&sort_dir=asc\&limit=20\&marker=6e1c351e-77ae-4fa6-a663-e9eb9100349a > > $ ./img1000-curl-v1.sh 2>/dev/null | grep '"images"' | wc -l > 51 > > $ time ./img1000-curl-v1.sh >/dev/null 2>&1 > real 0m6.189s > user 0m0.124s > sys 0m0.108s > > // v2 api with curl > > $ head -10 img1000-curl-v2.sh > #!/bin/bash > > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/schemas/image > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/schemas/member > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/images?limit=20 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/images?marker=8b8638a4-1419-4c90-9d99-c1d8cfae6c4f\&limit=20 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/images?marker=4bf1dd82-bc88-4c62-90b7-117f0c0efb9e\&limit=20 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/images?marker=4b741769-f371-45b9-ac91-69b6a02d37a5\&limit=20 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/images?marker=35b131f4-5019-48a7-a091-af7fc7df05fc\&limit=20 > curl -i -X GET -H 'X-Auth-Token: fc67d68b700747cba4dfc78a862b1064' -H > 'Content-Type: application/json' -H 'User-Agent: python-glanceclient' > http://10.20.0.1:9292/v2/images?marker=c9b95dfb-def8-4ff6-825d-70ba49555b5d\&limit=20 > > $ ./img1000-curl-v2.sh 2>/dev/null | grep '"images"' | wc -l > 51 > > $ time ./img1000-curl-v2.sh >/dev/null 2>&1 > real 0m3.446s > user 0m0.224s > sys 0m0.132s > > zhiyan > > On Thu, Jun 26, 2014 at 6:39 PM, Sam Morrison <sorri...@gmail.com> wrote: >> Hi Stuart, >> >> Yeah we’re using Icehouse which has this patch included. Glad to hear it’s >> not just our set up and to confirm that v2 API does seem to be slower. >> I’m just worried as there is talk of deprecating v2 API but currently it >> doesn’t seem anywhere near ready to be a suitable replacement for v1. >> >> Hopefully as more people start to look at v2 we can iron out some of these >> issues. >> >> Cheers, >> Sam >> >> >> >> On 25 Jun 2014, at 10:54 pm, stuart.mcla...@hp.com wrote: >> >>> Hi Sam, >>> >>> We're just starting to look at v2. >>> >>> We did see some really slow listing behaviour, backporting >>> Ie7458044edbe1ef64871bcf5ef4996ed3cc0f957 helped a lot -- but you'll >>> probably have that change already if you're using more recent code. >>> >>> Even with that it does seem slower in some cases, eg devstack listing 1000 >>> images: >>> >>> v1: >>> $ time ./.tox/py27/bin/glance --os-image-api-version 1 image-list > >>> /dev/null >>> real 0m6.058s >>> user 0m1.348s >>> sys 0m0.172s >>> >>> v2, no registry: >>> $ time ./.tox/py27/bin/glance --os-image-api-version 2 image-list > >>> /dev/null >>> real 0m15.487s >>> user 0m9.233s >>> sys 0m0.120s >>> >>> v2, with registry: >>> $ time glance --os-image-api-version 2 image-list > /tmp/out >>> real 0m21.904s >>> user 0m9.217s >>> sys 0m0.116s >>> >>> >>> -Stuart >>> >>> On Wed, 25 Jun 2014, Sam Morrison wrote: >>> >>>> We’ve recently enabled glance v2 API on our glance servers. We’ve noticed >>>> it is considerably slower than v1 to do things like glance image-list. (~4 >>>> seconds compared to ~1) >>>> Has anyone else experienced this or run any benchmarks comparing the two >>>> APIs? >>>> >>>> I should note we’re using glance registry for both v1 and v2 requests too. >>>> >>>> Also just wondering if people are actually using v2? >>>> For instance from what I can tell you can’t create an image with v2 via >>>> the client yet [1] >>>> >>>> Cheers, >>>> Sam >>>> >>>> >>>> [1] https://bugs.launchpad.net/python-glanceclient/+bug/1302351 >>>> _______________________________________________ >>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>> Post to : openstack@lists.openstack.org >>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> >> >> _______________________________________________ >> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> Post to : openstack@lists.openstack.org >> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
