[
https://issues.apache.org/jira/browse/CLOUDSTACK-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736662#comment-13736662
]
Koushik Das commented on CLOUDSTACK-4158:
-----------------------------------------
Log analysis
Time taken for the listAccount to complete ~3min20secs.
2013-08-07 23:14:52,996 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null)
===START=== 10.223.41.4 -- GET
command=listAccounts&listall=true&apikey=epDdgoWnO0n_4JTUzS81NUnkLs_mS4vaEgEYyRLyFyOnAEO5BLtFddtJ6uME22OWUh8NUHia4Lr_bt_UAC0ZHw&signature=hYAP0TmQI0BOhSllmVopl24VklE%3D
2013-08-07 23:18:15,759 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null)
===END=== 10.223.41.4 -- GET
command=listAccounts&listall=true&apikey=epDdgoWnO0n_4JTUzS81NUnkLs_mS4vaEgEYyRLyFyOnAEO5BLtFddtJ6uME22OWUh8NUHia4Lr_bt_UAC0ZHw&signature=hYAP0TmQI0BOhSllmVopl24VklE%3D
The select query on account_view is taking ~1min
2013-08-07 23:14:53,152 TRACE [db.Transaction.Statement]
(catalina-exec-22:null) Preparing: SELECT account_view.id, account_view.uuid,
account_view.account_name, account_view.type, account_view.state,
account_view.removed, account_view.cleanup_needed, account_view.network_domain,
account_view.domain_id, account_view.domain_uuid, account_view.domain_name,
account_view.domain_path, account_view.data_center_id,
account_view.data_center_uuid, account_view.data_center_name,
account_view.bytesReceived, account_view.bytesSent, account_view.vmLimit,
account_view.vmTotal, account_view.ipLimit, account_view.ipTotal,
account_view.ipFree, account_view.volumeLimit, account_view.volumeTotal,
account_view.snapshotLimit, account_view.snapshotTotal,
account_view.templateLimit, account_view.templateTotal,
account_view.stoppedVms, account_view.runningVms, account_view.projectLimit,
account_view.projectTotal, account_view.networkLimit,
account_view.networkTotal, account_view.vpcLimit, account_view.vpcTotal,
account_view.cpuLimit, account_view.cpuTotal, account_view.memoryLimit,
account_view.memoryTotal, account_view.primaryStorageLimit,
account_view.primaryStorageTotal, account_view.secondaryStorageLimit,
account_view.secondaryStorageTotal, account_view.job_id, account_view.job_uuid,
account_view.job_status, account_view.default FROM account_view WHERE
account_view.type != ? AND account_view.id != ? AND account_view.domain_path
LIKE ? AND account_view.removed IS NULL ORDER BY account_view.id ASC LIMIT
0, 30000
2013-08-07 23:15:57,318 TRACE [db.Transaction.Statement]
(catalina-exec-22:null) Closing:
com.mysql.jdbc.JDBC4PreparedStatement@5adb48a8: SELECT account_view.id,
account_view.uuid, account_view.account_name, account_view.type,
account_view.state, account_view.removed, account_view.cleanup_needed,
account_view.network_domain, account_view.domain_id, account_view.domain_uuid,
account_view.domain_name, account_view.domain_path,
account_view.data_center_id, account_view.data_center_uuid,
account_view.data_center_name, account_view.bytesReceived,
account_view.bytesSent, account_view.vmLimit, account_view.vmTotal,
account_view.ipLimit, account_view.ipTotal, account_view.ipFree,
account_view.volumeLimit, account_view.volumeTotal, account_view.snapshotLimit,
account_view.snapshotTotal, account_view.templateLimit,
account_view.templateTotal, account_view.stoppedVms, account_view.runningVms,
account_view.projectLimit, account_view.projectTotal,
account_view.networkLimit, account_view.networkTotal, account_view.vpcLimit,
account_view.vpcTotal, account_view.cpuLimit, account_view.cpuTotal,
account_view.memoryLimit, account_view.memoryTotal,
account_view.primaryStorageLimit, account_view.primaryStorageTotal,
account_view.secondaryStorageLimit, account_view.secondaryStorageTotal,
account_view.job_id, account_view.job_uuid, account_view.job_status,
account_view.default FROM account_view WHERE account_view.type != 5 AND
account_view.id != 1 AND account_view.domain_path LIKE _binary'/%' AND
account_view.removed IS NULL ORDER BY account_view.id ASC LIMIT 0, 30000
The count query on account_view is taking ~10secs
2013-08-07 23:15:57,321 TRACE [db.Transaction.Statement]
(catalina-exec-22:null) Preparing: select count(distinct id) from account_view
WHERE account_view.type != ? AND account_view.id != ? AND
account_view.domain_path LIKE ? AND account_view.removed IS NULL
2013-08-07 23:16:06,150 TRACE [db.Transaction.Statement]
(catalina-exec-22:null) Closing:
com.mysql.jdbc.JDBC4PreparedStatement@3daabc98: select count(distinct id) from
account_view WHERE account_view.type != 5 AND account_view.id != 1 AND
account_view.domain_path LIKE _binary'/%' AND account_view.removed IS NULL
Given that the db queries are getting over by 23:16:06 and the API is getting
completed at 23:18:15 means that creating the response is taking ~2mins. I see
that starting 4.2 additional data have been added to the account response
1. resource limits for cpu cores
2. resource limits for memory
3. resource limits for primary storage space and convert it from Bytes to GiB
4. resource limits for secondary storage space and convert it from Bytes to GiB
This obviously is increasing the overall time to generate the response.
> [Performance Testing] List API performance in comparison to 4.1 is not as good
> ------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4158
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4158
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: API
> Affects Versions: 4.2.0
> Environment: Simulator for creating mock resources, advanced zone, RVR
> Reporter: Sowmya Krishnan
> Assignee: Koushik Das
> Priority: Critical
> Fix For: 4.2.0
>
>
> Executed few List API calls for tracking performance runs for 4.2 and
> compared the results with 4.1. Most APIs are taking noticeably longer as
> compared to 4.1
> Examples:
> API Time taken in 4.1 Time taken in 4.2
>
> listAccounts 1m25.348s 2m22.628s
> listEvents 0m6.575s 0m20.277s
> Results so far are posted here (including comparitive results)
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Performance+Test+Execution+for+4.2
> Performance Test bed details:
> =======================
> Management server:
> Processor
> Dual core Intel(R) Xeon(R) CPU processor, 2.27GHz, ht enabled, 4 processor
> Operating System
> CentOS release 5.5 (Final), x86_64
> Configuration Parameters
> Following config parameters were used in both the management servers
> - Java heap size = 5 GB
> - db.cloud.maxActive = 250
> -
> db.cloud.url.params=prepStmtCacheSize=517&cachePrepStmts=true&prepStmtCacheSqlLimit=4096&includeInnodbStatusInDeadlockExceptions=true&logSlowQueries=true
> Setup:
> =====
> Advanced zone, 2 Management Servers, 124 Pods [Each Pod having 2 Clusters]
> 248 Clusters [Each cluster having 8 hosts and one primary storage]
> 2000 Hosts
> 4000 User accounts [Each account having one network]
> 4000 User instances
> ~8000 Virtual Routers [Since we are using Redundant Virtual Router offering
> Didn't notice any slow queries logged in DB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
