Hi, I have used git to get the cloudstack source from github to have a look at this problem. I could not find that part of code so I run a "grep -R usageRecordGeneric ./*" in the source folder and got no results. I can't find the code you've pasted earlier. Am I missing something?
Regards -----Original Message----- From: Nitin Mehta [mailto:nitin.me...@citrix.com] Sent: 31 May 2012 17:58 To: cloudstack-dev@incubator.apache.org Subject: RE: API usage records and SQL Hi Tamas, Asking for 500 records takes 5-6 seconds and 1000 records taking 12 seconds shows a linear increase and so doesn't seem like there is any performance degradation of any sort here. >From a quick glance at the code it seems that your analysis " It seems there >is a conversion/parsing algorithm somewhere in CS that struggles with the data >" might be correct. I see that after fetching the data from DB, CS iterates over each record and makes a DB call for setting the usage id (conversion from ID to UUID) depending on the usage type which Indicates the DB table to look for. Example below for (Object usageRecordGeneric : usageRecords) { if(usageRecord.getUsageType() == UsageTypes.RUNNING_VM || usageRecord.getUsageType() == UsageTypes.ALLOCATED_VM){ //VM Instance ID usageRecResponse.setUsageId(identityDao.getIdentityUuid("vm_instance", usageRecord.getUsageId().toString())); } else if(usageRecord.getUsageType() == UsageTypes.IP_ADDRESS){ //IP Address ID usageRecResponse.setUsageId(identityDao.getIdentityUuid("user_ip_address", usageRecord.getUsageId().toString())); } else if .......// So on for different usage type. So for fetching 1000 records its making 1 +1000 Db queries. Thanks, -Nitin -----Original Message----- From: Tamas Monos [mailto:tam...@veber.co.uk] Sent: Thursday, May 31, 2012 7:50 PM To: cloudstack-dev@incubator.apache.org Subject: API usage records and SQL Hi, I've spent lots of time trying to debug why does it take so long for the management server to answer the: "listUsageRecords&pagesize=1000&page=X" query. When I ask for 500 records I get results in about 5-6 seconds. If I increase the pagesize to 1000 I need to wait 12 seconds to start getting any response. This means to me somewhere there is a massive bottleneck either in mysql or the way cs generates the pages for response. I have tcpdumped and got proof that the sql server responds and delivers 1000 records in 300ms and after the management "hesitates" for about 10 seconds before sends the respond down to the client. It seems there is a conversion/parsing algorithm somewhere in CS that struggles with the data. I have and account with a month worth usage records (35k, 35 pages) and takes about 7 minutes (408sec) to get the all the data via "listUsageRecords&pagesize=1000&page=X". Every page adds a ~12 seconds delay (408/35=11.6). I have checked the server load and cpu utilization but nothing. Java is around 10% of CPU and the load is 0.06. This is just one account... Any advice or suggestion is appreciated. Regards Tamas Monos DDI +44(0)2034687012 Chief Technical Office +44(0)2034687000 Veber: The Hosting Specialists Fax +44(0)871 522 7057 http://www.veber.co.uk<http://www.veber.co.uk/> Follow us on Twitter: www.twitter.com/veberhost<http://www.twitter.com/veberhost> Follow us on Facebook: www.facebook.com/veberhost<http://www.facebook.com/veberhost>
