We can break this down into a list of measureable test cases. Many of these tests aren't just single host system tests. They require an integrated deployment to find and eliminate the bottleneck. I'm certain I'm missing additional items we would want to measure so consider this an off the top of my head incomplete list.
Rate of change tests: 1. Maximum rate of change per host machine -- this could be create, delete, migrate, snapshot, backup. 2. Maximum number of host machines per host controller machine that it can sustain at their maximum rate of change. 3. Maximum number of host machines per glance machine that it can sustain at their maximum rate of change. 4. Maximum number of requests per second and total buffer of requests on the message queue per machine. 5. Maximum number of storage volume operations per storage controller machine. Scale tests: 1. Maximum number of VMs on a host machine. 2. Maximum number of VMs per host controller. 3. Maximum number of storage volumes attached to a host machine. 4. Maximum number of storage volumes per storage controller machine. 5. Maximum number of VM records in the cloud database. 6. Maximum number of storage volume records in the cloud database. 7. Maximum number of VM images managed per glance machine. I understand that these maximum numbers will vary greatly depending on what "a machine", the size of the test VM image, the network configuration, and many other factors. I propose we get a test bed setup to measure our own CloudMIPS (call it whatever we want) like a BogoMIPS so we can measure the performance of the codebase over time. We can run the tests each weekend on the trunk as of 00:00 GMT Saturday and by tracking it against the code merged each week ensure we're headed in the right direction and if not we'll be consciously aware of which changes impacted the system performance. I'm up for organizing the effort -- finding hardware, DC space, and getting operational support to manage it. Is anyone interested in participating? Back on the AWS example I'll make some sweeping generalizations and round numbers off to make the math easy. I'll assume 50k hosts in the measured facility with a peak rate of change of 150,000/day. That is only a required rate of change per host machine of 1 VM / 8 hours. Even if they only have 12.5k hosts in that facility that is 1 VM / 2 hours per machine. If they have 20 VMs sustained then we're looking at 250k to 1M VMs for scale. Bret Piatt OpenStack.org Twitter: @bpiatt / Mobile 210-867-9338 Open Source Cloud Infrastructure: http://www.openstack.org -----Original Message----- From: openstack-bounces+bret=openstack....@lists.launchpad.net [mailto:openstack-bounces+bret=openstack....@lists.launchpad.net] On Behalf Of Ewan Mellor Sent: Thursday, December 30, 2010 7:48 AM To: Jay Pipes; openstack@lists.launchpad.net Subject: Re: [Openstack] Some insight into the number of instances Nova needs to spin up... Those are interesting figures, but it would also be interesting to know how many of those are kept running. Our 1M host target is irrelevant if all those instances are shut down again a day later. If Amazon are adding *and keeping* 70k VMs a day then that's a very different matter. Ewan. > -----Original Message----- > From: openstack-bounces+ewan.mellor=citrix....@lists.launchpad.net > [mailto:openstack-bounces+ewan.mellor=citrix....@lists.launchpad.net] > On Behalf Of Jay Pipes > Sent: 29 December 2010 16:47 > To: openstack@lists.launchpad.net > Subject: [Openstack] Some insight into the number of instances Nova > needs to spin up... > > Some insight into the number of instances being spun up *per day* on > AWS, *just in the US-East-1 region*: > > http://www.jackofallclouds.com/2010/12/recounting-ec2/ > > Avg in 2010: 70,528 instances spun up each day > Max: 150,800 instances in a single day > > Food for thought. > > What do we think Nova could/can support? I know we are aiming at > supporting clouds of 1M physical hosts. Perhaps we need to re-assess? > :) > > /me urges more prioritization of the openstack-ci (continuous > integration and performance/stress testing) project in Cactus... > > -jay > > _______________________________________________ > Mailing list: https://launchpad.net/~openstack > Post to : openstack@lists.launchpad.net > Unsubscribe : https://launchpad.net/~openstack > More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
