Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2011-01-03 Thread Thierry Carrez
Bret Piatt wrote: > We can break this down into a list of measureable test cases. Many of these > tests aren't just single host system tests. They require an integrated > deployment to find and eliminate the bottleneck. I'm certain I'm missing > additional items we would want to measure so consi

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-31 Thread Ewan Mellor
:17 > To: openstack@lists.launchpad.net > Cc: 'Jay Pipes'; Ewan Mellor; zait...@redhat.com; jor...@openstack.org > Subject: RE: [Openstack] Some insight into the number of instances Nova > needs to spin up... > > We can break this down into a list of measureable test cases.

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-31 Thread Ewan Mellor
.mellor=citrix@lists.launchpad.net] > On Behalf Of Erik Carlin > Sent: 30 December 2010 19:00 > To: Bret Piatt; openstack@lists.launchpad.net > Subject: Re: [Openstack] Some insight into the number of instances Nova > needs to spin up... > > Agree the scale limits will increase over time.

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-31 Thread Ewan Mellor
ts.launchpad.net > [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] > On Behalf Of Rick Clark > Sent: 30 December 2010 17:42 > To: Erik Carlin > Cc: openstack@lists.launchpad.net > Subject: Re: [Openstack] Some insight into the number of instances Nova > needs to spin up

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Erik Carlin
unces+bret=openstack@lists.launchpad.net >[mailto:openstack-bounces+bret=openstack....@lists.launchpad.net] On >Behalf >Of Erik Carlin >Sent: Thursday, December 30, 2010 12:15 PM >To: Rick Clark >Cc: openstack@lists.launchpad.net >Subject: Re: [Openstack] Some insight into the nu

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Bret Piatt
ack-bounces+bret=openstack@lists.launchpad.net] On Behalf Of Erik Carlin Sent: Thursday, December 30, 2010 12:15 PM To: Rick Clark Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Some insight into the number of instances Nova needs to spin up... I suggest we consider the limits of a sin

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Erik Carlin
I suggest we consider the limits of a single nova deployment, not across all regions. To Pete's point, at a certain scale, people will break into parallel, independent nova deployments. A single, global, deployment becomes untenable at scale. I agree with Pete that 1M hosts (and 45M Vms) is a bi

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Erik Carlin
Jay - Few comments below... Erik On 12/30/10 10:43 AM, "Jay Pipes" wrote: >On Wed, Dec 29, 2010 at 2:27 PM, Erik Carlin >wrote: >> We know Amazon is highly, highly elastic. While the instances launched >> per day is impressive, we know that many of those instances have a short >> life. > >OK

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Rick Clark
Actually it was 1M hosts and Ivthink 45 million vms. It was meant to be across all regions. Jason Seats set the number arbitrarily, but it is a good target to not let us forget about scaling while we design. I think eventually all loads will be more ephemeral. So, I think I agree with your num

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Erik Carlin
You are right. The 1M number was VMs not hosts. At least, that was from one scale discussion we had within Rackspace. I'm not sure what the "official" nova target limits are and I can't find anything on launchpad that defines it. If there is something, could someone please send me a link. I'm

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Jay Pipes
On Wed, Dec 29, 2010 at 2:27 PM, Erik Carlin wrote: > We know Amazon is highly, highly elastic.  While the instances launched > per day is impressive, we know that many of those instances have a short > life. OK, good point. But, this begs the question: what should Nova's priority be? Elasticit

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Bret Piatt
penstack-bounces+bret=openstack@lists.launchpad.net [mailto:openstack-bounces+bret=openstack@lists.launchpad.net] On Behalf Of Ewan Mellor Sent: Thursday, December 30, 2010 7:48 AM To: Jay Pipes; openstack@lists.launchpad.net Subject: Re: [Openstack] Some insight into the number of instances Nova nee

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Pete Zaitcev
On Wed, 29 Dec 2010 19:27:09 + Erik Carlin wrote: > The 1M host limit still seems reasonable to me. [] In my opinion, such numbers are completely out of whack. Google's Chubby article says that the busiest Chubby has 90,000 clients (not hosts!) and the biggest datacenter has 10,000 systems.

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-30 Thread Ewan Mellor
enstack] Some insight into the number of instances Nova > needs to spin up... > > Some insight into the number of instances being spun up *per day* on > AWS, *just in the US-East-1 region*: > > http://www.jackofallclouds.com/2010/12/recounting-ec2/ > > Avg in 2010

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-29 Thread Erik Carlin
We know Amazon is highly, highly elastic. While the instances launched per day is impressive, we know that many of those instances have a short life. I see Guy is now teaming up with CloudKick on this report. The EC2 instance ID enables precise measurement of instances launched, and CloudKick pr

[Openstack] Some insight into the number of instances Nova needs to spin up...

2010-12-29 Thread Jay Pipes
Some insight into the number of instances being spun up *per day* on AWS, *just in the US-East-1 region*: http://www.jackofallclouds.com/2010/12/recounting-ec2/ Avg in 2010: 70,528 instances spun up each day Max: 150,800 instances in a single day Food for thought. What do we think Nova could/ca