Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-29 Thread John R Pierce
top is not the be-all and end-all of analysis tools. I'm sure you know that, but it bears repeating. More importantly, in a virtualised environment the tools on the inside of the guest don't have a full picture of what's really going on. Indeed, you have hit the nail on the head. does anyon

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-29 Thread Mike Bresnahan
In an attempt to determine whether top(1) is lying about the CPU utilization, I did an experiment. I fired up a EC2 c1.xlarge instance and ran pgbench and a tight loop in parallel. -bash-4.0$ uname -a Linux domu-12-31-39-00-8d-71.compute-1.internal 2.6.31-302-ec2 #7-Ubuntu SMP Tue Oct 13 19:55:22

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-29 Thread Rodger Donaldson
Mike Bresnahan wrote: > I can understand that I will not get as much performance out of a EC2 instance as a dedicated server, but I don't understand why top(1) is showing 50% CPU utilization. If it were a memory speed problem wouldn't top(1) report 100% CPU utilization? A couple of points: top

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-28 Thread Jeff Davis
On Thu, 2010-01-28 at 22:45 +, Mike Bresnahan wrote: > I can understand that I will not get as much performance out of a EC2 instance > as a dedicated server, but I don't understand why top(1) is showing 50% CPU > utilization. One possible cause is lock contention, but I don't know if that exp

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-28 Thread Mike Bresnahan
Greg Smith 2ndquadrant.com> writes: > Looks to me like you're running into a general memory bandwidth issue > here, possibly one that's made a bit worse by how pgbench works. It's a > somewhat funky workload Linux systems aren't always happy with, although > one of your tests had the right co

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-28 Thread Greg Smith
Mike Bresnahan wrote: I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) instance in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a small database, I am unable to peg the CPUs no matter how many clients I throw at it. In fact, the CPU utilization never drop

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-28 Thread Mike Bresnahan
Jim Mlodgenski gmail.com> writes: > Let's start from the beginning. Have you tuned your postgresql.conf file? What do you have shared_buffers set to? That would have the biggest effect on a test like this.  shared_buffers = 128MB maintenance_work_mem = 256MB checkpoint_segments = 20 -- Sent v

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Jim Mlodgenski
On Wed, Jan 27, 2010 at 6:37 PM, Mike Bresnahan wrote: > Greg Smith 2ndquadrant.com> writes: > > Could you try this again with "top -c", which will label these > > postmaster processes usefully, and include the pgbench client itself in > > what you post? It's hard to sort out what's going on in

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
Greg Smith 2ndquadrant.com> writes: > Could you try this again with "top -c", which will label these > postmaster processes usefully, and include the pgbench client itself in > what you post? It's hard to sort out what's going on in these > situations without that style of breakdown. As a fur

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
> Could you try this again with "top -c", which will label these > postmaster processes usefully, and include the pgbench client itself in > what you post? It's hard to sort out what's going on in these > situations without that style of breakdown. I had run pgbench on a separate instance last

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
John R Pierce hogranch.com> writes: > more likely, he's disk IO bound, but hard to say as that iostat output > only showed a couple 2 second slices of work. the first output, which > shows average since system startup, seems to show the system has had > relatively high average wait times of 1

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread John R Pierce
I have seen behavior like this in the past on EC2. I believe your bottleneck may be pulling the data out of cache. I benchmarked this a while back and found that memory speeds are not much faster than disk speeds on EC2. I am not sure if that is true of Xen in general or if its just limited t

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Greg Smith
Mike Bresnahan wrote: top - 15:55:05 up 1:33, 2 users, load average: 2.44, 0.98, 0.44 Tasks: 123 total, 11 running, 112 sleeping, 0 stopped, 0 zombie Cpu(s): 18.9%us, 8.8%sy, 0.0%ni, 70.6%id, 0.0%wa, 0.0%hi, 1.7%si, 0.0%st Mem: 7348132k total, 1886912k used, 5461220k free,34

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
Jim Mlodgenski gmail.com> writes: > I have seen behavior like this in the past on EC2. I believe your bottleneck may be pulling the data out of cache. I benchmarked this a while back and found that memory speeds are not much faster than disk speeds on EC2. I am not sure if that is true of Xen in g

Re: [GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Jim Mlodgenski
On Wed, Jan 27, 2010 at 3:59 PM, Mike Bresnahan wrote: > I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) > instance > in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a > small > database, I am unable to peg the CPUs no matter how many clients I throw at >

[GENERAL] Amazon EC2 CPU Utilization

2010-01-27 Thread Mike Bresnahan
I have deployed PostgresSQL 8.4.1 on a Fedora 9 c1.xlarge (8x1 cores) instance in the Amazon E2 Cloud. When I run pgbench in read-only mode (-S) on a small database, I am unable to peg the CPUs no matter how many clients I throw at it. In fact, the CPU utilization never drops below 60% idle. I also