Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread David Boreham
2006-05-04 18:04:58 EDT USER=postgres DB=FIX1 [12427] PORT = [local] ERROR:  invalid memory alloc request size 18446744073709551613 Perhaps I'm off beam here, but any time I've seen an app try to allocate a gazillion bytes, it's due to some code incorrectly calculating the size of something

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Jim C. Nasby
On Fri, May 05, 2006 at 10:27:10AM -0400, mcelroy, tim wrote: > Sorry, been up all night and maybe provided too much information or not the Do you have any budget for support or training, either from the company selling you the app or a company that provides PostgreSQL support? I suspect some mone

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Jim C. Nasby
On Fri, May 05, 2006 at 01:09:53PM -0400, Michael Stone wrote: > On Fri, May 05, 2006 at 06:33:33PM +0200, G?briel ?kos wrote: > >if postgres tries to allocate more memory (because of settings enable > >it) than the kernel itself is configured for, then you will see similar > >error messages. >

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Jim C. Nasby
On Fri, May 05, 2006 at 10:40:33AM -0400, Michael Stone wrote: > You've got 1.4G in use, 5.3G of disk cache, 165M of buffers and 667M > free. That doesn't seem unreasonable. If an application needs more Actually, it indiciates a bunch of memory not being used, but IIRC Tim's database is approxim

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Michael Stone
On Fri, May 05, 2006 at 06:33:33PM +0200, Gábriel Ákos wrote: if postgres tries to allocate more memory (because of settings enable it) than the kernel itself is configured for, then you will see similar error messages. If you're talking about the shared memory limits, postgres will bomb out

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Gábriel Ákos
Michael Stone wrote: On Fri, May 05, 2006 at 10:45:21AM -0400, mcelroy, tim wrote: Thanks Michael. Are you saying the 'used' column is the irrelevant number? Is the number that is more pertinent is 1416880? Is that the actual amount of memory in use? Yes. I agree about the allocation of

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Craig A. James
mcelroy, tim wrote: Sorry, been up all night and maybe provided too much information or not the right information and only confused folks, tired I guess. When I say 'in use' I am referring to the 'used' column. Thanks all who have responded to this inquiry, I appreciate it. Here's free from

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Michael Stone
On Fri, May 05, 2006 at 10:45:21AM -0400, mcelroy, tim wrote: Thanks Michael. Are you saying the 'used' column is the irrelevant number? Is the number that is more pertinent is 1416880? Is that the actual amount of memory in use? Yes. I agree about the allocation of a bogus amount of memo

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: RE: [PERFORM] Memory and/or cache issues? Thanks for a great explanation Craig, makes more sense now. Tim  -Original Message- From:   Craig A. James [mailto:[EMAIL PROTECTED]] Sent:   Friday, May 05, 2006 10:51 AM To: mcelroy, tim Cc: pgsql-performance@postgresql.or

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: RE: [PERFORM] Memory and/or cache issues? Thanks Michael.  Are you saying the 'used' column is the irrelevant number?  Is the number that is more pertinent is 1416880?  Is that the actual amount of memory in use?  I agree about the allocation of a bogus amount of memory but the issue oc

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: Message On the boxes in question the settings are:   shared_buffers = 1000 work_mem = 1024   I have revised these on my DEV box and see some improvement (a quick thank you to Jim Nasby for his assistance with that):   shared_buffers = 2 work_mem = 8024   Regards, Tim

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Michael Stone
On Fri, May 05, 2006 at 10:27:10AM -0400, mcelroy, tim wrote: Sorry, been up all night and maybe provided too much information or not the right information and only confused folks, tired I guess. When I say 'in use' I am referring to the 'used' column. Which is a mostly irrelevant number. H

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: RE: [PERFORM] Memory and/or cache issues? Sorry, been up all night and maybe provided too much information or not the right information and only confused folks, tired I guess.  When I say 'in use' I am referring to the 'used' column.  Thanks all who have responded to this inquiry, I app

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Dave Dutcher
Title: Message For a standard config most of the memory used by Postgres is the shared buffers.  The shared buffers are a cache to store blocks read from the disk, so if you do a query, Postgres will allocate and fill the shared buffers up to the max amount you set in your postgresql.conf fi

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Michael Stone
On Fri, May 05, 2006 at 09:57:58AM -0400, mcelroy, tim wrote: Are you saying the kernel's disc cache may be getting whacked? No, I understand that PG should use as much memory as it can and the system as well. The main problem here is that with almost all the 8GB of RAM 'in use' when I try to d

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: RE: [PERFORM] Memory and/or cache issues? Are you saying the kernel's disc cache may be getting whacked?  No, I understand that PG should use as much memory as it can and the system as well.  The main problem here is that with almost all the 8GB of RAM 'in use' when I try to do a pg_du

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Tom Lane
"mcelroy, tim" <[EMAIL PROTECTED]> writes: > I see this on all the postgres installations, no matter what I > set the postgresql.conf settings to regarding memory allocation, once > postgres starts up 95% of the memory on the box is used. Is there a way > within Linux to 'see' what or who is actua

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: RE: [PERFORM] Memory and/or cache issues? Thanks Tom.  I thought the same thing and waded through the archives trying various fixes such as vacuum, vacuum full (both with analyze), reindex and still the same issue.  However, once the box was rebooted the backup went smooth and the data

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Alvaro Herrera
Tom Lane wrote: > "mcelroy, tim" <[EMAIL PROTECTED]> writes: > > pg_dump: ERROR: invalid memory alloc request size 18446744073709551613 > > That looks more like a corrupt-data problem than anything directly to do > with having or not having enough memory. The bit pattern is certainly suspicious,

Re: [PERFORM] Memory and/or cache issues?

2006-05-05 Thread Tom Lane
"mcelroy, tim" <[EMAIL PROTECTED]> writes: > pg_dump: ERROR: invalid memory alloc request size 18446744073709551613 That looks more like a corrupt-data problem than anything directly to do with having or not having enough memory. regards, tom lane ---

[PERFORM] Memory and/or cache issues?

2006-05-05 Thread mcelroy, tim
Title: Memory and/or cache issues? Good morning, First the stats:  I'm using PostgreSQL 8.0.1 (I know I should upgrade, cannot due to vendor app. restrictions...), RedHat 9 on a SUN V40Z with 8GB of memory.  I'm using the "out-of-the-box" settings in postgresql.conf.  I've been testing vario

Re: [PERFORM] Super-smack?

2006-05-05 Thread Bruce Momjian
Isn't Super Smack a breakfast cereal? :-) --- Tom Lane wrote: > I wrote: > > FWIW, my own experiments with tests like this suggest that PG is at > > worst about 2x slower than mysql for trivial queries. If you'd reported >

Re: [PERFORM] Performance Issues on Opteron Dual Core

2006-05-05 Thread Magnus Hagander
> > > > > FWIW, I've found problems running PostgreSQL on Windows in a > > > > > multi-CPU environment on w2k3. It runs fine for some > period, and > > > > > then CPU and throughput drop to zero. So far I've > been unable to > > > > > track down any more information than that, other than the >