Re: [PERFORM] Indexes with descending date columns

2006-03-16 Thread andrew
> I have a performance problem when traversing a table in index order with > multiple columns including a date column in date reverse order. Below > follows a simplified description of the table, the index and the > associated query > > \d prcdedit > prcdedit_prcd | character(20)

Re: [PERFORM] 1 TB of memory

2006-03-16 Thread Josh Berkus
Luke, > With a single 3 Gbyte/second infiniband connection to the device? Hey, take it easy! Jim's post was tongue-in-cheek. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 5: don't forget to increase your fre

Re: [PERFORM] 1 TB of memory

2006-03-16 Thread Luke Lonergan
Jim, On 3/16/06 10:44 PM, "Luke Lonergan" <[EMAIL PROTECTED]> wrote: > Plus - need more speed? Add 12 more servers, and you'd run at 12.8GB/s and > have 96TB of disk to work with, and you'd *still* spend less on HW and SW > than the SSD. And I forgot to mention that with these 16 servers you'd

Re: [PERFORM] 1 TB of memory

2006-03-16 Thread Luke Lonergan
Jim, > PostgreSQL tuned to the max and still too slow? Database too big to > fit into memory? Here's the solution! http://www.superssd.com/ > products/tera-ramsan/ With a single 3 Gbyte/second infiniband connection to the device? You'd be better off with 4 x $10K servers that do 800MB/s from dis

Re: [PERFORM] Background writer configuration

2006-03-16 Thread Kevin Grittner
>>> On Thu, Mar 16, 2006 at 12:15 pm, in message <[EMAIL PROTECTED]>, Evgeny Gridasov <[EMAIL PROTECTED]> wrote: > > please, could you post other settings from your postgresql.conf? Everything in postgresql.conf which is not commented out: listen_addresses = '*' # what IP interface(s)

Re: [PERFORM] Help optimizing a slow index scan

2006-03-16 Thread Dan Harris
Dan Harris wrote: explain analyze doh.. sorry to reply to my own post. But I messed up copying some of the fields into the select statement that you'll see in the "Sort Key" section of the analyze results. The mistake was mine. Everything else is "normal" between the query and the plan

[PERFORM] Help optimizing a slow index scan

2006-03-16 Thread Dan Harris
explain analyze select distinct eventmain.incidentid, eventmain.entrydate, eventgeo.long, eventgeo.lat, eventgeo.geox, eventgeo.geoy from eventmain, eventgeo where eventmain.incidentid = eventgeo.incidentid and ( long > -104.998027962962 and long < -104.985957781349 ) and ( lat > 39.707

Re: [PERFORM] 1 TB of memory

2006-03-16 Thread Matthew Nuzum
On 3/16/06, Jim Nasby <[EMAIL PROTECTED]> wrote: > PostgreSQL tuned to the max and still too slow? Database too big to > fit into memory? Here's the solution! http://www.superssd.com/ > products/tera-ramsan/ > > Anyone purchasing one will be expected to post benchmarks! :) Pricing is tight-lipped,

Re: [PERFORM] 1 TB of memory

2006-03-16 Thread Ron Peacetree
The US Dept of Homeland Security has at least two =10=TB SSDs. Rumor is they are being used for Carnivore or an offshoot/descendent of Carnivore. Good luck getting them to give you benchmark data. You need >deep< pockets to afford >= 1TB of SSD. (...and as the example shows, perhaps more money

[PERFORM] Indexes with descending date columns

2006-03-16 Thread Theo Kramer
Hi I have a performance problem when traversing a table in index order with multiple columns including a date column in date reverse order. Below follows a simplified description of the table, the index and the associated query \d prcdedit prcdedit_prcd | character(20) | prc

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Simon Riggs
On Thu, 2006-03-16 at 14:45 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > [BTW how do you add new indexes to system tables? I want to add one to > > pg_inherits but not sure where to look.] > > src/include/catalog/indexing.h > > Offhand I think adding a new entry is all you

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I was thinking that we would fill out the OpExpr with different > opclasses for each plan, so each one sees a different story. (I was > thinking there was a clauselist for each plan; if not, there could be.) This is backwards: there isn't a plan yet. If t

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Simon Riggs
On Thu, 2006-03-16 at 15:41 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > [BTW how do you add new indexes to system tables? I want to add one to > > pg_inherits but not sure where to look.] > > See src/include/catalog/indexing.h -- I don't remember if there's > anything else that needs

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Alvaro Herrera
Simon Riggs wrote: > [BTW how do you add new indexes to system tables? I want to add one to > pg_inherits but not sure where to look.] See src/include/catalog/indexing.h -- I don't remember if there's anything else that needs modification. -- Alvaro Herrerahttp:/

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Simon Riggs
On Thu, 2006-03-16 at 10:57 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Trying to get the information in the wrong place would be very > > expensive, I agree. But preparing that information when we have access > > to it and passing it through the plan would be much cheaper.

Re: [PERFORM] 1 TB of memory

2006-03-16 Thread Joshua D. Drake
Jim Nasby wrote: PostgreSQL tuned to the max and still too slow? Database too big to fit into memory? Here's the solution! http://www.superssd.com/products/tera-ramsan/ Anyone purchasing one will be expected to post benchmarks! :) And give us one :) -- Jim C. Nasby, Sr. Engineering Consulta

[PERFORM] 1 TB of memory

2006-03-16 Thread Jim Nasby
PostgreSQL tuned to the max and still too slow? Database too big to fit into memory? Here's the solution! http://www.superssd.com/ products/tera-ramsan/ Anyone purchasing one will be expected to post benchmarks! :) -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive So

Re: [PERFORM] Background writer configuration

2006-03-16 Thread Evgeny Gridasov
Kevin, please, could you post other settings from your postgresql.conf? interested in: bgwriter_delay shared_buffers checkpoint_segments checkpoint_timeout wal_buffers On Wed, 15 Mar 2006 13:43:45 -0600 "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > We were seeing clusters of query timeouts wi

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
On 3/16/06, Tom Lane <[EMAIL PROTECTED]> wrote: > What we want to find out is if there's a lot of select()s and/or > semop()s shown in the result. Ideally there wouldn't be any, but > I fear that's not what you'll find. OK, I'll try to do it on monday before our upgrade then see what happens with

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Sven Geisler
Hi Guillaume, Guillaume Smet schrieb: How much faster is the XEON DP? Well, on high load, PostgreSQL scales well on the DP (load at 40, queries slower but still performing well) and is awfully slow on the MP box. I know what you mean with awfully slow. I think, your application is facing con

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Tom Lane
"Guillaume Smet" <[EMAIL PROTECTED]> writes: > You mean strace -p pid with pid on some of the postgres process not on > the postmaster itself, does you? Right, pick a couple that are accumulating CPU time. > Do we need other options? strace will generate a *whole lot* of output to stderr. I usu

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
On 3/16/06, Tom Lane <[EMAIL PROTECTED]> wrote: > Can you try strace'ing some of the backend processes while the system is > behaving like this? I suspect what you'll find is a whole lot of > delaying select() calls due to high contention for spinlocks ... Tom, I think we can try to do it. You

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
On 3/16/06, Sven Geisler <[EMAIL PROTECTED]> wrote: > Did you compare 7.4 on a 4-way with 8.1 on a 2-way? I know there are too many parameters changing between the two servers but I can't really change anything before tuesday. On tuesday, we will be able to compare both servers with the same softw

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Trying to get the information in the wrong place would be very > expensive, I agree. But preparing that information when we have access > to it and passing it through the plan would be much cheaper. Where would that be? > The operator and the opclass are

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Sven Geisler
Hi Guillaume, Guillaume Smet schrieb: The server is a dell 6650 from end of 2004 with 4 xeon mp 2.2 and 2MB cache per proc. Here are the information from Dell: 4x PROCESSOR, 80532, 2.2GHZ, 2MB cache, 400Mhz, SOCKET F 8x DUAL IN-LINE MEMORY MODULE, 512MB, 266MHz You should provide det

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Tom Lane
"Guillaume Smet" <[EMAIL PROTECTED]> writes: > Here is a top output I had on november 17 when the server completely > hangs (several minutes for each page of the website) and it is typical > of this server behaviour: > 17:08:41 up 19 days, 15:16, 1 user, load average: 4.03, 4.26, 4.36 > 288 proc

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
On 3/16/06, Sven Geisler <[EMAIL PROTECTED]> wrote: > Hi Guillaume, > > I had a similar issue last summer. Could you please provide details > about your XEON MP server and some statistics (context-switches/load/CPU > usage)? I forgot the statistics: CPU load usually from 1 to 4. CPU usage < 40% fo

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
Sven, On 3/16/06, Sven Geisler <[EMAIL PROTECTED]> wrote: > What version of XEON MP does your server have? The server is a dell 6650 from end of 2004 with 4 xeon mp 2.2 and 2MB cache per proc. Here are the information from Dell: 4x PROCESSOR, 80532, 2.2GHZ, 2MB cache, 400Mhz, SOCKET F 8x DUAL IN

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
On 3/16/06, Richard Huxton wrote: > Very strange. Sure. I can't find any logical explanation for that but it is the behaviour we have for more than a year now (the site was migrated from Oracle to PostgreSQL on january 2005). We check iostat, vmstat and so on without any hint on why we have this

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Richard Huxton
Guillaume Smet wrote: Richard, You should be seeing context-switching jump dramatically if it's the "classic" multi-Xeon problem. There's a point at which it seems to just escalate without a corresponding jump in activity. No we don't have this problem of very high context switching in our ca

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Sven Geisler
Hi Guillaume, I had a similar issue last summer. Could you please provide details about your XEON MP server and some statistics (context-switches/load/CPU usage)? I tried different servers (x86) with different results. I saw a difference between XEON MP w/ and w/o EMT64. The memory bandwidth

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
Richard, > You should be seeing context-switching jump dramatically if it's the > "classic" multi-Xeon problem. There's a point at which it seems to just > escalate without a corresponding jump in activity. No we don't have this problem of very high context switching in our case even when the dat

Re: [PERFORM] BETWEEN optimizer problems with single-value

2006-03-16 Thread Simon Riggs
On Wed, 2006-03-15 at 21:05 -0500, Tom Lane wrote: > So we need a more accurate estimate for the boundary case. Agreed. > > Using 1.0e-10 isn't very useful... the selectivity for a range should > > never be less than the selectivity for an equality, so we should simply > > put in a test against o

Re: [PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Richard Huxton
Guillaume Smet wrote: Hello, We are experiencing performances problem with a quad Xeon MP and PostgreSQL 7.4 for a year now. I had a similar issue with a client the other week. Our context switch rate is not so high but the load of the server is blocked to 4 even on very high load and we ha

[PERFORM] PostgreSQL and Xeon MP

2006-03-16 Thread Guillaume Smet
Hello, We are experiencing performances problem with a quad Xeon MP and PostgreSQL 7.4 for a year now. Our context switch rate is not so high but the load of the server is blocked to 4 even on very high load and we have 60% cpu idle even in this case. Our database fits in RAM and we don't have any