Re: [PERFORM] HT on or off for E5-26xx ?

2012-11-07 Thread David Boreham
Well, the results are in and at least in this particular case conventional wisdom is overturned. Not a huge benefit, but throughput is definitely higher with HT enabled and nthreads >> ncores: HT off : bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48 starting vacuum...end. transaction t

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Tom Lane
Rodrigo Rosenfeld Rosas writes: > Ok, I could finally strip part of my database schema that will allow you > to run the explain query and reproduce the issue. > There is a simple SQL dump in plain format that you can restore both on > 9.1 and 9.2 and an example EXPLAIN query so that you can see

Re: [PERFORM] How to upgrade from 9.1 to 9.2 with replication?

2012-11-07 Thread Claudio Freire
On Wed, Nov 7, 2012 at 5:59 PM, Bruce Momjian wrote: >> > Is there any reason to upgrade the slave when you are going to do rsync >> > anyway? Of course you need to install the new binaries and libs, but it >> > seems running pg_upgrade on the standby is unnecessary. >> >> Just to speed up the rs

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Rodrigo Rosenfeld Rosas
Em 07-11-2012 14:58, Tom Lane escreveu: Merlin Moncure writes: On Wed, Nov 7, 2012 at 5:16 AM, Rodrigo Rosenfeld Rosas wrote: This was a great guess! Congrats, Merlin: Heh -- that was tom's guess, not mine. What this does is confirm the planner regression and that elevates the importance o

Re: [PERFORM] HT on or off for E5-26xx ?

2012-11-07 Thread Mark Kirkwood
On 08/11/12 02:33, David Boreham wrote: On 11/6/2012 9:16 PM, Mark Kirkwood wrote: I've been benchmarking a E5-4640 (4 socket) and hyperthreading off gave much better scaling behaviour in pgbench (gentle rise and flatten off), whereas with hyperthreading on there was a dramatic falloff afte

Re: [PERFORM] How to upgrade from 9.1 to 9.2 with replication?

2012-11-07 Thread Bruce Momjian
On Wed, Nov 7, 2012 at 03:44:13PM -0300, Claudio Freire wrote: > On Wed, Nov 7, 2012 at 3:36 PM, Bruce Momjian wrote: > >> Bring both down. > >> pg_upgrade master > >> Bring master up > >> pg_upgrade slave > > > > Is there any reason to upgrade the slave when you are going to do rsync > > anyway?

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Rodrigo Rosenfeld Rosas
Em 07-11-2012 14:34, Merlin Moncure escreveu: On Wed, Nov 7, 2012 at 5:16 AM, Rodrigo Rosenfeld Rosas wrote: Em 06-11-2012 19:11, Merlin Moncure escreveu: On Tue, Nov 6, 2012 at 1:45 PM, Tom Lane wrote: Rodrigo Rosenfeld Rosas writes: Em 06-11-2012 17:24, Tom Lane escreveu: Can you pu

Re: [PERFORM] How to upgrade from 9.1 to 9.2 with replication?

2012-11-07 Thread Claudio Freire
On Wed, Nov 7, 2012 at 3:36 PM, Bruce Momjian wrote: >> Bring both down. >> pg_upgrade master >> Bring master up >> pg_upgrade slave > > Is there any reason to upgrade the slave when you are going to do rsync > anyway? Of course you need to install the new binaries and libs, but it > seems runnin

Re: [PERFORM] How to upgrade from 9.1 to 9.2 with replication?

2012-11-07 Thread Bruce Momjian
On Fri, Oct 19, 2012 at 12:02:49PM -0300, Claudio Freire wrote: > > This unfortunately does require a new data copy to be pulled across to the > > slave. For the local copies this isn't so bad as wire speed is fast enough > > to make it reasonable; for the actual backup units at a remove it takes

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Tom Lane
Merlin Moncure writes: > On Wed, Nov 7, 2012 at 5:16 AM, Rodrigo Rosenfeld Rosas > wrote: >> This was a great guess! Congrats, Merlin: > Heh -- that was tom's guess, not mine. What this does is confirm the > planner regression and that elevates the importance of Tom's request > to get sample da

Re: [PERFORM] Unique values across a table of arrays - documents and tags

2012-11-07 Thread Florent Guillaume
On Wed, Nov 7, 2012 at 5:00 PM, Ivan Voras wrote: > On 07/11/2012 16:54, Florent Guillaume wrote: >> Maybe you could store the tags as fulltext words, query them using >> fulltext search, and use ts_stat to gather the list of words? Needs to >> be benched of course. >> You'll probably need to chan

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Merlin Moncure
On Wed, Nov 7, 2012 at 5:16 AM, Rodrigo Rosenfeld Rosas wrote: > Em 06-11-2012 19:11, Merlin Moncure escreveu: > >> On Tue, Nov 6, 2012 at 1:45 PM, Tom Lane wrote: >>> >>> Rodrigo Rosenfeld Rosas writes: Em 06-11-2012 17:24, Tom Lane escreveu: > > Can you put together a self-co

Re: [PERFORM] Unique values across a table of arrays - documents and tags

2012-11-07 Thread Ivan Voras
On 07/11/2012 16:54, Florent Guillaume wrote: > Maybe you could store the tags as fulltext words, query them using > fulltext search, and use ts_stat to gather the list of words? Needs to > be benched of course. > You'll probably need to change the config to avoid stemming and stop words. I have t

Re: [PERFORM] Unique values across a table of arrays - documents and tags

2012-11-07 Thread Florent Guillaume
Maybe you could store the tags as fulltext words, query them using fulltext search, and use ts_stat to gather the list of words? Needs to be benched of course. You'll probably need to change the config to avoid stemming and stop words. Florent On Wed, Nov 7, 2012 at 4:38 PM, Ivan Voras wrote: >

Re: [PERFORM] Unique values across a table of arrays - documents and tags

2012-11-07 Thread Ivan Voras
On 07/11/2012 16:34, François Beausoleil wrote: > Le 2012-11-07 à 10:21, Ivan Voras a écrit : >> Barring that, what would you suggest for efficiently handing a classic >> structure like this (meaning documents with tags)? > > Can you structure it as the "classic" many to many pattern: > > docume

Re: [PERFORM] Unique values across a table of arrays - documents and tags

2012-11-07 Thread François Beausoleil
Le 2012-11-07 à 10:21, Ivan Voras a écrit : > > This is unfortunately slow (because I know the load will increase and > this will be a common operation). > > The thing I was planning to do is create a separate table, with only the > unique tags, and possibly an array of documents which have thes

[PERFORM] Unique values across a table of arrays - documents and tags

2012-11-07 Thread Ivan Voras
Hello, I know I need to re-engineer this so it doesn't suck by design, so I'm wondering if there is some nifty PostgreSQL feature or best practice which may automagically do the best thing. I store information about documents which are tagged by string tags. The structure is very simple: CREATE

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-11-07 Thread Tom Lane
Denis writes: > Tom Lane-2 wrote >> Hmmm ... so the problem here isn't that you've got 2600 schemas, it's >> that you've got 183924 tables. That's going to take some time no matter >> what. > I wonder why pg_dump has to have deal with all these 183924 tables, if I > specified to dump only one sc

Re: [PERFORM] HT on or off for E5-26xx ?

2012-11-07 Thread David Boreham
On 11/7/2012 6:37 AM, Devrim GÜNDÜZ wrote: HT should be good for file servers, or say many of the app servers, or small web/mail servers. PostgreSQL relies on the CPU power, and since the HT CPUs don't have the same power as the original CPU, when OS submits a job to that particular HTed CPU, qu

Re: [PERFORM] HT on or off for E5-26xx ?

2012-11-07 Thread Devrim GÜNDÜZ
Hi, On Tue, 2012-11-06 at 20:31 -0700, David Boreham wrote: > Was assuming it was 6-core but I just noticed it has HT which is > currently enabled since I see 12 cores in /proc/cpuinfo > > Question for the performance experts : is it better to have H enabled > or disabled for this generation o

Re: [PERFORM] HT on or off for E5-26xx ?

2012-11-07 Thread David Boreham
On 11/6/2012 9:16 PM, Mark Kirkwood wrote: I've been benchmarking a E5-4640 (4 socket) and hyperthreading off gave much better scaling behaviour in pgbench (gentle rise and flatten off), whereas with hyperthreading on there was a dramatic falloff after approx number clients = number of (hype

Re: [PERFORM] dbt2 performance regresses from 9.1.6 to 9.2.1

2012-11-07 Thread Heikki Linnakangas
On 05.11.2012 16:32, Claudio Freire wrote: On Sun, Nov 4, 2012 at 7:23 PM, Dong Ye wrote: You are using prepared statements, this makes me think that this regression might be due to support for parameter specific plans for prepared statements. [1] Can you run the test on both versions without p

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Rodrigo Rosenfeld Rosas
Em 06-11-2012 19:48, Merlin Moncure escreveu: On Tue, Nov 6, 2012 at 12:57 PM, Rodrigo Rosenfeld Rosas wrote: I would strongly consider investigation of hstore type along with gist/gin index. select * from company_transaction where contract_attributes @> 'State=>Delaware, Paid=Y'; etc I'm no

Re: [PERFORM] Query completed in < 1s in PG 9.1 and ~ 700s in PG 9.2

2012-11-07 Thread Rodrigo Rosenfeld Rosas
Em 06-11-2012 19:11, Merlin Moncure escreveu: On Tue, Nov 6, 2012 at 1:45 PM, Tom Lane wrote: Rodrigo Rosenfeld Rosas writes: Em 06-11-2012 17:24, Tom Lane escreveu: Can you put together a self-contained test case to duplicate these results? I'm prepared to believe there's some sort of plan

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-11-07 Thread Denis
Tom Lane-2 wrote > Denis < > socsam@ > > writes: >> Here is the output of EXPLAIN ANALYZE. It took 5 seconds but usually it >> takes from 10 to 15 seconds when I am doing backup. > >> Sort (cost=853562.04..854020.73 rows=183478 width=219) (actual >> time=5340.477..5405.604 rows=183924 loops=1)