Re: [PERFORM] Massive update, memory usage

2010-10-28 Thread Emanuele Bracci Poste
Scusa, scadenze a parte, ma non vi è sembrato il caso di chiedere a chi sta gestendo il progetto prima di rimuovere una risorsa? Grazie comunque. Emanuele Il giorno 28/ott/2010, alle ore 23.48, Trenta sis ha scritto: > Well, I have solved executing with more RAM, and then works correctly > > Th

Re: [PERFORM] Massive update, memory usage

2010-10-28 Thread Trenta sis
Well, I have solved executing with more RAM, and then works correctly Thanks 2010/10/28 Cédric Villemain > 2010/10/28 Trenta sis : > > > > > > There are about 100.000 inserts and 30 updates. Without transaction > it > > seems that works, but with a transaction no. Witt about only 300.000

Re: [PERFORM] BBU Cache vs. spindles

2010-10-28 Thread Tom Lane
James Mansion writes: > Tom Lane wrote: >> The other and probably worse problem is that there's no application >> control over how soon changes to mmap'd pages get to disk. An msync >> will flush them out, but the kernel is free to write dirty pages sooner. >> So if they're depending for consiste

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Ben
whoops, didn't see the i=9 (linebreak! linebreak!) nonetheless that is a static constant constraint on the column i, and i was asking if constraint exclusions would work for dynamic constraints (like those derived from a table joined against.) so for example the bar table has only 0-9 in its h

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Joshua D. Drake
On Thu, 2010-10-28 at 12:59 -0700, Ben wrote: > On Oct 28, 2010, at 12:44 PM, Joshua D. Drake wrote: > > > > My tests show you are incorrect: > > > > > > part_test=# explain analyze select * from foo join bar using (i) where > > i=9; > >QUERY >

Re: [PERFORM] BBU Cache vs. spindles

2010-10-28 Thread James Mansion
Tom Lane wrote: The other and probably worse problem is that there's no application control over how soon changes to mmap'd pages get to disk. An msync will flush them out, but the kernel is free to write dirty pages sooner. So if they're depending for consistency on writes not happening until m

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Reid Thompson
On Thu, 2010-10-28 at 12:59 -0700, Ben wrote: > explain analyze select * from foo join bar using (i); vs explain analyze select * from foo join bar using (i) where i=9;

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Ben
On Oct 28, 2010, at 12:44 PM, Joshua D. Drake wrote: > > My tests show you are incorrect: > > > part_test=# explain analyze select * from foo join bar using (i) where > i=9; >QUERY > PLAN > -

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Joshua D. Drake
On Thu, 2010-10-28 at 12:25 -0700, Ben wrote: > i think we are talking about two different things here: the constraints on > the table, and the where-clause constraints in a query which may or may not > trigger constraint exclusion. i understand that table constraints have to be > constants --

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Ben
On Oct 28, 2010, at 11:50 AM, Joshua D. Drake wrote: >>> Yes the constraints have to be static. Not sure about the operator >>> question honestly. >> >> this seems to severely restrict their usefulness -- our queries are data >> warehouse analytical -type queries, so the constraints are usually

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Joshua D. Drake
On Thu, 2010-10-28 at 11:44 -0700, Ben wrote: > > Yes the constraints have to be static. Not sure about the operator > > question honestly. > > this seems to severely restrict their usefulness -- our queries are data > warehouse analytical -type queries, so the constraints are usually > data-d

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Ben
thanks for the prompt response. some comments / questions below : On Oct 28, 2010, at 10:31 AM, Joshua D. Drake wrote: >> ...constraint exclusion is able to eliminate table partitions. the I/O >> advantages of having queries target small subtables are the same as the I/O >> advantages of clust

Re: [PERFORM] how to get the total number of records in report

2010-10-28 Thread Merlin Moncure
On Thu, Oct 28, 2010 at 1:49 PM, AI Rumman wrote: > But I am using Postgresql 8.1 and it is not possible to write query as your > one here. with 8.1, you are limited to subquery approach, application derived count, plpgsql hacks, etc. merlin -- Sent via pgsql-performance mailing list (pgsql-pe

Re: [PERFORM] how to get the total number of records in report

2010-10-28 Thread AI Rumman
But I am using Postgresql 8.1 and it is not possible to write query as your one here. On Thu, Oct 28, 2010 at 11:05 PM, Robert Haas wrote: > On Tue, Oct 19, 2010 at 7:56 PM, Merlin Moncure > wrote: > > On Mon, Oct 18, 2010 at 1:16 AM, AI Rumman wrote: > >> At present for reporting I use follow

Re: [PERFORM] how to get the total number of records in report

2010-10-28 Thread Merlin Moncure
On Thu, Oct 28, 2010 at 1:05 PM, Robert Haas wrote: > On Tue, Oct 19, 2010 at 7:56 PM, Merlin Moncure wrote: >> On Mon, Oct 18, 2010 at 1:16 AM, AI Rumman wrote: >>> At present for reporting I use following types of query: >>> select crm.*, crm_cnt.cnt >>> from crm, >>> (select count(*) as cnt f

Re: [PERFORM] partitioning question 1

2010-10-28 Thread Joshua D. Drake
On Thu, 2010-10-28 at 09:36 -0700, Ben wrote: > hello -- > > my last email was apparently too long to respond to so i'll split it up into > shorter pieces. my first question : > > my understanding of how range partitioning and constraint exclusion works > leads me to believe that it does not b

Re: [PERFORM] how to get the total number of records in report

2010-10-28 Thread Robert Haas
On Tue, Oct 19, 2010 at 7:56 PM, Merlin Moncure wrote: > On Mon, Oct 18, 2010 at 1:16 AM, AI Rumman wrote: >> At present for reporting I use following types of query: >> select crm.*, crm_cnt.cnt >> from crm, >> (select count(*) as cnt from crm) crm_cnt; >> Here count query is used to find the to

[PERFORM] partitioning question 1

2010-10-28 Thread Ben
hello -- my last email was apparently too long to respond to so i'll split it up into shorter pieces. my first question : my understanding of how range partitioning and constraint exclusion works leads me to believe that it does not buy any query performance that a clustered index doesn't alr

Re: [PERFORM] Query much faster with enable_seqscan=0

2010-10-28 Thread Robert Haas
On Tue, Oct 12, 2010 at 10:28 PM, Samuel Gendler wrote: > > > On Tue, Sep 21, 2010 at 4:30 PM, Tom Lane wrote: >> >> Ogden writes: >> > SELECT tr.id, tr.sid >> >             FROM >> >             test_registration tr, >> >             INNER JOIN test_registration_result r on (tr.id = >> > r.test

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Robert Haas
On Thu, Oct 28, 2010 at 11:23 AM, Mladen Gogala wrote: > On 10/28/2010 10:53 AM, Richard Broersma wrote: >> >> On Thu, Oct 28, 2010 at 7:51 AM, Mladen Gogala >>  wrote: >> >>> Yyesss! Any time frame on that? Can you make it into 9.0.2? >> >> Maybe 9.1.0 or 9.2.0 :)  9.0's features are already fro

Re: [PERFORM] Massive update, memory usage

2010-10-28 Thread Cédric Villemain
2010/10/28 Trenta sis : > > > There are about 100.000 inserts and 30 updates. Without transaction it > seems that works, but with a transaction no. Witt about only 300.000 updates > it seems that can finish correctly, but last 20% is slow because is using > swap... > > Any tunning to do in this

Re: [PERFORM] How does PG know if data is in memory?

2010-10-28 Thread Cédric Villemain
2010/10/28 Francisco Reyes : > Greg Smith writes: > >> heard privately from two people who have done similar experiments on Linux >> and found closer to 8GB to be the point where performance started > > So on a machine with 72GB is 8GB still the recommended value? Yes, as a maximum, not a minimum.

Re: [PERFORM] Postgres insert performance and storage requirement compared to Oracle

2010-10-28 Thread Jesper Krogh
On 2010-10-28 15:13, Merlin Moncure wrote: On Wed, Oct 27, 2010 at 3:47 PM, Jesper Krogh wrote: On 2010-10-27 20:51, Merlin Moncure wrote: Yes, I am quite aware of how the o/s page cache works. All else being equal, I more compact database obviously would be preferred. However 'al

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Mladen Gogala
On 10/28/2010 10:53 AM, Richard Broersma wrote: On Thu, Oct 28, 2010 at 7:51 AM, Mladen Gogala wrote: Yyesss! Any time frame on that? Can you make it into 9.0.2? Maybe 9.1.0 or 9.2.0 :) 9.0's features are already frozen. Well, with all this global warming around us, index scans may still

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Richard Broersma
On Thu, Oct 28, 2010 at 7:51 AM, Mladen Gogala wrote: > Yyesss! Any time frame on that? Can you make it into 9.0.2? Maybe 9.1.0 or 9.2.0 :) 9.0's features are already frozen. -- Regards, Richard Broersma Jr. Visit the Los Angeles PostgreSQL Users Group (LAPUG) http://pugs.postgresql.org/lap

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Mladen Gogala
On 10/28/2010 10:42 AM, Robert Haas wrote: I can believe that MySQL is faster, because they probably don't need to do the bitmap heap scan. There is a much-anticipated feature called index-only scans that we don't have yet in PG, which would help cases like this a great deal. Yyesss! Any time f

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Robert Haas
On Tue, Oct 19, 2010 at 6:05 PM, Ozer, Pam wrote: > On mysql the same query only takes milliseconds not seconds.  That's a > big difference. I can believe that MySQL is faster, because they probably don't need to do the bitmap heap scan. There is a much-anticipated feature called index-only scan

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Robert Haas
On Thu, Oct 28, 2010 at 10:39 AM, Robert Haas wrote: >> Can someone tell me why after it runs the index scan it hen runs a bitmap >> heap scan?  It should not take this long to run should it?  If I limit the >> results it comes back in 300ms. > > It doesn't.  The EXPLAIN output shows it running th

Re: [PERFORM] Slow Query- Simple taking

2010-10-28 Thread Robert Haas
On Tue, Oct 19, 2010 at 2:21 PM, Ozer, Pam wrote: > I have the following query running on 8.4, which takes 3516 ms.  It is very > straight forward.  It brings back 116412 records.  The explain only takes > 1348ms > > select VehicleUsed.VehicleUsedId as VehicleUsedId , > > VehicleUsed.VehicleUsedDi

Re: [PERFORM] temporary tables, indexes, and query plans

2010-10-28 Thread Tom Lane
Jon Nelson writes: > On Wed, Oct 27, 2010 at 5:36 PM, Tom Lane wrote: >> It thinks it's faster, or there is some reason why it *can't* use the >> index, like a datatype mismatch.  You could tell which by trying "set >> enable_seqscan = off" to see if that will make it change to another >> plan;

Re: [PERFORM] temporary tables, indexes, and query plans

2010-10-28 Thread Jon Nelson
On Wed, Oct 27, 2010 at 5:36 PM, Tom Lane wrote: > Jon Nelson writes: >> I'd like to zoom out a little bit and, instead of focusing on the >> specifics, ask more general questions: .. >> - is there some way for me to determine /why/ the planner chooses a >> sequential scan over other options? > >

Re: [PERFORM] Select count(*), the sequel

2010-10-28 Thread Kenneth Marshall
On Wed, Oct 27, 2010 at 05:49:42PM -0400, Tom Lane wrote: > Kenneth Marshall writes: > > Just keeping the hope alive for faster compression. > > Is there any evidence that that's something we should worry about? > I can't recall ever having seen a code profile that shows the > pg_lzcompress.c cod

Re: [PERFORM] Postgres insert performance and storage requirement compared to Oracle

2010-10-28 Thread Merlin Moncure
On Wed, Oct 27, 2010 at 3:47 PM, Jesper Krogh wrote: > On 2010-10-27 20:51, Merlin Moncure wrote: > >>> Yes, I am quite aware of how the o/s page cache works.  All else being >>> equal, I more compact database obviously would be preferred.  However >>> 'all else' is not necessarily equal.  I can m

Re: [PERFORM] AIX slow buffer reads

2010-10-28 Thread André Volpato
- Mensagem original - | André Volpato wrote: | > | | > | If it is being spent in the bitmap index scan, try setting | > | effective_io_concurrency to 0 for Linux, and see what effect that | > | has. | > | > I disabled effective_io_concurrency at AIX but it made no changes on | > bitmap ind

Re: [PERFORM] Select count(*), the sequel

2010-10-28 Thread Pierre C
"Pierre C" wrote: in-page compression How would that be different from the in-page compression done by TOAST now? Or are you just talking about being able to make it more aggressive? -Kevin Well, I suppose lzo-style compression would be better used on data that is written a few times maxi

[PERFORM] Massive update, memory usage

2010-10-28 Thread Trenta sis
There are about 100.000 inserts and 30 updates. Without transaction it seems that works, but with a transaction no. Witt about only 300.000 updates it seems that can finish correctly, but last 20% is slow because is using swap... Any tunning to do in this configuration or it is correct? thank