Re: [GENERAL] [PERFORM] Inaccurate Explain Cost

2012-09-26 Thread Jeff Janes
On Wed, Sep 26, 2012 at 1:21 PM, hubert depesz lubaczewski wrote: > On Wed, Sep 26, 2012 at 02:38:09PM -0400, Robert Sosinski wrote: >> The first query shows a cost of 190,169.55 and runs in 199,806.951 ms. >> When I disable nested loop, I get a cost of 2,535,992.34 which runs in >> only 133,447.7

Re: [PERFORM] Inaccurate Explain Cost

2012-09-26 Thread Samuel Gendler
On Wed, Sep 26, 2012 at 1:21 PM, hubert depesz lubaczewski < dep...@depesz.com> wrote: > On Wed, Sep 26, 2012 at 02:38:09PM -0400, Robert Sosinski wrote: > > The first query shows a cost of 190,169.55 and runs in 199,806.951 ms. > > When I disable nested loop, I get a cost of 2,535,992.34 which ru

Re: [PERFORM] [GENERAL] Memory issues

2012-09-26 Thread Shiran Kleiderman
Hi Another thing that may help, When I restart the postgres db, then I have a little bit of "grace" time (the memory free field is also lifted a bit). I can run the crons and then after an hour or two the status returns to regular... "out of memory" errors. Thanks! On Wed, Sep 26, 2012 at 11:36

Re: [PERFORM] [GENERAL] Memory issues

2012-09-26 Thread Shiran Kleiderman
Hi I contact amazon with this issue. What can I check for the fragmented memory issue? Thanks (: On Wed, Sep 26, 2012 at 11:11 PM, Scott Marlowe wrote: > OK then I'm lost. It's got to either be a bug in how amazon ec2 > instances work or severely fragmented memory because you've got a TON > of

Re: [PERFORM] [GENERAL] Memory issues

2012-09-26 Thread Scott Marlowe
OK then I'm lost. It's got to either be a bug in how amazon ec2 instances work or severely fragmented memory because you've got a TON of kernel cache available. On Wed, Sep 26, 2012 at 3:00 PM, Shiran Kleiderman wrote: > Hi > Yes, same machine. > > Thanks for your help. > > > On Wed, Sep 26, 201

Re: [PERFORM] [GENERAL] Memory issues

2012-09-26 Thread Shiran Kleiderman
Hi Yes, same machine. Thanks for your help. On Wed, Sep 26, 2012 at 10:55 PM, Scott Marlowe wrote: > On Wed, Sep 26, 2012 at 10:41 AM, Shiran Kleiderman > wrote: > > Hi > > Thanks again. > > Right now, this is free -m and ps aux and non of the crons can run - > can't > > allocate memory. > > OK

Re: [PERFORM] [GENERAL] Memory issues

2012-09-26 Thread Scott Marlowe
On Wed, Sep 26, 2012 at 10:41 AM, Shiran Kleiderman wrote: > Hi > Thanks again. > Right now, this is free -m and ps aux and non of the crons can run - can't > allocate memory. OK, so is the machine you're running free -m on the same as the one running postgresql and the same one you're running cr

Re: [PERFORM] Inaccurate Explain Cost

2012-09-26 Thread hubert depesz lubaczewski
On Wed, Sep 26, 2012 at 02:38:09PM -0400, Robert Sosinski wrote: > The first query shows a cost of 190,169.55 and runs in 199,806.951 ms. > When I disable nested loop, I get a cost of 2,535,992.34 which runs in > only 133,447.790 ms. We have run queries on our database with a cost > of 200K cost b

Re: [PERFORM] Guide to Posting Slow Query Questions

2012-09-26 Thread Jeff Janes
On Wed, Sep 12, 2012 at 11:40 PM, Ants Aasma wrote: > On Wed, Sep 12, 2012 at 7:00 PM, Jeff Janes wrote: >> Regarding the wiki page on reporting slow queries: >> We currently recommend EXPLAIN ANALYZE over just EXPLAIN. Should we >> recommend EXPLAIN (ANALYZE, BUFFERS) instead? I know I very of

Re: [PERFORM] Inaccurate Explain Cost

2012-09-26 Thread Shaun Thomas
On 09/26/2012 01:38 PM, Robert Sosinski wrote: I seem to be getting an inaccurate cost from explain. Here are two examples for one query with two different query plans: Well, there's this: Nested Loop (cost=0.00..151986.53 rows=2817 width=4) (actual time=163.275..186869.844 rows=43904 loop

[PERFORM] Inaccurate Explain Cost

2012-09-26 Thread Robert Sosinski
Hey Everyone, I seem to be getting an inaccurate cost from explain. Here are two examples for one query with two different query plans: exchange_prod=# set enable_nestloop = on; SET exchange_prod=# exchange_prod=# explain analyze SELECT COUNT(DISTINCT "exchange_uploads"."id") FROM "exchange_u

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Nikolay Ulyanitsky
On 26 September 2012 19:09, FFW_Rude wrote: > Could you explain what you are asking me to do because i don't really know > what i'm doing... postgresql-contrib packages contains pgbench tool on Ubuntu. For example postgresql-contrib-9.1_9.1.3-2_i386.deb on Ubuntu 12.04 contains: /usr/lib/postgre

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
so installing postgresql-contrib stopped my server and i don't have pgbench in it. It is still pgbench command not found... Could you explain what you are asking me to do because i don't really know what i'm doing... Rude - Last Territory Ou écouter ?http://www.deezer.com/fr/music/last-territor

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Undertaker Rude
Ok i'm installing. Can't stop the server right now. I'm gonna have to get back to you tomorrow afternoon (have other tasks that need to run from now until tomorrow by 1pm) Rude - Last Territory Ou écouter ?http://www.deezer.com/fr/music/last-territory/the-last-hope-3617781 (Post-apocalyp

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Nikolay Ulyanitsky
On 26 September 2012 18:38, FFW_Rude wrote: > root@testBI:/etc/postgresql/9.1/main# hdparm -tT /dev/sda > Timing cached reads: 892 MB in 2.01 seconds = 444.42 MB/sec > Timing buffered disk reads: 190 MB in 3.02 seconds = 62.90 MB/sec It's OK for single HDD. > Is fsync off by default ? I

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
Hi, root@testBI:/etc/postgresql/9.1/main# hdparm -tT /dev/sda /dev/sda: Timing cached reads: 892 MB in 2.01 seconds = 444.42 MB/sec Timing buffered disk reads: 190 MB in 3.02 seconds = 62.90 MB/sec Is fsync off by default ? I have#fsync = on (so it's off right ?). pgbench is not found on my

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Nikolay Ulyanitsky
Hi, FFW_Rude 1. Benchmark the device with your PostgreSQL DB: # hdparm -tT /dev/sda /dev/sda: Timing cached reads: 6604 MB in 2.00 seconds = 3303.03 MB/sec Timing buffered disk reads: 1316 MB in 3.00 seconds = 438.18 MB/sec 2. Benchmark your PostgreSQL with pgbench: Set "fsync = off" on

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Julien Cigar
On 09/26/2012 16:41, FFW_Rude wrote: Ok done to 512Mb and 2048Mb I'm relaunching. See you in a few hours (so tommorrow) with 250 000 rows and proper indexes it should run in less than a second. be sure your indexes are set properly and that they're used (use EXPLAIN ANALYZE for that) within y

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
My bad. Did not see that part. I tried to elevate buffer and SHMAX was a problem. I'll give it another try and will keep you posted. Thank you. Rude - Last Territory Ou écouter ?http://www.deezer.com/fr/music/last-territory/the-last-hope-3617781 (Post-apocalyptic Metal)http://www.deezer.c

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Julien Cigar
On 09/26/2012 16:14, FFW_Rude wrote: Thank for you answer. shared_buffer is at 24Mb effective_cache_size at 2048Mb What do you mean properly ? That's not really helping a novice... from my previous mail: before looking further, please configure shared_buffers and effective_cache_size prope

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
Thank for you answer. shared_buffer is at 24Mb effective_cache_size at 2048Mb What do you mean properly ? That's not really helping a novice... -- View this message in context: http://postgresql.1045698.n5.nabble.com/Same-query-doing-slow-then-quick-tp5725486p5725505.html Sent from the Postg

Re: [PERFORM] Postgres becoming slow, only full vacuum fixes it

2012-09-26 Thread Kiriakos Tsourapas
Hi Kevin, On Sep 26, 2012, at 14:39, Kevin Grittner wrote: > > I am concerned that your initial email said that you had this > setting: > > autovacuum_naptime = 28800 > > This is much too high for most purposes; small, frequently-modified > tables won't be kept in good shape with this setting.

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Julien Cigar
On 09/26/2012 15:36, FFW_Rude wrote: Hi, Thank you for your answer. It was already at 16MB and i upped it just this morning to 64MB. Still no change that's normal, please configure shared_buffers and effective_cache_size properly Rude - Last Territory *Ou écouter ?* http://www.deezer.

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
Hi, Thank you for your answer. It was already at 16MB and i upped it just this morning to 64MB. Still no change Rude - Last Territory Ou écouter ?http://www.deezer.com/fr/music/last-territory/the-last-hope-3617781 (Post-apocalyptic Metal)http://www.deezer.com/fr/music/rude-undertaker (

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread Julien Cigar
On 09/26/2012 15:03, FFW_Rude wrote: Here is the answer to Ray Stell who send me the wiki page of Slow Query. I hope i detailed all you wanted (i basicly pasted the page and add my answers). Full Table and Index Schema: schema tables_adresses "Tables" tables_adresses.adresses_XX (id (serial), X

Re: [PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
Here is the answer to Ray Stell who send me the wiki page of Slow Query. I hope i detailed all you wanted (i basicly pasted the page and add my answers). Full Table and Index Schema: schema tables_adresses "Tables" tables_adresses.adresses_XX (id (serial), X(Double precision),Y (Double precision

Re: [PERFORM] Postgres becoming slow, only full vacuum fixes it

2012-09-26 Thread Kevin Grittner
[resending because I accidentally failed to include the list] Kiriakos Tsourapas wrote: > I am taking your suggestions one step at a time. > > I changed my configuration to a much more aggressive autovacuum > policy (0.5% for analyzing and 1% for autovacuum). > > autovacuum_naptime = 1min > aut

[PERFORM] Same query doing slow then quick

2012-09-26 Thread FFW_Rude
Hi, I'm new here so i hope i don't do mistakes. I'm having a serious performance issue in postgresql. I have tables containing adresses with X,Y GPS coordinates and tables with zoning and square of gps coordinates. Basicly it looks like adresses_01 (id,X,Y) gps_01 (id,x_min,x_max,y_min,y_max).

Re: [PERFORM] Postgres becoming slow, only full vacuum fixes it

2012-09-26 Thread Kiriakos Tsourapas
Dear all, I am taking your suggestions one step at a time. I changed my configuration to a much more aggressive autovacuum policy (0.5% for analyzing and 1% for autovacuum). autovacuum_naptime = 1min autovacuum_vacuum_threshold = 50 #autovacuum_analyze_threshold = 50 autovacuum_vacuum_scale_fac