[PERFORM] CREATE STATISTICS and join selectivity

2017-11-15 Thread Justin Pryzby
r reports to avoid this kind of thing and support our PG95 customers, but I tentatively would've expected PG10 MV stats to "know" that USING(site_id, sect_id) is no more selective than USING(sect_id), same as it knows that's true for WHERE site... AND sect Justin -- Sent vi

Re: [PERFORM] query performance issue

2017-11-15 Thread Justin Pryzby
;t drop one of the two conditions, you can make PG treat it as a single condition for purpose of determining expected selectivity, using a ROW() comparison like: ROW(qlfy_grp_id, cog_grp_id) = ROW(dlr_grp.dlr_grp_id, dlr_grp_dlr_xref_1.dlr_grp_id) If you're running PG96+ you may also be ab

Re: [PERFORM] overestimate on empty table

2017-11-11 Thread Justin Pryzby
On Fri, Nov 10, 2017 at 04:19:41PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > (or, the opposite of the more common problem) > > As the queued_alters table is typically empty (and autoanalyzed with > > relpages=0), I see "why": > > >

[PERFORM] overestimate on empty table

2017-11-10 Thread Justin Pryzby
=4) -> Hash (cost=92010.97..92010.97 rows=2121397 width=72) (never executed) -> Seq Scan on pg_attribute colcld (cost=0.00..92010.97 rows=2121397 width=72) (never executed) But is there a better way (I don't consider adding a row of junk to be a significant improvement). Thanks in advance for any suggestion. Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Performance loss upgrading from 9.3 to 9.6

2017-11-06 Thread Justin Pryzby
On 11/6/17, 9:21 AM, "Justin Pryzby" wrote: > see if statistics improve: > SELECT (SELECT sum(x) FROM unnest(most_common_freqs) x) frac_MCV, > tablename, attname, n_distinct, array_length(most_common_vals,1) n_mcv, > FROM pg_stats WHERE attname~'customers_

Re: [PERFORM] Performance loss upgrading from 9.3 to 9.6

2017-11-06 Thread Justin Pryzby
quent IDs). I wouldn't bother re-running the query unless you find that increasing stats target causes the plan to change. Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] performance drop after upgrade (9.6 > 10)

2017-10-26 Thread Justin Pryzby
in default to be documented in the release notes but can't see that it's. 77cd477c4ba885cfa1ba67beaa82e06f2e182b85 Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] performance drop after upgrade (9.6 > 10)

2017-10-24 Thread Justin Pryzby
r) and pg_dump |pg_restore the relevant tables (just be sure to specify the alternate host/port/user/etc as needed for the restore invocation). Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/ma

Re: [PERFORM] Partitioned table - scans through every partitions

2017-08-25 Thread Justin Pryzby
On Fri, Aug 25, 2017 at 03:36:29PM +, Aniko Belim wrote: > Hi, > > We have an issue with one of our partitioned tables. It has a column with > timestamp without time zone type, and we had to partition it daily. To do > that, we created the following constraints like this example: > CHECK (to

Re: [PERFORM] Performance Issue -- "Materialize"

2017-08-21 Thread Justin Pryzby
.09 rows=1 width=8) > Would you send explain ANALYZE and not just explain ? Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to yo

Re: [PERFORM] Very poor read performance, query independent

2017-07-18 Thread Justin Pryzby
id/1415981309.90631.YahooMailNeo%40web133205.mail.ir2.yahoo.com https://www.postgresql.org/message-id/CAHyXU0yXYpCXN4%3D81ZDRQu-oGzrcq2qNAXDpyz4oiQPPAGk4ew%40mail.gmail.com https://www.pythian.com/blog/performance-tuning-hugepages-in-linux/ http://structureddata.org/2012/06/18/linux-6-transparent-huge-pages-and-had

estimate correlation of index separately from table (Re: [PERFORM] index fragmentation on insert-only table with non-unique column)

2017-07-07 Thread Justin Pryzby
lation. - unclustered/uncorrelated tables: tables whose heap have low correlation already discouraged from index scan; this includes tables whose column is UPDATEd and not just INSERTed; - table with correlated heap AND index: csquared should still be ~0.99 and not change much; - correla

Re: [PERFORM] Re: join under-estimates with ineq conditions

2017-06-15 Thread Justin Pryzby
I never heard back but was hoping for some feedback/discussion about this 2nd problem/patch. just a reminder - Thanks On Thu, Jun 08, 2017 at 11:05:38AM -0500, Justin Pryzby wrote: > On Mon, Jun 05, 2017 at 05:02:32PM -0400, Tom Lane wrote: > > Justin Pryzby writes: > > >

Re: [PERFORM] Re: join under-estimates with ineq conditions

2017-06-08 Thread Justin Pryzby
On Mon, Jun 05, 2017 at 05:02:32PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > diff --git a/src/backend/utils/adt/selfuncs.c > > b/src/backend/utils/adt/selfuncs.c > > + if (nd1>vardata1->rel->rows) nd1=vardata1->rel->rows; > > + if (

[PERFORM] Re: join estimate of subqueries with range conditions and constraint exclusion

2017-05-30 Thread Justin Pryzby
On Wed, May 24, 2017 at 04:17:30PM -0500, Justin Pryzby wrote: > We got bitten again by what appears to be the same issue I reported (perhaps > poorly) here: > https://www.postgresql.org/message-id/20170326193344.GS31628%40telsasoft.com > I'm diagnosing a bad estimate/plan due to

[PERFORM] join estimate of subqueries with range conditions and constraint exclusion

2017-05-24 Thread Justin Pryzby
imate is perfect.. SELECT * FROM (SELECT * FROM t)a JOIN(SELECT * FROM t)b USING (col) WHERE col>const So my original question is basically still opened ... is it possible to get both good estimates/plans AND constraint exclusion ?? Thanks Justin -- Sent via pgsql-

Re: [PERFORM] Performance issue in PostgreSQL server...

2017-04-27 Thread Justin Pryzby
e-id/flat/520D6610.8040907%40emulex.com#520d6610.8040...@emulex.com > https://www.postgresql.org/message-id/flat/20160524173914.GA11880%40telsasoft.com#20160524173914.ga11...@telsasoft.com > https://www.postgresql.org/message-id/flat/n6cmpug13b9rk1srebjvhphg0lm8dou1kn%404ax.com#n6

[PERFORM] self join estimate and constraint exclusion

2017-04-14 Thread Justin Pryzby
#x27;2017-03-20') t2 USING (start_time, site_id); -- Underestimtes due to perceived independence of clause: |ts=# explain analyze SELECT * FROM (SELECT * FROM eric_enodeb_metrics a WHERE start_time>='2017-03-19' AND start_time<'2017-03-20') t1 JOIN (SELECT * FROM eric_enodeb_metrics b WHERE start_time>='2017-03-19' AND start_time<'2017-03-20') t2 USING (start_time, site_id); | Hash Join (cost=7308.59..14676.41 rows=14 width=1436) (actual time=30.352..64.004 rows=7869 loops=1) Thank you in advance for your any response. Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

[PERFORM] self join estimate and constraint exclusion

2017-03-26 Thread Justin Pryzby
t_time>='2017-03-19 23:00:00' AND start_time<'2017-03-20') t2 USING (start_time, site_id); -- Underestimtes due to perceived independence of clause: |ts=# explain analyze SELECT * FROM (SELECT * FROM eric_enodeb_metrics a WHERE start_time>='2017-03-19' AND start_

Re: [PERFORM] Performance issue in PostgreSQL server...

2017-03-05 Thread Justin Pryzby
me='domain_class_id' ; .. or if that's too verbose or you don't want to share the histogram or MCV list: SELECT correlation FROM pg_stats WHERE tablename='point' AND attname='domain_class_id' ; Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] no MCV list of tiny table with unique columns

2016-11-05 Thread Justin Pryzby
no stats since it's empty. With indices+analyze: Sort (cost=189014.28..189014.28 rows=1 width=785) (actual time=25063.831..25063.886 rows=328 loops=1) ... BTW: join_collapse_limit | 8 from_collapse_limit | 8 ..and changing them doesn't seem to have any effect. By my count th

Re: [PERFORM] no MCV list of tiny table with unique columns

2016-11-02 Thread Justin Pryzby
On Wed, Nov 02, 2016 at 04:05:46PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I believe the join is being (badly) underestimated, leading to a crappy plan > > involving multiple nested loop joins, which takes 2.5 hours instead of a > > handful of seconds; I believe t

[PERFORM] no MCV list of tiny table with unique columns

2016-11-02 Thread Justin Pryzby
| f | 0 |11 | 2 | switch | f | 0 | 7 | 2 | (2 rows) Thanks in advance. Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-08-13 Thread Justin Pryzby
Regarding this earlier thread: https://www.postgresql.org/message-id/flat/20160524173914.GA11880%40telsasoft.com#20160524173914.ga11...@telsasoft.com On Tue, May 24, 2016 at 10:39 AM, Justin Pryzby wrote: > Summary: Non-unique btree indices are returning CTIDs for rows with same > va

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-06-03 Thread Justin Pryzby
On Fri, Jun 03, 2016 at 06:26:33PM -0300, Claudio Freire wrote: > On Wed, May 25, 2016 at 11:00 AM, Justin Pryzby wrote: > >> > First, I found I was able to get 30-50min query results on full week's > >> > table by > >> > prefering a seq scan to a

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-25 Thread Justin Pryzby
s, with the offsets not consistently increasing (nor consistently > > decreasing): .. > > Which of those are the table, and which the index? Those weren't necessarily strace of the same process; I believe both of these were table data/heap, and didn't include any index access. > Something doesn't add up here. How could an index of an append-only > table possibly become that fragmented, when the highest amount of key > duplication is about 170? I'm certainly opened to alternate interpretations / conclusions :) Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-25 Thread Justin Pryzby
On Tue, May 24, 2016 at 09:16:20PM -0700, Peter Geoghegan wrote: > On Tue, May 24, 2016 at 10:39 AM, Justin Pryzby wrote: > > Postgres seems to assume that the high degree of correlation of the table > > column seen in pg_stats is how it will get data from the index scan, which

[PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-24 Thread Justin Pryzby
an't sort result by ctid for index tuples with same column value (_bt_steppage() or btgettuple())? Or maybe the problem could be mitigated by changing the behavior during INESRT? In the meantime, I'll be implementing a reindex job. Thanks, Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Clarification on using pg_upgrade

2016-03-04 Thread Justin Pryzby
e the data dir (under centos, /var/lib/pgsql/9.4~). Then pg_upgrade makes links in 9.5/. Renaming has the advantage that the old instances can't be accidentally started; and, makes it much easier to believe that it's safe to remove the 9.4~ afterwards. Justin -- Sent via pgsql

Re: [PERFORM] Guide to PG's capabilities for inlining, predicate hoisting, flattening, etc?

2011-11-03 Thread Justin Pitts
On Wed, Nov 2, 2011 at 11:13 AM, Robert Haas wrote: > […] Perhaps we could let people say > something like WITH x AS FENCE (...) when they want the fencing > behavior, and otherwise assume they don't (but give it to them anyway > if there's a data-modifying operation in there). > I would love t

Re: [PERFORM] issue related to logging facility of postgres

2011-09-01 Thread Justin Pitts
Syslog does that, I believe. Have a look at the man page for syslog.conf. On Wed, Jul 27, 2011 at 5:11 AM, shailesh singh wrote: > Hi, > I want to configure Logging of postgres in such a way that messages of > different severity should be logged in different log file. eg: all ERROR > message shou

Re: [PERFORM] table size is bigger than expected

2011-08-04 Thread Justin Pitts
On Thu, Aug 4, 2011 at 2:56 PM, Jian Shi wrote: > Hey, > >   I’m a new user of PostgreSQL. I found one of my tables is taking > unexpectedly large space: > > select > pg_size_pretty(pg_relation_size('archive_files')); > >  pg_size_pretty > > > > 1113 MB > > > the field “fname” sto

Re: [PERFORM] FUSION-IO io cards

2011-05-02 Thread Justin Pitts
On Fri, Apr 29, 2011 at 10:24 AM, Mark Steben wrote: > Just wondering if anyone has had any experience with this company and these > cards.  We're currently at postgres 8.3.11. td;dr Ask for a sample and test it out for yourself. I asked for, and received, a sample 80GB unit from Fusion to test

Re: [PERFORM] Intel SSDs that may not suck

2011-03-29 Thread Justin Pitts
The potential breakthrough here with the 320 is consumer grade SSD performance and price paired with high reliability. On Mon, Mar 28, 2011 at 7:54 PM, Andy wrote: > This might be a bit too little too late though. As you mentioned there really > isn't any real performance improvement for the Int

Re: [PERFORM] Shouldn't we have a way to avoid "risky" plans?

2011-03-23 Thread Justin Pitts
On Wed, Mar 23, 2011 at 1:12 PM, Josh Berkus wrote: > AFAICT, what's happening in this query is that PostgreSQL's statistics > on the device_nodes and several other tables are slightly out of date > (as in 5% of the table). What about some manner of query feedback mechanism ( along the lines of w

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Justin Pitts
I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. On Wed, Feb 16, 2011 at 9:55 AM, Greg Smith wrote: > Kevin Grittner wrote: >> >> In fact, I wonder whether we shouldn't leave a couple items you've >> excluded, since they are sometimes germane to problems pos

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-03 Thread Justin Pitts
Thank you. It appears I owe an apology also, for jumping to that conclusion. It was rash and unfair of me. I am sorry. On Wed, Feb 2, 2011 at 5:03 PM, Mladen Gogala wrote: > Justin Pitts wrote: >>> >>> With all >>> due respect, I consider myself smarter than the op

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-03 Thread Justin Pitts
> With all > due respect, I consider myself smarter than the optimizer.  I'm 6'4", 235LBS > so telling me that you disagree and that I am more stupid than a computer > program,  would not be a smart thing to do. Please, do not misunderestimate > me. I don't see computer programs make thinly veiled

Re: [PERFORM] High load,

2011-01-27 Thread Justin Pitts
> Number of logical CPUs: 16 (4x Quadcore Xeon E5520  @ 2.27GHz) > RAM: 16GB > Concurrent connections (according to our monitoring tool): 7 (min), 74 > (avg), 197 (max) Your current issue may be IO wait, but a connection pool isn't far off in your future either. > max_connections = 200 > work_mem

Re: [PERFORM] Compared MS SQL 2000 to Postgresql 9.0 on Windows

2010-12-19 Thread Justin Pitts
> If you strictly have an OLTP workload, with lots of simultaneous > connections issuing queries across small chunks of data, then > PostgreSQL would be a good match for SQL server. This matches my observations. In fact, PostgreSQL's MVCC seems to work heavily in my favor in OLTP workloads. > On

Re: [PERFORM] Select * is very slow

2010-11-08 Thread Justin Pitts
On Mon, Nov 8, 2010 at 1:16 AM, shaiju.ck wrote: > [] I have increased the shared_buffres to 1024MB, but no > improvement. I have noticed that the query "show shared_buffers" always show > 8MB.Why is this? Does it mean that changing the shared_buffers in config > file have no impact? Can anybo

Re: [PERFORM] temporary tables, indexes, and query plans

2010-10-29 Thread Justin Pitts
> Jason Pitts: > RE: changing default_statistics_target (or via ALTER TABLE SET STATS) > not taking effect until ANALYZE is performed. > > I did already know that, but it's probably good to put into this > thread. However, you'll note that this is a temporary table created at > the beginning of a t

Re: [PERFORM] temporary tables, indexes, and query plans

2010-10-29 Thread Justin Pitts
If you alter the default_statistics_target or any of the specific statistics targets ( via ALTER TABLE SET STATISTICS ) , the change will not have an effect until an analyze is performed. This is implied by http://www.postgresql.org/docs/9.0/static/planner-stats.html and http://www.postgresql.org/

Re: [PERFORM] Are Indices automatically generated for primary keys?

2010-08-18 Thread Justin Graf
On 8/18/2010 9:15 AM, Clemens Eisserer wrote: > Hi, > > >> they are generated automatically. >> > Thanks depesz! > The reason why I asked was because pgAdmin doesn't display the > automatically created indices, which confused me. > > Thanks, Clemens > PGAdmin caches all database layout locally, the

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-12 Thread Justin Pitts
>>> As others said, RAID6 is RAID5 + a hot spare. >> >> No. RAID6 is NOT RAID5 plus a hot spare. > > The original phrase was that RAID 6 was like RAID 5 with a hot spare > ALREADY BUILT IN. Built-in, or not - it is neither. It is more than that, actually. RAID 6 is like RAID 5 in that it uses pari

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-07 Thread Justin Pitts
> Yes, I know that.  I am very familiar with how RAID6 works.  RAID5 > with the hot spare already rebuilt / built in is a good enough answer > for management where big words like parity might scare some PHBs. > >> In terms of storage cost, it IS like paying for RAID5 + a hot spare, >> but the prote

Re: [PERFORM] raid10 write performance

2010-06-22 Thread Justin Graf
On 6/22/2010 4:31 AM, Grzegorz Jaśkiewicz wrote: > Hi folks, > > is there a general problem with raid10 performance postgresql on it? > We see very low performance on writes (2-3x slower than on less > performant servers). I wonder if it is solely problem of raid10 > configuration, or if it is post

Re: [PERFORM] too complex query plan for not exists query and multicolumn indexes

2010-03-22 Thread Justin Graf
Message from Corin at 03-19-2010 01:26:35 PM -- ***snip The intention of the query is to find rows with no "partner" row. The offset and limit are just to ignore the time needed to send the result to the client. --- I don't understand the point of OFFSET, limit will accomplish

Re: [PERFORM] Building multiple indexes concurrently

2010-03-22 Thread Justin Pitts
Yes. On Mar 18, 2010, at 5:20 PM, Hannu Krosing wrote: > On Thu, 2010-03-18 at 16:12 -0400, Justin Pitts wrote: >> It seems to me that a separate partition / tablespace would be a much >> simpler approach. > > Do you mean a separate partition/ tablespace for _each_ index

Re: [PERFORM] Building multiple indexes concurrently

2010-03-22 Thread Justin Pitts
It seems to me that a separate partition / tablespace would be a much simpler approach. On Mar 17, 2010, at 5:18 PM, Hannu Krosing wrote: > On Wed, 2010-03-17 at 16:49 -0400, Greg Smith wrote: >> Alvaro Herrera wrote: >>> Andres Freund escribió: >>> >>> I find it way much easier to believe

Re: [PERFORM] Testing FusionIO

2010-03-17 Thread Justin Pitts
warranty they have on the devices. FusionIO's claim _seems_ credible. I'd love to see some evidence to the contrary. On Mar 17, 2010, at 9:18 AM, Brad Nicholson wrote: > On Wed, 2010-03-17 at 09:11 -0400, Justin Pitts wrote: >> On Mar 17, 2010, at 9:03 AM, Brad Nicholson wrote

Re: [PERFORM] Testing FusionIO

2010-03-17 Thread Justin Pitts
On Mar 17, 2010, at 9:03 AM, Brad Nicholson wrote: > I've been hearing bad things from some folks about the quality of the > FusionIO drives from a durability standpoint. Can you be more specific about that? Durability over what time frame? How many devices in the sample set? How did FusionIO de

Re: [PERFORM] Testing FusionIO

2010-03-17 Thread Justin Pitts
On Mar 17, 2010, at 10:41 AM, Brad Nicholson wrote: > On Wed, 2010-03-17 at 09:52 -0400, Justin Pitts wrote: >> FusionIO is publicly claiming 24 years @ 5TB/day on the 80GB SLC device, >> which wear levels across 100GB of actual installed capacity. >> http://community.fus

Re: [PERFORM] perf problem with huge table

2010-02-10 Thread Justin Graf
On 2/10/2010 5:13 PM, rama wrote: > in that way, when i need to do a query for a long ranges (ie: 1 year) i just > take the rows that are contained to contab_y > if i need to got a query for a couple of days, i can go on ymd, if i need to > get some data for the other timeframe, i can do some co

Re: [PERFORM] How exactly PostgreSQL allocates memory for its needs?

2010-02-10 Thread Justin Graf
On 2/10/2010 12:10 AM, Anton Maksimenkov wrote: > Can anybody briefly explain me how each postgres process allocate > memory for it needs? > I mean, what is the biggest size of malloc() it may want? How many > such chunks? What is the average size of allocations? > > I think that at first it alloca

Re: [PERFORM] dump time increase by 1h with new kernel

2009-10-08 Thread Justin T Pryzby
On Thu, Oct 08, 2009 at 03:37:39PM -0500, Kevin Grittner wrote: > Justin T Pryzby wrote: > > On Thu, Oct 08, 2009 at 10:49:37AM -0700, Joshua D. Drake wrote: > >> Did your scheduler change between the kernel versions? > > No, it's deadline for both. > >

Re: [PERFORM] dump time increase by 1h with new kernel

2009-10-08 Thread Justin T Pryzby
On Thu, Oct 08, 2009 at 10:49:37AM -0700, Joshua D. Drake wrote: > On Thu, 2009-10-08 at 10:44 -0700, Justin T Pryzby wrote: > > Hi Everyone > Did your scheduler change between the kernel versions? No, it's deadline for both. Justin -- Sent via pgsql-performance mailing list (

Re: [PERFORM] dump time increase by 1h with new kernel

2009-10-08 Thread Justin T Pryzby
Hi Everyone On Fri, Oct 02, 2009 at 12:58:12PM -0700, Justin Pryzby wrote: > When we upgraded from linux-2.6.24 to linux-2.6.27, our pg_dump > duration increased by 20% from 5 hours to 6. My first attempt at On Sat, Oct 03, 2009 at 11:31:11PM -0600, Scott Marlowe wrote: > between the

[PERFORM] dump time increase by 1h with new kernel

2009-10-03 Thread Justin Pryzby
(for daily 20GB output compressed by pg_dump -Fc). Does anyone know what might be different which could cause such a drastic change? Thanks, Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

[PERFORM] dump time increase by 1h with new kernel

2009-10-02 Thread Justin Pryzby
(for daily 20GB output compressed by pg_dump -Fc). Does anyone know what might be different which could cause such a drastic change? Thanks, Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] cluster index on a table

2009-07-16 Thread Justin Pitts
ISTR that is the approach that MSSQL follows. Storing the full tuple in an index and not even having a data only page would also be an interesting approach to this (and perhaps simpler than a separate index file and data file if trying to keep the data in the order of the index). -- S

Re: [PERFORM] cluster index on a table

2009-07-16 Thread Justin Pitts
ell from a performance perspective. IOT in Oracle is a huge win in some cases, but a bit more clunky for others than Clustered Indexes in MSSQL. Both are highly useful. On 7/16/09 10:52 AM, "Justin Pitts" wrote: ISTR that is the approach that MSSQL follows. Storing the full tuple in a

Re: [PERFORM] cluster index on a table

2009-07-16 Thread Justin Pitts
Is there any interest in adding that (continual/automatic cluster order maintenance) to a future release? On Wed, Jul 15, 2009 at 8:33 PM, Scott Carey wrote: > If you have a lot of insert/update/delete activity on a table fillfactor can > help. > > I don’t believe that postgres will try and mainta

Re: [PERFORM] Bundling postgreSQL with my Java application

2009-07-06 Thread justin
Saurabh Dave wrote: >No offense intended - but have you looked at the documentation for postgresql.conf? >If you are going to include PostgreSQL in your application, I'd highly recommend you >understand what you are including. :-) I had a look into the documentation of postgres.conf

Re: [PERFORM] - Slow Query

2009-07-01 Thread justin
Rui Carvalho wrote: SELECT distinct on (bien.uid) bien.uid , bien.date_creation , bien.date_modification , bien.nom , bien.numero_voie , bien.mer , bien.proximite , bien.nom_voie , bien.type_voie , bien.lieudit , bien.arrondissement , bien.montagne , bien.complement_adresse , bien.xy_geo ,

Re: [PERFORM] How would you store read/unread topic status?

2009-06-23 Thread justin
Mathieu Nebra wrote: Hi all, I'm running a quite large website which has its own forums. They are currently heavily used and I'm getting performance issues. Most of them are due to repeated UPDATE queries on a "flags" table. This "flags" table has more or less the following fields: UserID - To

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
Message from mailto:peter.alb...@gmail.com Peter Alban peter.alb...@gmail.com at 06-21-2009 10:59:49 PM -- On Sun, Jun 21, 2009 at 10:01 PM, Justin Graf mailto:jus...@emproshunts.com wrote: Peter Alban wrote: duration: 2533.734 ms statement

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
Message from mailto:gryz...@gmail.com Grzegorz Jaśkiewicz gryz...@gmail.com at 06-21-2009 09:36:01 PM -- On Sun, Jun 21, 2009 at 9:01 PM, Justin grafjus...@emproshunts.com wrote: work_mem = 51024 # min 64, size in KB Thats allot memory dedicated to work mem if you have 30

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
Peter Alban wrote: duration: 2533.734 ms statement: SELECT news.url_text,news.title, http://comments.name comments.name, comments.createdate, comments.user_id, comments.comment FROM news, comments WHERE comments.cid=http://news.id news.id AND comments.published='1' GROUP BY news.url_tex

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
With out knowing how much memory for each of those settings and how much work_mem for each connection its kinda hard to tell what is going. Also need version for PG, OS, how big the tables are, Also would be nice to see the query itself with explain and analyze PG does not cache the results f

Re: [PERFORM] processor running queue - general rule of thumb?

2009-06-19 Thread justin
Alan McKay wrote: Hey folks, We did 4 tests, upping the load each time. The 3rd and 4th ones have all 8 cores pegged at about 95%. Yikes! In the first test the processor running queue spikes at 7 and maybe averages 4 or 5 In the last test it spikes at 33 with an average maybe 25. Looks to m

Re: [PERFORM] Postgres 8.x on Windows Server in production

2009-04-13 Thread Justin Pitts
you may as well turn indexing service off. Don't enable compression on the data or transaction log volumes either. Pay attention to Automatic Updates - you likely don't want your database server to restart every 4th Wednesday morning or so. Hope this helps, Justin 2009/4/13 Ognjen B

Re: [PERFORM] plpgsql arrays

2009-04-07 Thread justin
Matthew Wakeling wrote: On Tue, 7 Apr 2009, justin wrote: What would be the syntax for putting a single row from a cursor into a variable? I have tried: FETCH INTO left left_cursor; which says syntax error, and left = FETCH left_cursor

Re: [PERFORM] plpgsql arrays

2009-04-07 Thread justin
Matthew Wakeling wrote: What would be the syntax for putting a single row from a cursor into a variable? I have tried: FETCH INTO left left_cursor; which says syntax error, and left = FETCH left_cursor; which gives the error 'ERROR: missing datatype declaration at or near "="' Matthew

Re: [PERFORM] explanation of some configs

2009-02-10 Thread justin
Greg Smith wrote: On Tue, 10 Feb 2009, justin wrote: Not to be overly nick picking where is the version called out that it applies to. Stating Older version is vague It's at the bottom of the document. I just updated the "Performance Optimization" page to reflect that

Re: [PERFORM] explanation of some configs

2009-02-10 Thread justin
Greg Smith wrote: On Tue, 10 Feb 2009, justin wrote: http://www.varlena.com/GeneralBits/Tidbits/annotated_conf_e.html checkpoint_segments Maximum distance between automatic WAL checkpoints, in log file segments (each segment is normally 16 megabytes). Increase these settings if your

Re: [PERFORM] explanation of some configs

2009-02-10 Thread justin
if you are short on disk space or your environment has a significant risk of unexpected power-outs, as any un-checkpointed transactions will dropped on restart. Matthew Wakeling wrote: On Mon, 9 Feb 2009, justin wrote: Well then we have conflicting instructions in places on

Re: [PERFORM] explanation of some configs

2009-02-09 Thread justin
Matthew Wakeling wrote: On Sat, 7 Feb 2009, justin wrote: In a big databases a checkpoint could get very large before time had elapsed and if server cashed all that work would be rolled back. No. Once you commit a transaction, it is safe (unless you play with fsync or asynchronous commit

Re: [PERFORM] explanation of some configs

2009-02-07 Thread justin
Thomas Finneid wrote: Joshua D. Drake wrote: On Sat, 2009-02-07 at 01:52 +0100, Thomas Finneid wrote: >> effective_cache_size >> This is just a hint to tell the planner how much cache will generally be available. ok, but available for what? for storing the data/tables/rows in memory so it

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread justin
Bruce Momjian wrote: Matt Burke wrote: we'd have no choice other than replacing the server+shelf+disks. I want to see just how much better a high-end Areca/Adaptec controller is, but I just don't think I can get approval for a ?1000 card "because some guy on the internet said the

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-04 Thread justin
Scott Carey wrote: Sorry for the top post -- Assuming Linux -- 1: PERC 6 is still a bit inferior to other options, but not that bad. Its random IOPS is fine, sequential speeds are noticeably less than say the latest from Adaptec or Areca. In the archives there was big thread about this ve

Re: [PERFORM] Need help with 8.4 Performance Testing

2008-12-09 Thread justin
Tom Lane wrote: Scott Carey <[EMAIL PROTECTED]> writes: Which brings this back around to the point I care the most about: I/O per second will diminish as the most common database performance limiting factor in Postgres 8.4's lifetime, and become almost irrelevant in 8.5's. Becoming more CPU

[PERFORM] getting estimated cost to agree with actual

2008-06-02 Thread Justin
As i've been looking over the more complicated queries that i have written and gotten allot of help in redoing the quires from you all, thanks again. I have noticed that estimated Cost to do the query is way off from Actual. The queries don't run slow at least not to me. The Estimated Cost

Re: [PERFORM] [GENERAL] Ubuntu question

2008-05-08 Thread Justin
sorry all i accident cross posted fat fingered it Justin wrote: Q Master wrote: Hello, I had postgresql 7.4 on ubuntu and over one year ago I moved to 8.2 Till now I was backing up my db via pgadmin remotely from windows but now I want to do it from the ubuntu server. When I run the

Re: [PERFORM] [GENERAL] Ubuntu question

2008-05-08 Thread Justin
Q Master wrote: Hello, I had postgresql 7.4 on ubuntu and over one year ago I moved to 8.2 Till now I was backing up my db via pgadmin remotely from windows but now I want to do it from the ubuntu server. When I run the command pgdump it said that the database is 8.2 but the tool is 7.4 -

Re: [PERFORM] need to speed up query

2008-05-06 Thread Justin
PFC wrote: What is a "period" ? Is it a month, or something more "custom" ? Can periods overlap ? No periods can never overlap. If the periods did you would be in violation of many tax laws around the world. Plus it you would not know how much money you are making or losing. I

Re: [PERFORM] What constitutes a complex query

2008-05-06 Thread Justin
Craig James wrote: Justin wrote: This falls under the stupid question and i'm just curious what other people think what makes a query complex? There are two kinds: 1. Hard for Postgres to get the answer. this one 2. Hard for a person to comprehend. Which do you mean? Craig --

Re: [PERFORM] need to speed up query

2008-05-06 Thread Justin
it worked it had couple missing parts but it worked and ran in 3.3 seconds. *Thanks for this * i need to review the result and balance it to my results as the Accountant already went through and balanced some accounts by hand to verify my results <> You might want to consider a denormalized

[PERFORM] What constitutes a complex query

2008-05-06 Thread Justin
This falls under the stupid question and i'm just curious what other people think what makes a query complex? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] need to speed up query

2008-05-06 Thread Justin
PFC wrote: i've had to write queries to get trail balance values out of the GL transaction table and i'm not happy with its performance The table has 76K rows growing about 1000 rows per working day so the performance is not that great it takes about 20 to 30 seconds to get all the records

Re: [PERFORM] need to speed up query

2008-05-05 Thread Justin
Gregory Williamson wrote: Justin -- You wrote: > > i've had to write queries to get trail balance values out of the GL > transaction table and i'm not happy with its performance > > > The table has 76K rows growing about 1000 rows per working day so the > p

Re: [PERFORM] need to speed up query

2008-05-05 Thread Justin
yes the cross join is intentional. Thanks creating the two column index drop processing time to 15 to 17 seconds put per period down to 1 second Scott Marlowe wrote: You're joining these two tables: period, accnt, but I'm not seeing an on () clause or a where clause joining them. Is the cr

[PERFORM] need to speed up query

2008-05-05 Thread Justin
i've had to write queries to get trail balance values out of the GL transaction table and i'm not happy with its performance The table has 76K rows growing about 1000 rows per working day so the performance is not that great it takes about 20 to 30 seconds to get all the records for the table

Re: [PERFORM] Pros and Cons of 8.3.1

2008-05-02 Thread Justin
Gauri Kanekar wrote: Hi, Can anyone who have started using 8.3.1 list out the pros and cons. Thanx in advance ~ Gauri don't know for sure if it is windows to linux but we moved to 8.2 that was install on windows and moved to 8.3.1 on Ubuntu using the compiled version from Ubuntu We h

Re: [PERFORM] Please ignore ...

2008-04-30 Thread Justin
D'Arcy J.M. Cain wrote: On Thu, 01 May 2008 01:16:00 -0300 "Marc G. Fournier" <[EMAIL PROTECTED]> wrote: Someone on this list has one of those 'confirm your email' filters on their Argh! Why do people think that it is OK to make their spam problem everyone else's problem? Whenever I

[Fwd: Re: [PERFORM] Max shared_buffers]

2008-04-03 Thread Justin
Scott Marlowe wrote: On Thu, Apr 3, 2008 at 4:10 AM, sathiya psql <[EMAIL PROTECTED]> wrote: There is NO MAX It is according to your hardware you have, and the db you have. Not entirely true. on 32 bit OS / software, the limit is just under 2 Gig. I'd imagine that the limit on

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
Craig James wrote: Justin wrote: 2000 tps ??? do you have fsync turned off ? Dave No its turned on. Unless I'm seriously confused, something is wrong with these numbers. That's the sort of performance you expect from a good-sized RAID 10 six-disk array. With a single 720

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
2000 tps ??? do you have fsync turned off ? Dave No its turned on. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
Just out of curiosity: Last time I did research, the word seemed to be that xfs was better than ext2 or ext3. Is that not true? Why use ext2/3 at all if xfs is faster for Postgres? Criag Ext2 vs XFS on my setup there is difference in the performance between the two file systems but its

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
Well every thing worked right up to the point where i tried to mount the file system Warning: xfs_db: /dev/sdb1 contains a mounted file system fatal error -- couldn't initialize XFS library. think i'm missing something??? Craig Ringer wrote: Justin wrote: OK i'm showing

  1   2   >