[PERFORM] mid 2007 "best bang for the buck" hardware opinions

2007-08-08 Thread justin
I saw an interesting topic in the archives on best bang for the buck for $20k.. about a year old now. So whats the thoughts on a current combined rack/disks/cpu combo around the $10k-$15k point, currently? I can configure up a Dell poweredge 2900 for $9k, but am wondering if I'm missing out

Re: [PERFORM] mid 2007 "best bang for the buck" hardware opinions

2007-08-08 Thread justin
I can convince myself it is just a better option. The sense that there might be better options out there, I've no doubt in.. it is why I posted hoping for some solid leads on what & why. Scott Marlowe wrote: On 8/8/07, justin <[EMAIL PROTECTED]> wrote: I saw an interesting topi

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-13 Thread justin
Absolutely on the battery backup. I did not load the linux drivers from dell, it works so i figured i was not going to worry about it. This server is so oversized for its load its unreal. I have always gone way overboard on server specs and making sure its redundant. The difference in our

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-13 Thread justin
- Original Message - From: "Greg Smith" <[EMAIL PROTECTED]> To: Sent: Thursday, March 13, 2008 4:27 PM Subject: Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10 On Thu, 13 Mar 2008, Joshua D. Drake wrote: Greg Smith <[EMAIL PROTECTED]> wrote: wal_sync_method = open_sync The

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-14 Thread Justin
Is this on a 64 bit or 32 bit machine? We had the problem with a 32 bit linux box (not sure what flavor) just a few months ago. I would not create a filesystem on a partition of 2+TB Yes this machine is 64bit You do know that effective_cache_size is the size of the OS level cache. i.e. i

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-14 Thread Justin
Greg Smith wrote: On Fri, 14 Mar 2008, Justin wrote: I played with shared_buffer and never saw much of an improvement from 100 all the way up to 800 megs moved the checkpoints from 3 to 30 and still never saw no movement in the numbers. Increasing shared_buffers normally improves

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-15 Thread Justin
I decided to reformat the raid 10 into ext2 to see if there was any real big difference in performance as some people have noted here is the test results please note the WAL files are still on the raid 0 set which is still in ext3 file system format. these test where run with the fsync as

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
einstall the OS. Nor do i want to install some unstable library into the kernel. Dave Cramer wrote: On 16-Mar-08, at 3:04 PM, Craig James wrote: Dave Cramer wrote: On 16-Mar-08, at 2:19 AM, Justin wrote: I decided to reformat the raid 10 into ext2 to see if there was any real bi

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
Well every thing worked right up to the point where i tried to mount the file system Warning: xfs_db: /dev/sdb1 contains a mounted file system fatal error -- couldn't initialize XFS library. think i'm missing something??? Craig Ringer wrote: Justin wrote: OK i'm showing

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
2000 tps ??? do you have fsync turned off ? Dave No its turned on. -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
Craig James wrote: Justin wrote: 2000 tps ??? do you have fsync turned off ? Dave No its turned on. Unless I'm seriously confused, something is wrong with these numbers. That's the sort of performance you expect from a good-sized RAID 10 six-disk array. With a single 720

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-17 Thread Justin
Just out of curiosity: Last time I did research, the word seemed to be that xfs was better than ext2 or ext3. Is that not true? Why use ext2/3 at all if xfs is faster for Postgres? Criag Ext2 vs XFS on my setup there is difference in the performance between the two file systems but its

[Fwd: Re: [PERFORM] Max shared_buffers]

2008-04-03 Thread Justin
Scott Marlowe wrote: On Thu, Apr 3, 2008 at 4:10 AM, sathiya psql <[EMAIL PROTECTED]> wrote: There is NO MAX It is according to your hardware you have, and the db you have. Not entirely true. on 32 bit OS / software, the limit is just under 2 Gig. I'd imagine that the limit on

Re: [PERFORM] Please ignore ...

2008-04-30 Thread Justin
D'Arcy J.M. Cain wrote: On Thu, 01 May 2008 01:16:00 -0300 "Marc G. Fournier" <[EMAIL PROTECTED]> wrote: Someone on this list has one of those 'confirm your email' filters on their Argh! Why do people think that it is OK to make their spam problem everyone else's problem? Whenever I

Re: [PERFORM] Pros and Cons of 8.3.1

2008-05-02 Thread Justin
Gauri Kanekar wrote: Hi, Can anyone who have started using 8.3.1 list out the pros and cons. Thanx in advance ~ Gauri don't know for sure if it is windows to linux but we moved to 8.2 that was install on windows and moved to 8.3.1 on Ubuntu using the compiled version from Ubuntu We h

[PERFORM] need to speed up query

2008-05-05 Thread Justin
i've had to write queries to get trail balance values out of the GL transaction table and i'm not happy with its performance The table has 76K rows growing about 1000 rows per working day so the performance is not that great it takes about 20 to 30 seconds to get all the records for the table

Re: [PERFORM] need to speed up query

2008-05-05 Thread Justin
yes the cross join is intentional. Thanks creating the two column index drop processing time to 15 to 17 seconds put per period down to 1 second Scott Marlowe wrote: You're joining these two tables: period, accnt, but I'm not seeing an on () clause or a where clause joining them. Is the cr

Re: [PERFORM] need to speed up query

2008-05-05 Thread Justin
Gregory Williamson wrote: Justin -- You wrote: > > i've had to write queries to get trail balance values out of the GL > transaction table and i'm not happy with its performance > > > The table has 76K rows growing about 1000 rows per working day so the > p

Re: [PERFORM] need to speed up query

2008-05-06 Thread Justin
PFC wrote: i've had to write queries to get trail balance values out of the GL transaction table and i'm not happy with its performance The table has 76K rows growing about 1000 rows per working day so the performance is not that great it takes about 20 to 30 seconds to get all the records

[PERFORM] What constitutes a complex query

2008-05-06 Thread Justin
This falls under the stupid question and i'm just curious what other people think what makes a query complex? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] need to speed up query

2008-05-06 Thread Justin
it worked it had couple missing parts but it worked and ran in 3.3 seconds. *Thanks for this * i need to review the result and balance it to my results as the Accountant already went through and balanced some accounts by hand to verify my results <> You might want to consider a denormalized

Re: [PERFORM] What constitutes a complex query

2008-05-06 Thread Justin
Craig James wrote: Justin wrote: This falls under the stupid question and i'm just curious what other people think what makes a query complex? There are two kinds: 1. Hard for Postgres to get the answer. this one 2. Hard for a person to comprehend. Which do you mean? Craig --

Re: [PERFORM] need to speed up query

2008-05-06 Thread Justin
PFC wrote: What is a "period" ? Is it a month, or something more "custom" ? Can periods overlap ? No periods can never overlap. If the periods did you would be in violation of many tax laws around the world. Plus it you would not know how much money you are making or losing. I

Re: [PERFORM] [GENERAL] Ubuntu question

2008-05-08 Thread Justin
Q Master wrote: Hello, I had postgresql 7.4 on ubuntu and over one year ago I moved to 8.2 Till now I was backing up my db via pgadmin remotely from windows but now I want to do it from the ubuntu server. When I run the command pgdump it said that the database is 8.2 but the tool is 7.4 -

Re: [PERFORM] [GENERAL] Ubuntu question

2008-05-08 Thread Justin
sorry all i accident cross posted fat fingered it Justin wrote: Q Master wrote: Hello, I had postgresql 7.4 on ubuntu and over one year ago I moved to 8.2 Till now I was backing up my db via pgadmin remotely from windows but now I want to do it from the ubuntu server. When I run the

[PERFORM] getting estimated cost to agree with actual

2008-06-02 Thread Justin
As i've been looking over the more complicated queries that i have written and gotten allot of help in redoing the quires from you all, thanks again. I have noticed that estimated Cost to do the query is way off from Actual. The queries don't run slow at least not to me. The Estimated Cost

Re: [PERFORM] Need help with 8.4 Performance Testing

2008-12-09 Thread justin
Tom Lane wrote: Scott Carey <[EMAIL PROTECTED]> writes: Which brings this back around to the point I care the most about: I/O per second will diminish as the most common database performance limiting factor in Postgres 8.4's lifetime, and become almost irrelevant in 8.5's. Becoming more CPU

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-04 Thread justin
Scott Carey wrote: Sorry for the top post -- Assuming Linux -- 1: PERC 6 is still a bit inferior to other options, but not that bad. Its random IOPS is fine, sequential speeds are noticeably less than say the latest from Adaptec or Areca. In the archives there was big thread about this ve

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread justin
Bruce Momjian wrote: Matt Burke wrote: we'd have no choice other than replacing the server+shelf+disks. I want to see just how much better a high-end Areca/Adaptec controller is, but I just don't think I can get approval for a ?1000 card "because some guy on the internet said the

Re: [PERFORM] explanation of some configs

2009-02-07 Thread justin
Thomas Finneid wrote: Joshua D. Drake wrote: On Sat, 2009-02-07 at 01:52 +0100, Thomas Finneid wrote: >> effective_cache_size >> This is just a hint to tell the planner how much cache will generally be available. ok, but available for what? for storing the data/tables/rows in memory so it

Re: [PERFORM] explanation of some configs

2009-02-09 Thread justin
Matthew Wakeling wrote: On Sat, 7 Feb 2009, justin wrote: In a big databases a checkpoint could get very large before time had elapsed and if server cashed all that work would be rolled back. No. Once you commit a transaction, it is safe (unless you play with fsync or asynchronous commit

Re: [PERFORM] explanation of some configs

2009-02-10 Thread justin
if you are short on disk space or your environment has a significant risk of unexpected power-outs, as any un-checkpointed transactions will dropped on restart. Matthew Wakeling wrote: On Mon, 9 Feb 2009, justin wrote: Well then we have conflicting instructions in places on

Re: [PERFORM] explanation of some configs

2009-02-10 Thread justin
Greg Smith wrote: On Tue, 10 Feb 2009, justin wrote: http://www.varlena.com/GeneralBits/Tidbits/annotated_conf_e.html checkpoint_segments Maximum distance between automatic WAL checkpoints, in log file segments (each segment is normally 16 megabytes). Increase these settings if your

Re: [PERFORM] explanation of some configs

2009-02-10 Thread justin
Greg Smith wrote: On Tue, 10 Feb 2009, justin wrote: Not to be overly nick picking where is the version called out that it applies to. Stating Older version is vague It's at the bottom of the document. I just updated the "Performance Optimization" page to reflect that

Re: [PERFORM] plpgsql arrays

2009-04-07 Thread justin
Matthew Wakeling wrote: What would be the syntax for putting a single row from a cursor into a variable? I have tried: FETCH INTO left left_cursor; which says syntax error, and left = FETCH left_cursor; which gives the error 'ERROR: missing datatype declaration at or near "="' Matthew

Re: [PERFORM] plpgsql arrays

2009-04-07 Thread justin
Matthew Wakeling wrote: On Tue, 7 Apr 2009, justin wrote: What would be the syntax for putting a single row from a cursor into a variable? I have tried: FETCH INTO left left_cursor; which says syntax error, and left = FETCH left_cursor

Re: [PERFORM] processor running queue - general rule of thumb?

2009-06-19 Thread justin
Alan McKay wrote: Hey folks, We did 4 tests, upping the load each time. The 3rd and 4th ones have all 8 cores pegged at about 95%. Yikes! In the first test the processor running queue spikes at 7 and maybe averages 4 or 5 In the last test it spikes at 33 with an average maybe 25. Looks to m

Re: [PERFORM] How would you store read/unread topic status?

2009-06-23 Thread justin
Mathieu Nebra wrote: Hi all, I'm running a quite large website which has its own forums. They are currently heavily used and I'm getting performance issues. Most of them are due to repeated UPDATE queries on a "flags" table. This "flags" table has more or less the following fields: UserID - To

Re: [PERFORM] - Slow Query

2009-07-01 Thread justin
Rui Carvalho wrote: SELECT distinct on (bien.uid) bien.uid , bien.date_creation , bien.date_modification , bien.nom , bien.numero_voie , bien.mer , bien.proximite , bien.nom_voie , bien.type_voie , bien.lieudit , bien.arrondissement , bien.montagne , bien.complement_adresse , bien.xy_geo ,

Re: [PERFORM] Bundling postgreSQL with my Java application

2009-07-06 Thread justin
Saurabh Dave wrote: >No offense intended - but have you looked at the documentation for postgresql.conf? >If you are going to include PostgreSQL in your application, I'd highly recommend you >understand what you are including. :-) I had a look into the documentation of postgres.conf

[PERFORM] 'Fastest' PC's are slowest in the house

2005-05-31 Thread Justin Davis
ws 2000 - 11-13 secs     Thanks,   Justin Davis Rapid Systems, Inc. 800.356.8952   -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.322 / Virus Database: 267.3.0 - Release Date: 5/30/2005

[PERFORM] could not send data to client:

2005-06-16 Thread Justin Davis
et machine actively refused it.   2005-06-16 16:17:30 LOG:  unexpected EOF on client connection   Thanks,   Justin Davis Rapid Systems, Inc. 800.356.8952   -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.323 / Virus Database: 267.7.3/15 - Release Date: 6/14/2005

Re: [PERFORM] Testing FusionIO

2010-03-17 Thread Justin Pitts
On Mar 17, 2010, at 10:41 AM, Brad Nicholson wrote: > On Wed, 2010-03-17 at 09:52 -0400, Justin Pitts wrote: >> FusionIO is publicly claiming 24 years @ 5TB/day on the 80GB SLC device, >> which wear levels across 100GB of actual installed capacity. >> http://community.fus

Re: [PERFORM] Testing FusionIO

2010-03-17 Thread Justin Pitts
On Mar 17, 2010, at 9:03 AM, Brad Nicholson wrote: > I've been hearing bad things from some folks about the quality of the > FusionIO drives from a durability standpoint. Can you be more specific about that? Durability over what time frame? How many devices in the sample set? How did FusionIO de

Re: [PERFORM] Testing FusionIO

2010-03-17 Thread Justin Pitts
warranty they have on the devices. FusionIO's claim _seems_ credible. I'd love to see some evidence to the contrary. On Mar 17, 2010, at 9:18 AM, Brad Nicholson wrote: > On Wed, 2010-03-17 at 09:11 -0400, Justin Pitts wrote: >> On Mar 17, 2010, at 9:03 AM, Brad Nicholson wrote

Re: [PERFORM] Building multiple indexes concurrently

2010-03-22 Thread Justin Pitts
It seems to me that a separate partition / tablespace would be a much simpler approach. On Mar 17, 2010, at 5:18 PM, Hannu Krosing wrote: > On Wed, 2010-03-17 at 16:49 -0400, Greg Smith wrote: >> Alvaro Herrera wrote: >>> Andres Freund escribió: >>> >>> I find it way much easier to believe

Re: [PERFORM] Building multiple indexes concurrently

2010-03-22 Thread Justin Pitts
Yes. On Mar 18, 2010, at 5:20 PM, Hannu Krosing wrote: > On Thu, 2010-03-18 at 16:12 -0400, Justin Pitts wrote: >> It seems to me that a separate partition / tablespace would be a much >> simpler approach. > > Do you mean a separate partition/ tablespace for _each_ index

Re: [PERFORM] too complex query plan for not exists query and multicolumn indexes

2010-03-22 Thread Justin Graf
Message from Corin at 03-19-2010 01:26:35 PM -- ***snip The intention of the query is to find rows with no "partner" row. The offset and limit are just to ignore the time needed to send the result to the client. --- I don't understand the point of OFFSET, limit will accomplish

Re: [PERFORM] raid10 write performance

2010-06-22 Thread Justin Graf
On 6/22/2010 4:31 AM, Grzegorz Jaśkiewicz wrote: > Hi folks, > > is there a general problem with raid10 performance postgresql on it? > We see very low performance on writes (2-3x slower than on less > performant servers). I wonder if it is solely problem of raid10 > configuration, or if it is post

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-07 Thread Justin Pitts
> Yes, I know that.  I am very familiar with how RAID6 works.  RAID5 > with the hot spare already rebuilt / built in is a good enough answer > for management where big words like parity might scare some PHBs. > >> In terms of storage cost, it IS like paying for RAID5 + a hot spare, >> but the prote

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-12 Thread Justin Pitts
>>> As others said, RAID6 is RAID5 + a hot spare. >> >> No. RAID6 is NOT RAID5 plus a hot spare. > > The original phrase was that RAID 6 was like RAID 5 with a hot spare > ALREADY BUILT IN. Built-in, or not - it is neither. It is more than that, actually. RAID 6 is like RAID 5 in that it uses pari

Re: [PERFORM] Are Indices automatically generated for primary keys?

2010-08-18 Thread Justin Graf
On 8/18/2010 9:15 AM, Clemens Eisserer wrote: > Hi, > > >> they are generated automatically. >> > Thanks depesz! > The reason why I asked was because pgAdmin doesn't display the > automatically created indices, which confused me. > > Thanks, Clemens > PGAdmin caches all database layout locally, the

Re: [PERFORM] temporary tables, indexes, and query plans

2010-10-29 Thread Justin Pitts
If you alter the default_statistics_target or any of the specific statistics targets ( via ALTER TABLE SET STATISTICS ) , the change will not have an effect until an analyze is performed. This is implied by http://www.postgresql.org/docs/9.0/static/planner-stats.html and http://www.postgresql.org/

Re: [PERFORM] temporary tables, indexes, and query plans

2010-10-29 Thread Justin Pitts
> Jason Pitts: > RE: changing default_statistics_target (or via ALTER TABLE SET STATS) > not taking effect until ANALYZE is performed. > > I did already know that, but it's probably good to put into this > thread. However, you'll note that this is a temporary table created at > the beginning of a t

Re: [PERFORM] Select * is very slow

2010-11-08 Thread Justin Pitts
On Mon, Nov 8, 2010 at 1:16 AM, shaiju.ck wrote: > [] I have increased the shared_buffres to 1024MB, but no > improvement. I have noticed that the query "show shared_buffers" always show > 8MB.Why is this? Does it mean that changing the shared_buffers in config > file have no impact? Can anybo

Re: [PERFORM] Compared MS SQL 2000 to Postgresql 9.0 on Windows

2010-12-19 Thread Justin Pitts
> If you strictly have an OLTP workload, with lots of simultaneous > connections issuing queries across small chunks of data, then > PostgreSQL would be a good match for SQL server. This matches my observations. In fact, PostgreSQL's MVCC seems to work heavily in my favor in OLTP workloads. > On

Re: [PERFORM] High load,

2011-01-27 Thread Justin Pitts
> Number of logical CPUs: 16 (4x Quadcore Xeon E5520  @ 2.27GHz) > RAM: 16GB > Concurrent connections (according to our monitoring tool): 7 (min), 74 > (avg), 197 (max) Your current issue may be IO wait, but a connection pool isn't far off in your future either. > max_connections = 200 > work_mem

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-03 Thread Justin Pitts
> With all > due respect, I consider myself smarter than the optimizer.  I'm 6'4", 235LBS > so telling me that you disagree and that I am more stupid than a computer > program,  would not be a smart thing to do. Please, do not misunderestimate > me. I don't see computer programs make thinly veiled

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-03 Thread Justin Pitts
Thank you. It appears I owe an apology also, for jumping to that conclusion. It was rash and unfair of me. I am sorry. On Wed, Feb 2, 2011 at 5:03 PM, Mladen Gogala wrote: > Justin Pitts wrote: >>> >>> With all >>> due respect, I consider myself smarter than the op

Re: [PERFORM] high user cpu, massive SELECTs, no io waiting problem

2011-02-17 Thread Justin Pitts
I think adding UNION ALL SELECT 'postgres version', version(); might be a good thing. On Wed, Feb 16, 2011 at 9:55 AM, Greg Smith wrote: > Kevin Grittner wrote: >> >> In fact, I wonder whether we shouldn't leave a couple items you've >> excluded, since they are sometimes germane to problems pos

Re: [PERFORM] Shouldn't we have a way to avoid "risky" plans?

2011-03-23 Thread Justin Pitts
On Wed, Mar 23, 2011 at 1:12 PM, Josh Berkus wrote: > AFAICT, what's happening in this query is that PostgreSQL's statistics > on the device_nodes and several other tables are slightly out of date > (as in 5% of the table). What about some manner of query feedback mechanism ( along the lines of w

Re: [PERFORM] Intel SSDs that may not suck

2011-03-29 Thread Justin Pitts
The potential breakthrough here with the 320 is consumer grade SSD performance and price paired with high reliability. On Mon, Mar 28, 2011 at 7:54 PM, Andy wrote: > This might be a bit too little too late though. As you mentioned there really > isn't any real performance improvement for the Int

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-13 Thread Justin Graf
I recent just got a new server also from dell 2 weeks ago went with more memory slower CPU, and smaller harddrives have not run pgbench Dell PE 2950 III 2 Quad Core 1.866 Ghz 16 gigs of ram. 8 hard drives 73Gig 10k RPM SAS 2 drives in Mirrored for OS, Binaries, and WAL 6 in a

Re: [PERFORM] Benchmark: Dell/Perc 6, 8 disk RAID 10

2008-03-13 Thread Justin Graf
gui" then installed postgres 8.3 gutsy. it took about 3 hours to get the server setup. Message from mailto:[EMAIL PROTECTED] Will Weaver <[EMAIL PROTECTED]> at 03-13-2008 08:11:06 AM -- Justin, This may be a bit out of context, but did you run into any troubles gett

Re: [PERFORM] Postgres 8.x on Windows Server in production

2009-04-13 Thread Justin Pitts
you may as well turn indexing service off. Don't enable compression on the data or transaction log volumes either. Pay attention to Automatic Updates - you likely don't want your database server to restart every 4th Wednesday morning or so. Hope this helps, Justin 2009/4/13 Ognjen B

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
With out knowing how much memory for each of those settings and how much work_mem for each connection its kinda hard to tell what is going. Also need version for PG, OS, how big the tables are, Also would be nice to see the query itself with explain and analyze PG does not cache the results f

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
Peter Alban wrote: duration: 2533.734 ms statement: SELECT news.url_text,news.title, http://comments.name comments.name, comments.createdate, comments.user_id, comments.comment FROM news, comments WHERE comments.cid=http://news.id news.id AND comments.published='1' GROUP BY news.url_tex

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
Message from mailto:gryz...@gmail.com Grzegorz Jaśkiewicz gryz...@gmail.com at 06-21-2009 09:36:01 PM -- On Sun, Jun 21, 2009 at 9:01 PM, Justin grafjus...@emproshunts.com wrote: work_mem = 51024 # min 64, size in KB Thats allot memory dedicated to work mem if you have 30

Re: [PERFORM] same query in high number of times

2009-06-21 Thread Justin Graf
Message from mailto:peter.alb...@gmail.com Peter Alban peter.alb...@gmail.com at 06-21-2009 10:59:49 PM -- On Sun, Jun 21, 2009 at 10:01 PM, Justin Graf mailto:jus...@emproshunts.com wrote: Peter Alban wrote: duration: 2533.734 ms statement

Re: [PERFORM] cluster index on a table

2009-07-16 Thread Justin Pitts
Is there any interest in adding that (continual/automatic cluster order maintenance) to a future release? On Wed, Jul 15, 2009 at 8:33 PM, Scott Carey wrote: > If you have a lot of insert/update/delete activity on a table fillfactor can > help. > > I don’t believe that postgres will try and mainta

Re: [PERFORM] cluster index on a table

2009-07-16 Thread Justin Pitts
ell from a performance perspective. IOT in Oracle is a huge win in some cases, but a bit more clunky for others than Clustered Indexes in MSSQL. Both are highly useful. On 7/16/09 10:52 AM, "Justin Pitts" wrote: ISTR that is the approach that MSSQL follows. Storing the full tuple in a

Re: [PERFORM] cluster index on a table

2009-07-16 Thread Justin Pitts
ISTR that is the approach that MSSQL follows. Storing the full tuple in an index and not even having a data only page would also be an interesting approach to this (and perhaps simpler than a separate index file and data file if trying to keep the data in the order of the index). -- S

[PERFORM] dump time increase by 1h with new kernel

2009-10-02 Thread Justin Pryzby
(for daily 20GB output compressed by pg_dump -Fc). Does anyone know what might be different which could cause such a drastic change? Thanks, Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

[PERFORM] dump time increase by 1h with new kernel

2009-10-03 Thread Justin Pryzby
(for daily 20GB output compressed by pg_dump -Fc). Does anyone know what might be different which could cause such a drastic change? Thanks, Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [PERFORM] How exactly PostgreSQL allocates memory for its needs?

2010-02-10 Thread Justin Graf
On 2/10/2010 12:10 AM, Anton Maksimenkov wrote: > Can anybody briefly explain me how each postgres process allocate > memory for it needs? > I mean, what is the biggest size of malloc() it may want? How many > such chunks? What is the average size of allocations? > > I think that at first it alloca

Re: [PERFORM] perf problem with huge table

2010-02-10 Thread Justin Graf
On 2/10/2010 5:13 PM, rama wrote: > in that way, when i need to do a query for a long ranges (ie: 1 year) i just > take the rows that are contained to contab_y > if i need to got a query for a couple of days, i can go on ymd, if i need to > get some data for the other timeframe, i can do some co

[PERFORM] Optimization

2003-07-28 Thread Justin Long
imization stuff.   Thanks in advance, Justin Long  

[PERFORM] " ... though the pgsql version performance is not very great."

2003-09-03 Thread Justin Clift
ing useful to sink their teeth into. :-) Regards and best wishes, Justin Clift Original Message Subject: Re: [osdldbt-general] DBT1 and dynamic cache Date: 02 Sep 2003 09:46:29 -0700 From: Jenny Zhang <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] CC: [EMAIL PROTECTED] References: &l

Re: [PERFORM] FUSION-IO io cards

2011-05-02 Thread Justin Pitts
On Fri, Apr 29, 2011 at 10:24 AM, Mark Steben wrote: > Just wondering if anyone has had any experience with this company and these > cards.  We're currently at postgres 8.3.11. td;dr Ask for a sample and test it out for yourself. I asked for, and received, a sample 80GB unit from Fusion to test

Re: [PERFORM] table size is bigger than expected

2011-08-04 Thread Justin Pitts
On Thu, Aug 4, 2011 at 2:56 PM, Jian Shi wrote: > Hey, > >   I’m a new user of PostgreSQL. I found one of my tables is taking > unexpectedly large space: > > select > pg_size_pretty(pg_relation_size('archive_files')); > >  pg_size_pretty > > > > 1113 MB > > > the field “fname” sto

Re: [PERFORM] issue related to logging facility of postgres

2011-09-01 Thread Justin Pitts
Syslog does that, I believe. Have a look at the man page for syslog.conf. On Wed, Jul 27, 2011 at 5:11 AM, shailesh singh wrote: > Hi, > I want to configure Logging of postgres in such a way that messages of > different severity should be logged in different log file. eg: all ERROR > message shou

Re: [PERFORM] Guide to PG's capabilities for inlining, predicate hoisting, flattening, etc?

2011-11-03 Thread Justin Pitts
On Wed, Nov 2, 2011 at 11:13 AM, Robert Haas wrote: > […] Perhaps we could let people say > something like WITH x AS FENCE (...) when they want the fencing > behavior, and otherwise assume they don't (but give it to them anyway > if there's a data-modifying operation in there). > I would love t

Re: [PERFORM] Clarification on using pg_upgrade

2016-03-04 Thread Justin Pryzby
e the data dir (under centos, /var/lib/pgsql/9.4~). Then pg_upgrade makes links in 9.5/. Renaming has the advantage that the old instances can't be accidentally started; and, makes it much easier to believe that it's safe to remove the 9.4~ afterwards. Justin -- Sent via pgsql

[PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-24 Thread Justin Pryzby
an't sort result by ctid for index tuples with same column value (_bt_steppage() or btgettuple())? Or maybe the problem could be mitigated by changing the behavior during INESRT? In the meantime, I'll be implementing a reindex job. Thanks, Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-25 Thread Justin Pryzby
On Tue, May 24, 2016 at 09:16:20PM -0700, Peter Geoghegan wrote: > On Tue, May 24, 2016 at 10:39 AM, Justin Pryzby wrote: > > Postgres seems to assume that the high degree of correlation of the table > > column seen in pg_stats is how it will get data from the index scan, which

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-05-25 Thread Justin Pryzby
s, with the offsets not consistently increasing (nor consistently > > decreasing): .. > > Which of those are the table, and which the index? Those weren't necessarily strace of the same process; I believe both of these were table data/heap, and didn't include any index access. > Something doesn't add up here. How could an index of an append-only > table possibly become that fragmented, when the highest amount of key > duplication is about 170? I'm certainly opened to alternate interpretations / conclusions :) Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-06-03 Thread Justin Pryzby
On Fri, Jun 03, 2016 at 06:26:33PM -0300, Claudio Freire wrote: > On Wed, May 25, 2016 at 11:00 AM, Justin Pryzby wrote: > >> > First, I found I was able to get 30-50min query results on full week's > >> > table by > >> > prefering a seq scan to a

Re: [PERFORM] index fragmentation on insert-only table with non-unique column

2016-08-13 Thread Justin Pryzby
Regarding this earlier thread: https://www.postgresql.org/message-id/flat/20160524173914.GA11880%40telsasoft.com#20160524173914.ga11...@telsasoft.com On Tue, May 24, 2016 at 10:39 AM, Justin Pryzby wrote: > Summary: Non-unique btree indices are returning CTIDs for rows with same > va

[PERFORM] no MCV list of tiny table with unique columns

2016-11-02 Thread Justin Pryzby
| f | 0 |11 | 2 | switch | f | 0 | 7 | 2 | (2 rows) Thanks in advance. Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] no MCV list of tiny table with unique columns

2016-11-02 Thread Justin Pryzby
On Wed, Nov 02, 2016 at 04:05:46PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I believe the join is being (badly) underestimated, leading to a crappy plan > > involving multiple nested loop joins, which takes 2.5 hours instead of a > > handful of seconds; I believe t

Re: [PERFORM] no MCV list of tiny table with unique columns

2016-11-05 Thread Justin Pryzby
no stats since it's empty. With indices+analyze: Sort (cost=189014.28..189014.28 rows=1 width=785) (actual time=25063.831..25063.886 rows=328 loops=1) ... BTW: join_collapse_limit | 8 from_collapse_limit | 8 ..and changing them doesn't seem to have any effect. By my count th

Re: [PERFORM] Performance issue in PostgreSQL server...

2017-03-05 Thread Justin Pryzby
me='domain_class_id' ; .. or if that's too verbose or you don't want to share the histogram or MCV list: SELECT correlation FROM pg_stats WHERE tablename='point' AND attname='domain_class_id' ; Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

[PERFORM] self join estimate and constraint exclusion

2017-03-26 Thread Justin Pryzby
t_time>='2017-03-19 23:00:00' AND start_time<'2017-03-20') t2 USING (start_time, site_id); -- Underestimtes due to perceived independence of clause: |ts=# explain analyze SELECT * FROM (SELECT * FROM eric_enodeb_metrics a WHERE start_time>='2017-03-19' AND start_

[PERFORM] self join estimate and constraint exclusion

2017-04-14 Thread Justin Pryzby
#x27;2017-03-20') t2 USING (start_time, site_id); -- Underestimtes due to perceived independence of clause: |ts=# explain analyze SELECT * FROM (SELECT * FROM eric_enodeb_metrics a WHERE start_time>='2017-03-19' AND start_time<'2017-03-20') t1 JOIN (SELECT * FROM eric_enodeb_metrics b WHERE start_time>='2017-03-19' AND start_time<'2017-03-20') t2 USING (start_time, site_id); | Hash Join (cost=7308.59..14676.41 rows=14 width=1436) (actual time=30.352..64.004 rows=7869 loops=1) Thank you in advance for your any response. Justin -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Performance issue in PostgreSQL server...

2017-04-27 Thread Justin Pryzby
e-id/flat/520D6610.8040907%40emulex.com#520d6610.8040...@emulex.com > https://www.postgresql.org/message-id/flat/20160524173914.GA11880%40telsasoft.com#20160524173914.ga11...@telsasoft.com > https://www.postgresql.org/message-id/flat/n6cmpug13b9rk1srebjvhphg0lm8dou1kn%404ax.com#n6

[PERFORM] join estimate of subqueries with range conditions and constraint exclusion

2017-05-24 Thread Justin Pryzby
imate is perfect.. SELECT * FROM (SELECT * FROM t)a JOIN(SELECT * FROM t)b USING (col) WHERE col>const So my original question is basically still opened ... is it possible to get both good estimates/plans AND constraint exclusion ?? Thanks Justin -- Sent via pgsql-

[PERFORM] Re: join estimate of subqueries with range conditions and constraint exclusion

2017-05-30 Thread Justin Pryzby
On Wed, May 24, 2017 at 04:17:30PM -0500, Justin Pryzby wrote: > We got bitten again by what appears to be the same issue I reported (perhaps > poorly) here: > https://www.postgresql.org/message-id/20170326193344.GS31628%40telsasoft.com > I'm diagnosing a bad estimate/plan due to

Re: [PERFORM] Re: join under-estimates with ineq conditions

2017-06-08 Thread Justin Pryzby
On Mon, Jun 05, 2017 at 05:02:32PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > diff --git a/src/backend/utils/adt/selfuncs.c > > b/src/backend/utils/adt/selfuncs.c > > + if (nd1>vardata1->rel->rows) nd1=vardata1->rel->rows; > > + if (

Re: [PERFORM] Re: join under-estimates with ineq conditions

2017-06-15 Thread Justin Pryzby
I never heard back but was hoping for some feedback/discussion about this 2nd problem/patch. just a reminder - Thanks On Thu, Jun 08, 2017 at 11:05:38AM -0500, Justin Pryzby wrote: > On Mon, Jun 05, 2017 at 05:02:32PM -0400, Tom Lane wrote: > > Justin Pryzby writes: > > >

estimate correlation of index separately from table (Re: [PERFORM] index fragmentation on insert-only table with non-unique column)

2017-07-07 Thread Justin Pryzby
lation. - unclustered/uncorrelated tables: tables whose heap have low correlation already discouraged from index scan; this includes tables whose column is UPDATEd and not just INSERTed; - table with correlated heap AND index: csquared should still be ~0.99 and not change much; - correla

  1   2   >