Re: [PERFORM] Performance Optimization for Dummies 2 - the SQL

2006-10-08 Thread Jim C. Nasby
On Fri, Oct 06, 2006 at 02:53:35PM -0400, Merlin Moncure wrote: > On 10/6/06, Scott Marlowe <[EMAIL PROTECTED]> wrote: > >On Fri, 2006-10-06 at 11:44, Carlo Stonebanks wrote: > >> This didn't work right away, but DID work after running a VACUUM FULL. In > >> other words, i was still stuck with a se

Re: [PERFORM] Simple join optimized badly?

2006-10-09 Thread Jim C. Nasby
On Mon, Oct 09, 2006 at 06:07:29PM +, Chris Browne wrote: > [EMAIL PROTECTED] ("Craig A. James") writes: > > Mark Kirkwood wrote: > >>> The result? I can't use my function in any WHERE clause that > >>> involves any other conditions or joins. Only by itself. PG will > >>> occasionally decide

Re: [PERFORM] odd variances in count(*) times

2006-10-09 Thread Jim C. Nasby
o do with > >the gettimeofday() calls being implemented differently between Windows > >and Linux..). > > that was it. amd system now drop to .3 seconds, windows .6. (doing > time foo > psql -c bar > file). thanks... You can also turn timing on in psql. And FWIW,

Re: [PERFORM] autovacuum not working?

2006-10-09 Thread Jim C. Nasby
On Mon, Oct 09, 2006 at 09:27:30AM -0500, Medora Schauer wrote: > > From your attached config file: > > > > #autovacuum_vacuum_threshold = 1000 # min # of tuple updates before > > # vacuum > > > > Yup, that was it. Actually, not quite. Vacuum will update relp

Re: [PERFORM] Simple join optimized badly?

2006-10-09 Thread Jim C. Nasby
On Mon, Oct 09, 2006 at 11:33:03PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Mon at 04:18:27PM -0500] > > I can agree to that, but we'll never get any progress so long as every > > time hints are brought up the response is that they're evil and should > > never be

Re: [PERFORM] Simple join optimized badly?

2006-10-10 Thread Jim C. Nasby
On Mon, Oct 09, 2006 at 06:45:16PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > One of the big problems with doing set enable_...=off is that there's no > > way to embed that into something like a view, so you're almost forced &

Re: [PERFORM] Simple join optimized badly?

2006-10-10 Thread Jim C. Nasby
On Mon, Oct 09, 2006 at 03:41:09PM -0700, Joshua D. Drake wrote: > > > > One of the big problems with doing set enable_...=off is that there's no > > way to embed that into something like a view, so you're almost forced > > into putting into the application code itself, which makes matters even >

Re: [PERFORM] Simple join optimized badly?

2006-10-10 Thread Jim C. Nasby
de is bad even with documentation. Would you put something from the obfuscated C contest into production with comments describing what it does, or would you just write the code cleanly to begin with? -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB htt

Re: [PERFORM] Simple join optimized badly?

2006-10-10 Thread Jim C. Nasby
On Tue, Oct 10, 2006 at 10:14:48AM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > I'd rather have the ugly solution sooner rather than the elegant one > > later (if ever). > > The trouble with that is that we couldn't ever g

Re: [PERFORM] Postgre 8.0 Installation - Issues

2006-10-10 Thread Jim C. Nasby
Moving to -general. On Tue, Oct 10, 2006 at 04:17:06PM +0530, Ravindran G - TLS, Chennai. wrote: > All, > > We are facing few issues while we install Postgres 8.0 in Windows 2000 > Japanese OS. Installer kit name : postgresql-8.0-ja Is there a reason you're not using 8.1.4? 8.0 was the first wi

Re: [PERFORM] Simple join optimized badly?

2006-10-10 Thread Jim C. Nasby
On Tue, Oct 10, 2006 at 10:28:29AM -0700, Josh Berkus wrote: > Jim, > > > We've depricated things before, I'm sure we'll do it again. Yes, it's a > > pain, but it's better than not having anything release after release. > > And having a formal hint language would at least allow us to eventually >

Re: [PERFORM] Scrub one large table against another

2006-10-10 Thread Jim C. Nasby
On Tue, Oct 10, 2006 at 05:46:18PM -0600, Brendan Curran wrote: > > > Tom Lane wrote: > >Brendan Curran <[EMAIL PROTECTED]> writes: > >>Tom Lane wrote: > >>>Have you tried doing EXPLAIN ANALYZE of each of the INSERT/DELETE steps? > > > >>FIRST INSERT (Just the select is explained): > > > >EXPLAIN

Collect stats during seqscan (was: [PERFORM] Simple join optimized badly?)

2006-10-11 Thread Jim C. Nasby
On Wed, Oct 11, 2006 at 10:27:26AM -0400, Bucky Jordan wrote: > Also, I'm guessing this has already come up at some point, but what > about allowing PG to do some stat collection during queries? If you're > touching a lot of data (such as an import process) wouldn't it be more > efficient (and perh

Re: [PERFORM] Scrub one large table against another

2006-10-11 Thread Jim C. Nasby
On Wed, Oct 11, 2006 at 10:53:41AM -0600, Brendan Curran wrote: > Interestingly, and thank you to Tom and Jim, the explicit JOIN improved > performance tremendously (RESULTS BELOW). I converted the entire query > to use explicit joins instead of IN and EXISTS and discovered acceptable > performa

Re: FW: [PERFORM] Simple join optimized badly?

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 10:59:23PM +1300, Mark Kirkwood wrote: > H.J. Sanders wrote: > > > why not just like in some other (commercial) databases: > > > > a statement to say: use index > > > > I know this is against all though but if even the big ones can not resist > > the pressure

[PERFORM] Hints proposal

2006-10-12 Thread Jim C. Nasby
Posting here instead of hackers since this is where the thread got started... The argument has been made that producing a hints system will be as hard as actually fixing the optimizer. There's also been clamoring for an actual proposal, so here's one that (I hope) wouldn't be very difficult to imp

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 11:42:32AM -0400, Tom Lane wrote: > [ This is off-topic for -performance, please continue the thread in > -hackers ] > > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > These hints would outright force the planner to do things a certain

Re: FW: [PERFORM] Simple join optimized badly?

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 10:44:20AM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > If someone's going to commit to putting effort into improving the > > planner then that's wonderful. But I can't recall any significant > >

Re: [PERFORM] Hints proposal

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 09:26:24AM -0600, Joshua Marsh wrote: > On 10/12/06, Jim C. Nasby <[EMAIL PROTECTED]> wrote: > > > >Posting here instead of hackers since this is where the thread got > >started... > > > >The argument has been made that produci

Re: [PERFORM] Hints proposal

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 04:55:17PM +0100, Heikki Linnakangas wrote: > Bruce Momjian wrote: > >Because DB2 doesn't like hints, and the fact that they have gotten to a > >point where they feel they do not need them, I feel we too can get to a > >point where we don't need them either. The question is

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 09:40:30AM -0700, Josh Berkus wrote: > Jim, > > >>>These hints would outright force the planner to do things a certain way. > >>>... FROM table /* ACCESS {SEQSCAN | [[NO] BITMAP] INDEX index_name} */ > >>This proposal seems to deliberately ignore every point that has been >

Re: [PERFORM] Hints proposal

2006-10-12 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 09:42:55AM -0700, Jeff Davis wrote: > On Thu, 2006-10-12 at 10:14 -0500, Jim C. Nasby wrote: > > The syntax these hints is something arbitrary. I'm borrowing Oracle's > > idea of embedding hints in comments, but we can use some other method if &

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jim C. Nasby
On Thu, Oct 12, 2006 at 01:58:22PM -0700, Josh Berkus wrote: > > Unless you've got a time machine or a team of coders in your back > > pocket, I don't see how the planner will suddenly become perfect in > > 8.4... > > Since you're not a core code contributor, I really don't see why you > continue

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-13 Thread Jim C. Nasby
On Fri, Oct 13, 2006 at 03:57:23PM -0700, Josh Berkus wrote: > Jim, > > > Well, that's not what I said (my point being that until the planner and > > stats are perfect you need a way to over-ride them)... but I've also > > never said hints would be faster or easier than stats modification (I > > s

Re: [PERFORM] Performance Optimization for Dummies 2 - the SQL

2006-10-16 Thread Jim C. Nasby
I think there's 2 things that would help this case. First, partition on country. You can either do this on a table level or on an index level by putting where clauses on the indexes (index method would be the fastest one to test, since it's just new indexes). That should shrink the size of that ind

Re: [PERFORM] Performance Optimization for Dummies 2 - the SQL

2006-10-18 Thread Jim C. Nasby
On Mon, Oct 16, 2006 at 05:56:54PM -0400, Carlo Stonebanks wrote: > >I think there's 2 things that would help this case. First, partition on > > country. You can either do this on a table level or on an index level > > by putting where clauses on the indexes (index method would be the > > fastest o

Re: [ADMIN] [PERFORM] autovacuum on a -mostly- r/o table

2006-10-18 Thread Jim C. Nasby
On Sun, Oct 15, 2006 at 04:52:12PM +0200, Tobias Brox wrote: > Are there any logs that can help me, and eventually, are there any > ready-made scripts for checking when autovacuum is running, and > eventually for how long it keeps its transactions? I'll probably write > up something myself if not.

Re: [PERFORM] Optimization of this SQL sentence

2006-10-18 Thread Jim C. Nasby
On Tue, Oct 17, 2006 at 12:51:19PM -0400, Merlin Moncure wrote: > so, imo alexander is correct: > contacto varchar(255) > > ...is a false constraint, why exactly 255? is that were the dart landed? BTW, if we get variable-length varlena headers at some point, then setting certain limits might make

Re: [PERFORM] Optimization of this SQL sentence (SOLVED)

2006-10-18 Thread Jim C. Nasby
On Tue, Oct 17, 2006 at 12:25:39PM +0200, Ruben Rubio wrote: > First of all I have to say that I now the database is not ok. There was > a people before me that didn't do the thinks right. I would like to > normalize the database, but it takes too much time (there is is hundred > of SQLs to change

Re: [PERFORM] Postgresql 8.1.4 - performance issues for select on

2006-10-18 Thread Jim C. Nasby
On Wed, Oct 18, 2006 at 02:33:49PM -0700, Jeff Davis wrote: > On Wed, 2006-10-18 at 23:19 +0200, Dimitri Fontaine wrote: > > Le mercredi 18 octobre 2006 23:02, Ioana Danes a ??crit : > > > I tried the partitioning scenario but I've got into > > > the same problem. The max function is not using the

Re: [PERFORM] index growth problem

2006-10-18 Thread Jim C. Nasby
On Wed, Oct 18, 2006 at 03:20:19PM -0700, Graham Davis wrote: > I have a question about index growth. > > The way I understand it, dead tuples in indexes were not reclaimed by > VACUUM commands in the past. However, I've read in a few forum posts > that this was changed somewhere between 7.4 an

Re: [PERFORM] Postgresql 8.1.4 - performance issues for select on

2006-10-18 Thread Jim C. Nasby
On Wed, Oct 18, 2006 at 03:32:15PM -0700, Jeff Davis wrote: > On Wed, 2006-10-18 at 17:10 -0500, Jim C. Nasby wrote: > > Sorry, don't have the earlier part of this thread, but what about... > > > > SELECT greatest(max(a), max(b)) ... > > > > ? > > To

Re: [PERFORM] [GENERAL] UDF and cache

2006-10-18 Thread Jim C. Nasby
And PLEASE do not post something to 3 lists; it's a lot of extra traffic for no reason. Moving to -hackers. On Wed, Oct 18, 2006 at 05:15:13PM -0400, jungmin shin wrote: > Hello all, > > I read a paper, which is Query optimization in the presence of Foreign > Functions. > And the paper , there

Re: [PERFORM] index growth problem

2006-10-18 Thread Jim C. Nasby
On Wed, Oct 18, 2006 at 03:39:56PM -0700, Graham Davis wrote: > So I guess any changes that were made to make VACUUM and FSM include > indexes > does not remove the necessity to reindex (as long as we don't want index > sizes to bloat and grow larger than they need be). > Is that correct? Not in

Re: [PERFORM] [HACKERS] UDF and cache

2006-10-18 Thread Jim C. Nasby
On Wed, Oct 18, 2006 at 05:15:13PM -0400, jungmin shin wrote: > Hello all, > > I read a paper, which is Query optimization in the presence of Foreign > Functions. > And the paper , there is a paragraph like below. > > In order to reduce the number of invocations, caching the results of > invoca

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 03:54:28PM +0200, Tobias Brox wrote: > I just came to think about /proc/sys/swappiness ... > > When this one is set to a high number (say, 100 - which is maximum), the > kernel will aggressively swap out all memory that is not beeing > accessed, to allow more memory for cac

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:00:54PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Thu at 10:28:31AM -0500] > > I think it'd be much better to experiment with using much larger > > shared_buffers settings. The conventional wisdom there is from 7.x days > > when you really

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:39:22PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Thu at 11:31:26AM -0500] > > Yeah, test setups are a good thing to have... > > We would need to replicate the production traffic as well to do reliable > tests. Well, we'll get to that o

Re: [PERFORM] Swappiness setting on a linux pg server

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:53:49PM +0200, Tobias Brox wrote: > [Jim C. Nasby - Thu at 11:45:32AM -0500] > > > > The issue with pg_xlog is you don't need bandwidth... you need super-low > > > > latency. The best way to accomplish that is to get a battery-backed RAID

Re: [PERFORM] DB Performance decreases due to often written/accessed

2006-10-19 Thread Jim C. Nasby
On Thu, Oct 19, 2006 at 06:19:16PM +0100, Richard Huxton wrote: > OK - these plans look about the same, but the time is greatly different. > Both have rows=140247 as the estimated number of rows in tbl_reg. Either > you have many more rows in the second case (in which case you're not > running

Re: [PERFORM] Vacuum and Memory Loss

2006-10-23 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 09:45:59AM +0100, Richard Huxton wrote: > Mike wrote: > >Hello friends, > > > >I am responsible for maintaining a high volume website using postgresql > >8.1.4. Given the amount of reads and writes, I vacuum full the server a > >few times a week around 1, 2 AM shutting down

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Jim C. Nasby
On Sat, Oct 21, 2006 at 12:12:59AM +0930, Shane Ambler wrote: > Generally more disks at slower speed - 2 10K disks in raid 0 is faster > than 1 15K disk. More disks also allow more options. Not at writing they're not (unless you're using RAID0... ugh). -- Jim Nasby

Re: [PERFORM] Best COPY Performance

2006-10-23 Thread Jim C. Nasby
that I hadn't used psql in the first place is that > I'm loading the data into partitioned tables, and the loader keeps > several COPY connections open at a time to load the data into the > right table. I guess I could just as easily keep several psql pipes > open, but it see

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Jim C. Nasby
On Sat, Oct 21, 2006 at 08:43:05AM -0700, John Philips wrote: > I heard some say that the transaction log should be on > it's own array, others say it doesn't hurt to have it > on the same array as the OS. Is it really worthwhile > to put it on it's own array? It all depends on the controller and

Re: [PERFORM] Best COPY Performance

2006-10-24 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 03:37:47PM -0700, Craig A. James wrote: > Jim C. Nasby wrote: > >http://stats.distributed.net used to use a perl script to do some > >transformations before loading data into the database. IIRC, when we > >switched to using C we saw 100x improvement i

Re: [PERFORM] Best COPY Performance

2006-10-24 Thread Jim C. Nasby
On Tue, Oct 24, 2006 at 09:17:08AM -0400, Worky Workerson wrote: > >http://stats.distributed.net used to use a perl script to do some > >transformations before loading data into the database. IIRC, when we > >switched to using C we saw 100x improvement in speed, so I suspect t

Re: [PERFORM] Problems using a function in a where clause

2006-10-24 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 04:54:00PM -0300, Mara Dalponte wrote: > Hello, > > I have a query with several join operations and applying the same > filter condition over each involved table. This condition is a complex > predicate over an indexed timestamp field, depending on some > parameters. > To

Re: [PERFORM] Copy database performance issue

2006-10-24 Thread Jim C. Nasby
On Mon, Oct 23, 2006 at 05:51:40PM -0400, Steve wrote: > Hello there; > > I've got an application that has to copy an existing database to a new > database on the same machine. > > I used to do this with a pg_dump command piped to psql to perform the > copy; however the database is 18 gigs larg

Re: [PERFORM] Best COPY Performance

2006-10-25 Thread Jim C. Nasby
On Tue, Oct 24, 2006 at 10:36:04PM -0700, Craig A. James wrote: > Jim C. Nasby wrote: > >Well, given that perl is using an entire CPU, it sounds like you should > >start looking either at ways to remove some of the overhead from perl, > >or to split that perl into multiple

Re: [PERFORM] Best COPY Performance

2006-10-25 Thread Jim C. Nasby
On Wed, Oct 25, 2006 at 08:03:38AM -0400, Worky Workerson wrote: > I'm just doing CSV style transformations (and calling a lot of > functions along the way), but the end result is a straight bulk load > of data into a blank database. And we've established that Postgres > can do *way* better than w

Re: [PERFORM] Problems using a function in a where clause

2006-10-25 Thread Jim C. Nasby
On Wed, Oct 25, 2006 at 07:55:38AM -0300, Mara Dalponte wrote: > On 10/24/06, Jim C. Nasby <[EMAIL PROTECTED]> wrote: > >On Mon, Oct 23, 2006 at 04:54:00PM -0300, Mara Dalponte wrote: > >> Hello, > >> > >> I have a query with several join operations and ap

Re: [PERFORM] Best COPY Performance

2006-10-25 Thread Jim C. Nasby
On Wed, Oct 25, 2006 at 11:25:01AM -0400, Worky Workerson wrote: > >I'm guessing the high bursts are checkpoints. Can you check your log > >files for pg and see if you are getting warnings about checkpoint > >frequency? You can get some mileage here by increasing wal files. > > Nope, nothing in

Re: [PERFORM] commit so slow program looks frozen

2006-10-25 Thread Jim C. Nasby
On Wed, Oct 25, 2006 at 04:32:16PM -0400, Carlo Stonebanks wrote: > >> I have a question for you: did you have a long running query keeping open > a transaction? I've just noticed the same problem here, but things cleaned > up immediately when I aborted the long-running transaction. > > No, the o

Re: [PERFORM] Configuration Issue ?

2006-10-26 Thread Jim C. Nasby
On Wed, Oct 25, 2006 at 05:31:29PM -0400, Mark Lonsdale wrote: > Set my sort_mem to 8192 You really need to look at what your workload is before trying to tweak sort_mem. With 8G of memory, sort_mem=40 (~400MB) with only 10 active connections might be a good setting. It's usually better to get

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-10-26 Thread Jim C. Nasby
On Thu, Oct 26, 2006 at 04:06:09PM +0100, Gavin Hamill wrote: > On Thu, 26 Oct 2006 10:47:21 -0400 > Tom Lane <[EMAIL PROTECTED]> wrote: > > > Gavin Hamill <[EMAIL PROTECTED]> writes: > > > Nodes 2 and 3 take only the tables necessary to run our search (10 > > > out of the full 130) and are much l

Re: [PERFORM] Stored procedure slower than sql?

2006-10-26 Thread Jim C. Nasby
09:21:37AM -0700, Matthew Peters wrote: > Parameterized. > > IE (very simplified) > > CREATE OR REPLACE FUNCTION my_function(IN param1 BIGINT, IN param2 > INTEGER) > RETURNS my_type > SECURITY DEFINER > AS > $$ > /* my_type = (a,b,c) */ >

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-10-26 Thread Jim C. Nasby
On Thu, Oct 26, 2006 at 09:35:56PM +0100, Gavin Hamill wrote: > On Thu, 26 Oct 2006 14:17:29 -0500 > "Jim C. Nasby" <[EMAIL PROTECTED]> wrote: > > > Are you sure that there's nothing else happening on the machine that > > could affect the vacuum tim

Re: [PERFORM] query slows down drastically with increased number of fields

2006-10-26 Thread Jim C. Nasby
On Thu, Oct 26, 2006 at 03:03:38PM -0700, George Pavlov wrote: > i have wondered myself. i wouldn't do it through pgAdmin (not sure what > the best test it, but i thought psql from the same machine might be > better--see below). anyway, the funny thing is that if you concatenate > them the time dro

Re: [PERFORM] Context switch storm

2006-11-14 Thread Jim C. Nasby
On Tue, Nov 14, 2006 at 09:17:08AM -0500, Merlin Moncure wrote: > On 11/14/06, Cosimo Streppone <[EMAIL PROTECTED]> wrote: > >I must say I lowered "shared_buffers" to 8192, as it was before. > >I tried raising it to 16384, but I can't seem to find a relationship > >between shared_buffers and perfor

Re: [PERFORM] possible improvement between G4 and G5

2004-04-06 Thread Jim C. Nasby
RS6000 hardware for database stuff (and that gets expensive if you're paying per CPU!). -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows:

[PERFORM] Poor performance of group by query

2004-04-16 Thread Jim C. Nasby
iled with -O3. Also, if I set enable_hashagg = false, it runs in less than a second. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Wh

[PERFORM] Horribly slow hash join

2004-04-16 Thread Jim C. Nasby
=254056 width=48) (actual time=786.515..1289.101 rows=49091 loops=1) Filter: (project_id = 8) Total runtime: 3548.087 ms Even though the second case is only a select, it seems clear that something's wrong... -- Jim C. Nasby, Database Consultant

Re: [PERFORM] Horribly slow hash join

2004-04-16 Thread Jim C. Nasby
ested someplace you can grab it. On Fri, Apr 16, 2004 at 12:34:11PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > Note the time for the hash join step: > > Have you ANALYZEd these tables lately? > > It looks to me like it's hashing o

Re: [PERFORM] Horribly slow hash join

2004-04-20 Thread Jim C. Nasby
Dammit, I somehow deleted a bunch of replies to this. Did a TODO ever come out of this? -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows

Re: [PERFORM] Equivalent praxis to CLUSTERED INDEX?

2004-08-31 Thread Jim C. Nasby
of write > traffic, the same kind of reorganization you would normally expect in a > BTree index. This isn't true, at least in 9i. You can create whatever indexes you want on an index-organized table. I believe that the index stores the PK value instead of the ROWID. -- Jim

Re: [PERFORM] Equivalent praxis to CLUSTERED INDEX?

2004-08-31 Thread Jim C. Nasby
ll benefit some means of indicating what range of PK values are on a page might be needed. It's not as beneficial as a true IOT since you don't get the benefit of storing your tuples inline with your B-Tree. I'm sure there's a ton of things I'm missing, especially since I'm

Re: [PERFORM] Data Warehouse Reevaluation - MySQL vs Postgres -- merge tables

2004-09-14 Thread Jim C. Nasby
g a set of tools would make life much easier. I just looked but didn't see anything on GBorg. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today

Re: [PERFORM] Partitioning

2004-09-16 Thread Jim C. Nasby
r each project. Storing project_id in that table is an extra 4 bytes... doesn't sound like much until you consider that the table has over 130M rows right now. So it would be nice to have an easy way to partition the table based on unique project_id's and not waste space in the partit

Re: [PERFORM] Article about PostgreSQL and RAID in Brazil

2004-09-16 Thread Jim C. Nasby
mp on seperate drives? Specifically, we have a box with 8 drives, 2 in a mirror with the OS and WAL and pg_temp; the rest in a raid10 with the database on it. Do you think it would have been better to make one big raid10? What if it was raid5? And what if it was only 6 drives total? -- Jim C. Nasby

Re: [PERFORM] Article about PostgreSQL and RAID in Brazil

2004-09-16 Thread Jim C. Nasby
seperate drive would still be a losing proposition. BTW, my experience with our setup is that the raid10 is almost always the IO bottleneck, and not the mirror with everything else on it. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.

Re: [PERFORM] Caching of Queries

2004-09-27 Thread Jim C. Nasby
s. This means MVCC is used and no locking is required. Even if locks were required, they would be shared read locks which wouldn't block each other. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 W

Re: [PERFORM] Caching of Queries

2004-09-30 Thread Jim C. Nasby
plan caching, because your connect overhead will swamp the planning cost. This does not mean you have to use something like pgpool (which makes some rather questionable claims IMO); any decent web application language/environment will support connection pooling. -- Jim C. Nasby, Database Consultant

Re: [PERFORM] Caching of Queries

2004-10-01 Thread Jim C. Nasby
nection. Pretty much all web apps have one connection > per process, which is persistent (i.e. not dropped and remade for each > request), but not shared between processes, therefore not pooled. OK, that'd work too... the point is if you're re-connecting all the time it doesn

Re: [PERFORM] Caching of Queries

2004-10-04 Thread Jim C. Nasby
me specific queries as being cachable is an excellent idea; perfect for 'static data' scenarios. What I don't know is how much will be saved. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windo

Re: [PERFORM] Performance suggestions for an update-mostly database?

2004-10-04 Thread Jim C. Nasby
atabase Solutions > San Francisco > > ---(end of broadcast)--- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED]) > -- Jim C. Nasby, Database

Re: [PERFORM] Does PostgreSQL run with Oracle?

2004-10-21 Thread Jim C. Nasby
Basically, for a given workload, it would take 2x the number of Sun CPUs as RS/6000 CPUs. The difference in Oracle licensing costs was usually enough to pay for the new hardware in one year. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! w

Re: [PERFORM] OS desicion

2004-10-21 Thread Jim C. Nasby
eeBSD will schedule disk I/O based on process priority, while linux won't. This can be very handy for things like vacuum. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you wan

Re: [PERFORM] Anything to be gained from a 'Postgres Filesystem'?

2004-10-21 Thread Jim C. Nasby
all that easy to get right. :-) ) > > /* Steinar */ > -- > Homepage: http://www.sesse.net/ > > ---(end of broadcast)------- > TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] > -- Jim C. Nasby, Database Consultant

Re: [PERFORM] Sequential Scan with LIMIT

2004-10-28 Thread Jim C. Nasby
mething that the > statistics we keep are too crude to detect. Isn't that exactly what pg_stats.correlation is? -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to g

Re: [PERFORM] Sequential Scan with LIMIT

2004-10-28 Thread Jim C. Nasby
On Thu, Oct 28, 2004 at 07:49:28PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > On Sun, Oct 24, 2004 at 04:11:53PM -0400, Tom Lane wrote: > >> The test case you are showing is probably suffering from nonrandom > >> placem

[PERFORM] seqscan strikes again

2004-11-09 Thread Jim C. Nasby
ual time=0.009..3.368 rows=2439 loops=1) -> Hash (cost=3.11..3.11 rows=10 width=4) (actual time=0.061..0.061 rows=0 loops=1) -> Index Scan using alert_type_pkey on alert_type t (cost=0.00..3.11 rows=10 width=4) (actual time=0.018..0.038 rows=1

Re: [PERFORM] seqscan strikes again

2004-11-10 Thread Jim C. Nasby
query only has a problem if it's run on a > >large date/time window, which normally doesn't happen. > > Try increasing your statistics target for the column and then rerunning > analyze. > > Sincerely, > > Joshua D. Drake -- Jim C. Nasby, Database

Re: [PERFORM] Analyzer is clueless

2004-11-17 Thread Jim C. Nasby
ere it seems the planer doesn't think it'll be getting a unique value or a small set of values even though stats indicates that it should be. One final question... would there be interest in a process that would dynamically update the histogram settings for tables based on how distinc

Re: [PERFORM] memcached and PostgreSQL

2004-11-24 Thread Jim C. Nasby
te never happens. > > Having pgmemcache delete, not replace data addresses this second issue. > -sc > > -- > Sean Chittenden > > > ---(end of broadcast)--- > TIP 3: if posting/reading through Usenet, please send

Re: [PERFORM] Postgres vs. DSpam

2004-11-29 Thread Jim C. Nasby
.Net .NZ Ltd, PO Box 11-053, Manners St, Wellington > WEB: http://catalyst.net.nz/ PHYS: Level 2, 150-154 Willis St > DDI: +64(4)803-2201 MOB: +64(272)DEBIAN OFFICE: +64(4)499-2267 >These PRESERVES should be FORCE-FED to PENTAGON OFFICIALS!! > --

Re: [PERFORM] Normalization or Performance

2004-12-02 Thread Jim C. Nasby
stem files and a 6x200G RAID10 for the database (all SATA drives). The largest table 120M rows and 825,000 8k pages. I can scan 1/5th of that table via an index scan in about a minute. (The schema can be found at http://minilink.org/cvs.distributed.net/l3.sql. -- Jim C. Nasby, Database Consultant

Re: [PERFORM] Partitioned table performance

2004-12-21 Thread Jim C. Nasby
's probably possible to fix it though, at least for > cases where the child tables have rowtypes identical to the parent. > > regards, tom lane > > ---(end of broadcast)--- > TIP 3: if posting/reading through Us

Re: [PERFORM] Partitioned table performance

2004-12-21 Thread Jim C. Nasby
++---+-----+--+- public | email_contrib | project_id | 0 | 4 | 6 | {205,5,25,8,24,3} | {0.461133,0.4455,0.0444333,0.0418667,0.0049,0.00216667} | | 0.703936 -- Jim

Re: [PERFORM] PostgreSQL vs. Oracle vs. Microsoft

2005-01-10 Thread Jim C. Nasby
find too many real-world examples where you could do something with a PostgreSQL procedural language that you couldn't do with PL/SQL. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: "

Re: [PERFORM] Performance delay

2005-01-13 Thread Jim C. Nasby
>>'01-10-2005 23:59' order by id limit 30) as t; > >> > >>it will cost me about 3+ secs > > > > > >The difference will be that in the final case you only make 30 calls > >to long2ip() whereas in the first two you call it 30

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Jim C. Nasby
> within a single server (doesn't it? I thought it did, I know it was > discussed w/ the guy from Cox Communications and I thought he was using > it :). No, PostgreSQL doesn't support any kind of partitioning, unless you write it yourself. I think there's some work being done

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Jim C. Nasby
s for locking and consistency across machines. So you better have fast access to the drive array, and the array better have caching of some kind. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where d

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-20 Thread Jim C. Nasby
kind of limited clustering scheme that could be implemented without a great amount of effort by the core developers. In that case I think there's a good chance you could find people willing to work on it. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer

[PERFORM] Odd number of rows expected

2005-01-21 Thread Jim C. Nasby
et is set to 1000. Basically, it seems that it doesn't understand that each row in log will match up with at most one row in bucket. There is a unique index on bucket(rrs_id, end_time), so it should be able to tell this. -- Jim C. Nasby, Database Consultant

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-22 Thread Jim C. Nasby
uld be a source of contention on > multi-processor machines running lots of concurrent update/deletes. > > -- > greg > > > ---(end of broadcast)--- > TIP 6: Have you searched our list archives? > >http://arc

Re: [PERFORM] Cheaper VACUUMing

2005-01-23 Thread Jim C. Nasby
l carefully read the owner's manual." > <http://www.eviloverlord.com/> > > ---(end of broadcast)--- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to [EMAIL PROTECTED

Re: [PERFORM] Odd number of rows expected

2005-01-23 Thread Jim C. Nasby
On Sat, Jan 22, 2005 at 10:18:00PM -0500, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > (SELECT b.bucket_id AS rrs_bucket_id, s.* > > FROM rrs.bucket b > > JOIN page_log

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-23 Thread Jim C. Nasby
hanged areas of large tables would make for a significant > reduction in the cost of VACUUM. FWIW, that's already on the TODO. See also http://lnk.nu/archives.postgresql.org/142.php. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy!

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-24 Thread Jim C. Nasby
thout numbers this is a bunch of hand-waving, but I don't think it's valid to assume that minimizing the amount of work you do in a transaction means better throughput without considering what it will cost to do the work you're putting off until later. -- Jim C. Nasby, Database C

Re: [PERFORM] Automagic tuning

2005-01-31 Thread Jim C. Nasby
xamples. Are there any examples of how you can take numbers from pg_stats_* or explain analize and turn them into configuration settings (such and random page cost)? -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team #182

<    3   4   5   6   7   8   9   >