Re: [HACKERS] Schemas: status report, call for developers

2002-05-02 Thread Jeffrey W. Baker
On Tue, Apr 30, 2002 at 09:41:47PM +0300, Oleg Bartunov wrote: > I think DBD::Pg driver very much depends on system tables. > Hope, Jeffrey (current maintainer) is online. These changes may break DBD::Pg. What is the expected time of this release? I will review my code for impact. Thanks for t

Re: [HACKERS] Schemas: status report, call for developers

2002-05-02 Thread Jeffrey W. Baker
On Thu, May 02, 2002 at 05:28:36PM +0300, Oleg Bartunov wrote: > On Wed, 1 May 2002, Jeffrey W. Baker wrote: > > > On Tue, Apr 30, 2002 at 09:41:47PM +0300, Oleg Bartunov wrote: > > > I think DBD::Pg driver very much depends on system tables. > > > Hope, Jeffrey

[HACKERS] bitmap scans, btree scans, and tid order

2005-05-15 Thread Jeffrey W. Baker
About this time last year I was holding forth on the value of visiting the heap in TID order, even when index scan tuples are randomly ordered. Today I decided to start working on the problem stated in this TODO item: Fetch heap pages matching index entries in sequential order

Re: [HACKERS] bitmap scans, btree scans, and tid order

2005-05-16 Thread Jeffrey W. Baker
On Mon, 2005-05-16 at 09:53 -0400, Tom Lane wrote: > Jeffrey Baker <[EMAIL PROTECTED]> writes: > > Change the planner/executor to use the bitmap scan in all cases where > > index order is unimportant. From my reading of the current code, the > > bitmap scan is only used in case of a join. > > T

Re: [HACKERS] bitmap scans, btree scans, and tid order

2005-05-16 Thread Jeffrey W. Baker
On Mon, 2005-05-16 at 14:35 -0400, Tom Lane wrote: > "Jeffrey W. Baker" <[EMAIL PROTECTED]> writes: > > It's also possible that changing the btree scan to work in > > groups of tuples instead of single tuples would make more sense, which > > is why I vent

[HACKERS] Bitmap scan cost model (was Re: bitmap scans, btree scans, and tid order)

2005-05-17 Thread Jeffrey W. Baker
On Mon, 2005-05-16 at 14:35 -0400, Tom Lane wrote: > "Jeffrey W. Baker" <[EMAIL PROTECTED]> writes: > > On Mon, 2005-05-16 at 09:53 -0400, Tom Lane wrote: > >> This is a fallacy, and I think your concern is largely mistaken. Have > >> you experi

Re: [HACKERS] Bitmap scan cost model (was Re: bitmap scans, btree

2005-05-18 Thread Jeffrey W. Baker
On Wed, 2005-05-18 at 11:27 -0400, Tom Lane wrote: > "Jeffrey W. Baker" <[EMAIL PROTECTED]> writes: > > Obviously in this case sequential scan was (would have been) a huge win. > > Incrementing random_page_cost from 4 (the default) to 5 causes the > > planner t

Re: [HACKERS] Consumer-grade vs enterprise-grade disk drives

2005-05-31 Thread Jeffrey W. Baker
On Mon, 2005-05-30 at 21:38 -0700, Luke Lonergan wrote: > Tom, > > This is a story that is evolving. Anyone else use StorageReview? Great > comprehensive drive benchmarks: > http://www.storagereview.com/ > > Check the comparisons between 15K RPM SCSI drives and the 2004 Western > Digital 10K

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-07-14 Thread Jeffrey W. Baker
On Fri, 2005-06-24 at 09:37 -0400, Tom Lane wrote: > ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > > ... So I'll post the new results: > > > checkpoint_ | writeback | > > segments| cache | open_sync | fsync=false | O_DIRECT only | > > fsync_direct | open_direct > > +-

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-07-14 Thread Jeffrey W. Baker
On Fri, 2005-06-24 at 10:19 -0500, Jim C. Nasby wrote: > On Fri, Jun 24, 2005 at 09:37:23AM -0400, Tom Lane wrote: > > ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > > > ... So I'll post the new results: > > > > > checkpoint_ | writeback | > > > segments| cache | open_sync | fsync=false

Re: [HACKERS] ORDER BY

2005-07-25 Thread Jeffrey W. Baker
On Mon, 2005-07-25 at 19:08 -0300, Marc G. Fournier wrote: > > On Mon, 25 Jul 2005, Jeffrey W. Baker wrote: > > > On Mon, 2005-07-25 at 18:11 -0300, Marc G. Fournier wrote: > >> Just curious as to whether or not a warning or something should be issued > >> in a

Re: [HACKERS] Simplifying wal_sync_method

2005-08-08 Thread Jeffrey W. Baker
On Mon, 2005-08-08 at 17:03 -0400, Tom Lane wrote: > > That's a decision that hasn't got a shred of evidence to justify > imposing it on every platform. This option has its uses on Linux, however. In my testing it's good for a large speedup (20%) on a 10-client pgbench, and a minor improvement w

Re: [HACKERS] data on devel code perf dip

2005-08-21 Thread Jeffrey W. Baker
On Sun, 2005-08-21 at 20:37 -0400, Tom Lane wrote: > The whole thing's pretty bizarre. I hate to sound obvious, but does the missing performance return if you back the patch out? It seemed to have been decided on Tue, 16 Aug 2005 15:45:30 -0700 that the performance was the same before and after.

[HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-17 Thread Jeffrey W. Baker
Greetings all, We have noticed a way to make a major improvement in Pg's performance on our workload, and I would like to get your thoughts before I go off to work up a patch. The basic problem is that Pg seeks far too much when performing an index scan. I have saved an strace of a backend which

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-19 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 07:58, Tom Lane wrote: > "Jeffrey W. Baker" <[EMAIL PROTECTED]> writes: > > We have noticed a way to make a major improvement in Pg's performance on > > our workload, and I would like to get your thoughts before I go off to > > wo

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-19 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 11:56, Tom Lane wrote: > "Jeffrey W. Baker" <[EMAIL PROTECTED]> writes: > > Are you saying that index scan results are sorted by something other > > than the index key? Because in my scheme they would still be sorted by > > index key. >

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-19 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 13:10, Tom Lane wrote: > "Glen Parker" <[EMAIL PROTECTED]> writes: > >> Not unless you add yet another sort step after the fetch step, that is > >> the idea becomes > >> 1. read index to get candidate TIDs > >> 2. sort TIDs into physical order > >> 3. read heap in physical ord

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-20 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 12:55, Tom Lane wrote: > "Glen Parker" <[EMAIL PROTECTED]> writes: > > What am I missing? Why is a performance bottle neck of this magnitude not > > on the same list of priorities as PITR, replication, and Win32? > > It's higher on my personal to-do list than most of those ;

Re: [HACKERS] New project launched : PostgreSQL GUI Installer for

2006-01-30 Thread Jeffrey W. Baker
On Mon, 2006-01-30 at 19:52 -0800, Joshua D. Drake wrote: > > > > > >On my Debian systems, I can install PostgreSQL quite readily via the > >command "apt-get install postgresql-8.1", which can get GUIed at least > >somewhat if I run aptitude, synaptic, or such... > > > > > Yes Christopher, you ca

Re: [HACKERS] New project launched : PostgreSQL GUI Installer for

2006-01-30 Thread Jeffrey W. Baker
On Mon, 2006-01-30 at 20:53 -0800, Joshua D. Drake wrote: > > Oracle's graphical installer is a material impediment to Oracle > > adoption. The installer only works on systems where particular versions > > of Java and Motif libraries are available. On 64-bit Opteron systems it > > only works with

Re: [HACKERS] [GENERAL] New project launched : PostgreSQL GUI

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 03:50 -0400, Marc G. Fournier wrote: > On Tue, 31 Jan 2006, Tino Wildenhain wrote: > > > Devrim GUNDUZ schrieb: > >> Hi, > >> > > ... > >>> Are you going to work with the underlying system's package manager, or > >>> put everything in /usr/local? > >> > >> > >> We'll work

Re: [HACKERS] Configuration WAS: New project launched : PostgreSQL

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 12:11 -0800, Josh Berkus wrote: > Jeffery, > > > > PostgreSQL *desperately* needs a better means of dealing with > > > configuration (though I guess I shouldn't be pushing too hard for this > > > since the current state of affairs brings me business). Any > > > improvement in

Re: [HACKERS] Configuration WAS: New project launched : PostgreSQL

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 17:06 -0600, Jim C. Nasby wrote: > On Tue, Jan 31, 2006 at 12:36:31PM -0800, Jeffrey W. Baker wrote: > > Random page cost - should possible to determine this at runtime > > Before worrying about random_page_cost at all it makes a lot more sense > to look

Re: [HACKERS] New project launched : PostgreSQL GUI Installer for

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 13:02 -0600, Jim C. Nasby wrote: > On Tue, Jan 31, 2006 at 11:46:03AM +, Andreas Pflug wrote: > > Tino Wildenhain wrote: > > > > > > > >Figuring out the correct values for some of the buffers and costs > > >is still a bit tough. Otoh, I guess there is no easy way to predi

Re: [HACKERS] Any conclusion on the Xeon context-switching issue?

2006-02-27 Thread Jeffrey W. Baker
On Mon, 2006-02-27 at 16:54 +, Richard Huxton wrote: > Tom Lane wrote: > > Richard Huxton writes: > >> Subject says it all really. I've got a new client who seems to be > >> suffering from it, and I'm not sure if any conclusion was reached. > > > > What's he using? 8.1 seems to have alleviat

Re: [HACKERS] Database file compatability

2005-09-27 Thread Jeffrey W. Baker
On Mon, 2005-09-26 at 17:27 -0500, Jim C. Nasby wrote: > If a database is created with a 64 bit version of initdb, would a 32bit > backend be able to talk to it? Likewise, would a backend compiled by a > different compiler be able to? > > If there was some kind of incompatability, would the backen

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Jeffrey W. Baker
On Tue, 2005-09-27 at 13:15 -0400, Ron Peacetree wrote: > That Btree can be used to generate a physical reordering of the data > in one pass, but that's the weakest use for it. The more powerful > uses involve allowing the Btree to persist and using it for more > efficient re-searches or combinin

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-28 Thread Jeffrey W. Baker
On Wed, 2005-09-28 at 12:03 -0400, Ron Peacetree wrote: > >From: "Jeffrey W. Baker" <[EMAIL PROTECTED]> > >Sent: Sep 27, 2005 1:26 PM > >To: Ron Peacetree <[EMAIL PROTECTED]> > >Subject: Re: [HACKERS] [PERFORM] A Better External Sort? > > >

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-29 Thread Jeffrey W. Baker
On Thu, 2005-09-29 at 10:06 -0700, Luke Lonergan wrote: > Josh, > > On 9/29/05 9:54 AM, "Josh Berkus" wrote: > > > Following an index creation, we see that 95% of the time required is the > > external sort, which averages 2mb/s. This is with seperate drives for > > the WAL, the pg_tmp, the tabl

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-29 Thread Jeffrey W. Baker
On Thu, 2005-09-29 at 11:03 -0700, Josh Berkus wrote: > Jeff, > > > Josh, do you happen to know how many passes are needed in the multiphase > > merge on your 60GB table? > > No, any idea how to test that? I would just run it under the profiler and see how many times beginmerge() is called. -jw

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-03 Thread Jeffrey W. Baker
On Mon, 2005-10-03 at 13:34 -0700, Josh Berkus wrote: > Michael, > > > >Realistically, you can't do better than about 25MB/s on a > > > single-threaded I/O on current Linux machines, > > > > What on earth gives you that idea? Did you drop a zero? > > Nope, LOTS of testing, at OSDL, GreenPlum and

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-03 Thread Jeffrey W. Baker
On Mon, 2005-10-03 at 14:16 -0700, Josh Berkus wrote: > Jeff, > > > > Nope, LOTS of testing, at OSDL, GreenPlum and Sun. For comparison, A > > > Big-Name Proprietary Database doesn't get much more than that either. > > > > I find this claim very suspicious. I get single-threaded reads in > > ex

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-05 Thread Jeffrey W. Baker
On Wed, 2005-10-05 at 12:14 -0400, Ron Peacetree wrote: > I've now gotten verification from multiple working DBA's that DB2, Oracle, and > SQL Server can achieve ~250MBps ASTR (with as much as ~500MBps ASTR in > setups akin to Oracle RAC) when attached to a decent (not outrageous, but > decent) HD

Re: [HACKERS] ice-broker scan thread

2005-11-29 Thread Jeffrey W. Baker
On Tue, 2005-11-29 at 09:45 -0500, Pollard, Mike wrote: > Anyway, what I did was the following. When doing a sequential scan, we > were starting at the beginning of the table and scanning forward. If I > threw up some threads to read ahead, then my user thread and my read > ahead threads would t