Where "*" ==
{print | save to PDF | save to format | display on screen}
Anyone know of one?
TiA
Ron
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
I'm putting in as much time as I can afford thinking about pg related
performance issues. I'm doing it because of a sincere desire to help
understand and solve them, not to annoy people.
If I didn't believe in pg, I would't be posting thoughts about how to
make it better.
It's probably worth s
First I wanted to verify that pg's IO rates were inferior to The Competition.
Now there's at least an indication that someone else has solved similar
problems. Existence proofs make some things easier ;-)
Is there any detailed programmer level architectual doc set for pg? I know
"the best doc is
I've now gotten verification from multiple working DBA's that DB2, Oracle, and
SQL Server can achieve ~250MBps ASTR (with as much as ~500MBps ASTR in
setups akin to Oracle RAC) when attached to a decent (not outrageous, but
decent) HD subsystem...
I've not yet had any RW DBA verify Jeff Baker's su
t: Oct 4, 2005 8:24 AM
To: Simon Riggs <[EMAIL PROTECTED]>
Cc: Tom Lane <[EMAIL PROTECTED]>, Ron Peacetree <[EMAIL PROTECTED]>,
pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
On Tue, Oct 04, 2005 at 12:24:54PM +0100, Simon Riggs wrote:
>
OK, change "performance" to "single thread performance" and we
still have a valid starting point for a discussion.
Ron
-Original Message-
From: Gregory Maxwell <[EMAIL PROTECTED]>
Sent: Oct 3, 2005 8:19 PM
To: Ron Peacetree <[EMAIL PROTECTED]>
Subject:
Let's pretend we get a 24HD HW RAID solution like that J Baker
says he has access to and set it up as a RAID 10. Assuming
it uses two 64b 133MHz PCI-X busses and has the fastest HDs
available on it, Jeff says he can hit ~1GBps of XFS FS IO rate
with that set up (12*83.3MBps= 1GBps).
Josh says th
Jeff, are those _burst_ rates from HD buffer or _sustained_ rates from
actual HD media? Rates from IO subsystem buffer or cache are
usually considerably higher than Average Sustained Transfer Rate.
Also, are you measuring _raw_ HD IO (bits straight off the platters, no
FS or other overhead) or _c
ut
Sent: Oct 1, 2005 12:19 PM
Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
On Sat, Oct 01, 2005 at 10:22:40AM -0400, Ron Peacetree wrote:
> Assuming we get the abyssmal physical IO performance fixed...
> (because until we do, _nothing_ is going to help us as much)
I'm still
so
that I can.)
Ron
-Original Message-
From: Andrew Dunstan <[EMAIL PROTECTED]>
Sent: Oct 1, 2005 11:19 AM
To: Ron Peacetree <[EMAIL PROTECTED]>
Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
Ron Peacetree wrote:
>The good news is all this means it's easy
*blink* Tapes?! I thought that was a typo...
If our sort is code based on sorting tapes, we've made a mistake. HDs
are not tapes, and Polyphase Merge Sort and it's brethren are not the
best choices for HD based sorts.
Useful references to this point:
Knuth, Vol 3 section 5.4.9, (starts p356 of 2
less ROI.
Ron
-Original Message-----
From: Josh Berkus
Sent: Sep 30, 2005 4:41 PM
To: Ron Peacetree <[EMAIL PROTECTED]>
Cc: pgsql-hackers@postgresql.org, pgsql-performance@postgresql.org
Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
Ron,
> That 11MBps was your =bulk load
could replace the
present sorting code with infinitely fast sorting code and we'd
still be scrod performance wise.
So why does basic IO suck so badly?
Ron
-Original Message-
From: Josh Berkus
Sent: Sep 30, 2005 1:23 PM
To: Ron Peacetree <[EMAIL PROTECTED]>
Cc: pg
>From: Josh Berkus
>Sent: Sep 29, 2005 12:54 PM
>Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
>
>The biggest single area where I see PostgreSQL external
>sort sucking is on index creation on large tables. For
>example, for free version of TPCH, it takes only 1.5 hours to
>load a 60GB
>From: Zeugswetter Andreas DAZ SD <[EMAIL PROTECTED]>
>Sent: Sep 29, 2005 9:28 AM
>Subject: RE: [HACKERS] [PERFORM] A Better External Sort?
>
>>In my original example, a sequential scan of the 1TB of 2KB
>>or 4KB records, => 250M or 500M records of data, being sorted
>>on a binary value key will
ntial passes, returning the first value
>>>on the first pass and the second value on the second pass.
>>> This will be faster than the method you propose.
>>
>>Ron Peacetree:
>>1= No that was not my main example. It was the simplest example
>>used to frame
In the interest of efficiency and "not reinventing the wheel", does anyone know
where I can find C or C++ source code for a Btree variant with the following
properties:
A= Data elements (RIDs) are only stored in the leaves, Keys (actually
KeyPrefixes; see "D" below) and Node pointers are only stor
>From: "Jeffrey W. Baker" <[EMAIL PROTECTED]>
>Sent: Sep 27, 2005 1:26 PM
>To: Ron Peacetree <[EMAIL PROTECTED]>
>Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
>
>On Tue, 2005-09-27 at 13:15 -0400, Ron Peacetree wrote:
>
>>That Btree can
>From: "Jeffrey W. Baker" <[EMAIL PROTECTED]>
>Sent: Sep 29, 2005 12:27 AM
>To: Ron Peacetree <[EMAIL PROTECTED]>
>Cc: pgsql-hackers@postgresql.org, pgsql-performance@postgresql.org
>Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
>
>You are e
If I've done this correctly, there should not be anywhere near
the number of context switches we currently see while sorting.
Each unscheduled context switch represents something unexpected
occuring or things not being where they are needed when they are
needed. Reducing such circumstances to the
>From: Josh Berkus
>ent: Sep 27, 2005 12:15 PM
>To: Ron Peacetree <[EMAIL PROTECTED]>
>Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
>
>I've somehow missed part of this thread, which is a shame since this is
>an area of primary concern for me.
>
&g
SECOND ATTEMPT AT POST. Web mailer appears to have
eaten first one. I apologize in advance if anyone gets two
versions of this post.
=r
>From: Tom Lane <[EMAIL PROTECTED]>
>Sent: Sep 26, 2005 9:42 PM
>Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
>
>So far, you've blithely assumed th
>From: Dann Corbit <[EMAIL PROTECTED]>
>Sent: Sep 26, 2005 5:13 PM
>To: Ron Peacetree <[EMAIL PROTECTED]>, pgsql-hackers@postgresql.org,
> pgsql-performance@postgresql.org
>Subject: RE: [HACKERS] [PERFORM] A Better External Sort?
>
>I think that the btre
>From: Ron Peacetree <[EMAIL PROTECTED]>
>Sent: Sep 24, 2005 6:30 AM
>Subject: Re: [HACKERS] [PERFORM] Releasing memory during External sorting?
>
>... the amount of IO done is the most
>important of the things that you should be optimizing for in
>choosing an
From: Dann Corbit <[EMAIL PROTECTED]>
Sent: Sep 23, 2005 5:38 PM
Subject: RE: [HACKERS] [PERFORM] Releasing memory during External sorting?
>_C Unleashed_ also explains how to use a callback function to perform
>arbitrary radix sorts (you simply need a method that returns the
>[bucketsize] most si
membered from my CS classes back in college :)
-- Mark
On Fri, 2005-09-23 at 13:17 -0400, Tom Lane wrote:
> Ron Peacetree <[EMAIL PROTECTED]> writes:
> > 2= No optimal external sorting algorithm should use more than 2 passes.
> > 3= Optimal external sorting algorithms should
From: Simon Riggs <[EMAIL PROTECTED]>
Sent: Sep 23, 2005 5:37 AM
Subject: [PERFORM] Releasing memory during External sorting?
>I have concerns about whether we are overallocating memory for use in
>external sorts. (All code relating to this is in tuplesort.c)
>
A decent external sorting algorithm,
From: Tom Lane <[EMAIL PROTECTED]>
Sent: Sep 23, 2005 2:15 PM
Subject: Re: [PERFORM] Releasing memory during External sorting?
>Mark Lewis <[EMAIL PROTECTED]> writes:
>> operations != passes. If you were clever, you could probably write a
>> modified bubble-sort algorithm that only made 2 passes
28 matches
Mail list logo
