Tom Lane wrote:
> Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes:
>
>>CREATE INDEX on a 1,8B row table (5 int columns - index created on the
>>first row about 300M distinct values):
>
>
>>before: 11h 51min
>>after: 3h 11min(!)
>
>
> Cool. Does it seem to be I/O bound now? Would you be will
"Zeugswetter Andreas DCP SD" <[EMAIL PROTECTED]> wrote:
> Ok, we cannot reuse a dead tuple. Maybe we can reuse the space of a dead
> tuple by reducing the tuple to it's header info.
I was just working about your idea. In my work, bgwriter truncates
dead tuples and leaves only their headers. I'll
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> Convention is that it goes near the top of the file, correct?
It goes at the bottom of the file header block comment ... if there
isn't a block comment at the start of the file, there should be. We
have a thousand or so examples to follow ;-)
Tom Lane wrote:
Mark Kirkwood <[EMAIL PROTECTED]> writes:
Good points! I had not noticed this test case. Probably NULL is better
Would setting it to 'BLCKSZ - (fixed index header stuff)' be better,
No, I don't think so, because that will just make it harder to recognize
what's what (reme
On Wed, 8 Mar 2006, Jim C. Nasby wrote:
On Wed, Mar 08, 2006 at 11:27:23PM -0300, Alvaro Herrera wrote:
Jim C. Nasby wrote:
+1. I'd also argue that it should be in contrib as well.
I can submit a patch for contrib or the entire tree if desired.
Convention is that it goes near the top of the
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> On Wed, Mar 08, 2006 at 06:55:59PM -0500, Greg Stark wrote:
> >
> > "Luke Lonergan" <[EMAIL PROTECTED]> writes:
> >
> > > > I am pretty sure from this thread that PostgreSQL is not doing #1, and I
> > > > have no idea if it is doing #2.
> > >
> > >
On Wed, Mar 08, 2006 at 11:27:23PM -0300, Alvaro Herrera wrote:
> Jim C. Nasby wrote:
>
> > +1. I'd also argue that it should be in contrib as well.
> >
> > I can submit a patch for contrib or the entire tree if desired.
> > Convention is that it goes near the top of the file, correct?
>
> Corre
Jim C. Nasby wrote:
> +1. I'd also argue that it should be in contrib as well.
>
> I can submit a patch for contrib or the entire tree if desired.
> Convention is that it goes near the top of the file, correct?
Correct.
--
Alvaro Herrerahttp://www.CommandPrompt.
Dann Corbit wrote:
I do not clearly understand the sorting code in PostgreSQL. If I did
have a good grasp of it, I would take a go at improving it.
"Show me the code" (and the benchmarks).
Seriously. We see regular discussions on this and similar topics, but I
haven't seen a patch that
On Wed, Mar 08, 2006 at 07:37:14PM -0300, Alvaro Herrera wrote:
> On the $PostgreSQL$ issue, I think it's best to have the tag everywhere,
> so if these files were the only ones that had it, I'd rather add them to
> the rest ...
+1. I'd also argue that it should be in contrib as well.
I can submi
On Wed, Mar 08, 2006 at 04:57:52PM -0500, Neil Conway wrote:
> On Wed, 2006-03-08 at 08:20 -0800, David Fetter wrote:
> > The previous discussion showed that there is a wide diversity of
> > opinions on what The Right Delimiter and The Right NULL String(TM)
> > are.
>
> Barring a more convincing j
On Wed, Mar 08, 2006 at 06:55:59PM -0500, Greg Stark wrote:
>
> "Luke Lonergan" <[EMAIL PROTECTED]> writes:
>
> > > I am pretty sure from this thread that PostgreSQL is not doing #1, and I
> > > have no idea if it is doing #2.
> >
> > Yep. Even Knuth says that the tape goo is only interesting f
> -Original Message-
> From: Jim C. Nasby [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 08, 2006 5:44 PM
> To: Dann Corbit
> Cc: Tom Lane; Luke Lonergan; Simon Riggs; pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] Merge algorithms for large numbers of "tapes"
>
> On Wed, Mar 08
On Wed, Mar 08, 2006 at 03:35:53PM -0800, Dann Corbit wrote:
> I think I did not explain it clearly enough. Suppose that you have a
> set of rows you need to sort. Instead of loading the whole row into
> memory, just load the columns (or parts of columns) that are being
> sorted. I hope that it
An interesting read at http://www.vldb.org/conf/1997/P376.PDFOn 3/8/06, Dann Corbit <
[EMAIL PROTECTED]> wrote:> -Original Message-> From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]> Sent: Wednesday, March 08, 2006 3:56 PM> To: Luke Lonergan> Cc: Dann Corbit; Tom Lane; Jim C. Nasby; Simo
On Wed, Mar 08, 2006 at 10:49:16AM -0800, Luke Lonergan wrote:
> Jim,
>
> On 3/8/06 9:49 AM, "Jim C. Nasby" <[EMAIL PROTECTED]> wrote:
>
> > On Wed, Mar 08, 2006 at 11:20:50AM -0500, Tom Lane wrote:
>
> >> Not sure that follows. In particular, the entire point of the recent
> >> changes has bee
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 08, 2006 3:56 PM
> To: Luke Lonergan
> Cc: Dann Corbit; Tom Lane; Jim C. Nasby; Simon Riggs; pgsql-
> [EMAIL PROTECTED]
> Subject: Re: [HACKERS] Merge algorithms for large numbers of "tapes"
>
"Luke Lonergan" <[EMAIL PROTECTED]> writes:
> > I am pretty sure from this thread that PostgreSQL is not doing #1, and I
> > have no idea if it is doing #2.
>
> Yep. Even Knuth says that the tape goo is only interesting from a
> historical perspective and may not be relevant in an era of disk d
Ben Chelf <[EMAIL PROTECTED]> writes:
> >>>#ifdef STATIC_ANALYSIS
> >>>#define ereport(elevel, rest) \
> >>>(errstart(elevel, __FILE__, __LINE__, PG_FUNCNAME_MACRO) ? \
> >>> (errfinish rest) : (void) 0), (elevel >= ERROR ? exit(0) : 0)
> >>>#else
> >>>/* Normal def */
> >>>#endif
> -Original Message-
> From: Tom Lane [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 08, 2006 3:17 PM
> To: Dann Corbit
> Cc: Jim C. Nasby; Luke Lonergan; Simon Riggs;
pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] Merge algorithms for large numbers of "tapes"
>
> "Dann Corbit
"Dann Corbit" <[EMAIL PROTECTED]> writes:
> Here are some suggestions of things that I know work really, really
> well:
> #1. Two pass merge (none of that silly poly-tape merge goo)
This amounts to an assumption that you have infinite work_mem, in which
case you hardly need an external sort at al
Bruce Momjian writes:
> Intended. None of the other include files in that directory had it.
> Either all should, or none. Should we add it to all of them?
All. There's no good excuse for not having a version tag on every
source-code file. The contrib stuff is pretty bad about this, but
hey,
Bruce Momjian wrote:
> Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > Log Message:
> > > ---
> > > Remove Christof Petig copyright on include file, per author request.
> >
> > Huh, I thought what he actually told was that the file was released
> > under BSD license. Maybe I missed i
Alvaro Herrera wrote:
> Alvaro Herrera wrote:
> > Bruce Momjian wrote:
> > > Log Message:
> > > ---
> > > Remove Christof Petig copyright on include file, per author request.
> >
> > Huh, I thought what he actually told was that the file was released
> > under BSD license. Maybe I missed
Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > Log Message:
> > ---
> > Remove Christof Petig copyright on include file, per author request.
>
> Huh, I thought what he actually told was that the file was released
> under BSD license. Maybe I missed it, but I didn't see him asking to
> r
Alvaro Herrera wrote:
> Bruce Momjian wrote:
> > Log Message:
> > ---
> > Remove Christof Petig copyright on include file, per author request.
>
> Huh, I thought what he actually told was that the file was released
> under BSD license. Maybe I missed it, but I didn't see him asking to
> r
Bruce Momjian wrote:
> Log Message:
> ---
> Remove Christof Petig copyright on include file, per author request.
Huh, I thought what he actually told was that the file was released
under BSD license. Maybe I missed it, but I didn't see him asking to
remove the copyright.
We certainly hav
There are some articles here that are worth reading if you want to sort
fast:
http://research.microsoft.com/barc/SortBenchmark/
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:pgsql-hackers-
> [EMAIL PROTECTED] On Behalf Of Dann Corbit
> Sent: Wednesday, March 08, 2006 1:59 PM
> To
On 3/8/06, Josh Berkus wrote:
Actually, I thougth that Neil/eDB did this with their copy. Is there anyway to get a copy of that "training configuration"?
I think we have a backup of it somewhere. I'll look into it.
-- Jonah H. Harris, Database Internals ArchitectEnterpriseDB
Folks,
> As for Coverity, if the elevel that's passed to the ereport is really a
> constant, the above #ifdef should absolutely do the trick for us so we
> know to stop analyzing on that path...Let me know if it doesn't actually
> do that ;)
Um, I think the answer is to train Coverity, not change
> -Original Message-
> From: Luke Lonergan [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 08, 2006 1:52 PM
> To: Dann Corbit; Tom Lane; Jim C. Nasby
> Cc: Simon Riggs; pgsql-hackers@postgreSQL.org
> Subject: Re: [HACKERS] Merge algorithms for large numbers of "tapes"
>
> Dann,
>
> On
On Wed, 2006-03-08 at 08:20 -0800, David Fetter wrote:
> The previous discussion showed that there is a wide diversity of
> opinions on what The Right Delimiter and The Right NULL String(TM)
> are.
Barring a more convincing justification for why we need this feature,
I'm inclined to side with Tom:
Dann,
On 3/8/06 12:39 PM, "Dann Corbit" <[EMAIL PROTECTED]> wrote:
> Here are some suggestions of things that I know work really, really
> well:
Can you point to an example? That might help move the discussion along.
The reason to interject about the tape goo in this discussion is that we
seem
etmorelli wrote:
> Tom,
>
> sorry, but the address that you wrote tells that there isn?t any patch to
> apply. Is this patch Itagaki's one? How could I pick it?
>
> By the way, don?t worry about the whole idea. It's an experiment that shall
> be improved in the future, I hope.
>
> Best rega
I do not clearly understand the sorting code in PostgreSQL. If I did
have a good grasp of it, I would take a go at improving it.
Here are some suggestions of things that I know work really, really
well:
#1. Two pass merge (none of that silly poly-tape merge goo)
#2. Load ONLY the keys that ar
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> But do fewer/longer sorted runs translate into not merging back to disk?
> I thought that was controlled by if we had to be able to rewind the
> result set.
A plain SELECT ... ORDER BY doesn't assume that anymore. It is still
required for some cases su
Martijn van Oosterhout wrote:
On Tue, Mar 07, 2006 at 05:39:18PM -0500, Tom Lane wrote:
Martijn van Oosterhout writes:
#ifdef STATIC_ANALYSIS
#define ereport(elevel, rest) \
(errstart(elevel, __FILE__, __LINE__, PG_FUNCNAME_MACRO) ? \
(errfinish rest) : (void) 0), (elevel >
Tom,
sorry, but the address that you wrote tells that there isn´t any patch to
apply. Is this patch Itagaki's one? How could I pick it?
By the way, don´t worry about the whole idea. It's an experiment that shall
be improved in the future, I hope.
Best regards,
Eduardo Morelli
Jim,
On 3/8/06 9:49 AM, "Jim C. Nasby" <[EMAIL PROTECTED]> wrote:
> On Wed, Mar 08, 2006 at 11:20:50AM -0500, Tom Lane wrote:
>> Not sure that follows. In particular, the entire point of the recent
>> changes has been to extend the range in which we can use a single merge
>> pass --- that is, w
On Sun, Mar 05, 2006 at 01:23:10PM -0500, Bruce Momjian wrote:
> > "Allow commenting of variables in postgresql.conf to restore them to
> > defaults
> > Currently, if a variable is commented out, it keeps the previous
> > uncommented value until a server restarted."
> > I take that to apply to th
On Wed, 2006-03-08 at 10:21 -0500, Tom Lane wrote:
> Simon Riggs <[EMAIL PROTECTED]> writes:
> > 1. Earlier we had some results that showed that the heapsorts got slower
> > when work_mem was higher and that concerns me most of all right now.
>
> Fair enough, but that's completely independent of t
On Wed, 2006-03-08 at 10:45 -0500, Tom Lane wrote:
> Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes:
> > CREATE INDEX on a 1,8B row table (5 int columns - index created on the
> > first row about 300M distinct values):
>
> > before: 11h 51min
> > after: 3h 11min(!)
>
> Cool. Does it seem to be
On Wed, Mar 08, 2006 at 11:20:50AM -0500, Tom Lane wrote:
> "Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> > If we do have to fail to disk, cut back to 128MB, because having 8x that
> > certainly won't make the sort run anywhere close to 8x faster.
>
> Not sure that follows. In particular, the enti
On Wed, 2006-03-08 at 08:33 -0500, Stephen Frost wrote:
> Greetings,
>
> * Simon Riggs ([EMAIL PROTECTED]) wrote:
> > work_mem= 1 GB benefit at 8 TB
> > work_mem= 256MB benefit at 0.5 TB
> > (based upon runs on average twice size of memory, and each logical tape
> > requiring
David Fetter <[EMAIL PROTECTED]> writes:
> Not everybody's editor/mailer/whatever does this right, and it makes
> things fragile. Another way to do this is to change the delimter to a
> printable character like '|', but that raises hackles, too.
Frankly if you're passing you data through an edi
Tom,
* Tom Lane ([EMAIL PROTECTED]) wrote:
> I've got the same problem with this that I do with the recently-proposed
> patch to fail queries with estimated cost > X --- to wit, I think it
> will result in a net *reduction* in system reliability not an improvement.
> Any such feature changes the p
On Wed, Mar 08, 2006 at 11:26:00AM -0500, Tom Lane wrote:
> David Fetter <[EMAIL PROTECTED]> writes:
> > On Wed, Mar 08, 2006 at 11:03:00AM -0500, Tom Lane wrote:
> >> Did anyone provide a convincing use case for this?
>
> > I've had one so far, and it was enough to cause me to make a
> > special
David Fetter <[EMAIL PROTECTED]> writes:
> On Wed, Mar 08, 2006 at 11:03:00AM -0500, Tom Lane wrote:
>> Did anyone provide a convincing use case for this?
> I've had one so far, and it was enough to cause me to make a special
> patched version of pg_dump. To get some idea of how drastic that was,
"Jim C. Nasby" <[EMAIL PROTECTED]> writes:
> If we do have to fail to disk, cut back to 128MB, because having 8x that
> certainly won't make the sort run anywhere close to 8x faster.
Not sure that follows. In particular, the entire point of the recent
changes has been to extend the range in which
On Wed, Mar 08, 2006 at 11:10:04AM -0500, Neil Conway wrote:
> On Wed, 2006-03-08 at 07:47 -0800, David Fetter wrote:
> > From the earlier discussion, it appears that there is a variety of
> > opinions on what the COPY delimiter should be in pg_dump. This patch
> > allows people to set it and the
On Wed, Mar 08, 2006 at 11:03:00AM -0500, Tom Lane wrote:
> David Fetter <[EMAIL PROTECTED]> writes:
> > From the earlier discussion, it appears that there is a variety of
> > opinions on what the COPY delimiter should be in pg_dump. This
> > patch allows people to set it and the NULL string.
>
>
On Wed, Mar 08, 2006 at 07:28:16AM -0800, Luke Lonergan wrote:
> > I think this would be extremely dangerous, as it would encourage
> > processes to take more than their fair share of available resources.
>
> I agree - in fact, we currently have no structured concept of "fair share of
> available
On Wed, 2006-03-08 at 07:47 -0800, David Fetter wrote:
> From the earlier discussion, it appears that there is a variety of
> opinions on what the COPY delimiter should be in pg_dump. This patch
> allows people to set it and the NULL string.
I'm still not convinced there is a reasonable use-case
David Fetter <[EMAIL PROTECTED]> writes:
> From the earlier discussion, it appears that there is a variety of
> opinions on what the COPY delimiter should be in pg_dump. This patch
> allows people to set it and the NULL string.
Did anyone provide a convincing use case for this? It's of zero valu
Scott Marlowe <[EMAIL PROTECTED]> writes:
> For same reasons, i.e. a need for precision, I find it hard to accept
> the idea of mixing positive and negative units in the same interval.
The semantics are perfectly well defined, so I don't buy this.
regards, tom lane
-
Stephen Frost <[EMAIL PROTECTED]> writes:
> I suppose I could put quotas in place or something but I don't really
> have a problem with the database as a whole using up a bunch of disk
> space (hence why it's got alot of room to grow into), I just would have
> liked a "this will chew up more disk s
On Wed, 2006-03-08 at 06:07, Markus Schaber wrote:
> Hi, Scott,
>
> Scott Marlowe wrote:
>
> >>But it isn't '-2 months, -1 day'. I think what you are saying is what I
> >>am saying, that we should make the signs consistent.
> > Pretty much. It just seems wrong to have different signs in what is
Folks,
>From the earlier discussion, it appears that there is a variety of
opinions on what the COPY delimiter should be in pg_dump. This patch
allows people to set it and the NULL string. Thanks to Gavin Sherry
for help with the pointers :)
I didn't patch pg_dumpall, but it would be trivial if
Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes:
> CREATE INDEX on a 1,8B row table (5 int columns - index created on the
> first row about 300M distinct values):
> before: 11h 51min
> after: 3h 11min(!)
Cool. Does it seem to be I/O bound now? Would you be willing to do it
over with oprofile t
Tom Lane wrote:
I wrote:
Stefan Kaltenbrunner <[EMAIL PROTECTED]> writes:
samples %symbol name
24915704 96.2170 ltsReleaseBlock
We probably need to tweak things so this doesn't get called during the
"final merge" pass. Looking at it now.
I've committed a fix for this into CV
On Wed, 8 Mar 2006, Hannu Krosing wrote:
> ??hel kenal p??eval, E, 2006-03-06 kell 17:25, kirjutas Bruce Momjian:
> > Hannu Krosing wrote:
> > > ?hel kenal p?eval, E, 2006-03-06 kell 12:12, kirjutas Bruce Momjian:
> > > > Added to TODO:
> > > >
> > > > o Prevent parent tables from altering o
Tom,
On 3/8/06 7:21 AM, "Tom Lane" <[EMAIL PROTECTED]> wrote:
> Simon Riggs <[EMAIL PROTECTED]> writes:
>> 1. Earlier we had some results that showed that the heapsorts got slower
>> when work_mem was higher and that concerns me most of all right now.
>
> Fair enough, but that's completely indep
Mark Kirkwood <[EMAIL PROTECTED]> writes:
>> Good points! I had not noticed this test case. Probably NULL is better
> Would setting it to 'BLCKSZ - (fixed index header stuff)' be better,
No, I don't think so, because that will just make it harder to recognize
what's what (remember that BLCKSZ i
Simon Riggs <[EMAIL PROTECTED]> writes:
> 1. Earlier we had some results that showed that the heapsorts got slower
> when work_mem was higher and that concerns me most of all right now.
Fair enough, but that's completely independent of the merge algorithm.
(I don't think the Nyberg results necessa
Greetings,
* Simon Riggs ([EMAIL PROTECTED]) wrote:
> work_mem= 1 GBbenefit at 8 TB
> work_mem= 256MB benefit at 0.5 TB
> (based upon runs on average twice size of memory, and each logical tape
> requiring 256KB memory, i.e. min(work_mem/4, 6) * work_mem * 2, which
> for wo
On Tue, 2006-03-07 at 18:14 -0500, Tom Lane wrote:
> BTW, I was just looking over Knuth's discussion of sorting again, and
> realized that there is still something more that could be done within
> the existing sort framework. We currently use standard polyphase merge
> (his Algorithm 5.4.2D), whic
Ühel kenal päeval, E, 2006-03-06 kell 17:25, kirjutas Bruce Momjian:
> Hannu Krosing wrote:
> > ?hel kenal p?eval, E, 2006-03-06 kell 12:12, kirjutas Bruce Momjian:
> > > Added to TODO:
> > >
> > > o Prevent parent tables from altering or dropping constraints
> > > like CHECK that ar
Hi, Scott,
Scott Marlowe wrote:
>>But it isn't '-2 months, -1 day'. I think what you are saying is what I
>>am saying, that we should make the signs consistent.
> Pretty much. It just seems wrong to have different signs in what is
> essentially a single unit.
>
> We don't say 42 degrees, -12 m
Gourish Singbal wrote:
> Just Curious, which postgresql server version bug has been fixed ?. is it
> existing on 8.1.3 and under which circumstances ?.
No, this particular bug existed only on the CVS HEAD branch, as it was
introduced there recently.
--
Alvaro Herrera
I have applied all the patches in the patch queue, and am starting to
look at the patches_hold queue, which are patches submitted after the
feature freeze.
http://momjian.postgresql.org/cgi-bin/pgpatches_hold
--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraos
Michael Glaesemann wrote:
>
> On Mar 8, 2006, at 13:01 , Bruce Momjian wrote:
>
> > I have updated the /contrib and PL regression tests for
> > escape_string_warning. I can't test all the regressions but the build
> > farm will tell us soon enough.
>
> I see 'em turning green :) Thanks, Bruce.
On Mar 8, 2006, at 13:01 , Bruce Momjian wrote:
I have updated the /contrib and PL regression tests for
escape_string_warning. I can't test all the regressions but the build
farm will tell us soon enough.
I see 'em turning green :) Thanks, Bruce.
Michael Glaesemann
grzm myrealbox com
--
On Tue, Mar 07, 2006 at 05:39:18PM -0500, Tom Lane wrote:
> Martijn van Oosterhout writes:
> > #ifdef STATIC_ANALYSIS
> > #define ereport(elevel, rest) \
> > (errstart(elevel, __FILE__, __LINE__, PG_FUNCNAME_MACRO) ? \
> > (errfinish rest) : (void) 0), (elevel >= ERROR ? exit(0)
73 matches
Mail list logo