Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-26 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes: > would this break any application pulling a numeric field as binary > over the protocol? No. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-26 Thread Merlin Moncure
On 9/24/07, Gregory Stark <[EMAIL PROTECTED]> wrote: > > > We previously discussed compressing the numeric data type for small values: > > http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php > > We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse > the weight

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-26 Thread Zdenek . Kotala
Andrew Dunstan wrote: Zdenek Kotala wrote: Tom Lane wrote: Gregory Stark <[EMAIL PROTECTED]> writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-25 Thread Andrew Dunstan
Zdenek Kotala wrote: Tom Lane wrote: Gregory Stark <[EMAIL PROTECTED]> writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought t

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-25 Thread Zdenek Kotala
Tom Lane wrote: Gregory Stark <[EMAIL PROTECTED]> writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse the weight and

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-24 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: "Tom Lane" <[EMAIL PROTECTED]> writes: > Gregory Stark <[EMAIL PROTECTED]> writes: > >> I think we also should move the NumericData and declaration to numeric.c and >> make the Numeric type an opaque pointer for the rest of the source >> tree. > > I don't ag

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-24 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > We previously discussed compressing the numeric data type for small values: > http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php > We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse > the weight and sign/dscale s

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-24 Thread Gregory Stark
We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse the weight and sign/dscale so we could do this sometime without introducing

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-07-17 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: > We've changed the on-disk database format in 8.3, so we have an > opportunity to change other th

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-27 Thread Josh Berkus
Tom, > That sounds good, but there are corner cases where it wouldn't work --- > consider a page containing a single maximum-length tuple. Certainly any mature upgrade-in-place tool will require a "checker" which you run first which determines if you have a prohibitive corner case. Besides, I

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-22 Thread Zdenek Kotala
Heikki Linnakangas wrote: Since we're discussing upgrades, let me summarize the discussions we had over dinner in Ottawa for the benefit of all: Thanks for summary. As before, someone just needs to step up and do it. I'm now working on proposal. I hope that it will ready soon.

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-21 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > * Page format changes that grow data size are problematic, because there > can be pages that can't be expanded to the new format because there's > not enough space. However, it would be possible to write a pre-upgrade > program to look for the pro

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-21 Thread Heikki Linnakangas
Since we're discussing upgrades, let me summarize the discussions we had over dinner in Ottawa for the benefit of all: * pg_migrator is a sound approach to handling catalog changes. * Handling any page format change that doesn't grow the space taken by data is straightforward. Note that all 8.

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-21 Thread Zdenek Kotala
Andrew Sullivan wrote: On Wed, Jun 20, 2007 at 12:34:21PM -0400, Robert Treat wrote: FWIW pg_migrator is a pretty good swing at an in-place upgrade tool for 8.1->8.2. Unfortunately until the PGDG decides that in-place upgrade is a constraint their willing to place on development, I see them a

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-20 Thread Andrew Sullivan
On Wed, Jun 20, 2007 at 12:34:21PM -0400, Robert Treat wrote: > FWIW pg_migrator is a pretty good swing at an in-place upgrade tool for > 8.1->8.2. Unfortunately until the PGDG decides that in-place upgrade is a > constraint their willing to place on development, I see them a good > chicken/eg

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-20 Thread Robert Treat
On Tuesday 19 June 2007 10:15, Tom Lane wrote: > Zdenek Kotala <[EMAIL PROTECTED]> writes: > > I'm little bit confused when we introduce new page layout version? I > > expect that new version become with changes with pageheader, tuple > > header or data encoding (varlen/TOAST ...). But in case when

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-19 Thread Tom Lane
Zdenek Kotala <[EMAIL PROTECTED]> writes: > I'm little bit confused when we introduce new page layout version? I > expect that new version become with changes with pageheader, tuple > header or data encoding (varlen/TOAST ...). But in case when there is > new data type internal implementation, t

What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-19 Thread Zdenek Kotala
Simon Riggs wrote: The objections to applying this patch originally were: 1. it changes on-disk format (we've done this, so argument is void) I'm little bit confused when we introduce new page layout version? I expect that new version become with changes with pageheader, tuple header or dat

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Gregory Stark
"Bruce Momjian" <[EMAIL PROTECTED]> writes: > Would someone please explain why we are considering this so far past > features freeze, and who suggtested that the 8.3->8.4 upgrade being a binary > upgrade was anything more than a pipe dream? Simon just updated a patch he had originally submitted o

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Would someone please explain why we are considering this so far past > features freeze, and who suggtested that the 8.3->8.4 upgrade being a binary > upgrade was anything more than a pipe dream? Well, Greg had left further squeezing of numerics as an ope

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Bruce Momjian
Simon Riggs wrote: > On Mon, 2007-06-18 at 11:55 -0400, Tom Lane wrote: > > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > Since this is your idea, would you like to do this, or should I? > > > > Go for it. > > OK > > > I'm not actually convinced this is worth spending time on, > > as Greg St

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > I think representing zero as compactly as possible is worth the trouble, Either of these proposals can do that in 2 bytes. regards, tom lane ---(end of broadcast)--- TIP 4: Have yo

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 12:44 -0400, Tom Lane wrote: > I wrote: > > Gregory Stark <[EMAIL PROTECTED]> writes: > >> Anything shorter than the shortest possible numeric representation can > >> implicitly be interpreted as some alternate compact representation. I > >> already had a patch that stored sma

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
I wrote: > Gregory Stark <[EMAIL PROTECTED]> writes: >> Anything shorter than the shortest possible numeric representation can >> implicitly be interpreted as some alternate compact representation. I >> already had a patch that stored small integers in a single >> NumericDigit without any numeric h

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > Anything shorter than the shortest possible numeric representation can > implicitly be interpreted as some alternate compact representation. I already > had a patch that stored small integers in a single NumericDigit without any > numeric header at all.

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > I had a thought though: it's possible to reduce the header overhead for > typical-size numbers without giving up the ability to store large ones. > This is because the POS/NEG/NAN sign possibilities leave one unused bit > pattern. Hence: I had a whack an

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 11:55 -0400, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > Since this is your idea, would you like to do this, or should I? > > Go for it. OK > I'm not actually convinced this is worth spending time on, > as Greg Stark's 1-byte-varlena patch already save

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > Since this is your idea, would you like to do this, or should I? Go for it. I'm not actually convinced this is worth spending time on, as Greg Stark's 1-byte-varlena patch already saved more for typical numerics than this will.

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 17:49 +0200, Andreas Pflug wrote: > I wonder if the currently waiting patch isn't Good Enough for > 999. % of use cases, and "all" others can use numeric > instead of numeric(1000,800) or so. Especially since there are many > patches waiting that do need furth

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Andreas Pflug <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> In any case, no capability is lost, unlike the original proposal; and >> this would be much less invasive than the original patch since there's >> no need to play tricks with the content of the digit array. > I wonder if the currently

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 11:32 -0400, Tom Lane wrote: > Andreas Pflug <[EMAIL PROTECTED]> writes: > > Simon Riggs wrote: > >> The objections to applying this patch originally were: > >> 2. it would restrict number of digits to 508 and there are allegedly > >> some people that want to store > 508 digit

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Andreas Pflug
Tom Lane wrote: > Andreas Pflug <[EMAIL PROTECTED]> writes: > >> Simon Riggs wrote: >> >>> The objections to applying this patch originally were: >>> 2. it would restrict number of digits to 508 and there are allegedly >>> some people that want to store > 508 digits. >>> >>> >> If 50

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Andreas Pflug <[EMAIL PROTECTED]> writes: > Simon Riggs wrote: >> The objections to applying this patch originally were: >> 2. it would restrict number of digits to 508 and there are allegedly >> some people that want to store > 508 digits. >> > If 508 digits are not enough, are1000 digits be suff

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Michael Paesold
Andreas Pflug wrote: Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store > 508 digits. If 508 digits are not enough, are1000 digits be sufficient? Both limits appear quit

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 10:54 -0400, Tom Lane wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > We've changed the on-disk database format in 8.3, so we have an > > opportunity to change other things also. There is a patch thats been on > > the patch queue for some time called numeric508, submitt

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 16:56 +0200, Andreas Pflug wrote: > Simon Riggs wrote: > > The objections to applying this patch originally were: > > 2. it would restrict number of digits to 508 and there are allegedly > > some people that want to store > 508 digits. > > > If 508 digits are not enough, ar

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Andreas Pflug
Simon Riggs wrote: > The objections to applying this patch originally were: > 2. it would restrict number of digits to 508 and there are allegedly > some people that want to store > 508 digits. > If 508 digits are not enough, are1000 digits be sufficient? Both limits appear quite arbitrary to me

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > We've changed the on-disk database format in 8.3, so we have an > opportunity to change other things also. There is a patch thats been on > the patch queue for some time called numeric508, submitted Dec 2005; I thought that idea had been rejected long si

[HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
We've changed the on-disk database format in 8.3, so we have an opportunity to change other things also. There is a patch thats been on the patch queue for some time called numeric508, submitted Dec 2005; I've updated this patch now for 8.3 to remove bit rot (an hour's work). This is posted to pgsq