Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: >> I would suggest that *all* of those TODOs are premature in the absence >> of experimental evidence about the effect of varying the parameters. > Isn't it obvious that the "right" value is going to depend extraord

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > I would suggest that *all* of those TODOs are premature in the absence > of experimental evidence about the effect of varying the parameters. > If we end up finding out that the existing settings are about right > anyway across a range of test cases, who n

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Luke Lonergan
Hi Tom, On 4/12/07 1:40 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > I would suggest that *all* of those TODOs are premature in the absence > of experimental evidence about the effect of varying the parameters. > If we end up finding out that the existing settings are about right > anyway across a

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Tom Lane
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > On 4/12/07 9:24 AM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: >> At this point I would be happy just to set the TOAST threshold to a >> value defined as optimal, rather than as the most minimal use of TOAST >> possible. > I agree that's a good startin

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Luke Lonergan
Hi Bruce, On 4/12/07 9:24 AM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > Luke Lonergan wrote: >> Hi Bruce, >> >> How about these: >> >> - Allow specification of TOAST size threshold in bytes on a per column basis >> - Enable storage of columns in separate TOAST tables >> - Enable use of multi

Re: [HACKERS] TOASTing smaller things

2007-04-12 Thread Bruce Momjian
Luke Lonergan wrote: > Hi Bruce, > > How about these: > > - Allow specification of TOAST size threshold in bytes on a per column basis > - Enable storage of columns in separate TOAST tables > - Enable use of multi-row compression method(s) for TOAST tables At this point I would be happy just to

Re: [HACKERS] TOASTing smaller things

2007-04-10 Thread Luke Lonergan
Hi Bruce, How about these: - Allow specification of TOAST size threshold in bytes on a per column basis - Enable storage of columns in separate TOAST tables - Enable use of multi-row compression method(s) for TOAST tables - Luke On 3/26/07 5:39 PM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: >

Re: [HACKERS] TOASTing smaller things

2007-03-26 Thread Bruce Momjian
Luke Lonergan wrote: > I advocate the following: > > - Enable specification of TOAST policy on a per column basis > > As a first step, then: > > - Enable vertical partitioning of tables using per-column specification of > storage policy. > How are these different from ALTER TABLE SET STORAGE?

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
Hi Hannu, On 3/22/07 3:21 PM, "Hannu Krosing" <[EMAIL PROTECTED]> wrote: > Maybe monetdb ( http://monetdb.cwi.nl/ ) can be of some inspiration ? > > btw, it may be a good idea to have a verion of bizgresMPP which has > monetdb as partition db, if monetdb is as efficient as they tell it is . Yep

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Hannu Krosing
Ühel kenal päeval, N, 2007-03-22 kell 10:19, kirjutas Luke Lonergan: > Andreas, > > On 3/22/07 9:40 AM, "Andreas Pflug" <[EMAIL PROTECTED]> wrote: > > > Wouldn't it be enough to enable having the toast table on a different > > table space? > > Yes, but the ultimate goal would allow the allocatio

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
Chris, > Hmm. Are you trying to do something sort of like CStore? > > http://db.csail.mit.edu/projects/cstore/ > > That seems to have some relevant ideas... I think "something like" is a good way to put it. As you know Cstore was a prototype for Vertica and these are in the same class as Syba

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Chris Browne
[EMAIL PROTECTED] ("Luke Lonergan") writes: > Andreas, > > On 3/22/07 9:40 AM, "Andreas Pflug" <[EMAIL PROTECTED]> wrote: > >> Wouldn't it be enough to enable having the toast table on a different >> table space? > > Yes, but the ultimate goal would allow the allocation of a storage mechanism > tha

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
Andreas, On 3/22/07 9:40 AM, "Andreas Pflug" <[EMAIL PROTECTED]> wrote: > Wouldn't it be enough to enable having the toast table on a different > table space? Yes, but the ultimate goal would allow the allocation of a storage mechanism that is unique to each column. The most frequently used mec

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Andreas Pflug
Luke Lonergan wrote: > I advocate the following: > > - Enable specification of TOAST policy on a per column basis > > As a first step, then: > > - Enable vertical partitioning of tables using per-column specification of > storage policy. > Wouldn't it be enough to enable having the toast table o

Re: [HACKERS] TOASTing smaller things

2007-03-22 Thread Luke Lonergan
I advocate the following: - Enable specification of TOAST policy on a per column basis As a first step, then: - Enable vertical partitioning of tables using per-column specification of storage policy. - Luke On 3/21/07 1:12 PM, "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > > Is this a TODO?

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Jim C. Nasby
On Wed, Mar 21, 2007 at 12:37:36PM -0400, Chris Browne wrote: > 4. A different mechanism would be to add a fifth storage column > strategy (the present four are PLAIN, EXTENDED, EXTERNAL, MAIN), let's > say, TOAST. > > At present, the 4 values are essentially advisory; columns get TOASTed > if th

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Chris Browne
[EMAIL PROTECTED] (Tom Lane) writes: > Chris Browne <[EMAIL PROTECTED]> writes: >> #define TOAST_DENOMINATOR 17 >>/* Use this as the divisor; current default behaviour falls from >> TOAST_DENOMINATOR = 4 */ > >> #define TOAST_TUPLE_THRESHOLD^I\ >> ^IMAXALIGN_DOWN((BLCKSZ - \ >> ^I^I^I^I MA

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Bruce Momjian
Is this a TODO? --- Jan Wieck wrote: > On 3/21/2007 2:05 PM, Tom Lane wrote: > > Chris Browne <[EMAIL PROTECTED]> writes: > >> #define TOAST_DENOMINATOR 17 > >>/* Use this as the divisor; current default behaviour fall

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Jan Wieck
On 3/21/2007 2:05 PM, Tom Lane wrote: Chris Browne <[EMAIL PROTECTED]> writes: #define TOAST_DENOMINATOR 17 /* Use this as the divisor; current default behaviour falls from TOAST_DENOMINATOR = 4 */ #define TOAST_TUPLE_THRESHOLD^I\ ^IMAXALIGN_DOWN((BLCKSZ - \ ^I^I^I^I MAXALIGN(sizeof(Pag

Re: [HACKERS] TOASTing smaller things

2007-03-21 Thread Tom Lane
Chris Browne <[EMAIL PROTECTED]> writes: > #define TOAST_DENOMINATOR 17 >/* Use this as the divisor; current default behaviour falls from > TOAST_DENOMINATOR = 4 */ > #define TOAST_TUPLE_THRESHOLD^I\ > ^IMAXALIGN_DOWN((BLCKSZ - \ > ^I^I^I^I MAXALIGN(sizeof(PageHeaderData) + 3 * sizeof(Ite

[HACKERS] TOASTing smaller things

2007-03-21 Thread Chris Browne
In some of our applications, we have cases where it would be very nice if we could activate TOAST at some sort of lower threshold than the usual 2K that is true now. Let me note the current code that controls the threshold: /* * These symbols control toaster activation. If a tuple is larger tha