On Wed, Dec 18, 2019 at 11:37 AM Robert Haas wrote:
> If nobody has further comments or objections, I plan to commit these
> in early January.
Done.
Which, I think, wraps up the work I felt needed to be done here.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreS
On Tue, Dec 17, 2019 at 4:12 PM Robert Haas wrote:
> Hearing no further comments, I went ahead and pushed 0002 today. That
> turned out to have a bug, so I pushed a fix for that. Hopefully the
> buildfarm will agree that it's fixed.
>
> Meanwhile, here are the remaining patches again, rebased over
On Fri, Nov 22, 2019 at 10:41 AM Robert Haas wrote:
> Thanks for the review. Updated patches attached. This version is more
> complete than the last set of patches I posted. It looks like this:
>
> 0001 - Lets a table AM that needs a toast table choose the AM that
> will be used to implement the
On Thu, Nov 21, 2019 at 5:37 AM Peter Eisentraut
wrote:
> Partial review: The 0001 patch seems very sensible. Some minor comments
> on that:
Thanks for the review. Updated patches attached. This version is more
complete than the last set of patches I posted. It looks like this:
0001 - Lets a t
On 2019-11-08 17:59, Robert Haas wrote:
OK. Could you see what you think of the attached patches? 0001 does
some refactoring of toast_fetch_datum() and toast_fetch_datum_slice()
to make them look more like each other and clean up a bunch of stuff
that I thought was annoying, and 0002 then pulls o
On Mon, Nov 11, 2019 at 8:51 AM Peter Eisentraut
wrote:
> Compared to the previous patch (v7) where the API just had a "use this
> AM for TOAST" field and the other extreme of pushing TOAST entirely
> inside the heap AM, this seems like the worst of both worlds, with the
> maximum additional compl
On 2019-11-08 17:59, Robert Haas wrote:
On Wed, Nov 6, 2019 at 12:00 PM Andres Freund wrote:
I'd like an AM to have the *option* of implementing something better, or
at least go in the direction of making that possible.
OK. Could you see what you think of the attached patches? 0001 does
some
Hi Craig,
Please find my response inline below.
On Sun, Nov 10, 2019 at 2:39 PM Craig Ringer wrote:
>
> On Thu, 7 Nov 2019 at 22:45, Ashutosh Sharma wrote:
>>
>
> In fact, I suspect this is PostgreSQL successfully protecting itself from an
> unsafe situation.
>
> Does the host have thin-provis
On Thu, 7 Nov 2019 at 22:45, Ashutosh Sharma wrote:
> On Thu, Nov 7, 2019 at 7:35 PM Robert Haas wrote:
> >
> > On Thu, Nov 7, 2019 at 1:15 AM Ashutosh Sharma
> wrote:
> > > @Robert, Myself and Prabhat have tried running the test-cases that
> > > caused the checkpointer process to crash earlier
On Wed, Nov 6, 2019 at 12:00 PM Andres Freund wrote:
> I'd like an AM to have the *option* of implementing something better, or
> at least go in the direction of making that possible.
OK. Could you see what you think of the attached patches? 0001 does
some refactoring of toast_fetch_datum() and t
On Thu, Nov 7, 2019 at 7:35 PM Robert Haas wrote:
>
> On Thu, Nov 7, 2019 at 1:15 AM Ashutosh Sharma wrote:
> > @Robert, Myself and Prabhat have tried running the test-cases that
> > caused the checkpointer process to crash earlier multiple times but we
> > are not able to reproduce it both with
On Thu, Nov 7, 2019 at 1:15 AM Ashutosh Sharma wrote:
> @Robert, Myself and Prabhat have tried running the test-cases that
> caused the checkpointer process to crash earlier multiple times but we
> are not able to reproduce it both with and without the patch. However,
> from the stack trace shared
On 2019-11-06 18:00, Andres Freund wrote:
I'd like an AM to have the *option* of implementing something better, or
at least go in the direction of making that possible.
I don't think the presented design prevents that. An AM can just return
false from relation_needs_toast_table in all cases a
On Thu, Nov 7, 2019 at 10:57 AM Prabhat Sahu
wrote:
>
>
>
> On Tue, Nov 5, 2019 at 4:48 PM Ashutosh Sharma wrote:
>>
>> From the stack trace shared by Prabhat, I understand that the checkpointer
>> process panicked due to one of the following two reasons:
>>
>> 1) The fsync() failed in the first
On Tue, Nov 5, 2019 at 4:48 PM Ashutosh Sharma
wrote:
> From the stack trace shared by Prabhat, I understand that the checkpointer
> process panicked due to one of the following two reasons:
>
> 1) The fsync() failed in the first attempt itself and the reason for the
> failure was not due to file
Hi,
On 2019-11-06 11:49:10 -0500, Robert Haas wrote:
> On Wed, Nov 6, 2019 at 11:25 AM Andres Freund wrote:
> > I think it's more than just that. It's also that I think presenting a
> > hardcoded value to the outside of / above an AM is architecturally
> > wrong. If anything this is an implementa
Hi,
On 2019-11-06 10:38:58 -0500, Robert Haas wrote:
> Yeah. I've been nervous about trying to proceed with this patch
> because Andres seemed confident there was a better approach than what
> I did here, but as I wrote about back on September 12th, it doesn't
> seem like his idea will work. I'm n
On Wed, Nov 6, 2019 at 11:25 AM Andres Freund wrote:
> I think it's more than just that. It's also that I think presenting a
> hardcoded value to the outside of / above an AM is architecturally
> wrong. If anything this is an implementation detail of the AM, that the
> AM ought to be concerned wit
Hi,
On 2019-11-06 10:01:40 +0100, Peter Eisentraut wrote:
> On 2019-10-04 20:32, Robert Haas wrote:
> > Here's the last patch back, rebased over that renaming. Although I
> > think that Andres (and Tom) are probably right that there's room for
> > improvement here, I currently don't see a way arou
On Wed, Nov 6, 2019 at 4:01 AM Peter Eisentraut
wrote:
> This patch seems sound as far as the API restructuring goes.
Thanks. And thanks for weighing in.
> If I may summarize the remaining discussion: This patch adds a field
> toast_max_chunk_size to TableAmRoutine, to take the place of the
> h
On 2019-10-04 20:32, Robert Haas wrote:
Here's the last patch back, rebased over that renaming. Although I
think that Andres (and Tom) are probably right that there's room for
improvement here, I currently don't see a way around the issues I
wrote about
inhttp://postgr.es/m/ca+tgmoa0zfcacpojcssp
>From the stack trace shared by Prabhat, I understand that the checkpointer
process panicked due to one of the following two reasons:
1) The fsync() failed in the first attempt itself and the reason for the
failure was not due to file being dropped or truncated i.e. fsync failed
with the error oth
On Wed, Oct 30, 2019 at 9:46 PM Robert Haas wrote:
> On Wed, Oct 30, 2019 at 3:49 AM Prabhat Sahu <
> prabhat.s...@enterprisedb.com> wrote:
>
>> While testing the Toast patch(PG+v7 patch) I found below server crash.
>> System configuration:
>> VCPUs: 4, RAM: 8GB, Storage: 320GB
>>
>> This issue i
On Wed, Oct 30, 2019 at 3:49 AM Prabhat Sahu
wrote:
> While testing the Toast patch(PG+v7 patch) I found below server crash.
> System configuration:
> VCPUs: 4, RAM: 8GB, Storage: 320GB
>
> This issue is not frequently reproducible, we need to repeat the same
> testcase multiple times.
>
I wonde
Hi All,
While testing the Toast patch(PG+v7 patch) I found below server crash.
System configuration:
VCPUs: 4, RAM: 8GB, Storage: 320GB
This issue is not frequently reproducible, we need to repeat the same
testcase multiple times.
CREATE OR REPLACE FUNCTION toast_chunks_cnt_func(p1 IN text)
RE
On Fri, Sep 6, 2019 at 10:59 AM Robert Haas wrote:
> On Thu, Sep 5, 2019 at 4:07 PM Andres Freund wrote:
> > Yea, makes sense to me.
>
> OK, done. Here's the remaining patches again, with a slight update to
> the renaming patch (now 0002). In the last version, I renamed
> toast_insert_or_update
On Fri, Aug 2, 2019 at 6:42 PM Andres Freund wrote:
> The obvious problem with this is the slice fetching logic. For slices
> with an offset of 0, it's obviously trivial to implement. For the higher
> slice logic, I'd assume we'd have to fetch the first slice by estimating
> where the start chunk
On Thu, Sep 5, 2019 at 4:07 PM Andres Freund wrote:
> Yea, makes sense to me.
OK, done. Here's the remaining patches again, with a slight update to
the renaming patch (now 0002). In the last version, I renamed
toast_insert_or_update to heap_toast_insert_or_update but did not
rename toast_delete
On 2019-09-05 15:27:28 -0400, Robert Haas wrote:
> On Thu, Sep 5, 2019 at 3:10 PM Andres Freund wrote:
> > On 2019-09-05 13:42:40 -0400, Robert Haas wrote:
> > > Done, thanks. Here's the rest again with the additional rename added
> > > to 0003 (formerly 0004). I think it's probably OK to go ahead
On Thu, Sep 5, 2019 at 3:36 PM Tom Lane wrote:
> Andres Freund writes:
> > Well, I still dislike making the toast chunk size configurable in a
> > halfhearted manner.
>
> It's hard to make it fully configurable without breaking our on-disk
> storage, because of the lack of any explicit representa
Andres Freund writes:
> Well, I still dislike making the toast chunk size configurable in a
> halfhearted manner.
It's hard to make it fully configurable without breaking our on-disk
storage, because of the lack of any explicit representation of the chunk
size in TOAST data. You have to "just kn
On Thu, Sep 5, 2019 at 3:10 PM Andres Freund wrote:
> On 2019-09-05 13:42:40 -0400, Robert Haas wrote:
> > Done, thanks. Here's the rest again with the additional rename added
> > to 0003 (formerly 0004). I think it's probably OK to go ahead with
> > that stuff, too, but I'll wait a bit to see if
Hi,
On 2019-09-05 13:42:40 -0400, Robert Haas wrote:
> Done, thanks. Here's the rest again with the additional rename added
> to 0003 (formerly 0004). I think it's probably OK to go ahead with
> that stuff, too, but I'll wait a bit to see if anyone wants to raise
> more objections.
Well, I still
On Thu, Sep 5, 2019 at 10:52 AM Alvaro Herrera wrote:
> I agree, and can we move forward with this 0001? The idea here is to
> change no code (as also suggested by Tom elsewhere), and it's the
> largest patch in this series by a mile. I checked --color-moved=zebra
> and I think the patch looks f
Hi,
On 2019-08-05 15:36:59 -0400, Robert Haas wrote:
> On Fri, Aug 2, 2019 at 6:42 PM Andres Freund wrote:
> > Hm. This leaves toast_insert_or_update() as a name. That makes it sound
> > like it's generic toast code, rather than heap specific?
>
> I'll rename it to heap_toast_insert_or_update().
Robert Haas writes:
> I don't feel entirely convinced that there's any rush to do all of
> this right now, and the more I change the harder it is to make sure
> that I haven't broken anything. How strongly do you feel about this
> stuff?
FWIW, I agree with your comment further up that this patch
On Fri, Aug 2, 2019 at 6:42 PM Andres Freund wrote:
> Hm, those all include writing, right? And for read-only we don't expect
> any additional overhead, correct? The write overhead is probably too
> large show a bit of function call overhead - but if so, it'd probably be
> on unlogged tables? And
Hi,
On 2019-08-01 12:23:42 -0400, Robert Haas wrote:
> Roughly, on both master and with the patches, the first one takes
> about 4.2 seconds, the second 7.5, and the third 1.2. The third one
> is the fastest because it doesn't do any compression. Since it does
> less irrelevant work than the oth
On Thu, Aug 1, 2019 at 1:53 PM Andres Freund wrote:
> Could you wait until I either had a chance to look again, or until, say,
> Monday if I don't get around to it? I'll try to get to it earlier than
> that.
Sure, no problem.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterpr
Hi,
On 2019-08-01 12:23:42 -0400, Robert Haas wrote:
> Barring objections, I plan to commit the whole series of patches here
> (latest rebase attached). They are not perfect and could likely be
> improved in various ways, but I think they are an improvement over
> what we have now, and it's not l
On Mon, Jul 8, 2019 at 9:06 PM Robert Haas wrote:
> On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu
> wrote:
> > I have tested the TOAST patches(v3) with different storage options
> like(MAIN, EXTERNAL, EXTENDED, etc.), and
> > combinations of compression and out-of-line storage options.
> > I have
On Tue, Jun 25, 2019 at 2:19 AM Prabhat Sahu
wrote:
> I have tested the TOAST patches(v3) with different storage options like(MAIN,
> EXTERNAL, EXTENDED, etc.), and
> combinations of compression and out-of-line storage options.
> I have used a few dummy tables with various tuple count say 10k, 20
On Wed, Jun 12, 2019 at 4:17 AM Robert Haas wrote:
> On Tue, May 21, 2019 at 2:10 PM Robert Haas wrote:
> > Updated and rebased patches attached.
>
> And again.
Hi Robert,
Thus spake GCC:
detoast.c: In function ‘toast_fetch_datum’:
detoast.c:308:12: error: variable ‘toasttupDesc’ set but not u
On Tue, Jun 11, 2019 at 9:47 PM Robert Haas wrote:
> On Tue, May 21, 2019 at 2:10 PM Robert Haas wrote:
> > Updated and rebased patches attached.
>
> And again.
>
Hi Robert,
I have tested the TOAST patches(v3) with different storage options
like(MAIN, EXTERNAL, EXTENDED, etc.), and
combination
44 matches
Mail list logo
