Re: Undo logs

2019-12-02 Thread Robert Haas
On Sat, Nov 30, 2019 at 9:25 PM Michael Paquier wrote: > On Thu, Feb 07, 2019 at 07:35:31AM +0530, Andres Freund wrote: > > It was JUST added ... :) thought I saw you reply on the other thread > > about it, but I was wrong... > > Six months later without any activity, I am marking this entry as >

Re: Undo logs

2019-11-30 Thread Michael Paquier
On Thu, Feb 07, 2019 at 07:35:31AM +0530, Andres Freund wrote: > It was JUST added ... :) thought I saw you reply on the other thread > about it, but I was wrong... Six months later without any activity, I am marking this entry as returned with feedback. The latest patch set does not apply anymo

Re: Undo logs

2019-02-06 Thread Andres Freund
On February 7, 2019 7:34:11 AM GMT+05:30, Michael Paquier wrote: >On Thu, Feb 07, 2019 at 07:25:57AM +0530, Andres Freund wrote: >> We now have the target version as a field, that should make such >> moves unnecessary, right? > >Oh, I missed this stuff. Thanks for pointing it out. It was JUS

Re: Undo logs

2019-02-06 Thread Michael Paquier
On Thu, Feb 07, 2019 at 07:25:57AM +0530, Andres Freund wrote: > We now have the target version as a field, that should make such > moves unnecessary, right? Oh, I missed this stuff. Thanks for pointing it out. -- Michael signature.asc Description: PGP signature

Re: Undo logs

2019-02-06 Thread Andres Freund
On February 7, 2019 7:21:49 AM GMT+05:30, Michael Paquier wrote: >On Thu, Feb 07, 2019 at 03:21:09AM +1100, Thomas Munro wrote: >> Correct. Originally the target was 12 but that was a bit too >ambitious. > >Could it be possible to move the patch set into the first PG-13 commit >fest then? We

Re: Undo logs

2019-02-06 Thread Michael Paquier
On Thu, Feb 07, 2019 at 03:21:09AM +1100, Thomas Munro wrote: > Correct. Originally the target was 12 but that was a bit too ambitious. Could it be possible to move the patch set into the first PG-13 commit fest then? We could use this CF as recipient for now, even if the schedule for next devel

Re: Undo logs

2019-02-06 Thread Thomas Munro
On Thu, Feb 7, 2019 at 1:16 AM Alvaro Herrera wrote: > On 2019-Feb-04, Thomas Munro wrote: > > On Mon, Feb 4, 2019 at 3:55 PM Michael Paquier wrote: > > > On Sun, Feb 03, 2019 at 02:23:16AM -0800, Andres Freund wrote: > > > > This thread is curently marked as returned with feedback, set so > > >

Re: Undo logs

2019-02-06 Thread Alvaro Herrera
On 2019-Feb-04, Thomas Munro wrote: > On Mon, Feb 4, 2019 at 3:55 PM Michael Paquier wrote: > > On Sun, Feb 03, 2019 at 02:23:16AM -0800, Andres Freund wrote: > > > This thread is curently marked as returned with feedback, set so > > > 2018-12-01. Given there've been several new versions submitte

Re: Undo logs

2019-02-03 Thread Dilip Kumar
On Sun, Feb 3, 2019 at 3:53 PM Andres Freund wrote: > > Hi, > > This thread is curently marked as returned with feedback, set so > 2018-12-01. Given there've been several new versions submitted since, is > that accurate? > As part of this thread we have been reviewing and fixing the comment for un

Re: Undo logs

2019-02-03 Thread Thomas Munro
On Mon, Feb 4, 2019 at 3:55 PM Michael Paquier wrote: > On Sun, Feb 03, 2019 at 02:23:16AM -0800, Andres Freund wrote: > > This thread is curently marked as returned with feedback, set so > > 2018-12-01. Given there've been several new versions submitted since, is > > that accurate? > > From the l

Re: Undo logs

2019-02-03 Thread Michael Paquier
On Sun, Feb 03, 2019 at 02:23:16AM -0800, Andres Freund wrote: > This thread is curently marked as returned with feedback, set so > 2018-12-01. Given there've been several new versions submitted since, is > that accurate? From the latest status of this thread, there have been new patches but no re

Re: Undo logs

2019-02-03 Thread Andres Freund
Hi, This thread is curently marked as returned with feedback, set so 2018-12-01. Given there've been several new versions submitted since, is that accurate? - Andres

Re: Undo logs

2019-01-09 Thread Dilip Kumar
On Wed, Jan 9, 2019 at 11:40 AM Dilip Kumar wrote: > On Tue, Jan 8, 2019 at 2:11 PM Amit Kapila > wrote: > >> >> >> 3. >> + work_txn.urec_next = uur->uur_next; >> + work_txn.urec_xidepoch = uur->uur_xidepoch; >> + work_txn.urec_progress = uur->uur_progress; >> + work_txn.urec_prevurp = uur->uur_

Re: Undo logs

2019-01-08 Thread Dilip Kumar
On Tue, Jan 8, 2019 at 2:11 PM Amit Kapila wrote: > > Few more comments: > > Few comments: > > 1. > + * undorecord.c > + * encode and decode undo records > + * > + * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group > > Chang

Re: Undo logs

2019-01-08 Thread Amit Kapila
On Sat, Jan 5, 2019 at 11:29 AM Dilip Kumar wrote: > > On Tue, Jan 1, 2019 at 4:37 PM Amit Kapila wrote: > > > > Thanks, the new changes look mostly okay to me, but I have few comments: > > 1. > > + /* > > + * WAL log, for log switch. This is required to identify the log switch > > + * during re

Re: Undo logs

2019-01-01 Thread Amit Kapila
On Sun, Dec 23, 2018 at 3:49 PM Dilip Kumar wrote: > > On Wed, Dec 12, 2018 at 3:03 PM Amit Kapila wrote: > > For addressing these issues related to multilog I have changed the > design as we discussed offlist. > 1) Now, at Do time we identify the log switch as you mentioned above > (identify whi

Re: Undo logs

2018-12-12 Thread Amit Kapila
On Wed, Dec 12, 2018 at 11:18 AM Dilip Kumar wrote: > > On Sat, Dec 8, 2018 at 7:52 PM Amit Kapila wrote: >> >> >> I think I see the problem in the discard mechanism when the log is >> spread across multiple logs. Basically, if the second log contains >> undo of some transaction prior to the tra

Re: Undo logs

2018-12-11 Thread Dilip Kumar
On Sat, Dec 8, 2018 at 7:52 PM Amit Kapila wrote: > On Tue, Dec 4, 2018 at 3:00 PM Dilip Kumar wrote: > > > > On Sat, Dec 1, 2018 at 12:58 PM Amit Kapila > wrote: > > > > > > > > > 13. > > > PrepareUndoInsert() > > > { > > > .. > > > if (!UndoRecPtrIsValid(prepared_urec_ptr)) > > > + urecptr =

Re: Undo logs

2018-12-08 Thread Amit Kapila
On Tue, Dec 4, 2018 at 3:00 PM Dilip Kumar wrote: > > On Sat, Dec 1, 2018 at 12:58 PM Amit Kapila wrote: > > > > > > 13. > > PrepareUndoInsert() > > { > > .. > > if (!UndoRecPtrIsValid(prepared_urec_ptr)) > > + urecptr = UndoRecordAllocate(urec, 1, upersistence, txid); > > + else > > + urecptr =

Re: Undo logs

2018-11-30 Thread Amit Kapila
On Thu, Nov 29, 2018 at 6:27 PM Dilip Kumar wrote: > > On Mon, Nov 26, 2018 at 2:13 PM Amit Kapila wrote: > > > > 10. > > + if (!UndoRecPtrIsValid(multi_prep_urp)) > > + urecptr = UndoRecordAllocateMulti(urec, 1, upersistence, txid); > > + else > > + urecptr = multi_prep_urp; > > + > > + size = U

Re: Undo logs

2018-11-29 Thread Dilip Kumar
On Mon, Nov 26, 2018 at 2:13 PM Amit Kapila wrote: > > On Tue, Nov 20, 2018 at 7:37 PM Dilip Kumar wrote: > > Along with that I have merged latest changes in zheap branch committed > > by Rafia Sabih for cleaning up the undo buffer information in abort > > path. > > > > Thanks, few more comments:

Re: Undo logs

2018-11-26 Thread Amit Kapila
On Tue, Nov 20, 2018 at 7:37 PM Dilip Kumar wrote: > Along with that I have merged latest changes in zheap branch committed > by Rafia Sabih for cleaning up the undo buffer information in abort > path. > Thanks, few more comments: 1. @@ -2627,6 +2653,7 @@ AbortTransaction(void) AtEOXact_HashTa

Re: Undo logs

2018-11-20 Thread Dilip Kumar
On Sat, Nov 17, 2018 at 5:12 PM Amit Kapila wrote: > > On Fri, Nov 16, 2018 at 9:46 AM Dilip Kumar wrote: > > > > On Thu, Nov 15, 2018 at 12:14 PM Dilip Kumar wrote: > > > > > Updated patch (merged latest code from the zheap main branch [1]). > > > > Review comments: > --

Re: Undo logs

2018-11-17 Thread Amit Kapila
On Fri, Nov 16, 2018 at 9:46 AM Dilip Kumar wrote: > > On Thu, Nov 15, 2018 at 12:14 PM Dilip Kumar wrote: > > > Updated patch (merged latest code from the zheap main branch [1]). > Review comments: --- 1. UndoRecordPrepareTransInfo() { .. + /* + * The absence of prev

Re: Undo logs

2018-11-15 Thread Dilip Kumar
On Thu, Nov 15, 2018 at 12:14 PM Dilip Kumar wrote: > Updated patch (merged latest code from the zheap main branch [1]). The main chain is related to removing relfilenode and tablespace id from the undo record and store reloid. Earlier, we kept it thinking that we will perform rollback without dat

Re: Undo logs

2018-11-14 Thread Dilip Kumar
On Wed, Nov 14, 2018 at 3:48 PM Dilip Kumar wrote: > > On Wed, Nov 14, 2018 at 2:58 PM Amit Kapila wrote: > > I think you can keep it with XXX instead of Fixme as there is nothing to > > fix. > Changed > > > > Both the patches 0003-undo-interface-v4.patch and > > 0004-undo-interface-test-v4.patc

Re: Undo logs

2018-11-14 Thread Dilip Kumar
On Wed, Nov 14, 2018 at 2:58 PM Amit Kapila wrote: > I think you can keep it with XXX instead of Fixme as there is nothing to fix. Changed > > Both the patches 0003-undo-interface-v4.patch and > 0004-undo-interface-test-v4.patch appears to be same except for the > name? My bad, please find the upd

Re: Undo logs

2018-11-14 Thread Amit Kapila
On Wed, Nov 14, 2018 at 2:42 PM Dilip Kumar wrote: > > On Sat, Nov 10, 2018 at 9:27 AM Amit Kapila wrote: > > > > On Mon, Nov 5, 2018 at 5:10 PM Dilip Kumar wrote: > > > > > [review for undo record layer (0003-undo-interface-v3)] > > > > I might sound repeating myself, but just to be clear, I wa

Re: Undo logs

2018-11-13 Thread Dilip Kumar
On Mon, Nov 5, 2018 at 5:09 PM Dilip Kumar wrote: > > On Mon, Sep 3, 2018 at 11:26 AM Dilip Kumar wrote: > > Thomas has already posted the latest version of undo log patches on > 'Cleaning up orphaned files using undo logs' thread[1]. So I have > rebased the undo-interface patch also. This patc

Re: Undo logs

2018-11-13 Thread Amit Kapila
On Wed, Oct 17, 2018 at 3:27 PM Amit Kapila wrote: > > On Mon, Oct 15, 2018 at 6:09 PM Amit Kapila wrote: > > > > On Sun, Sep 2, 2018 at 12:19 AM Thomas Munro > > wrote: > > > I have also pushed a new WIP version of the lower level undo log > > > storage layer patch set to a public branch[1]. I

Re: Undo logs

2018-11-09 Thread Amit Kapila
On Mon, Nov 5, 2018 at 5:10 PM Dilip Kumar wrote: > [review for undo record layer (0003-undo-interface-v3)] I might sound repeating myself, but just to be clear, I was involved in the design of this patch as well and I have given a few high-level inputs for this patch. I have used this interface

Re: Undo logs

2018-10-17 Thread Amit Kapila
On Mon, Oct 15, 2018 at 6:09 PM Amit Kapila wrote: > > On Sun, Sep 2, 2018 at 12:19 AM Thomas Munro > wrote: > > I have also pushed a new WIP version of the lower level undo log > > storage layer patch set to a public branch[1]. I'll leave the earlier > > branch[2] there because the record-level

Re: Undo logs

2018-10-15 Thread Amit Kapila
On Sun, Sep 2, 2018 at 12:19 AM Thomas Munro wrote: > > On Fri, Aug 31, 2018 at 10:24 PM Dilip Kumar > wrote: > > On Fri, Aug 31, 2018 at 3:08 PM, Dilip Kumar wrote: > > > As Thomas has already mentioned upthread that we are working on an > > > undo-log based storage and he has posted the patch

Re: Undo logs

2018-09-02 Thread Dilip Kumar
On Sun, Sep 2, 2018 at 12:18 AM, Thomas Munro wrote: > On Fri, Aug 31, 2018 at 10:24 PM Dilip Kumar > wrote: >> On Fri, Aug 31, 2018 at 3:08 PM, Dilip Kumar wrote: >> > As Thomas has already mentioned upthread that we are working on an >> > undo-log based storage and he has posted the patch sets

Re: Undo logs

2018-09-01 Thread Thomas Munro
On Fri, Aug 31, 2018 at 10:24 PM Dilip Kumar wrote: > On Fri, Aug 31, 2018 at 3:08 PM, Dilip Kumar wrote: > > As Thomas has already mentioned upthread that we are working on an > > undo-log based storage and he has posted the patch sets for the lowest > > layer called undo-log-storage. > > > > Th

Re: Undo logs

2018-08-31 Thread Dilip Kumar
On Fri, Aug 31, 2018 at 3:08 PM, Dilip Kumar wrote: > Hello hackers, > > As Thomas has already mentioned upthread that we are working on an > undo-log based storage and he has posted the patch sets for the lowest > layer called undo-log-storage. > > This is the next layer which sits on top of the

Re: Undo logs

2018-08-31 Thread Dilip Kumar
Hello hackers, As Thomas has already mentioned upthread that we are working on an undo-log based storage and he has posted the patch sets for the lowest layer called undo-log-storage. This is the next layer which sits on top of the undo log storage, which will provide an interface for prepare, in

Re: Undo logs

2018-05-30 Thread Thomas Munro
Hi Simon, On Mon, May 28, 2018 at 11:40 PM, Simon Riggs wrote: > On 24 May 2018 at 23:22, Thomas Munro wrote: >> The lowest level piece of this work is a physical undo log manager, > >> 1. Efficient appending of new undo data from many concurrent >> backends. Like logs. >> 2. Efficient discar

Re: Undo logs

2018-05-28 Thread Simon Riggs
On 24 May 2018 at 23:22, Thomas Munro wrote: > As announced elsewhere[1][2][3], at EnterpriseDB we are working on a > proposal to add in-place updates with undo logs to PostgreSQL. The > goal is to improve performance and resource usage by recycling space > better. Cool > The lowest level piec

Re: Undo logs

2018-05-27 Thread James Sewell
Exciting stuff! Really looking forward to having a play with this. James Sewell, *Chief Architect* Suite 112, Jones Bay Wharf, 26-32 Pirrama Road, Pyrmont NSW 2009 *P *(+61) 2 8099 9000 <(+61)%202%208099%209000> *W* www.jirotech.com *F * (+61) 2 8099 9099 <(+61)%202%208099%209000> On 25 May