On Tue, May 11, 2021 at 09:26:36AM +0530, Amit Kapila wrote: > On Mon, May 10, 2021 at 7:30 PM Justin Pryzby <pry...@telsasoft.com> wrote: > > > > Can these be merged: > > Allow logical replication to stream long transactions to standbys (Dilip > > Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > Improve the logical replication API to allow streaming large in-progress > > transactions (Tomas Vondra, Dilip Kumar, Amit Kapila) > > > > I think we should merge the below two: > Add support for streaming to built-in logical replication (Vondra, > Dilip Kumar, Amit Kapila, Ajin Cherian, Nikhil Sontakke, Stas Kelvich) > > Allow logical replication to stream long transactions to standbys > (Dilip Kumar, Tomas Vondra, Amit Kapila, Nikhil Sontakke) > > and say something like: > Allow logical replication to stream long in-progress transactions to standbys > Previously transactions that exceeded logical_decoding_work_mem were > written to disk until the transaction completed.
OK, updated as you suggested. > Also, the commits that lead to the above feature is: > 2020-09-03 [464824323] Add support for streaming to built-in logical > replicatio > 2020-08-08 [7259736a6] Implement streaming mode in ReorderBuffer. Yes, these were already listed above that item. > * Improve the logical replication API to allow streaming large > in-progress transactions (Tomas Vondra, Dilip Kumar, Amit Kapila) > The output functions begin with "stream". test_decoding also supports these. > > In the above, I think it is better to say Enhance instead of Improve. OK, word changed. > * Add system view pg_stat_replication_slots to report replication slot > activity (Sawada Masahiko, Amit Kapila) > > I think for this item, we can give Vignesh C as author credit because > he helped a lot enhance/stabilize this feature after the initial > commit. See > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=f5fc2f5b23d1b1dff60f8ca5dc211161df47eda4 > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=cca57c1d9bf7eeba5b81115e0b82651cf3d8e4ea > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=592f00f8dec68038301467a904ac514eddabf6cd OK, I have added him: Add system view pg_stat_replication_slots to report replication slot activity (Sawada Masahiko, Amit Kapila, Vignesh C) > * Allow multiple xacts during table sync in logical replication (Peter > Smith, Amit Kapila, and Takamichi Osumi) > > IMPORTANT? > > Yes, this is a big enough improvement in the initial table sync > mechanism of logical replication that it is worth mentioning in > Release Notes. One of the main user-visible effects this commit can > have is that it will reduce the time for which we need to hold WAL on > the server during the initial table sync phase. Then it can avoid > large copies in case there is any error during the time when > apply-worker and tablesync-worker synchronizes the apply location. OK, thanks. > * Enable logical replication to handle two phase commits (Ajin Cherian) > This is controlled via pg_create_logical_replication_slot(). > > We still haven't supported two-phase commits for built-in logical > replication but we have exposed APIs for decoding plugins to achieve > that. Using these APIs, we are planning to support it for built-in > logical replication but that will be probably done in PG-15. So, I > suggest we can modify it to something like: > > * Enhance logical decoding APIs to handle two-phase commits > This is controlled via pg_create_logical_replication_slot(). OK, updated as you suggested. > Most of the work for this feature is done via commits: > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=a271a1b50e9bec07e2ef3a05e38e7285113e4ce6 > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=0aa8a01d04c8fe200b7a106878eebc3d0af9105c > > So, I think the author credit should be given to Ajin Cherian, Amit > Kapila, Nikhil Sontakke, and Stas Kelvich OK, the new item text is: <listitem> <!-- Author: Amit Kapila <akap...@postgresql.org> 2020-12-30 [0aa8a01d0] Extend the output plugin API to allow decoding of prepar Author: Amit Kapila <akap...@postgresql.org> 2021-01-04 [a271a1b50] Allow decoding at prepare time in ReorderBuffer. Author: Amit Kapila <akap...@postgresql.org> 2021-03-03 [19890a064] Add option to enable two_phase commits via pg_create_log --> <para> Enhance logical decoding APIs to handle two-phase commits (Ajin Cherian, Amit Kapila, Nikhil Sontakke, Stas Kelvich) </para> <para> This is controlled via pg_create_logical_replication_slot(). </para> </listitem> > * I think we should mention about commit: > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=d7eb52d7181d83cf2363570f7a205b8eb1008dbc > > This allows Logical decoding to work efficiently in presence of a > large amount of DDLs. This was a complaint from many users for long > time. Apart from the discussion thread, see some of the earlier > complaints: > https://www.postgresql.org/message-id/flat/20f3de7675f83176253f607b5e199b228406c21c.ca...@cybertec.at > https://www.postgresql.org/message-id/flat/CAHoiPjzea6N0zuCi=+f9v_j94nfsy6y8SU7-=bp4=7qw6_i=r...@mail.gmail.com > > and most recently at: > https://www.postgresql.org/message-id/DB8PR10MB36126BC0DC3570637FDFED12EB4F9%40DB8PR10MB3612.EURPRD10.PROD.OUTLOOK.COM OK, I added this: <listitem> <!-- Author: Amit Kapila <akap...@postgresql.org> 2020-10-15 [d7eb52d71] Execute invalidation messages for each XLOG_XACT_INVALID --> <para> Allow logical decoding to more efficently process cache invalidation messages </para> <para> This allows Logical decoding to work efficiently in presence of a large amount of DDL. </para> </listitem> > * Speed truncation of small tables on large shared buffer servers (Kirk > Jamison) > > This optimization applies during recovery. Shouldn't we mention that > in some way? Yes, new text: Speed truncation of small tables during recovery on clusters with a large number of shared buffers (Kirk Jamison) -- Bruce Momjian <br...@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
