On Tue, Jul 6, 2021 at 4:21 AM Amit Kapila <amit.kapil...@gmail.com> wrote: > > On Fri, Jul 2, 2021 at 8:29 PM Simon Riggs <simon.ri...@enterprisedb.com> > wrote: > > > > On Fri, Jul 2, 2021 at 5:34 AM Craig Ringer > > <craig.rin...@enterprisedb.com> wrote: > > > > > > > > If you don't think the sorts of use cases I presented are worth the > > > trouble that's fair enough. I'm not against adding it on the commit > > > record. It's just that with logical decoding moving toward a streaming > > > model I suspect only having it at commit time may cause us some pain > > > later. > > > > I think you have some good ideas about how to handle larger > > transactions with streaming. As a separate patch it might be worth > > keeping track of transaction size and logging something when a > > transaction gets too large. > > > > If we want this additional information for streaming mode in logical > replication then can't we piggyback it on the very first record > written for a transaction when this info is required? Currently, we do > something similar for logging top_level_xid for subtransaction in > XLogRecordAssemble().
It's possible, but I'm struggling to believe anybody would accept that as an approach because it breaks simplicity, modularity and makes it harder to search for this info in the WAL. I was imagining that we'd keep track of amount of WAL written by a transaction and when it reaches a certain size generate a "streaming info" record as an early warning that we have a big transaction coming down the pipe. I'm feeling that a simple patch is expanding well beyond its original scope and timeline. How can we do this simply? -- Simon Riggs http://www.EnterpriseDB.com/
