Hello All,
Im working on my postgres FDW extension, to support logical replication I
need to use Custom WAL resource manager. In postgres extensions have the
flexibility to register their resource managers in RmgrTable[]. But Like
RmgrTable[] we have another resource manager related table RmgrDesc
On Tue, 2023-02-14 at 19:09 +0530, Pradeep Kumar wrote:
> GetRmgrDesc() are widely used in XLogDumpDisplayRecord() and
> XLogDumpDisplayStats() in pg_waldump.c. In function GetRmgrDesc() for
> custom resource managers by the default they are assign
> 1) rm_desc = default_desc
> 2) r
Jeff Davis writes:
> I'm happy with this patch and I committed it.
wrasse is less happy:
ccache /opt/developerstudio12.6/bin/cc -m64 -Xa -v -O findtimezone.o initdb.o
localtime.o -L../../../src/port -L../../../src/common -L../../../src/fe_utils
-lpgfeutils -L../../../src/common -lpgcommon -L
On Thu, Apr 7, 2022 at 11:45 AM Jeff Davis wrote:
> I'm happy with this patch and I committed it. Changes from last
> version:
Hi Jeff, I think there's a typo [1] in pg_waldump.c, we might miss to
take account of the last custom resource mgr stats. Please fix it.
diff --git a/src/bin/pg_waldump/
Hi,
On 2022-04-07 00:45:34 -0700, Andres Freund wrote:
> On 2022-04-06 23:56:40 -0700, Andres Freund wrote:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=wrasse&dt=2022-04-07%2006%3A49%3A14
>
> ccache /opt/developerstudio12.6/bin/cc -m64 -Xa -v -O findtimezone.o
> initdb.o localtim
Hi,
On 2022-04-06 23:56:40 -0700, Andres Freund wrote:
> On 2022-04-06 23:35:05 -0700, Andres Freund wrote:
> > Causes plenty new warnings here:
>
> And my machine isn't alone. There's also:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lapwing&dt=2022-04-07%2006%3A40%3A14
>
> rmgrde
Hi,
On 2022-04-06 23:35:05 -0700, Andres Freund wrote:
> Causes plenty new warnings here:
And my machine isn't alone. There's also:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lapwing&dt=2022-04-07%2006%3A40%3A14
rmgrdesc.c:44:1: error: missing braces around initializer
[-Werror=mis
Hi,
On 2022-04-06 23:15:27 -0700, Jeff Davis wrote:
> I'm happy with this patch and I committed it.
That caused breakage with WAL_DEBUG enabled. Fixed that. But it was
half-broken before, at least since 70e81861fad, because 'rmid' didn't refer to
the current record's rmid anymore, but to rmid fro
On Tue, 2022-04-05 at 15:12 +0530, Bharath Rupireddy wrote:
> Yes, I meant this. If we do this, then a global variable
> current_max_rmid can be maintained (by RegisterCustomRmgr) and the
> other functions RmgrStartup, RmgrCleanup and RegisterCustomRmgr can
> avoid for-loops with full range RM_MAX_
On Tue, Apr 5, 2022 at 5:55 AM Jeff Davis wrote:
>
> On Mon, 2022-04-04 at 11:15 +0530, Bharath Rupireddy wrote:
> > 1) Why can't rmid be chosen by the extensions in sequential order
> > from
> > (129 till 255), say, to start with a columnar extension can choose
> > 129, another extension can regi
On Mon, 2022-04-04 at 11:15 +0530, Bharath Rupireddy wrote:
> 1) Why can't rmid be chosen by the extensions in sequential order
> from
> (129 till 255), say, to start with a columnar extension can choose
> 129, another extension can register 130 and so on right?
I'm not sure what you mean by "chos
On Sun, 2022-04-03 at 21:33 -0700, Andres Freund wrote:
> I still think the easiest and fastest would be to just make RmgrTable
> longer,
> and not const. When registering, copy the caller provided struct into
> the
> respective RmgrData element. Yes, we'd waste a bit of space at the
> end of the
On Mon, Apr 4, 2022 at 8:33 AM Jeff Davis wrote:
>
> I still may change it to go back to two RmgrTables (one for builtin and
> one for custom) to remove the lingering performance doubts. Other than
> that, and some cleanup, this is pretty close to the version I intend to
> commit.
I just had a qu
Hi,
On 2022-03-31 14:20:51 -0700, Jeff Davis wrote:
> If you are still concerned, I can switch back to separate tables to
> eliminate the indirection for built-in rmgrs. Separate rmgr tables
> still require a branch (to decide which table to access), but it should
> be a highly predictable one.
I
On Mon, 2022-03-28 at 11:00 -0700, Andres Freund wrote:
> > +/*
> > + * Start up all resource managers.
> > + */
> > +void
> > +StartupResourceManagers()
>
> (void)
Fixed.
> Random idea: Might be worth emitting the id->name mapping just after
> a redo
> location is determined, to make it easier
On Mon, 2022-03-28 at 11:00 -0700, Andres Freund wrote:
> I think this has been discussed before, but to me it's not obvious
> that it's a
> good idea to change RmgrTable from RmgrData to RmgrData *. That adds
> an
> indirection, without obvious benefit.
I did some performance tests. I created a n
Hi,
On 2022-03-23 21:43:08 -0700, Jeff Davis wrote:
> /* must be kept in sync with RmgrData definition in xlog_internal.h */
> #define PG_RMGR(symname,name,redo,desc,identify,startup,cleanup,mask,decode)
> \
> - { name, redo, desc, identify, startup, cleanup, mask, decode },
> + &(struc
On Fri, 2022-02-04 at 22:56 +0800, Julien Rouhaud wrote:
> > Right, which I guess raises another question: if the maximum is
> > UINT8_MAX, which BTW I find perfectly reasonable, why are we not
> > just
> > defining this as an array of size 256? There's no point in adding
> > code
> > complexity to
On Fri, Feb 04, 2022 at 09:53:09AM -0500, Robert Haas wrote:
> On Fri, Feb 4, 2022 at 9:48 AM Julien Rouhaud wrote:
> > I guess the idea was to have a compromise between letting rmgr authors
> > choose
> > arbitrary ids to avoid any conflicts, especially with private
> > implementations,
> > wit
On Fri, Feb 4, 2022 at 9:48 AM Julien Rouhaud wrote:
> I guess the idea was to have a compromise between letting rmgr authors choose
> arbitrary ids to avoid any conflicts, especially with private implementations,
> without wasting too much memory. But those approaches would be pretty much
> inco
Hi,
On Fri, Feb 04, 2022 at 09:10:42AM -0500, Robert Haas wrote:
> On Thu, Feb 3, 2022 at 12:34 AM Julien Rouhaud wrote:
> > I agree that having dozen of custom rmgrs doesn't seem likely, but I also
> > have
> > no idea of how much overhead you get by not doing a direct array access. I
> > thin
On Thu, Feb 3, 2022 at 12:34 AM Julien Rouhaud wrote:
> I agree that having dozen of custom rmgrs doesn't seem likely, but I also have
> no idea of how much overhead you get by not doing a direct array access. I
> think it would be informative to benchmark something like simple OLTP write
> workl
Hi,
On Tue, Feb 01, 2022 at 03:38:32PM -0800, Jeff Davis wrote:
> On Tue, 2022-02-01 at 20:45 +0800, Julien Rouhaud wrote:
> >
> > One last thing, did you do some benchmark with a couple custom rmgr
> > to see how
> > much the O(n) access is showing up in profiles?
>
> What kind of a test case w
On Tue, 2022-02-01 at 20:45 +0800, Julien Rouhaud wrote:
> On Tue, Feb 01, 2022 at 12:39:38PM +0800, Julien Rouhaud wrote:
> >
> > Other than that the patch looks good to me, as you said we just
> > need a decision
> > on whether custom rmgrs are wanted or not.
>
> One last thing, did you do some
On Tue, Feb 01, 2022 at 12:39:38PM +0800, Julien Rouhaud wrote:
>
> Other than that the patch looks good to me, as you said we just need a
> decision
> on whether custom rmgrs are wanted or not.
One last thing, did you do some benchmark with a couple custom rmgr to see how
much the O(n) access i
Hi,
On Mon, Jan 31, 2022 at 06:36:59PM -0800, Jeff Davis wrote:
> On Tue, 2022-01-18 at 17:53 +0800, Julien Rouhaud wrote:
>
> > There's a bit of 0003 present in 002:
>
> I just squashed 0002 and 0003 together. Not large enough to keep
> separate.
Agreed.
> > + elog(LOG, "registering custome
On Tue, 2022-01-18 at 17:53 +0800, Julien Rouhaud wrote:
> > Review on patch 0001 would be helpful. It makes decoding just
> > another
> > method of rmgr, which makes a lot of sense to me from a code
> > organization standpoint regardless of the other patches. Is there
> > any
> > reason not to do
Hi,
On Mon, Jan 17, 2022 at 12:42:06AM -0800, Jeff Davis wrote:
> On Mon, 2021-11-08 at 15:36 -0800, Jeff Davis wrote:
> > The attached patch (against v14, so it's easier to test columnar) is
> > somewhat like a simplified version of [3] combined with refactoring
> > to
> > make decoding a part of
On Mon, 2021-11-08 at 15:36 -0800, Jeff Davis wrote:
> The attached patch (against v14, so it's easier to test columnar) is
> somewhat like a simplified version of [3] combined with refactoring
> to
> make decoding a part of the rmgr.
New patches attached (v3). Essentially the content as v2, but s
On Mon, Nov 8, 2021 at 6:36 PM Jeff Davis wrote:
> Extensible rmgr would enable the table AM to support its own
> redo/decode hooks and WAL format, so that it could support crash
> recovery, physical replication, and logical replication.
Without taking a position on your implementation, which I h
On Mon, 2021-11-08 at 15:36 -0800, Jeff Davis wrote:
> The attached patch (against v14, so it's easier to test columnar) is
> somewhat like a simplified version of [3] combined with refactoring
> to
> make decoding a part of the rmgr.
I created a wiki page here:
https://wiki.postgresql.org/wiki/E
Motivation:
I'm working on a columnar compression AM[0]. Currently, it uses generic
xlog, which works for crash recovery and physical replication, but not
logical decoding/replication.
Extensible rmgr would enable the table AM to support its own
redo/decode hooks and WAL format, so that it could
32 matches
Mail list logo
