Re: Switch to multi-inserts for pg_depend

2020-09-05 Thread Michael Paquier
On Fri, Sep 04, 2020 at 10:15:57AM +0900, Michael Paquier wrote: > Thanks, fixed. With the two comment fixes included, I have looked at both patches again today, and applied them. -- Michael signature.asc Description: PGP signature

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Michael Paquier
On Thu, Sep 03, 2020 at 10:50:49AM -0400, Alvaro Herrera wrote: > I'm not sure you need the second sentence in this comment; keeping the > "delay initialization until ..." part seems sufficient. If you really > want to highlight that initialization is costly, maybe just say "delay > costly initial

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Alvaro Herrera
I agree, this version looks much better, thanks. Two very minor things: On 2020-Sep-03, Michael Paquier wrote: > @@ -76,11 +77,23 @@ recordMultipleDependencies(const ObjectAddress *depender, > > dependDesc = table_open(DependRelationId, RowExclusiveLock); > > + /* > + * Alloca

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Daniel Gustafsson
> On 3 Sep 2020, at 12:19, Michael Paquier wrote: > > On Thu, Sep 03, 2020 at 09:47:07AM +0200, Daniel Gustafsson wrote: >> I think this version is a clear improvement. Nothing more sticks out from a >> read-through. > > Thanks for taking the time to look at it, Daniel. We of course could > st

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Michael Paquier
On Thu, Sep 03, 2020 at 09:47:07AM +0200, Daniel Gustafsson wrote: > I think this version is a clear improvement. Nothing more sticks out from a > read-through. Thanks for taking the time to look at it, Daniel. We of course could still try to figure out how we could group all dependencies withou

Re: Switch to multi-inserts for pg_depend

2020-09-03 Thread Daniel Gustafsson
> On 3 Sep 2020, at 07:35, Michael Paquier wrote: > > On Tue, Sep 01, 2020 at 11:53:36AM +0200, Daniel Gustafsson wrote: >> On 14 Aug 2020, at 20:23, Alvaro Herrera wrote: >> >>> The logic to keep track number of used slots used is baroque, though -- that >>> could use a lot of simplification.

Re: Switch to multi-inserts for pg_depend

2020-09-02 Thread Michael Paquier
On Tue, Sep 01, 2020 at 11:53:36AM +0200, Daniel Gustafsson wrote: > On 14 Aug 2020, at 20:23, Alvaro Herrera wrote: > >> The logic to keep track number of used slots used is baroque, though -- that >> could use a lot of simplification. > > What if slot_init was an integer which increments toget

Re: Switch to multi-inserts for pg_depend

2020-09-01 Thread Daniel Gustafsson
> On 14 Aug 2020, at 20:23, Alvaro Herrera wrote: > The logic to keep track number of used slots used is baroque, though -- that > could use a lot of simplification. What if slot_init was an integer which increments together with the loop variable until max_slots is reached? If so, maybe it sho

Re: Switch to multi-inserts for pg_depend

2020-08-31 Thread Michael Paquier
On Sat, Aug 15, 2020 at 10:50:37AM +0900, Michael Paquier wrote: > What are you suggesting here? A new API layer to manage a set of > slots? It has been a couple of weeks, and I am not really sure what is the suggestion here. So I would like to move on with this patch set as the changes are stra

Re: Switch to multi-inserts for pg_depend

2020-08-14 Thread Michael Paquier
On Fri, Aug 14, 2020 at 02:23:16PM -0400, Alvaro Herrera wrote: > It seems a bit silly to worry about allocating just the exact amount > needed; the current approach looked fine to me. Okay, thanks. > The logic to keep track > number of used slots used is baroque, though -- that could use a lot o

Re: Switch to multi-inserts for pg_depend

2020-08-14 Thread Alvaro Herrera
On 2020-Aug-14, Michael Paquier wrote: > Regarding the maximum number of slots allocated. Do people like the > current approach taken by the patch to do a single loop of the > dependency entries at the cost of more allocating perhaps too much for > the array holding the set of TupleTableSlots (th

Re: Switch to multi-inserts for pg_depend

2020-08-14 Thread Michael Paquier
On Thu, Aug 13, 2020 at 11:45:52AM -0400, Alvaro Herrera wrote: > MAX_CATALOG_INSERT_BYTES sounds decent to me. I mentioned dependency.h > because I was uncaffeinatedly thinking that this was used with API > defined there -- but in reality it's used with indexing.h functions, and > it seems to me

Re: Switch to multi-inserts for pg_depend

2020-08-13 Thread Alvaro Herrera
On 2020-Aug-13, Michael Paquier wrote: > On Thu, Aug 13, 2020 at 05:35:14AM -0400, Alvaro Herrera wrote: > > Next to the API definition I guess, is that dependency.h? > > We need something more central, see also MAX_PGATTRIBUTE_INSERT_BYTES > and MAX_PGSHDEPEND_INSERT_BYTES. And the definition s

Re: Switch to multi-inserts for pg_depend

2020-08-13 Thread Michael Paquier
On Thu, Aug 13, 2020 at 05:35:14AM -0400, Alvaro Herrera wrote: > Next to the API definition I guess, is that dependency.h? We need something more central, see also MAX_PGATTRIBUTE_INSERT_BYTES and MAX_PGSHDEPEND_INSERT_BYTES. And the definition should be named something like MAX_CATALOG_INSERT_B

Re: Switch to multi-inserts for pg_depend

2020-08-13 Thread Alvaro Herrera
On 2020-Aug-13, Michael Paquier wrote: > Okay. Would src/include/catalog/catalog.h be a suited location for > this flag or somebody has a better idea? Next to the API definition I guess, is that dependency.h? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Developmen

Re: Switch to multi-inserts for pg_depend

2020-08-12 Thread Michael Paquier
On Wed, Aug 12, 2020 at 07:52:42PM -0400, Alvaro Herrera wrote: > Yeah. As I understand, the only reason to have this number is to avoid > an arbitrarily large number of entries created as a single multi-insert > WAL record ... but does that really ever happen? I guess if you create > a table wit

Re: Switch to multi-inserts for pg_depend

2020-08-12 Thread Alvaro Herrera
On 2020-Aug-11, Robert Haas wrote: > On Tue, Aug 11, 2020 at 1:59 AM Michael Paquier wrote: > > On Mon, Aug 10, 2020 at 05:32:21PM -0700, Andres Freund wrote: > > > Do we really want to end up with several separate defines for different > > > type of catalog batch inserts? That doesn't seem like

Re: Switch to multi-inserts for pg_depend

2020-08-11 Thread Robert Haas
On Tue, Aug 11, 2020 at 1:59 AM Michael Paquier wrote: > On Mon, Aug 10, 2020 at 05:32:21PM -0700, Andres Freund wrote: > > Do we really want to end up with several separate defines for different > > type of catalog batch inserts? That doesn't seem like a good > > thing. Think there should be a si

Re: Switch to multi-inserts for pg_depend

2020-08-10 Thread Michael Paquier
On Mon, Aug 10, 2020 at 05:32:21PM -0700, Andres Freund wrote: > Do we really want to end up with several separate defines for different > type of catalog batch inserts? That doesn't seem like a good > thing. Think there should be a single define for all catalog bulk > inserts. Unlikely so, but I

Re: Switch to multi-inserts for pg_depend

2020-08-10 Thread Andres Freund
Hi, On 2020-08-07 15:16:19 +0900, Michael Paquier wrote: > From cd117fa88938c89ac953a5e3c036828337150b07 Mon Sep 17 00:00:00 2001 > From: Michael Paquier > Date: Fri, 7 Aug 2020 10:57:40 +0900 > Subject: [PATCH 1/2] Use multi-inserts for pg_depend > > This is a follow-up of the work done in e393

Re: Switch to multi-inserts for pg_depend

2020-08-06 Thread Michael Paquier
On Fri, Aug 07, 2020 at 03:16:19PM +0900, Michael Paquier wrote: > I am adding this thread to the next commit fest. Thoughts are > welcome. Forgot to mention that this is based on some initial work from Daniel, and that we have discussed the issue before I worked on it. -- Michael signature.asc