On Fri, Oct 29, 2021 at 10:13:44AM +0200, Ronan Dunklau wrote:
> We could use a single query on the primary (using the primary's checkpoint
> LSN
> instead) but it feels a bit convoluted just to avoid a query on the standby.
Cheating with pg_walfile_name() running on the primary is fine by me.
Le vendredi 29 octobre 2021, 04:27:51 CEST Michael Paquier a écrit :
> On Thu, Oct 28, 2021 at 03:55:12PM +0200, Ronan Dunklau wrote:
> > Interesting ideas, thanks. For the record, the time drops from ~4.5s to 3s
> > on average on my machine.
> > I think if you reduce the size of the generate_serie
On Thu, Oct 28, 2021 at 03:55:12PM +0200, Ronan Dunklau wrote:
> Interesting ideas, thanks. For the record, the time drops from ~4.5s to 3s on
> average on my machine.
> I think if you reduce the size of the generate_series batches, this should
> probably be reduced everywhere. With what we do t
Le jeudi 28 octobre 2021, 14:31:36 CEST Michael Paquier a écrit :
> On Wed, Oct 27, 2021 at 10:11:00AM +0200, Ronan Dunklau wrote:
> > Sorry I sent an intermediary version of the patch, here is the correct
> > one.
>
> While looking at this patch, I have figured out a simple way to make
> the test
On Wed, Oct 27, 2021 at 10:11:00AM +0200, Ronan Dunklau wrote:
> Sorry I sent an intermediary version of the patch, here is the correct one.
While looking at this patch, I have figured out a simple way to make
the tests faster without impacting the coverage. The size of the WAL
segments archived
Le mercredi 27 octobre 2021, 10:00:40 CEST Ronan Dunklau a écrit :
> Le mercredi 27 octobre 2021, 04:17:28 CEST Kyotaro Horiguchi a écrit :
> > +my @walfiles = glob "$slot_dir/*";
> >
> > This is not used.
>
> Sorry, fixed in attached version.
>
> > Each pg_receivewal run stalls for about 10 or
Le mercredi 27 octobre 2021, 04:17:28 CEST Kyotaro Horiguchi a écrit :
> +my @walfiles = glob "$slot_dir/*";
>
> This is not used.
>
Sorry, fixed in attached version.
> Each pg_receivewal run stalls for about 10 or more seconds before
> finishing, which is not great from the standpoint of recen
At Wed, 27 Oct 2021 11:17:28 +0900 (JST), Kyotaro Horiguchi
wrote in
> Each pg_receivewal run stalls for about 10 or more seconds before
> finishing, which is not great from the standpoint of recently
> increasing test run time.
>
> Maybe we want to advance LSN a bit, after taking $nextlsn then
At Tue, 26 Oct 2021 11:01:46 +0200, Ronan Dunklau
wrote in
> Le mardi 26 octobre 2021, 08:27:47 CEST Ronan Dunklau a écrit :
> > Yes, I will try to simplify the logic of the patch I sent last week. I'll
> > keep you posted here soon.
>
> I was able to simplify it quite a bit, by using only one
Le mardi 26 octobre 2021, 08:27:47 CEST Ronan Dunklau a écrit :
> Yes, I will try to simplify the logic of the patch I sent last week. I'll
> keep you posted here soon.
I was able to simplify it quite a bit, by using only one standby for both test
scenarios.
This test case verify that after a ti
Le mardi 26 octobre 2021, 03:15:40 CEST Michael Paquier a écrit :
> I have changed the patch per Ronan's suggestion to have the version
> check out of GetSlotInformation(), addressed what you have reported,
> and the result looked good. So I have applied this part.
Thanks !
>
> What remains on t
On Mon, Oct 25, 2021 at 05:46:57PM +0530, Bharath Rupireddy wrote:
> StreamLog() isn't reached for create and drop slot cases, see [1]. I
> suggest to remove replication_slot != NULL and have Assert(slot_name)
> in GetSlotInformation():
> /*
> * Try to get the starting point from t
On Mon, Oct 25, 2021 at 4:19 PM Michael Paquier wrote:
> > 6) Why do we need these two assignements?
> > I think we can just get rid of lsn_loc and tli_loc, initlaize
> > *restart_lsn = InvalidXLogRecPtr and *restart_tli = 0 at the start of
> > the function and directly assign the requrested value
On Mon, Oct 25, 2021 at 02:40:05PM +0530, Bharath Rupireddy wrote:
> 2) I think we should tweak the below error message
> to
> pg_log_error("could not read replication slot \"%s\": %s",
> "IDENTIFY_SYSTEM", PQerrorMessage(conn));
> Having slot name in the error message
On Mon, Oct 25, 2021 at 12:21 PM Michael Paquier wrote:
> Attached is the updated patch I am finishing with, which is rather
> clean now. I have tweaked a couple of things while on it, and
> documented better the new GetSlotInformation() in streamutil.c.
Thanks for the v11 patch, here are some c
On Mon, Oct 25, 2021 at 10:24:46AM +0200, Ronan Dunklau wrote:
> However, if we want to support the case of "just check if the slot exists",
> we
> need to make sure the command is actually executed, and check the version
> before calling the function, which would make the check executed twice.
Le lundi 25 octobre 2021, 10:10:13 CEST Michael Paquier a écrit :
> On Mon, Oct 25, 2021 at 09:50:01AM +0200, Ronan Dunklau wrote:
> > Le lundi 25 octobre 2021, 09:40:10 CEST Michael Paquier a écrit :
> >> On Mon, Oct 25, 2021 at 09:15:32AM +0200, Ronan Dunklau wrote:
> >>> Does it make sense thoug
On Mon, Oct 25, 2021 at 09:50:01AM +0200, Ronan Dunklau wrote:
> Le lundi 25 octobre 2021, 09:40:10 CEST Michael Paquier a écrit :
>> On Mon, Oct 25, 2021 at 09:15:32AM +0200, Ronan Dunklau wrote:
>>> Does it make sense though ? The NULL slot_name case handling is pretty
>>> straight forward has it
Le lundi 25 octobre 2021, 09:40:10 CEST Michael Paquier a écrit :
> On Mon, Oct 25, 2021 at 09:15:32AM +0200, Ronan Dunklau wrote:
> > Does it make sense though ? The NULL slot_name case handling is pretty
> > straight forward has it will be handled by string formatting, but in the
> > case of a nu
On Mon, Oct 25, 2021 at 09:15:32AM +0200, Ronan Dunklau wrote:
> Does it make sense though ? The NULL slot_name case handling is pretty
> straight forward has it will be handled by string formatting, but in the case
> of a null restart_lsn, we have no way of knowing if the command was issued at
Le lundi 25 octobre 2021, 00:43:11 CEST Michael Paquier a écrit :
> On Sun, Oct 24, 2021 at 07:20:57PM +0530, Bharath Rupireddy wrote:
> > Thanks. I've no further comments on the v10 patch.
>
> Okay, thanks. I have applied this part, then.
> --
> Michael
Thank you all for your work on this patch
Le lundi 25 octobre 2021, 08:51:23 CEST Michael Paquier a écrit :
> On Thu, Oct 21, 2021 at 10:36:42AM +0200, Ronan Dunklau wrote:
> > Done. I haven't touched the timeline switch test patch for now, but I
> > still
> > include it here for completeness.
>
> As 0001 and 0002 have been applied, I hav
On Thu, Oct 21, 2021 at 10:36:42AM +0200, Ronan Dunklau wrote:
> Done. I haven't touched the timeline switch test patch for now, but I still
> include it here for completeness.
As 0001 and 0002 have been applied, I have put my hands on 0003, and
found a couple of issues upon review.
+ Asse
On Sun, Oct 24, 2021 at 07:20:57PM +0530, Bharath Rupireddy wrote:
> Thanks. I've no further comments on the v10 patch.
Okay, thanks. I have applied this part, then.
--
Michael
signature.asc
Description: PGP signature
On Sun, Oct 24, 2021 at 12:21 PM Michael Paquier wrote:
>
> On Sun, Oct 24, 2021 at 09:08:01AM +0530, Bharath Rupireddy wrote:
> > pg_get_replication_slots holds the ReplicationSlotControlLock until
> > the end of the function so it can be assured that *slot contents will
> > not change. In ReadRe
On Sun, Oct 24, 2021 at 09:08:01AM +0530, Bharath Rupireddy wrote:
> pg_get_replication_slots holds the ReplicationSlotControlLock until
> the end of the function so it can be assured that *slot contents will
> not change. In ReadReplicationSlot, the ReplicationSlotControlLock is
> released immedia
On Sun, Oct 24, 2021 at 4:40 AM Michael Paquier wrote:
> On Sat, Oct 23, 2021 at 10:46:30PM +0530, Bharath Rupireddy wrote:
> > 2) As I previously mentioned, we are not copying the slot contents
> > while holding the spinlock, it's just we are taking the memory address
> > and releasing the lock,
On Sat, Oct 23, 2021 at 10:46:30PM +0530, Bharath Rupireddy wrote:
> 1) It's better to initialize nulls with false, we can avoid setting
> them to true. The instances where the columns are not nulls is going
> to be more than the columns with null values, so we could avoid some
> of nulls[i] = fals
On Sat, Oct 23, 2021 at 1:14 PM Michael Paquier wrote:
>
> On Thu, Oct 21, 2021 at 10:36:42AM +0200, Ronan Dunklau wrote:
> > Done. I haven't touched the timeline switch test patch for now, but I still
> > include it here for completeness.
>
> Thanks. I have applied and back-patched 0001, then lo
On Thu, Oct 21, 2021 at 10:36:42AM +0200, Ronan Dunklau wrote:
> Done. I haven't touched the timeline switch test patch for now, but I still
> include it here for completeness.
Thanks. I have applied and back-patched 0001, then looked again at
0002 that adds READ_REPLICATION_SLOT:
- Change the T
Le jeudi 21 octobre 2021, 09:21:44 CEST Michael Paquier a écrit :
> On Thu, Oct 21, 2021 at 08:29:54AM +0200, Ronan Dunklau wrote:
> > Ok, do you want me to propose a different patch for previous versions ?
>
> That's not necessary. Thanks for proposing.
>
> > Do you mean restart_lsn as the poin
On Thu, Oct 21, 2021 at 08:29:54AM +0200, Ronan Dunklau wrote:
> Ok, do you want me to propose a different patch for previous versions ?
That's not necessary. Thanks for proposing.
> Do you mean restart_lsn as the pointer argument to the function, or
> restart_lsn as the field returned by the c
Le jeudi 21 octobre 2021, 07:35:08 CEST Michael Paquier a écrit :
> On Wed, Oct 20, 2021 at 02:58:26PM +0200, Ronan Dunklau wrote:
> > After sending the previous patch suite, I figured it would be worthwhile
> > to
> > also have tests covering timeline switches, which was not covered before.
>
> T
On Wed, Oct 20, 2021 at 02:58:26PM +0200, Ronan Dunklau wrote:
> After sending the previous patch suite, I figured it would be worthwhile to
> also have tests covering timeline switches, which was not covered before.
That seems independent to me. I'll take a look.
> So please find attached a ne
Le mercredi 20 octobre 2021 11:40:18 CEST, vous avez écrit :
> > +# Setup the slot, and connect to it a first time
> > +$primary->run_log(
> > + [ 'pg_receivewal', '--slot', $slot_name, '--create-slot' ],
> > + 'creating a replication slot');
> > +$primary->psql('postgres',
> > + 'INSERT INTO
Le mercredi 20 octobre 2021, 07:13:15 CEST Michael Paquier a écrit :
> On Tue, Oct 19, 2021 at 05:32:55PM +0200, Ronan Dunklau wrote:
> > Following recommendations, I stripped most of the features from the patch.
> > For now we support only physical replication slots, and only provide the
> > two f
On Tue, Oct 19, 2021 at 05:32:55PM +0200, Ronan Dunklau wrote:
> Following recommendations, I stripped most of the features from the patch.
> For
> now we support only physical replication slots, and only provide the two
> fields
> of interest (restart_lsn, restart_tli) in addition to the slot
Hello,
Following recommendations, I stripped most of the features from the patch. For
now we support only physical replication slots, and only provide the two fields
of interest (restart_lsn, restart_tli) in addition to the slot type (fixed at
physical) to not paint ourselves in a corner.
I al
On Mon, Sep 06, 2021 at 04:17:28PM +0900, Michael Paquier wrote:
> Using READ_REPLICATION_SLOT as the command name is fine, and it could
> be extended with more fields if necessary, implemented now with only
> what we think is useful. Returning errors on cases that are still not
> supported yet is
On Mon, Sep 06, 2021 at 12:37:29PM +0530, Bharath Rupireddy wrote:
> -1 for READ_PHYSICAL_REPLICATION_SLOT or failing on the server. What
> happens if we have another slot type "PHYSIOLOGICAL" or "FOO" or "BAR"
> some other? IMO, READ_REPLICATION_SLOT should just return info of all
> slots. The cli
Le vendredi 3 septembre 2021 17:49:34 CEST, vous avez écrit :
> On Fri, Sep 3, 2021 at 3:28 PM Ronan Dunklau wrote:
> > There is still the concern raised by Bharath about being able to select
> > the
> > way to fetch the replication slot information for the user, and what
> > should or should not
On Mon, Sep 6, 2021 at 12:20 PM Ronan Dunklau wrote:
>
> Le lundi 6 septembre 2021, 06:22:30 CEST Michael Paquier a écrit :
> > On Fri, Sep 03, 2021 at 11:58:27AM +0200, Ronan Dunklau wrote:
> > > 0002 for pg_receivewal tried to simplify the logic of information to
> > > return, by using a dedicat
Le lundi 6 septembre 2021, 06:22:30 CEST Michael Paquier a écrit :
> On Fri, Sep 03, 2021 at 11:58:27AM +0200, Ronan Dunklau wrote:
> > 0002 for pg_receivewal tried to simplify the logic of information to
> > return, by using a dedicated struct for this. This accounts for Bahrath's
> > demands to r
On Fri, Sep 03, 2021 at 11:58:27AM +0200, Ronan Dunklau wrote:
>
> 0002 for pg_receivewal tried to simplify the logic of information to return,
> by using a dedicated struct for this. This accounts for Bahrath's demands to
> return every possible field.
> In particular, an is_logical field simpl
On Fri, Sep 3, 2021 at 3:28 PM Ronan Dunklau wrote:
> There is still the concern raised by Bharath about being able to select the
> way to fetch the replication slot information for the user, and what should or
> should not be included in the documentation. I am in favor of documenting the
> proce
Le jeudi 2 septembre 2021, 10:37:20 CEST Michael Paquier a écrit :
> On Thu, Sep 02, 2021 at 10:08:26AM +0200, Ronan Dunklau wrote:
> > I could see the use for sending active_pid for use within pg_basebackup:
> > at
> > least we could fail early if the slot is already in use. But at the same
> > ti
On Thu, Sep 02, 2021 at 10:08:26AM +0200, Ronan Dunklau wrote:
> I could see the use for sending active_pid for use within pg_basebackup: at
> least we could fail early if the slot is already in use. But at the same
> time,
> maybe it won't be in use anymore once we need it.
There is no real co
Le jeudi 2 septembre 2021, 09:28:29 CEST Michael Paquier a écrit :
> On Thu, Sep 02, 2021 at 02:45:54PM +0900, Kyotaro Horiguchi wrote:
> > At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy
> > wrote in If I read the patch
> > correctly the situation above is warned by the following message the
On Thu, Sep 02, 2021 at 02:45:54PM +0900, Kyotaro Horiguchi wrote:
> At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy
> wrote in
> If I read the patch correctly the situation above is warned by the
> following message then continue to the next step giving up to identify
> start position from
Le jeudi 2 septembre 2021, 08:42:22 CEST Bharath Rupireddy a écrit :
> On Thu, Sep 2, 2021 at 11:15 AM Kyotaro Horiguchi
>
> wrote:
> > At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy
> > wrote in>
> > > On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau
wrote:
> > > > > 7) How about we let p
On Thu, Sep 2, 2021 at 11:15 AM Kyotaro Horiguchi
wrote:
>
> At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy
> wrote in
> > On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau
> > wrote:
> > > > 7) How about we let pg_receivewal use READ_REPLICATION_SLOT as an
> > > > option?
> > >
> > > From
At Wed, 1 Sep 2021 10:30:05 +0530, Bharath Rupireddy
wrote in
> On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau wrote:
> > > 7) How about we let pg_receivewal use READ_REPLICATION_SLOT as an option?
> >
> > From my point of view, I already expected it to use something like that when
> > using a r
On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau wrote:
> > 7) How about we let pg_receivewal use READ_REPLICATION_SLOT as an option?
>
> From my point of view, I already expected it to use something like that when
> using a replication slot. Maybe an option to turn it off could be useful ?
IMO, pg_
On Mon, Aug 30, 2021 at 11:55:42AM +0200, Ronan Dunklau wrote:
> Le vendredi 27 août 2021, 05:44:32 CEST Michael Paquier a écrit :
>> + if (slot == NULL || !slot->in_use)
>>
>>
Le mardi 31 août 2021, 13:17:22 CEST Bharath Rupireddy a écrit :
> On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau
wrote:
> > Thank you for this review ! Please see the other side of the thread where
> > I
> > answered Michael Paquier with a new patchset, which includes some of your
> > proposed mo
On Tue, Aug 31, 2021 at 4:47 PM Bharath Rupireddy
wrote:
>
> On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau wrote:
> > Thank you for this review ! Please see the other side of the thread where I
> > answered Michael Paquier with a new patchset, which includes some of your
> > proposed modification
On Mon, Aug 30, 2021 at 3:26 PM Ronan Dunklau wrote:
> Thank you for this review ! Please see the other side of the thread where I
> answered Michael Paquier with a new patchset, which includes some of your
> proposed modifications.
Thanks for the updated patches. Here are some comments on v3-000
Le samedi 28 août 2021, 14:10:25 CEST Bharath Rupireddy a écrit :
> On Thu, Aug 26, 2021 at 5:45 PM Ronan Dunklau
wrote:
> > order to fail early if the replication slot doesn't exist, so please find
> > attached v2 for that.
>
> Thanks for the patches. Here are some comments:
>
Thank you for t
Le vendredi 27 août 2021, 05:44:32 CEST Michael Paquier a écrit :
> On Thu, Aug 26, 2021 at 02:14:27PM +0200, Ronan Dunklau wrote:
> > Following the discussion at [1], I refactored the implementation into
> > streamutil and added a third patch making use of it in pg_basebackup
> > itself in order t
On Thu, Aug 26, 2021 at 5:45 PM Ronan Dunklau wrote:
> order to fail early if the replication slot doesn't exist, so please find
> attached v2 for that.
Thanks for the patches. Here are some comments:
1) While the intent of these patches looks good, I have following
concern with new replication
On Thu, Aug 26, 2021 at 02:14:27PM +0200, Ronan Dunklau wrote:
> Following the discussion at [1], I refactored the implementation into
> streamutil and added a third patch making use of it in pg_basebackup itself
> in
> order to fail early if the replication slot doesn't exist, so please find
>
Le jeudi 29 juillet 2021, 11:09:40 CEST Ronan Dunklau a écrit :
> Patch 0001 adds the new READ_REPLICATION_SLOT command.
> It returns for a given slot the type, restart_lsn, flush_lsn,
> restart_lsn_timeline and flush_lsn_timeline.
> The timelines are determined by reading the current timeline hist
Le jeudi 29 juillet 2021, 07:32:37 CEST Kyotaro Horiguchi a écrit :
> I didn't thought in details. But I forgot that ordinary SQL commands
> have been prohibited in physical replication connection. So we need a
> new replication command but it's not that a big deal.
Thank you for your feedback !
At Wed, 28 Jul 2021 12:57:39 +0200, Ronan Dunklau
wrote in
> Le mercredi 28 juillet 2021, 08:22:30 CEST Kyotaro Horiguchi a écrit :
> > At Tue, 27 Jul 2021 07:50:39 +0200, Ronan Dunklau
> > wrote in
> > > I don't know if it should be the default, toggled by a command line flag,
> > > or if we e
Le mercredi 28 juillet 2021, 08:22:30 CEST Kyotaro Horiguchi a écrit :
> At Tue, 27 Jul 2021 07:50:39 +0200, Ronan Dunklau
> wrote in
> > Hello,
> >
> > I've notived that pg_receivewal logic for deciding which LSN to start
> >
> > streaming at consists of:
> > - looking up the latest WAL file
At Tue, 27 Jul 2021 07:50:39 +0200, Ronan Dunklau
wrote in
> Hello,
>
> I've notived that pg_receivewal logic for deciding which LSN to start
> streaming at consists of:
> - looking up the latest WAL file in our destination folder, and resume from
> here
> - if there isn't, use the curren
Hello,
I've notived that pg_receivewal logic for deciding which LSN to start
streaming at consists of:
- looking up the latest WAL file in our destination folder, and resume from
here
- if there isn't, use the current flush location instead.
This behaviour surprised me when using it with a
67 matches
Mail list logo
