Re: Restart pg_usleep when interrupted

2024-08-07 Thread Bertrand Drouvot
Hi, On Wed, Aug 07, 2024 at 09:11:19AM -0500, Sami Imseih wrote: > > > > On Aug 7, 2024, at 1:00 AM, Bertrand Drouvot > > wrote: > > > > add t (in microseconds) to x” > > > I was attempting to be more verbose in the comment, > but what you ha

Re: Restart pg_usleep when interrupted

2024-08-07 Thread Bertrand Drouvot
seconds since the Epoch. It means that we’re currently about 237 years away from the limit. So even, if we were to say add 2 "recents" of them we are still about 183 years away from the limit. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-09 Thread Bertrand Drouvot
build correctly + what about? Test tracking of committed transactions during BUILDING_SNAPSHOT Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Restart pg_usleep when interrupted

2024-08-09 Thread Bertrand Drouvot
sewhere. Perhaps we should move it > to pgsleep.c and rename it to something to the effect of > pg_usleep_non_interruptable(). Yeah, I had the same thought in [1], so +1. [1]: https://www.postgresql.org/message-id/ZpDhS4nFX66ItAze%40ip-10-97-1-34.eu-west-3.compute.internal Regards, -- Bertrand

Re: Restart pg_usleep when interrupted

2024-08-11 Thread Bertrand Drouvot
Hi, On Sat, Aug 10, 2024 at 08:27:56AM -0500, Sami Imseih wrote: > > > v9 has some has some minor corrections to the comments. > Thanks! 1 === + * Unlike pg_usleep, This function continues s/This/this/? Apart from the above, LGTM. Regards, -- Bertrand Drouvot PostgreSQL

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-11 Thread Bertrand Drouvot
tate(builder) == SNAPBUILD_BUILDING_SNAPSHOT && info != XLOG_HEAP_INPLACE) || ctx->fast_forward) return; That way we'd still rely on what's being done in the XLOG_HEAP_INPLACE case ( should it change in the future). 6 === v3-0002 LGTM. Regards, -- Ber

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread Bertrand Drouvot
r my proposal) so that we can jump directly in the appropriate case. I think that way the code is easier to maintain instead of having to deal with the same info values in distinct places. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Restart pg_usleep when interrupted

2024-08-12 Thread Bertrand Drouvot
Hi, On Mon, Aug 12, 2024 at 10:19:56AM -0500, Sami Imseih wrote: > v10 attached in the previous message addresses > Bertrands last comment and replaces “This” with “this" > Thanks, v10 LGTM. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databa

Re: Restart pg_usleep when interrupted

2024-08-12 Thread Bertrand Drouvot
s compare to [1]) though. [0]: https://commitfest.postgresql.org/49/5027/ [1]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=f1889729dd Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-12 Thread Bertrand Drouvot
Not sure what happened but it looks like your reply in [0] is not part of the initial thread [1], but created a new thread instead, making the whole conversation difficult to follow. [0]: https://www.postgresql.org/message-id/tencent_8DEC9842690A9B6AFD52D4659EF0700E9409%40qq.com [1]: https://www.p

Re: Historic snapshot doesn't track txns committed in BUILDING_SNAPSHOT state

2024-08-13 Thread Bertrand Drouvot
fficult to follow. > > I'm not sure what happened but I attach the new thread to the CF: Unfortunately your last reply did start a new email thread again. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Restart pg_usleep when interrupted

2024-08-13 Thread Bertrand Drouvot
of interrupts and Sami's patch provides a way to get "accurate" delay in case of interrupts. I think both have their own benefit. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

define PG_REPLSLOT_DIR

2024-08-14 Thread Bertrand Drouvot
ind/filemap.o Also, I think we could do the same for: pg_notify pg_serial pg_subtrans pg_wal pg_multixact pg_tblspc pg_logical And I volunteer to do so if you think that makes sense. Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Sou

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Thu, Aug 15, 2024 at 08:40:42PM -0400, Alvaro Herrera wrote: > On 2024-Aug-14, Bertrand Drouvot wrote: > > > Out of curiosity, checking the sizes of affected files (O2, no debug): > > > > with patch: > > > >textdata bss dec hex

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Fri, Aug 16, 2024 at 01:31:11PM +0900, Yugo Nagata wrote: > On Wed, 14 Aug 2024 11:32:14 + > Bertrand Drouvot wrote: > > Looking forward to your feedback, > > /* restore all slots by iterating over all on-disk entries */ > - replication_dir = All

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Mon, Aug 19, 2024 at 04:11:31PM +0530, Ashutosh Bapat wrote: > On Wed, Aug 14, 2024 at 5:02 PM Bertrand Drouvot > wrote: > > > > Hi hackers, > > > > while working on a replication slot tool (idea is to put it in contrib, not > > shared yet), I reali

Re: Restart pg_usleep when interrupted

2024-08-19 Thread Bertrand Drouvot
Hi, On Thu, Aug 15, 2024 at 04:13:29PM -0500, Nathan Bossart wrote: > On Wed, Aug 14, 2024 at 06:00:06AM +0000, Bertrand Drouvot wrote: > > I gave it more thoughts and I don't think we have to choose between the two. > > The 1 Hz approach reduces the number of interru

Re: Avoid orphaned objects dependencies, take 3

2024-08-19 Thread Bertrand Drouvot
Hi, On Wed, Jul 10, 2024 at 07:31:06AM +, Bertrand Drouvot wrote: > So, to sum up: > > A. Locking is now done exclusively with LockNotPinnedObject(Oid classid, Oid > objid) > so that it's now always clear what object we want to acquire a lock for. It > means &g

Re: define PG_REPLSLOT_DIR

2024-08-19 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 09:26:59AM +0530, Ashutosh Bapat wrote: > On Mon, Aug 19, 2024 at 7:47 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Mon, Aug 19, 2024 at 04:11:31PM +0530, Ashutosh Bapat wrote: > > > On Wed, Aug 14, 2024 a

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
> Doing multiple commits with individual definitions for each path would > be the way to go for me. All that is mechanical, still that feels > slightly cleaner. Right, that's what v2 has done. If there is a need for v3 then I'll add one dedicated patch for Ashutosh's proposal

Re: define PG_REPLSLOT_DIR

2024-08-20 Thread Bertrand Drouvot
Hi, On Tue, Aug 20, 2024 at 05:47:57PM +0900, Michael Paquier wrote: > On Mon, Aug 19, 2024 at 02:11:55PM +0000, Bertrand Drouvot wrote: > > I made the changes for pg_tblspc in pg_combinebackup.c as the number of > > occurences > > are greater that the "pg_wal&q

Re: [Proposal] Add accumulated statistics for wait event

2018-07-26 Thread Bertrand DROUVOT
Hello Guys, As you mentioned Oracle like active session history sampling in this thread, I just want to let you know that I am working on a brand new extension to provide this feature. You can find the extension here: https://github.com/pgsentinel/pgsentinel Basically, you could see it as sampli

Re: Add contrib/pg_logicalsnapinspect

2024-10-13 Thread Bertrand Drouvot
Hi, On Fri, Oct 11, 2024 at 04:48:26PM -0700, Masahiko Sawada wrote: > On Fri, Oct 11, 2024 at 11:15 AM Masahiko Sawada > wrote: > > > > On Fri, Oct 11, 2024 at 6:15 AM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On

Re: Missing deconstruct_array_builtin usage

2024-10-13 Thread Bertrand Drouvot
Hi, On Fri, Oct 11, 2024 at 05:43:04PM -0700, Masahiko Sawada wrote: > Hi, > > On Thu, Oct 10, 2024 at 10:37 PM Bertrand Drouvot > wrote: > > > > Hi hackers, > > > > While working on [1], I noticed that we missed using > > deconstruct_array_bu

Re: Add contrib/pg_logicalsnapinspect

2024-10-13 Thread Bertrand Drouvot
> */ > -static bool > -SnapBuildRestore(SnapBuild *builder, XLogRecPtr lsn) > +bool > +SnapBuildRestoreSnapshot(SnapBuildOnDisk *ondisk, const char *path, > + MemoryContext context, bool missing_ok) > > nit - I think it's better to describe parameters in the same order

Re: Add contrib/pg_logicalsnapinspect

2024-10-08 Thread Bertrand Drouvot
That's right. > but it still seems a bit awkward to write this way. > Since so much else has changed, how about also improving this in > passing, like below: > > if (fd < 0) > { > if (errno == ENOENT) > return false; > > ereport(ERROR, > (errcode_for

Re: per backend I/O statistics

2024-10-08 Thread Bertrand Drouvot
Hi, On Tue, Oct 08, 2024 at 01:46:23PM +0900, Michael Paquier wrote: > On Mon, Oct 07, 2024 at 09:54:21AM +0000, Bertrand Drouvot wrote: > > On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote: > >> This would rely on the fact that we would use the ProcNumber for t

Re: Add contrib/pg_logicalsnapinspect

2024-10-09 Thread Bertrand Drouvot
Hi, On Tue, Oct 08, 2024 at 10:52:11AM -0700, Masahiko Sawada wrote: > On Tue, Oct 8, 2024 at 9:25 AM Bertrand Drouvot > wrote: > > Thank you for updating the patch! I have some comments on v12 patch: Thanks for looking at it! > --- > + if (ondisk.builder.

Re: Add contrib/pg_logicalsnapinspect

2024-10-09 Thread Bertrand Drouvot
Hi, On Wed, Oct 09, 2024 at 11:41:44AM +1100, Peter Smith wrote: > On Wed, Oct 9, 2024 at 3:25 AM Bertrand Drouvot > wrote: > > > > I don't get the point here. The examples just show another way to use the > > functions, > > the ouput is more "anecdotal

Missing deconstruct_array_builtin usage

2024-10-10 Thread Bertrand Drouvot
ently (as already mentioned by Peter in [2]). [1]: https://www.postgresql.org/message-id/ZwdK1UYdLn/zFMHy%40ip-10-97-1-34.eu-west-3.compute.internal [2]: https://www.postgresql.org/message-id/2914356f-9e5f-8c59-2995-5997fc48bcba%40enterprisedb.com Looking forward to your feedback, Regards, -

Re: BF mamba failure

2024-10-11 Thread Bertrand Drouvot
Hi, On Fri, Oct 11, 2024 at 11:07:29AM +0300, Kouber Saparev wrote: > На чт, 10.10.2024 г. в 17:42 Bertrand Drouvot > написа: > > > > > Does the error message looks like (added in [1]): > > > > " > > trying to drop stats entry already dropped

Re: Add contrib/pg_logicalsnapinspect

2024-10-10 Thread Bertrand Drouvot
Hi, On Thu, Oct 10, 2024 at 12:05:10AM -0700, Masahiko Sawada wrote: > On Wed, Oct 9, 2024 at 8:32 PM Bertrand Drouvot > wrote: > > So I think that having construct_array_builtin()/deconstruct_array_builtin() > > taking care of XIDOID is the way to go. If that makes sens

Re: Track the amount of time waiting due to cost_delay

2024-10-11 Thread Bertrand Drouvot
Hi, On Thu, Sep 19, 2024 at 07:54:21AM +, Bertrand Drouvot wrote: > Hi, > > On Wed, Sep 18, 2024 at 04:04:53PM -0500, Nathan Bossart wrote: > > On Thu, Sep 05, 2024 at 04:59:54AM +, Bertrand Drouvot wrote: > > > Please find attached v6, a mandatory rebas

Re: per backend I/O statistics

2024-10-07 Thread Bertrand Drouvot
Hi, On Fri, Sep 20, 2024 at 12:53:43PM +0900, Michael Paquier wrote: > On Wed, Sep 04, 2024 at 04:45:24AM +0000, Bertrand Drouvot wrote: > > On Tue, Sep 03, 2024 at 04:07:58PM +0900, Kyotaro Horiguchi wrote: > >> As an additional benefit of this approach, the client can set

Re: per backend I/O statistics

2024-10-07 Thread Bertrand Drouvot
Hi, On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote: > On Tue, Sep 17, 2024 at 01:56:34PM +0000, Bertrand Drouvot wrote: > > No problem at all! (I re-explained because I'm not always 100% sure that my > > explanations are crystal clear ;-) ) > > We

Re: Add contrib/pg_logicalsnapinspect

2024-10-07 Thread Bertrand Drouvot
fault" so that [-Wswitch] would report a warning in case of enumeration value not handled in the switch (v11 keeps a remark on top of the SnapBuildState enum definition though). > It needs a function call but I believe it won't be a problem in this use case. Agree. [1]: https://www.po

Re: BF mamba failure

2024-10-16 Thread Bertrand Drouvot
Hi, On Wed, Oct 16, 2024 at 09:43:48AM +0900, Michael Paquier wrote: > On Fri, Oct 11, 2024 at 08:18:58AM +0000, Bertrand Drouvot wrote: > > On Fri, Oct 11, 2024 at 11:07:29AM +0300, Kouber Saparev wrote: > >> Unfortunately not, we are running 15.4 and planning to upgrade very

Re: Refactor GetLockStatusData() by skipping unused backends and groups

2024-10-21 Thread Bertrand Drouvot
ssert to check that we overall iterated over FP_LOCK_SLOTS_PER_BACKEND? 4 === Then the patch does move existing code around and just add a call to FAST_PATH_SLOT() to get fast-path lock slot index based on the group and slot indexes we are iterating on. That does make sense to me. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-18 Thread Bertrand Drouvot
t cause a wait. What about? Detach before wait does not cause a wait (and so wakeup produces an error) Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add contrib/pg_logicalsnapinspect

2024-10-09 Thread Bertrand Drouvot
Hi, On Wed, Oct 09, 2024 at 10:21:31AM -0700, Masahiko Sawada wrote: > On Wed, Oct 9, 2024 at 1:12 AM Bertrand Drouvot > wrote: > > One option could be (did not test it) to add this switch in > > construct_array_builtin(): > > > > + case XIDOID: >

Re: BF mamba failure

2024-10-10 Thread Bertrand Drouvot
ded in [1]): " trying to drop stats entry already dropped: kind=%s dboid=%u objoid=%u refcount=%u", " by any chance? If so, would you mind to share it? Regards, [1]: https://postgr.es/c/d92573adcb -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add contrib/pg_logicalsnapinspect

2024-10-11 Thread Bertrand Drouvot
Hi, On Thu, Oct 10, 2024 at 05:38:43PM -0700, Masahiko Sawada wrote: > On Thu, Oct 10, 2024 at 6:10 AM Bertrand Drouvot > wrote: > > The patches mostly look good to me. Here are some minor comments: Thanks for looking at it! > > + sp

Re: Add contrib/pg_logicalsnapinspect

2024-10-11 Thread Bertrand Drouvot
id a few lines you could initialise the > nulls array during the declaration. > bool nulls[PG_GET_LOGICAL_SNAPSHOT_META_COLS] = {0}; > > This seems a common pattern in other source code, and it replaces the > need for the subsequent memset. Okay, fine by me, let's do it for

Re: define pg_structiszero(addr, s, r)

2024-10-29 Thread Bertrand Drouvot
Hi, On Tue, Oct 29, 2024 at 11:39:03AM +0200, Heikki Linnakangas wrote: > On 29/10/2024 09:54, Bertrand Drouvot wrote: > > > https://godbolt.org/z/x9hPWjheq. > > > > Yeah, I also think that's fine. Peter Smith did some testing in [1] > > comparing > >

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-29 Thread Bertrand Drouvot
Hi, On Tue, Oct 29, 2024 at 05:20:42PM +0900, Michael Paquier wrote: > On Tue, Oct 29, 2024 at 07:59:31AM +0000, Bertrand Drouvot wrote: > > On Tue, Oct 29, 2024 at 01:12:25PM +0900, Michael Paquier wrote: > >> Under a wait of N seconds, > >> we could still have

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-29 Thread Bertrand Drouvot
Hi, On Tue, Oct 29, 2024 at 01:12:25PM +0900, Michael Paquier wrote: > On Mon, Oct 28, 2024 at 07:17:28AM +0000, Bertrand Drouvot wrote: > > I think that we can not be 100% sure that the s1 wait will finish before the > > s2 detach (easy reproducible with gdb attached on s1 or an

Re: define pg_structiszero(addr, s, r)

2024-10-29 Thread Bertrand Drouvot
Hi, On Mon, Oct 28, 2024 at 04:32:51PM +0200, Heikki Linnakangas wrote: > On 18/09/2024 21:57, Bertrand Drouvot wrote: > > On Wed, Sep 18, 2024 at 10:03:21AM +0200, Peter Eisentraut wrote: > > > On 18.09.24 06:16, Bertrand Drouvot wrote: > > > > +#def

Re: define pg_structiszero(addr, s, r)

2024-10-29 Thread Bertrand Drouvot
hi, On Tue, Oct 29, 2024 at 10:23:37AM +0100, Peter Eisentraut wrote: > On 29.10.24 08:54, Bertrand Drouvot wrote: > > +static inline bool > > +pg_mem_is_all_zeros(const char *p, size_t len) > > This should be a void * pointer please. Thanks for looking at it! Yeah better

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Bertrand Drouvot
Hi, On Wed, Oct 30, 2024 at 09:09:54AM +0100, Peter Eisentraut wrote: > On 29.10.24 14:58, Bertrand Drouvot wrote: > > hi, > > > > On Tue, Oct 29, 2024 at 10:23:37AM +0100, Peter Eisentraut wrote: > > > On 29.10.24 08:54, Bertrand Drouvot wrot

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-30 Thread Bertrand Drouvot
Hi, On Wed, Oct 30, 2024 at 06:21:04PM +0900, Michael Paquier wrote: > On Wed, Oct 30, 2024 at 07:26:07AM +0000, Bertrand Drouvot wrote: > > 1 === Nit > > > > s/Permutations like this one/Permutations like the following commented out > > one/ ? > > > > 2

Re: per backend I/O statistics

2024-11-04 Thread Bertrand Drouvot
Hi, On Thu, Oct 31, 2024 at 05:09:56AM +, Bertrand Drouvot wrote: > === OPTIONS === > > So, based on this, I think that we could: > > Option 1: "move" the existing PGSTAT_KIND_IO to variable-numbered and let this > KIND take care of the aggregated view (pg_stat_i

Re: define pg_structiszero(addr, s, r)

2024-11-04 Thread Bertrand Drouvot
Hi, On Tue, Nov 05, 2024 at 12:24:48AM +1300, David Rowley wrote: > On Sat, 2 Nov 2024 at 01:50, Bertrand Drouvot > wrote: > > > > On Fri, Nov 01, 2024 at 09:47:05PM +1300, David Rowley wrote: > > > I've attached what I thought a more optimal version might look

Re: define pg_structiszero(addr, s, r)

2024-11-05 Thread Bertrand Drouvot
use SIMD instructions to check multiple bytes is zero in a single operation. Maybe just an idea to keep in mind and experiment if we feel the need later on. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define pg_structiszero(addr, s, r)

2024-11-04 Thread Bertrand Drouvot
d chars is better than char for byte level memory operations (no sign extension issues). Though I agree that using char in this function is not an issue as p is only compared with 0. This is more a matter of taste here. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Da

Re: define pg_structiszero(addr, s, r)

2024-11-04 Thread Bertrand Drouvot
Hi, On Tue, Nov 05, 2024 at 01:31:58PM +0900, Michael Paquier wrote: > On Mon, Nov 04, 2024 at 05:17:54PM +0000, Bertrand Drouvot wrote: > > Hi, > > > > On Tue, Nov 05, 2024 at 12:24:48AM +1300, David Rowley wrote: > > > On Sat, 2 Nov 2024 at 01:50,

Re: relfilenode statistics

2024-11-04 Thread Bertrand Drouvot
Hi, On Mon, Nov 04, 2024 at 02:51:10PM -0500, Robert Haas wrote: > On Mon, Nov 4, 2024 at 4:27 AM Bertrand Drouvot > wrote: > > Then I think we should go with the "sometimes I don't know the relation OID > > so I want to use the relfilenumber instead, without chang

Re: per backend I/O statistics

2024-11-05 Thread Bertrand Drouvot
Hi, On Mon, Nov 04, 2024 at 10:01:50AM +, Bertrand Drouvot wrote: > And why not add more per-backend stats in the future? (once the I/O part is > done). > > I think that's one more reason to go with option 2 (and implementing a brand > new > PGSTAT_KIND_BACKEND kind)

Re: Clear padding in PgStat_HashKey keys

2024-11-04 Thread Bertrand Drouvot
at will add padding could miss this thread and spend some time to figure out why his patch does break existing stats. 3/ when dealing with structs that are used as hash key I think we should not wait for padding to be introduced. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Clear padding in PgStat_HashKey keys

2024-11-04 Thread Bertrand Drouvot
Hi, On Mon, Nov 04, 2024 at 04:25:00PM +0900, Michael Paquier wrote: > On Sun, Nov 03, 2024 at 04:25:41AM +0000, Bertrand Drouvot wrote: > > We are using sizeof(PgStat_HashKey) in pgstat_cmp_hash_key() and we compute > > the > > hash hash key in pgstat_hash_hash_key() usi

Re: relfilenode statistics

2024-11-04 Thread Bertrand Drouvot
Hi, On Tue, Sep 10, 2024 at 05:30:32AM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Sep 05, 2024 at 04:48:36AM +, Bertrand Drouvot wrote: > > Please find attached a mandatory rebase. > > > > In passing, checking if based on the previous discussion (and given

Re: relfilenode statistics

2024-11-04 Thread Bertrand Drouvot
On Mon, Nov 04, 2024 at 09:27:38AM +, Bertrand Drouvot wrote: > On Tue, Sep 10, 2024 at 05:30:32AM +0000, Bertrand Drouvot wrote: > > Hi, > > > > On Thu, Sep 05, 2024 at 04:48:36AM +0000, Bertrand Drouvot wrote: > > > Please find attached a mandatory rebase. >

Re: define pg_structiszero(addr, s, r)

2024-11-04 Thread Bertrand Drouvot
Hi, On Fri, Nov 01, 2024 at 12:50:10PM +, Bertrand Drouvot wrote: > Hi, > > On Fri, Nov 01, 2024 at 09:47:05PM +1300, David Rowley wrote: > > On Fri, 1 Nov 2024 at 20:49, Michael Paquier wrote: > > I've attached what I thought a more optimal version might look like

Re: Clear padding in PgStat_HashKey keys

2024-11-04 Thread Bertrand Drouvot
Hi, On Mon, Nov 04, 2024 at 06:49:04PM +0900, Michael Paquier wrote: > On Mon, Nov 04, 2024 at 08:52:04AM +0000, Bertrand Drouvot wrote: > > Yeah, but not only the relfilenode ones. All kinds were affected as random > > data > > was in the padding bytes for all of them. >

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-30 Thread Bertrand Drouvot
Hi, On Wed, Oct 30, 2024 at 12:40:05PM +0900, Michael Paquier wrote: > On Tue, Oct 29, 2024 at 02:06:11PM +0000, Bertrand Drouvot wrote: > > Yeah, agree that it would make sense to document in the test what has been > > discovered here. > > How about something like the atta

Re: define pg_structiszero(addr, s, r)

2024-11-01 Thread Bertrand Drouvot
Hi, On Fri, Nov 01, 2024 at 09:47:05PM +1300, David Rowley wrote: > On Fri, 1 Nov 2024 at 20:49, Michael Paquier wrote: > > > > On Fri, Nov 01, 2024 at 07:44:22AM +, Bertrand Drouvot wrote: > > > Worth to add a comment as to why pg_memory_is_all_zeros() shoul

Re: per backend I/O statistics

2024-10-30 Thread Bertrand Drouvot
Hi, On Tue, Oct 08, 2024 at 04:28:39PM +, Bertrand Drouvot wrote: > > > On Fri, Sep 20, 2024 at 01:26:49PM +0900, Michael Paquier wrote: > > > > Okay, per the above and the persistency of the stats. > > Great, I'll work on an updated patch version then. >

Re: define pg_structiszero(addr, s, r)

2024-10-30 Thread Bertrand Drouvot
_mem_is_all_zeros(pagebytes , (BLCKSZ / sizeof(size_t > > Extra space not required after pagebytes. Fat finger here, thanks! Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 88f

Clear padding in PgStat_HashKey keys

2024-11-02 Thread Bertrand Drouvot
.eu-west-3.compute.internal Looking forward to your feedback, Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From 0a590aad0ff014dff60e5ee73bef2c5d7a659804 Mon Sep 17 00:00:00 2001 From: Bertrand Drouvot Date: Sat, 2 N

Re: Avoid orphaned objects dependencies, take 3

2024-10-28 Thread Bertrand Drouvot
Hi, On Mon, Aug 19, 2024 at 03:35:14PM +, Bertrand Drouvot wrote: > Hi, > > On Wed, Jul 10, 2024 at 07:31:06AM +, Bertrand Drouvot wrote: > > So, to sum up: > > > > A. Locking is now done exclusively with LockNotPinnedObject(Oid classid, > > Oid objid)

Re: Add isolation test template in injection_points for wait/wakeup/detach

2024-10-28 Thread Bertrand Drouvot
failing for the same reason. It's not ideal, but instead of removing this first permutation test what about adding a "sleep2" step in it (doing say, SELECT pg_sleep(1);) and call this new step before the detach2 one? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define pg_structiszero(addr, s, r)

2024-11-11 Thread Bertrand Drouvot
Hi, On Mon, Nov 11, 2024 at 06:19:50AM +, Bertrand Drouvot wrote: > Hi, > > On Sat, Nov 09, 2024 at 04:15:04AM +, Bertrand Drouvot wrote: > > Hi, > > > > On Fri, Nov 08, 2024 at 11:18:09PM +1300, David Rowley wrote: > > > I tried with [1] and the

Re: define pg_structiszero(addr, s, r)

2024-11-08 Thread Bertrand Drouvot
t; > > Btw, gcc seems a bit smarter than clang when it comes to optimizing > > the code depending on the size of the structures. gcc gives up on > > SIMD if it's sure that the structure on which we are going to use the > > allzero check won't need it at all, and cl

Re: define pg_structiszero(addr, s, r)

2024-11-10 Thread Bertrand Drouvot
Hi, On Sat, Nov 09, 2024 at 04:15:04AM +, Bertrand Drouvot wrote: > Hi, > > On Fri, Nov 08, 2024 at 11:18:09PM +1300, David Rowley wrote: > > I tried with [1] and the > > latest gcc does not seem to be smart enough to figure this out. I > > tried adding some add

Re: Proper object locking for GRANT/REVOKE

2024-11-10 Thread Bertrand Drouvot
Hi, On Sat, Nov 09, 2024 at 01:43:13PM +0100, Peter Eisentraut wrote: > On 31.10.24 15:26, Bertrand Drouvot wrote: > > + address = get_object_address(objtype, lfirst(cell), &relation, lockmode, > > false); > > + Assert(relation == NULL); > > > > Worth to

Re: define pg_structiszero(addr, s, r)

2024-11-11 Thread Bertrand Drouvot
Hi, On Tue, Nov 12, 2024 at 12:28:53PM +0900, Michael Paquier wrote: > On Mon, Nov 11, 2024 at 05:07:51PM +0000, Bertrand Drouvot wrote: > > To handle the "what about the len check if the function is not inlined?", I > > can't think about a good approach. > >

Re: Refactor GetLockStatusData() by skipping unused backends and groups

2024-10-23 Thread Bertrand Drouvot
perhaps placing it inside the FAST_PATH_SLOT() macro could be an option... > Or current assertion check is enough? Thought? Given that it's already done in FAST_PATH_GET_BITS(), I think that's fine as it is and v2 LGTM. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Bertrand Drouvot
e_t *) p)[7] != 0)) > return false; > } > } > > See v1_allzeros_small.c attached. Thanks for looking at it! It's not enough, as that would not fix the second reason mentioned in [1]. [1]: https://www.postgresql.org/message-id/ZzLxAJuGzyqA7cUo%40ip-10

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Bertrand Drouvot
ry_is_all_zeros_v12() then, thoughts? [1]: https://www.postgresql.org/message-id/CAApHDvp2jx_%3DpFbgj-O1_ZmzP9WOZKfwLzZrS_%3DZmbsqMQQ59g%40mail.gmail.com [2]: https://godbolt.org/z/8s44GKqcc Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Servi

Re: Speed up TransactionIdIsCurrentTransactionId() with lots of subxacts

2024-11-13 Thread Bertrand Drouvot
t;childXids = NULL; - s->nChildXids = 0; - s->maxChildXids = 0; + /* nothing to do */ Yeah that works but then some CurrentTransactionIds[] elements do not reflect the "reality" anymore (until the top level transaction finishes, or until a new subtransaction is crea

Re: small pg_combinebackup improvements

2024-10-31 Thread Bertrand Drouvot
t;); s/pg_combinebackup fails/pg_combinebackup fails due to an unexpected incremental file/? > Review appreciated. My plan is to commit at least to master and > possibly back-patch. Opinions on whether to back-patch are also > appreciated. I'm not sure about 0001 but I think 0002 deserves a back patch as I think it fits into the "low-risk fixes" category [0]. [0]: https://www.postgresql.org/support/versioning/ Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Proper object locking for GRANT/REVOKE

2024-10-31 Thread Bertrand Drouvot
dExtended()). The fact that we use lockmode != NoLock in the patch produces a lock followed by a "retry" in RangeVarGetRelidExtended() and so we get the "relation "intra_grant_inplace" does not exist" error. I think that the new behavior is still correct and in fact is more "appropriate" ( I mean that's the kind of error I expect to see from a user point of view). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define pg_structiszero(addr, s, r)

2024-11-01 Thread Bertrand Drouvot
ry_is_all_zeros() should not be used here? Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: small pg_combinebackup improvements

2024-11-01 Thread Bertrand Drouvot
Hi, On Thu, Oct 31, 2024 at 12:06:25PM -0400, Robert Haas wrote: > On Thu, Oct 31, 2024 at 11:41 AM Bertrand Drouvot > wrote: > > I'm not sure about 0001 but I think 0002 deserves a back patch as I think > > it fits > > into the "low-risk fixes" catego

Re: define pg_structiszero(addr, s, r)

2024-10-31 Thread Bertrand Drouvot
see [0]) and it looks like it's significantly slower using the new inline function. We could try to write a more elaborate version of pg_memory_is_all_zeros(), but as it looks like there is only one use case, then it's probably better to not implement (revert) this change here and "just" add a comment as to why pg_memory_is_all_zeros() should not be used here, thoughts? [0]: https://godbolt.org/z/xqnW4MPY5 Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define pg_structiszero(addr, s, r)

2024-11-08 Thread Bertrand Drouvot
LCKSZ of 121: gcc: then SIMD instructions are used (I think that's because sizeof(AlignedBlock) is 128 which is a multiple of 16 (SIMD xmm register size)). While clang uses SIMD instructions in both cases (more complex code with more branches at least in the 120 case). Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: define pg_structiszero(addr, s, r)

2024-11-08 Thread Bertrand Drouvot
de will read up to 48 bytes beyond the struct's memory (which is 16 bytes): This is fine: " movdqu xmm0, XMMWORD PTR [rdi] " But I don't think it is: " movdqu xmm2, XMMWORD PTR [rdi+16] movdqu xmm1, XMMWORD PTR [rdi+32] movdqu xmm3, XMMWORD PTR [rdi+48] " g

Re: pg_combinebackup --incremental

2024-11-08 Thread Bertrand Drouvot
25: error: continue statement not within a loop 766 | continue; | ^~~~ reconstruct.c:768:28: error: ‘s’ undeclared (first use in this function) 768 | read_block(s, offset, buffer); |^ make[3]: *** [../../../src/Makefile.global:961: recons

Re: define pg_structiszero(addr, s, r)

2024-11-12 Thread Bertrand Drouvot
Hi, On Tue, Nov 12, 2024 at 03:56:13PM +0900, Michael Paquier wrote: > On Tue, Nov 12, 2024 at 06:09:04AM +0000, Bertrand Drouvot wrote: > > I think that the 64b len check done in v11 is mandatory for safety reasons. > > > > The loop above reads 64 bytes at once, so would r

Re: define pg_structiszero(addr, s, r)

2024-09-18 Thread Bertrand Drouvot
Hi, On Wed, Sep 18, 2024 at 10:03:21AM +0200, Peter Eisentraut wrote: > On 18.09.24 06:16, Bertrand Drouvot wrote: > > +#define pg_structiszero(addr, s, r) > >

Re: Add contrib/pg_logicalsnapinspect

2024-09-24 Thread Bertrand Drouvot
Hi, On Tue, Sep 24, 2024 at 09:15:31AM +0530, shveta malik wrote: > On Fri, Sep 20, 2024 at 12:22 PM Bertrand Drouvot > wrote: > > > > > > Please find attached v8, that: > > > > Thank You for the patch. In one of my tests, I noticed that I got > negative

Re: Add contrib/pg_logicalsnapinspect

2024-09-24 Thread Bertrand Drouvot
s to be done" -- the word "needs" is > incorrect here. > > How about: > "...if a change needs to be made to SnapBuildState." Thanks, used this one in v9. [1]: https://www.postgresql.org/message-id/CAJpy0uCppUNdod4F3NaPpMCtrySdw1S0T1d8CA-2c4CX%3DShMOQ%40mail.gm

Re: Add contrib/pg_logicalsnapinspect

2024-09-19 Thread Bertrand Drouvot
it that way instead of implementing the enum -> text mapping in snapbuild.h (as the mapping is only used by the module). [1]: https://www.postgresql.org/message-id/CAJpy0uDJ65QHUZfww7n6TBZAGp-SP74P5U3fUorV%2B%3DbaaRu6Dw%40mail.gmail.com [2]: https://www.postgresql.org/message-id/CAA4eK1JgW1o9wOTwgR

Re: Add contrib/pg_logicalsnapinspect

2024-09-18 Thread Bertrand Drouvot
l/snapshots directory > (i.e. use server keyword, similar to how pg_ls_logicalsnapdir , > pg_ls_logicalmapdir explains it) Agree, done that way in v6. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com >From a

Re: Track the amount of time waiting due to cost_delay

2024-09-19 Thread Bertrand Drouvot
Hi, On Wed, Sep 18, 2024 at 04:04:53PM -0500, Nathan Bossart wrote: > On Thu, Sep 05, 2024 at 04:59:54AM +0000, Bertrand Drouvot wrote: > > Please find attached v6, a mandatory rebase due to catversion bump conflict. > > I'm removing the catversion bump from the patch as it ge

Re: Add contrib/pg_logicalsnapinspect

2024-09-18 Thread Bertrand Drouvot
d.xcnt; (*not xcnt_space*) > + * TransactionId * catchange.xcnt; > + * > + */ > +typedef struct SnapBuildOnDisk > > was directly beneath the comment: > -/* --- > - * Snapshot serialization support > - * --- >

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Bertrand Drouvot
So, the Int64GetDatum() output would still stay in the "xid" range. Keeping xid in the .sql makes it clear that we are dealing with transaction ID. Regards, -- Bertrand Drouvot PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Add contrib/pg_logicalsnapinspect

2024-09-25 Thread Bertrand Drouvot
not open file \"%s\": %m", > path))); > > Couldn't you just use the second branch for all errno's? Yeah, I think it comes from copying/pasting from SnapBuildRestore() too "quickly". v10 attached uses the second branch only. Regards, -- Bertrand

define pg_structiszero(addr, s, r)

2024-09-17 Thread Bertrand Drouvot
o is created in pgstat_internal.h as it looks like that "only" the statistics related code would benefit of it currently (could be moved to other header file later on if needed). [1]: https://www.postgresql.org/message-id/20230105002733.ealhzubjaiqis6ua%40awork3.anarazel.de Looki

Re: define pg_structiszero(addr, s, r)

2024-09-18 Thread Bertrand Drouvot
Hi, On Wed, Sep 18, 2024 at 03:07:15PM +0900, Michael Paquier wrote: > On Wed, Sep 18, 2024 at 04:16:12AM +0000, Bertrand Drouvot wrote: > > The macro is created in pgstat_internal.h as it looks like that "only" the > > statistics related code would benefit of it cur

Re: per backend I/O statistics

2024-11-07 Thread Bertrand Drouvot
Hi, On Thu, Nov 07, 2024 at 09:50:59AM +0900, Michael Paquier wrote: > On Wed, Nov 06, 2024 at 01:51:02PM +0000, Bertrand Drouvot wrote: > > That's not needed, the patch I'm working on stores the proc number in the > > objid field of the key. > > Relying on the pr

<    1   2   3   4   5   6   7   8   9   >