Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-29 Thread Maksim Milyutin
On 29.04.2024 23:59, Thomas Munro wrote: On Tue, Apr 30, 2024 at 7:17 AM Thomas Munro wrote: On Tue, Apr 30, 2024 at 6:47 AM Maksim Milyutin wrote: Should not we call at the end the StrategyFreeBuffer() function to add target buffer to freelist and not miss it after invalidation? Please ta

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-29 Thread Thomas Munro
On Tue, Apr 30, 2024 at 7:17 AM Thomas Munro wrote: > On Tue, Apr 30, 2024 at 6:47 AM Maksim Milyutin wrote: > >> Should not we call at the end the StrategyFreeBuffer() function to add > >> target buffer to freelist and not miss it after invalidation? > > > Please take a look at this issue, curr

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-29 Thread Thomas Munro
On Tue, Apr 30, 2024 at 6:47 AM Maksim Milyutin wrote: >> Should not we call at the end the StrategyFreeBuffer() function to add >> target buffer to freelist and not miss it after invalidation? > Please take a look at this issue, current implementation of > EvictUnpinnedBuffer() IMO is erroneou

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-29 Thread Maksim Milyutin
On 14.04.2024 21:16, Maksim Milyutin wrote: On 07.04.2024 02:07, Thomas Munro wrote: So this is the version I plan to commit. +bool +EvictUnpinnedBuffer(Buffer buf) +{ ... +/* This will return false if it becomes dirty or someone else pins it. */ +result = InvalidateVictimBuffer(desc)

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-14 Thread Maksim Milyutin
On 07.04.2024 02:07, Thomas Munro wrote: So this is the version I plan to commit. +bool +EvictUnpinnedBuffer(Buffer buf) +{ ... +/* This will return false if it becomes dirty or someone else pins it. */ +result = InvalidateVictimBuffer(desc); + +UnpinBuffer(desc); + +return resu

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Thomas Munro
On Mon, Apr 8, 2024 at 11:53 AM Melanie Plageman wrote: > I've reviewed v6. I think you should mention in the docs that it only > works for shared buffers -- so specifically not buffers containing > blocks of temp tables. Thanks for looking! The whole pg_buffercache extension is for working with

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Thomas Munro
On Mon, Apr 8, 2024 at 12:10 PM Andres Freund wrote: > On 2024-04-07 11:07:58 +1200, Thomas Munro wrote: > > I thought of a better name for the bufmgr.c function though: > > InvalidateUnpinnedBuffer(). That name seemed better to me after I > > festooned it with warnings about why exactly it's inh

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 11:07:58 +1200, Thomas Munro wrote: > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I > festooned it with warnings about why exactly it's inherently racy and > only for testing use. I still dislike

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Melanie Plageman
On Sat, Apr 6, 2024 at 7:08 PM Thomas Munro wrote: > > On second thoughts, I think the original "invalidate" terminology was > fine, no need to invent a new term. > > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I >

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-06 Thread Thomas Munro
On second thoughts, I think the original "invalidate" terminology was fine, no need to invent a new term. I thought of a better name for the bufmgr.c function though: InvalidateUnpinnedBuffer(). That name seemed better to me after I festooned it with warnings about why exactly it's inherently rac

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-03 Thread Thomas Munro
On Fri, Mar 8, 2024 at 6:20 AM Maxim Orlov wrote: > Quite an interesting patch, in my opinion. I've decided to work on it a bit, > did some refactoring (sorry) and add > basic tests. Also, I try to take into account as much as possible notes on > the patch, mentioned by Cédric Villemain. Thank

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-03-07 Thread Maxim Orlov
Quite an interesting patch, in my opinion. I've decided to work on it a bit, did some refactoring (sorry) and add basic tests. Also, I try to take into account as much as possible notes on the patch, mentioned by Cédric Villemain. > and maybe better to go with FlushOneBuffer() ? It's a good idea,

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-02-26 Thread Thomas Munro
[Sorry to those who received this message twice -- the first time got bounced by the list because of a defunct email address in the CC list.] Here is a rebase of Palak's v2 patch. I didn't change anything except for the required resource manager API change, a pgindent run, and removal of a stray

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-14 Thread Cédric Villemain
Hi Palak, there is currently even more interest in your patch as it should help building tests for on-going development around cache/read management/effects. Do you expect to be able to follow-up in the coming future ? Thank you, Cédric On 04/01/2024 00:15, Jim Nasby wrote: On 1/3/24 10:25

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-03 Thread Jim Nasby
On 1/3/24 10:25 AM, Cédric Villemain wrote: Hi Palak, I did a quick review of the patch: +CREATE FUNCTION pg_buffercache_invalidate(IN int, IN bool default true) +RETURNS bool +AS 'MODULE_PATHNA

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-01-03 Thread Cédric Villemain
Hi Palak, I did a quick review of the patch: +CREATE FUNCTION pg_buffercache_invalidate(IN int, IN bool default true) +RETURNS bool +AS 'MODULE_PATHNAME', 'pg_buffercache_invalidate' +LANGUAGE C PARALLEL SAFE; --> Not enforced anywhere, but you can also add a comment to the function, for end u

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-11-22 Thread Cédric Villemain
Le 01/07/2023 à 00:09, Thomas Munro a écrit : On Fri, Jun 30, 2023 at 10:47 PM Palak Chaturvedi wrote: We also talked a bit about how one might control the kernel page cache in more fine-grained ways for testing purposes, but it seems like the pgfincore project has that covered with its pgfad

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-31 Thread Palak Chaturvedi
Hii, Thanks for your feedback. We have decided to add a role for the extension to solve that problem. And concerning to pg_unwarm table I think we can create a new function to do that but I think a general user would not require to clear a table from buffercache. We can use bufferid and where state

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-28 Thread Cary Huang
Hello I had a look at the patch and tested it on CI bot, it compiles and tests fine both autoconf and meson. I noticed that the v2 patch contains the v1 patch file as well. Not sure if intended but put there my accident. > I don't think "invalidating" is the right terminology. Note that we alr

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-18 Thread Thomas Munro
On Wed, Jul 19, 2023 at 12:45 PM Andres Freund wrote: > I don't think "invalidating" is the right terminology. Note that we already > have InvalidateBuffer() - but it's something we can't allow users to do, as it > throws away dirty buffer contents (it's used for things like dropping a > table). >

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-18 Thread Andres Freund
Hi, I wanted this feature a couple times before... On 2023-07-03 13:56:29 +0530, Palak Chaturvedi wrote: > +PG_FUNCTION_INFO_V1(pg_buffercache_invalidate); > +Datum > +pg_buffercache_invalidate(PG_FUNCTION_ARGS) I don't think "invalidating" is the right terminology. Note that we already have In

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-11 Thread Palak Chaturvedi
Can you please review the new patch of the extension with implemented force variable. On Tue, 11 Jul 2023 at 18:08, Palak Chaturvedi wrote: > > Hey Nitin, > >Will > >there be a scenario where the buffer is dirty and its reference count > >is zero? > There might be a buffer that has been dirtied b

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-11 Thread Palak Chaturvedi
Hey Nitin, >Will >there be a scenario where the buffer is dirty and its reference count >is zero? There might be a buffer that has been dirtied but is not pinned or being used currently by a process. So checking the refcount and then dirty buffers helps. >First, The TryInvalidateBuffer() tries to f

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-05 Thread Nitin Jadhav
+1 for the idea. It's going to be more useful to test and understand the buffer management of PostgreSQL and it can be used to explicitly free up the buffers if there are any such requirements. I had a quick look over the patch. Following are the comments. First, The TryInvalidateBuffer() tries t

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-04 Thread jian he
On Sat, Jul 1, 2023 at 6:09 AM Thomas Munro wrote: > > > It should be restricted by role, but I wonder which role it should be. > Testing for superuser is now out of fashion. > as pg_buffercache/pg_buffercache--1.2--1.3.sql. You need pg_maintain privilege to use pg_buffercache. The following quer

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-04 Thread jian he
On Tue, Jul 4, 2023 at 5:45 PM Japin Li wrote: > > On Tue, 04 Jul 2023 at 17:00, jian he wrote: > > the following will also crash. no idea why. > > begin; > > select count(*) from onek; > > select relpages from pg_class where relname = 'onek'; --queryA > > > > SELECT count(*) FROM pg

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-04 Thread Japin Li
On Tue, 04 Jul 2023 at 17:00, jian he wrote: > the following will also crash. no idea why. > begin; > select count(*) from onek; > select relpages from pg_class where relname = 'onek'; --queryA > > SELECT count(*) FROM pg_buffercache WHERE relfilenode = > pg_relation_filenode('onek':

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-04 Thread jian he
the following will also crash. no idea why. begin; select count(*) from onek; select relpages from pg_class where relname = 'onek'; --queryA SELECT count(*) FROM pg_buffercache WHERE relfilenode = pg_relation_filenode('onek'::regclass); --queryB insert into onek values(default);

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-04 Thread Japin Li
On Mon, 03 Jul 2023 at 16:26, Palak Chaturvedi wrote: > Hi Thomas, > Thank you for your suggestions. I have added the sql in the meson > build as well. > > On Sat, 1 Jul 2023 at 03:39, Thomas Munro wrote: >> >> On Fri, Jun 30, 2023 at 10:47 PM Palak Chaturvedi >> wrote: >> > pgbench=# select

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-03 Thread Palak Chaturvedi
hi, I don't think we need to check the usage count. Because we are clearing all the buffers that are not pinned. Checking the usage count is for buffer replacement since we are not replacing it does not matter. On Mon, 3 Jul 2023 at 21:16, jian he wrote: > > On Mon, Jul 3, 2023 at 4:26 PM Palak Ch

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-03 Thread jian he
On Mon, Jul 3, 2023 at 4:26 PM Palak Chaturvedi wrote: > > Hi Thomas, > Thank you for your suggestions. I have added the sql in the meson > build as well. > > On Sat, 1 Jul 2023 at 03:39, Thomas Munro wrote: > > > > On Fri, Jun 30, 2023 at 10:47 PM Palak Chaturvedi > > wrote: > > > pgbench=# sel

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-03 Thread Palak Chaturvedi
Hi Thomas, Thank you for your suggestions. I have added the sql in the meson build as well. On Sat, 1 Jul 2023 at 03:39, Thomas Munro wrote: > > On Fri, Jun 30, 2023 at 10:47 PM Palak Chaturvedi > wrote: > > pgbench=# select count(pg_buffercache_invalidate(bufferid)) from > > pg_buffercache wher

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-06-30 Thread Thomas Munro
On Fri, Jun 30, 2023 at 10:47 PM Palak Chaturvedi wrote: > pgbench=# select count(pg_buffercache_invalidate(bufferid)) from > pg_buffercache where relfilenode = > pg_relation_filenode('pgbench_accounts'::regclass); Hi Palak, Thanks for working on this! I think this will be very useful for testi

Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-06-30 Thread Palak Chaturvedi
I hope this email finds you well. I am excited to share that I have extended the functionality of the `pg_buffercache` extension by implementing buffer invalidation capability, as requested by some PostgreSQL contributors for improved testing scenarios. This marks my first time submitting a patch