Re: Count and log pages set all-frozen by vacuum

2024-11-02 Thread Nitin Jadhav
se the flags parameter to return old_vmbits, making it an in-out parameter. Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft On Fri, Nov 1, 2024 at 4:00 AM Melanie Plageman wrote: > > Thanks for the review! > > On Thu, Oct 31, 2024 at 2:13 PM Masahiko Sawada

Re: Enhancing Memory Context Statistics Reporting

2024-10-26 Thread Nitin Jadhav
> * Shared memory is full, release lock and write to file from next > * iteration > */ >context_id++; >if (context_id == 29) >{ What if there are exactly 29 entries in the memory context? In that case, creating the file would be

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2024-10-07 Thread Nitin Jadhav
://www.postgresql.org/message-id/flat/CAOtHd0ApHna7_p6mvHoO%2BgLZdxjaQPRemg3_o0a4ytCPijLytQ%40mail.gmail.com#74ae447064932198495aa6d722fdc092 [2]: https://www.postgresql.org/message-id/CALj2ACVxX2ii=66rypxrweze2esbripmj0ahfrfhuexjcc7...@mail.gmail.com Best Regards, Nitin Jadhav Azure Database for

Re: Inconsistency in reporting checkpointer stats

2024-10-01 Thread Nitin Jadhav
h "SLRU" for > consistency, > as uppercase is typically used for SLRU. > - I updated the commit message. Thanks for updating. Looks good to me. Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft On Tue, Oct 1, 2024 at 9:28 PM Fujii Masao wrote: > > > >

Re: Inconsistency in reporting checkpointer stats

2024-09-22 Thread Nitin Jadhav
ver, I’m unsure if this effort is justified. While it makes sense for normal buffers to display the percentage, the additional code required might not provide significant value to users. Therefore, I have removed this in the attached v6 patch. If it is really required, I am happy to make the above

Re: Extensible storage manager API - SMGR hook Redux

2024-09-21 Thread Nitin Jadhav
your thoughts on this? Also, let me know if this topic has already been discussed and if any conclusions were reached. Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft On Sat, Jan 13, 2024 at 1:27 AM Tristan Partin wrote: > > Thought I would show off what is possible wit

Re: Inconsistency in reporting checkpointer stats

2024-07-18 Thread Nitin Jadhav
buffers data in a distinct field. I have created the new commitfest entry here https://commitfest.postgresql.org/49/5130/. Kindly share if any comments. Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft On Tue, Jan 30, 2024 at 1:20 PM Michael Paquier wrote: > > On Sat,

Address the -Wuse-after-free warning in ATExecAttachPartition()

2024-07-08 Thread Nitin Jadhav
/flat/202311151802.ngj2la66jwgi%40alvherre.pgsql#4fc5622772ba0244c1ad203f5fc56701 Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft v1-0001-Address-the-Wuse-after-free-warning-in-ATExecAttachP.patch Description: Binary data

Re: Show WAL write and fsync stats in pg_stat_io

2024-07-06 Thread Nitin Jadhav
e representation. Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft On Mon, Jun 17, 2024 at 8:23 PM Melanie Plageman wrote: > > On Thu, Jun 13, 2024 at 5:24 AM Nazir Bilal Yavuz wrote: > > > > On Sun, 9 Jun 2024 at 18:05, Nitin Jadhav > > wrote: > &g

Re: Show WAL write and fsync stats in pg_stat_io

2024-06-09 Thread Nitin Jadhav
ers in making educated choices when fine-tuning their database. However, if we include the hit count of WAL buffers in this, it may lead to misleading interpretations for database tuning. If there’s something I’ve overlooked that’s already been discussed, please feel free to correct me. Best Regar

Re: Use WALReadFromBuffers in more places

2024-06-08 Thread Nitin Jadhav
’. - The operations performed by logical_read_xlog_page() and read_local_xlog_page_guts() are identical. It might be beneficial to create a shared function to minimize code repetition. Best Regards, Nitin Jadhav Azure Database for PostgreSQL Microsoft On Mon, May 13, 2024 at 12:17 PM Bharath

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2023-07-05 Thread Nitin Jadhav
rty buffer before > invalidating. > +If the buffer is still dirty it returns false. > +*/ > +bool The star(*) and space are missing here. Please refer to the style of function comments and change accordingly. Thanks & Regards, Nitin Jadhav On Fri, Jun 30, 2023 at 4:17 PM Palak Chaturvedi w

Re: Improve WALRead() to suck data directly from WAL buffers when possible

2023-03-11 Thread Nitin Jadhav
GB wal_writer_delay=1ms ./pgbench --initialize --scale=300 postgres WAL buffers hit=37214 WAL buffers miss=844 Please share your thoughts. Thanks & Regards, Nitin Jadhav On Tue, Mar 7, 2023 at 12:39 PM Bharath Rupireddy wrote: > > On Tue, Mar 7, 2023 at 3:30 AM Nathan Bos

Re: Inconsistency in reporting checkpointer stats

2023-02-19 Thread Nitin Jadhav
there's no need for 2 separate patches for these changes. > I will make it a single patch while sharing the next patch. Clubbed both patches into one. Thanks & Regards, Nitin Jadhav On Tue, Feb 14, 2023 at 6:08 AM Andres Freund wrote: > > > On 2022-12-21 17:14:12 +0530, Niti

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-08 Thread Nitin Jadhav
0002, only for > HEAD, as that would be a new check. > > Remains > 0002, that I am letting sleep to see if there's interest for it, or > perhaps more ideas around it. Makes sense and the patch looks good to me. Thanks & Regards, Nitin Jadhav On Wed, Feb 8, 2023 at 1:29

Re: Add a test case related to the error "cannot fetch toast data without an active snapshot"

2023-02-07 Thread Nitin Jadhav
ate_series(1, 1000))); DO $$ DECLARE v_r record; DECLARE vref_cursor REFCURSOR; BEGIN OPEN vref_cursor FOR SELECT data FROM toasted; LOOP fetch vref_cursor into v_r; INSERT INTO toasted(data) VALUES(v_r.data); COMMIT; END LOOP; END;$$; Thanks & Regards, Nitin Jadhav On Fri, Jan 27, 2023 at 6:26

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-04 Thread Nitin Jadhav
are your thoughts. Thanks & Regards, Nitin Jadhav On Sat, Feb 4, 2023 at 1:07 AM Tom Lane wrote: > > Nitin Jadhav writes: > > My concern is if we do this, then we will end up having some policies > > (which can be read from pg_show_all_settings()) in guc.sql and some in >

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-03 Thread Nitin Jadhav
v1 patch and using this (as true) while creating the table tab_settings_flags in guc.sq and just remove (NO_SHOW_ALL && !NO_RESET_ALL) check from guc.sql. I don't think doing this is a problem as we can retain the support of existing signatures of the pg_show_all_settings function as

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-30 Thread Nitin Jadhav
s now. Similarly why can't we have a GUC marked as GUC_NO_SHOW_ALL but not GUC_NOT_IN_CONFIG. For me it makes sense if a GUC is marked as NO_SHOW_ALL and it can be present in a sample file. It's up to the author/developer to choose which all flags are applicable to the newly introduced GUCs. Ple

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-29 Thread Nitin Jadhav
t/CALj2ACXZMOGEtjk%2Beh0Zeiz%3D46ETVkr0koYAjWt8SoJUJJUe9g%40mail.gmail.com#04705e421e0dc63b1f0c862ae4929e6f Thanks & Regards, Nitin Jadhav On Wed, Jan 18, 2023 at 12:31 PM Nitin Jadhav wrote: > > > We would miss the names of the parameters that are marked as NO_SHOW, > > missing from pg_settin

Re: Inconsistency in reporting checkpointer stats

2023-01-27 Thread Nitin Jadhav
e. The v1 patch corresponds to what you are suggesting. But the question is not about tracking slru buffers, it is about separating this information from main buffers count during checkpoint. I think there is enough discussion done upthread to exclude slru buffers from CheckpointStats.ckpt_bufs_written

Re: Improve GetConfigOptionValues function

2023-01-27 Thread Nitin Jadhav
hanks & Regards, Nitin Jadhav On Wed, Jan 25, 2023 at 9:23 PM Tom Lane wrote: > > Nitin Jadhav writes: > > I agree that the developer can use both GUC_NO_SHOW_ALL and > > GUC_EXPLAIN knowingly or unknowingly for a single GUC. If used by > > mistake

Add a test case related to the error "cannot fetch toast data without an active snapshot"

2023-01-27 Thread Nitin Jadhav
* from detecting the problem. But it's better than nothing, and for sure * we shouldn't expend code on masking the problem more.) */ Thanks & Regards, Nitin Jadhav

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
veloper can use both GUC_NO_SHOW_ALL and GUC_EXPLAIN knowingly or unknowingly for a single GUC. If used by mistake then according to the existing code (without patch), GUC_NO_SHOW_ALL takes higher precedence whether it is marked first or last in the code. I am more convinced with this behaviour as I

Re: Improve GetConfigOptionValues function

2023-01-25 Thread Nitin Jadhav
atch as it reduces the code size. Thanks & Regards, Nitin Jadhav On Mon, Jan 23, 2023 at 9:51 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > LGTM. I've marked it RfC. > > After looking at this, it seemed to me that the factorization > wasn't quite right

Re: Improve GetConfigOptionValues function

2023-01-23 Thread Nitin Jadhav
o not lose track of it. Added https://commitfest.postgresql.org/42/4140/ Thanks & Regards, Nitin Jadhav On Mon, Jan 23, 2023 at 11:30 AM Bharath Rupireddy wrote: > > On Thu, Jan 19, 2023 at 3:27 PM Nitin Jadhav > wrote: > > > > > Possibly a better answer is to refactor

Re: Improve GetConfigOptionValues function

2023-01-19 Thread Nitin Jadhav
hed a patch for the same. Please share the comments if any. Thanks & Regards, Nitin Jadhav On Wed, Jan 18, 2023 at 9:44 PM Tom Lane wrote: > > Nitin Jadhav writes: > > GetConfigOptionValues function extracts the config parameters for the > > given variable irrespective of wh

Re: Improve GetConfigOptionValues function

2023-01-19 Thread Nitin Jadhav
the number of such parameters (GUC_NO_SHOW_ALL and GUC_SUPERUSER_ONLY) are less, we may not see improvements in performance. We can treat it as a kind of refactoring. Thanks & Regards, Nitin Jadhav On Wed, Jan 18, 2023 at 1:47 PM Bharath Rupireddy wrote: > > On Wed, Jan 18, 2023

Re: Improve GetConfigOptionValues function

2023-01-17 Thread Nitin Jadhav
Attaching the patch. On Wed, Jan 18, 2023 at 1:21 PM Nitin Jadhav wrote: > > Hi, > > GetConfigOptionValues function extracts the config parameters for the > given variable irrespective of whether it results in noshow or not. > But the parent function show_all_settings

Improve GetConfigOptionValues function

2023-01-17 Thread Nitin Jadhav
w. So a return statement in GetConfigOptionValues() when noshow is set to true is needed. Attached the patch for the same. Please share your thoughts. Thanks & Regards, Nitin Jadhav

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-17 Thread Nitin Jadhav
TTINGS)) || ((conf->flags & GUC_SUPERUSER_ONLY) && !has_privs_of_role(GetUserId(), ROLE_PG_READ_ALL_SETTINGS))) *noshow = true; else *noshow = false; } - - - } On Mon, Jan 16, 2023 at 7:58 AM Michael Paquier wrote: > > On S

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-14 Thread Nitin Jadhav
;all_params_array' in 003_check_guc.pl whenever we add such GUCs. I am not able to choose any of the above options as each has some disadvantages but if no other options exist, then I would like to go with option-3 as it validates more than the one currently doing. Please share if any other b

Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-01-13 Thread Nitin Jadhav
work on the patch if a fix is required. [1]: SELECT name FROM pg_settings WHERE NOT 'NOT_IN_SAMPLE' = ANY (pg_settings_get_flags(name)) AND name <> 'config_file' ORDER BY 1; Thanks & Regards, Nitin Jadhav

Re: Split index and table statistics into different types of stats

2023-01-09 Thread Nitin Jadhav
mmon to table/index stats? +1 to keep common functions/code between table and index stats. Only the data structure should be different as the goal is to shrink the current memory usage. Thanks & Regards, Nitin Jadhav On Thu, Jan 5, 2023 at 3:35 PM Drouvot, Bertrand wrote: > > Hi, &g

Re: Inconsistency in reporting checkpointer stats

2022-12-21 Thread Nitin Jadhav
; write=0.045 s, sync=0.161 s, total=0.244 s; sync files=25, longest=0.146 s, average=0.007 s; distance=66130 kB, estimate=66130 kB; lsn=0/5557C78, redo lsn=0/5557C40 Thanks & Regards, Nitin Jadhav On Tue, Dec 20, 2022 at 11:08 PM Andres Freund wrote: > > On 2022-12-20 08:18:36 -0500, R

Re: Inconsistency in reporting checkpointer stats

2022-12-20 Thread Nitin Jadhav
ist ProcessSyncRequests() up into CheckPointGuts() to make it clearer that it applies to all the SLRU mini-buffer-pools as well as the main buffer pool. Rearrange things so that data collected in CheckpointStats includes SLRU activity. Thanks & Regards, Nitin Jadhav On Tue, Dec 20, 2022 at 2:38

Re: Inconsistency in reporting checkpointer stats

2022-12-15 Thread Nitin Jadhav
fdata, aka outside the context of a checkpoint or > shutdown sequence. Sorry. I missed adding braces. Fixed in the v2 patch attached. Thanks & Regards, Nitin Jadhav On Thu, Dec 15, 2022 at 3:16 AM Michael Paquier wrote: > > On Wed, Dec 14, 2022 at 04:54:53PM +0530, Bhar

Re: Inconsistency in reporting checkpointer stats

2022-12-15 Thread Nitin Jadhav
is before the end of the checkpoint. Please share if you have any other ideas. On Wed, Dec 14, 2022 at 4:55 PM Bharath Rupireddy wrote: > > On Wed, Dec 14, 2022 at 1:02 PM Nitin Jadhav > wrote: > > > > Hi, > > > > While working on checkpoint related stuff, I

Inconsistency in reporting checkpointer stats

2022-12-13 Thread Nitin Jadhav
same. Please share your thoughts. Thanks & Regards, Nitin Jadhav v1-0001-Fix-inconsistency-in-checkpointer-stats.patch Description: Binary data

Re: Introduce a new view for checkpointer related stats

2022-12-13 Thread Nitin Jadhav
The patch looks good to me. Thanks & Regards, Nitin Jadhav On Fri, Dec 2, 2022 at 11:20 AM Bharath Rupireddy wrote: > > On Wed, Nov 30, 2022 at 5:15 PM Bharath Rupireddy > wrote: > > > > I don't have a strong opinion about changing column names. However, if >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-11-15 Thread Nitin Jadhav
message, we should increment all three here, buffers_total, buffers_processed and buffers_written. So the total number of buffers calculated earlier may not always be the same. If this looks good, I will update this in the next patch. Thanks & Regards, Nitin Jadhav On Fri, Nov 4, 2022 at

Re: Skipping schema changes in publication

2022-08-18 Thread Nitin Jadhav
I spent some time on understanding the proposal and the patch. Here are a few comments wrt the test cases. > +ALTER PUBLICATION testpub_reset ADD TABLE pub_sch1.tbl1; > + > +-- Verify that tables associated with the publication are dropped after RESET > +\dRp+ testpub_reset > +ALTER PUBLICATION te

Re: Generalize ereport_startup_progress infrastructure

2022-08-10 Thread Nitin Jadhav
"log_long_running_operations = startup, postmaster" will enable logging for startup and postmaster operations and disables logging of other long running operations. With this the number of GUCs will be limited to 2 and it is simple and easy for the user. Thanks &a

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-07-28 Thread Nitin Jadhav
erage of 5 checkpoints. Here are the details. With patch: 2.457 s Without patch: 2.334 s Please share your thoughts. Thanks & Regards, Nitin Jadhav On Thu, Jul 7, 2022 at 5:34 AM Andres Freund wrote: > > Hi, > > On 2022-06-13 19:08:35 +0530, Nitin Jadhav wrote: > >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-06-13 Thread Nitin Jadhav
WHEN 3 THEN 'checkpointing replication slots' > + WHEN 4 THEN 'checkpointing logical replication > snapshot files' > + WHEN 5 THEN 'checkpointing logical rewrite mapping > files' > + WHEN

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-06-13 Thread Nitin Jadhav
* views add up to a measurable part of > the size of an empty database: Thank you so much for sharing the detailed analysis. We can remove a few fields which are not so important to make it simple. Thanks & Regards, Nitin Jadhav On Sat, Mar 19, 2022 at 5:45 AM Andres Freund wrote: > &g

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-06-05 Thread Nitin Jadhav
Hi, Here is the update patch which fixes the previous comments discussed in this thread. I am sorry for the long gap in the discussion. Kindly let me know if I have missed any of the comments or anything new. Thanks & Regards, Nitin Jadhav On Fri, Mar 18, 2022 at 4:52 PM Nitin Jadhav w

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-18 Thread Nitin Jadhav
_walinspect.c:415:16: error: ‘XLogReaderState’ {aka ‘struct XLogReaderState’} has no member named ‘blocks’ 415 | record->blocks[block_id].hole_length); |^~ make: *** [../../src/Makefile.global:941: pg_walinspect.o] Error 1 Thanks & Regards, Nitin Jadhav On Th

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-18 Thread Nitin Jadhav
so much to display "how the checkpoint > was initially started" rather than "how the checkpoint is really behaving > right > now". The whole point of having a progress view is to have something dynamic > that reflects the current activity. As of now

Re: Query about time zone patterns in to_char

2022-03-15 Thread Nitin Jadhav
th that, so it appears correct > to me. > > Therefore, I have committed it. Thank you so much. Thanks & Regards, Nitin Jadhav On Tue, Mar 15, 2022 at 2:22 AM Robert Haas wrote: > > On Fri, Jul 9, 2021 at 10:44 AM Tomas Vondra > wrote: > > Yeah, does not seem to be w

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-14 Thread Nitin Jadhav
the current checkpoint starts with a different set of flags and when there is a new request (with CHECKPOINT_IMMEDIATE), it just processes the pending operations quickly to take up next requests. If we update this information in the 'flags' field of the view, it says that the current checkpoint

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-11 Thread Nitin Jadhav
escribed by the 'flags' field and there is no new request, then the value of this field is 'Yes' (Not throttling) which again confuses the user. 'new request' - The value will be set to Yes/No if any new checkpoint requests. This just indicates whether new requests have

Fix typo in progress reporting doc

2022-03-11 Thread Nitin Jadhav
Hi, I have observed that the table naming conventions used in 'progress-reporting.html' are not consistent across different sections. For some cases "Phases" (Table 28.37. CREATE INDEX Phases) is used and for some cases "phases" (Table 28.35. ANALYZE phases) is used. I have attached a patch to cor

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-11 Thread Nitin Jadhav
been explicitly requested, the user may not understand that it affects current checkpoint behaviour unless the user knows the internals of the checkpoint. How about naming the field to 'throttled' (Yes/ No) since our objective is to show that the current checkpoint is throttled or not. Th

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-11 Thread Nitin Jadhav
' END || CASE WHEN (S.param2 & 128) > 0 AND (S.param2 & -256) > 0 THEN ', ' ELSE '' END || CASE WHEN (S.param2 & 256) > 0 THEN 'time' ELSE '' END || '}' Basically, a separate CASE statement is used t

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-08 Thread Nitin Jadhav
current checkpoint behaviour gets affected by only CHECKPOINT_IMMEDIATE flag and all other checkpoint requests done in case of createdb(), dropdb(), etc gets called with CHECKPOINT_IMMEDIATE flag. I have updated this in the v5 patch. Please share your thoughts. Thanks & Regards, Nitin Jadhav On Thu, Mar

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-08 Thread Nitin Jadhav
comments. > > I will update in the next patch. The current format matches with the server log message for the checkpoint start in LogCheckpointStart(). Just to be consistent, I have not changed the code. I have taken care of the rest of the comments in v5 patch for which there was clarit

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-08 Thread Nitin Jadhav
ause some operation (for which the checkpoint is necessary) is forced the checkpoint". Fixed other comments as per the discussion above. Please find the v5 patch attached and share your thoughts. Thanks & Regards, Nitin Jadhav On Mon, Mar 7, 2022 at 7:45 P

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-07 Thread Nitin Jadhav
ted so that we can match the behaviour with checkpoint end message? Please share your thoughts. --- > I think we can add a couple of more information to this view - > start_time for buffer write operation and start_time for buffer sync > operation. These are two very time consuming tasks

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-07 Thread Nitin Jadhav
ing like the > backend that requested checkpoint will wait until it's completion > + Wait for completion before returning. > > 13) "removing unneeded or flushing needed logical rewrite mapping files" > + The checkpointer process is currently removing/flu

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-04 Thread Nitin Jadhav
thing it's still only showing the initial > checkpoint flags, so it's *not* showing what the checkpoint is really > doing, only what the checkpoint may be doing if nothing else happens. > It just feels wrong. You could even use that ckpt_flags info to know > that at least one

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-03 Thread Nitin Jadhav
st_lsn :s/pg_get_first_and_last_valid_wal_record_lsn/pg_wal_first_and_last_lsn :s/pg_get_wal_record_info/pg_wal_record_info :s/pg_get_wal_stats/pg_wal_stats 5) Even 'pg_get_wal_stats' and 'pg_get_wal_stats_till_end_of_wal' can be clubbed as one function. The above comments are

Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Nitin Jadhav
feel we should support this. Thoughts? Please find the patch attached. Thanks & Regards, Nitin Jadhav v1-0001-add-checkpoint_requested-flag-to-the-log-message.patch Description: Binary data

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-02 Thread Nitin Jadhav
er once the > > pg_stat_progress_checkpoint view gets in. > > I don't think that's really necessary to give the pid list. > > If you requested a new checkpoint, it doesn't matter if it's only your backend > that triggered it, another backend or a few other d

Refactor statistics collector, backend status reporting and command progress reporting

2022-03-01 Thread Nitin Jadhav
#x27; structure. Please share your thoughts. If required, I would like to work on the patch. Thanks & Regards, Nitin Jadhav

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-01 Thread Nitin Jadhav
684800 + (S.param4::float / 100)) AS start_time' works fine. The value '946684800' is equal to ((POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY). I am not sure whether it is good practice to use this way. Kindly share your thoughts. Thanks & Regards, Nitin Jadha

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
!ImmediateCheckpointRequested() && >IsCheckpointOnSchedule(progress)) I understand that the checkpointer considers flags as well as the shmem flags and if CHECKPOINT_IMMEDIATE flag is set, it affects the current checkpoint operation (No further delay) but does no

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
, val); > + } > > Any specific reason for recording the timelineID in checkpoint stats > table? Will this ever change in our case? The timelineID is used to decide whether the current operation is checkpoint or restartpoint. There is a field in the view to display this in

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
ling pgstat progress reporting when in bootstrap processing mode > / startup/end-of-recovery makes very little sense (see upthread) and > should be removed, regardless of whether seperate functions stay. Removed since log based reporting is not part of the current patch. > > diff --git a/

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-24 Thread Nitin Jadhav
ch will make it correct. My opinion about maintaining a separate field to show upcoming checkpoint flags is it makes the view complex. Please share your thoughts. Thanks & Regards, On Thu, Feb 24, 2022 at 10:45 PM Matthias van de Meent wrote: > > On Wed, 23 Feb 2022 at 14:28, Nitin Jadh

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
increases the code readability than calling it separately in all the scenarios. I am planning to include log based reporting in the next patch. Even after that if using the same function is not recommended, I am happy to change. Thanks & Regards, Nitin Jadhav On Wed, Feb 23, 2022 at 12:13

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
gt; + else > + checkpoint_progress_update_param(flags, > PROGRESS_CHECKPOINT_KIND, > + > PROGRESS_CHECKPOINT_KIND_UNKNOWN); > + } > +} > > -- > With Regards, > Ashutosh Sharma. > > On Thu, Feb 10, 2022 at 12:23 PM Nitin Jadhav >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Nitin Jadhav
o support this. I am planning to take care in the next patch. --- > If pg_is_in_recovery() is true, then it's a restartpoint, otherwise it's a > restartpoint if the checkpoint's timeline is different from the current > timeline? Fixed. Sharing the v2 patch. Kindly ha

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-21 Thread Nitin Jadhav
FROM (SELECT -2::bigint /* 0x/FFFE > */ AS my_bigint_lsn) AS stat(my_int64); Thanks for sharing. It works. I will include this in the next patch. On Sat, Feb 19, 2022 at 11:02 AM Julien Rouhaud wrote: > > Hi, > > On Fri, Feb 18, 2022 at 08:07:0

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-18 Thread Nitin Jadhav
ther then then kind of checkpoint? > You should still be able to compute the information while giving a bit more > information for the same memory usage. Can you please describe more about how checkpoint/restartpoint can be confirmed using the timeline id. Thanks & Regards, Nit

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
syncing files may complete before pg_is_in_recovery() returns false. But there are some cleanup operations happen as part of the checkpoint. During this scenario, we may get false value for pg_is_in_recovery(). Please refer following piece of code which is present in CreateRestartpoint(). if (!Re

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
rted due to the expiration of a > >checkpoint_timeout< interval "The checkpoint is started because checkpoint_timeout expired". > > + The checkpoint operation is forced even if no XLOG activity has > > occurred > > + since the last one. > > + Some operation fo

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-17 Thread Nitin Jadhav
> pg_is_in_recovery()? AFAIK we cannot use pg_is_in_recovery() to predict whether it is a checkpoint or restartpoint because if the system exits from recovery mode during restartpoint then any query to pg_stat_progress_checkpoint view will return it as a checkpoint which is ideally not correct.

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-16 Thread Nitin Jadhav
; which takes int64 (not Timestamptz) as argument and then returns string representing the elapsed time. This function can be called in the view. Is it safe/advisable to use int64 type here rather than Timestamptz for this purpose? 'checkpoint start location' (lsn = uint64) - I feel we cannot

Re: Refactor CheckpointWriteDelay()

2022-02-15 Thread Nitin Jadhav
r the additional information. I feel the information related to the pending shutdown request should also be added. Thanks & Regards, Nitin Jadhav On Thu, Feb 10, 2022 at 5:32 PM Bharath Rupireddy wrote: > > Hi, > > Given that CheckpointWriteDelay() is really a hot code path i.e.

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-15 Thread Nitin Jadhav
this. If there are any changes to the existing mechanism to make it generic to support basic data types, I would like to discuss this in the new thread. On Thu, Feb 10, 2022 at 12:22 PM Nitin Jadhav wrote: > > > > We need at least a trace of the number of buffers to sync (num_to_scan) >

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-09 Thread Nitin Jadhav
uring shutdown and end-of-recovery cases as we cannot access pg_stat_progress_checkpoint in those cases. I am working on this to use log_startup_progress_interval mechanism to log the progress in the server logs. Kindly review the patch and share your thoughts. On Fri, Jan 28, 2022 at 12:24 PM Bharath Rupired

Re: Report checkpoint progress in server logs

2022-01-20 Thread Nitin Jadhav
the sync > progress locally, and we may need some refactoring here. I agree to provide above mentioned information as part of showing the progress of current checkpoint operation. I am currently looking into the code to know if any other information can be added. Thanks & Regards, Nitin Jadhav On

Re: Multi-Column List Partitioning

2021-11-02 Thread Nitin Jadhav
> I noticed that there's no commitfest entry for this. Will you please > add this to the next one? I have added it to Nov commitfest. Thanks & Regards, Nitin Jadhav On Fri, Oct 29, 2021 at 1:40 PM Amit Langote wrote: > > Hi Nitin, > > On Fri, Oct 22, 2021 at 6:

Re: when the startup process doesn't (logging startup delays)

2021-10-29 Thread Nitin Jadhav
7;t see > how to avoid the race condition otherwise. Right. There is a possibility of race conditions. In that case the above changes look good to me. Thanks & Regards, Nitin Jadhav On Fri, Oct 29, 2021 at 6:10 PM Robert Haas wrote: > > On Fri, Oct 29, 20

Re: when the startup process doesn't (logging startup delays)

2021-10-29 Thread Nitin Jadhav
new one) but the flag is already set to true by the previous timer. Hence the next call to ereport_startup_progress() logs a message. So I feel just setting 'startup_progress_timer_expired' to false in begin_startup_progress_phase() would work. Please correct me if I am wrong. Thanks & Regards, Nitin Jadh

Re: when the startup process doesn't (logging startup delays)

2021-10-19 Thread Nitin Jadhav
for this since StartupXLOG() gets called during the startup process, bootstrap process or standalone backend. As per earlier discussion we need support for this in the case of startup process and standalone backend. Hence guarded this with '!IsBootstrapProcessingMode()'. Also verified the behavio

Re: when the startup process doesn't (logging startup delays)

2021-09-29 Thread Nitin Jadhav
or this one. I agree that has_startup_progress_timeout_expired() is better than the existing function name. So I changed the function name accordingly. > Please make sure to add ereport_startup_progress() as a translation > trigger in src/backend/nls.mk. I have added ereport_startup_progress()

Re: when the startup process doesn't (logging startup delays)

2021-09-28 Thread Nitin Jadhav
utes through the code flow. So I did not worry about adding logic to disable the feature. I will take care of this in the next patch. The answer for the question of "I don't understand why you posted the previous version of the patch without testing that it works" is, for the value of -

Re: when the startup process doesn't (logging startup delays)

2021-09-27 Thread Nitin Jadhav
and removed -1. Please find the patch attached. On Thu, Sep 23, 2021 at 9:44 PM Robert Haas wrote: > > On Wed, Sep 22, 2021 at 10:28 AM Nitin Jadhav > wrote: > > Thanks Justin for the detailed explanation. Done the necessary changes. > > Not really. The documentation he

Re: when the startup process doesn't (logging startup delays)

2021-09-22 Thread Nitin Jadhav
I think the convention is to for GUC global vars to be initialized with the > same default as in guc.c, so both should be 1, like: > > +int log_startup_progress_interval = 10 * 1000 /* 10sec */ Thanks Justin for the detailed explanation. Done the necessary changes. Please fin

Re: when the startup process doesn't (logging startup delays)

2021-09-13 Thread Nitin Jadhav
1ms. I tried values measured in milliseconds just for testing > > > purposes and didn't initially understand why it wasn't working. I > > > don't think there's any reason it can't work. > > > > As suggested, I have changed it to GUC_UNIT_MS a

Re: when the startup process doesn't (logging startup delays)

2021-09-07 Thread Nitin Jadhav
-1 turns this feature off."), > + GUC_UNIT_MS, > The unit is incorrect here, as that would default to 10ms, contrary to > what the documentation says about 10s. Kindly refer the previous few discussions wrt this point. On Tue, Sep 7, 2021 at 10:58 AM Michael Paquier wrote: >

Re: when the startup process doesn't (logging startup delays)

2021-09-07 Thread Nitin Jadhav
using AmStartupProcess() for this purpose is right. On Tue, Sep 7, 2021 at 10:58 AM Michael Paquier wrote: > > On Fri, Sep 03, 2021 at 09:23:27PM -0500, Justin Pryzby wrote: > > On Fri, Sep 03, 2021 at 01:23:56PM +0530, Nitin Jadhav wrote: > > > Please find the updated patch at

Re: when the startup process doesn't (logging startup delays)

2021-09-03 Thread Nitin Jadhav
d the updated patch attached. On Wed, Aug 18, 2021 at 12:23 PM Nitin Jadhav wrote: > > > Anything that's not used in other files should be declared static in > > the file itself, and not present in the header. Once you fix this for > > reset_startup_progress_timeout, the

Re: Partition Check not updated when insert into a partition

2021-08-27 Thread Nitin Jadhav
); + } Can we avoid using the extra variable 'partOid' and directly pass 'lfirst_oid(l)' to CacheInvalidateRelcacheByRelid(). Thanks & Regards, Nitin Jadhav On Fri, Aug 27, 2021 at 2:50 PM Amit Langote wrote: > > On Thu, Aug 5, 2021 at 11:32 AM Amit Langote wrote: > &

Re: Multi-Column List Partitioning

2021-08-27 Thread Nitin Jadhav
this part going to be updated in the next patch? Yes. The code changes related to partition-wise join are in progress. I will handle these in the next patch. Thanks & Regards, Nitin Jadhav On Thu, Aug 26, 2021 at 2:40 AM Zhihong Yu wrote: > > > > On Wed, Aug 25, 2021 at 5:41 AM

Re: when the startup process doesn't (logging startup delays)

2021-08-17 Thread Nitin Jadhav
> Anything that's not used in other files should be declared static in > the file itself, and not present in the header. Once you fix this for > reset_startup_progress_timeout, the header won't need to include > datatype/timestamp.h any more, which is good, because we don't want > header files to d

Re: when the startup process doesn't (logging startup delays)

2021-08-12 Thread Nitin Jadhav
s of startup_progress_timeout_has_expired(). Please let me now if this is not the one you wanted me to correct. > You mentioned previously that you would add documentation, but I do > not see it here. Sorry. I missed this. I have added the documentation in the attached patch. On Tue, Aug 10, 2021

Re: when the startup process doesn't (logging startup delays)

2021-08-10 Thread Nitin Jadhav
s trying to convey in case-2. I agree that it is better to consider "now + interval" in such a case instead of trying to adjust the drift. Please find the updated patch attached. On Tue, Aug 10, 2021 at 1:06 AM Robert Haas wrote: > > On Mon, Aug 9, 2021 at 11:20 AM Nitin Jadhav &

  1   2   >