Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-22 Thread Ashutosh Sharma
POINT_KIND_FORCE); + else + checkpoint_progress_update_param(flags, PROGRESS_CHECKPOINT_KIND, + PROGRESS_CHECKPOINT_KIND_UNKNOWN); + } +} -- With Regards, Ashutosh Sharma. On Thu, Feb 10, 2022 at 12:23 PM Nitin Jadhav wrote: > >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-22 Thread Ashutosh Sharma
I'm not sure about the current status, but found it while playing around with the latest changes a bit, so thought of sharing it here. + + strategy + + + This is used for copying the database directory. Currently, we have + two strategies the WAL_LOG_BLO

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Ashutosh Sharma
} Any specific reason for recording the timelineID in checkpoint stats table? Will this ever change in our case? -- With Regards, Ashutosh Sharma. On Wed, Feb 23, 2022 at 6:59 PM Nitin Jadhav wrote: > > > I will make use of pgstat_progress_update_multi_param() in the next > >

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-02 Thread Ashutosh Sharma
n't we make use of an existing WAL reader function - read_local_xlog_page()? -- With Regards, Ashutosh Sharma. On Fri, Feb 25, 2022 at 4:33 PM Bharath Rupireddy wrote: > > On Wed, Feb 16, 2022 at 9:04 AM Ashutosh Sharma wrote: > > I don't think that's the use

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-02 Thread Ashutosh Sharma
On Wed, Mar 2, 2022 at 10:37 PM Bharath Rupireddy wrote: > > On Wed, Mar 2, 2022 at 8:12 PM Ashutosh Sharma wrote: > > > > Some review comments on v5 patch (v5-0001-pg_walinspect.patch) > > Thanks for reviewing. > > > +-- > > +-- pg_get_wal_reco

Re: Make mesage at end-of-recovery less scary.

2022-03-03 Thread Ashutosh Sharma
On Wed, Mar 2, 2022 at 7:47 AM Kyotaro Horiguchi wrote: > > At Sat, 19 Feb 2022 09:31:33 +0530, Ashutosh Sharma > wrote in > > The changes looks good. thanks.! > > Thanks! > > Some recent core change changed WAL insertion speed during the TAP > test and revealed

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-03 Thread Ashutosh Sharma
people would find it useful to know how much time is being taken by the checkpoint in I/O operation phase. thoughts? -- With Regards, Ashutosh Sharma. On Wed, Mar 2, 2022 at 4:45 PM Nitin Jadhav wrote: > > Thanks for reviewing. > > > > > I suggested upthread to store t

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-03 Thread Ashutosh Sharma
I think we should also see if we can allow end users to input timeline information with the pg_walinspect functions. This will help the end users to get information about WAL records from previous timeline which can be helpful in case of restored servers. -- With Regards, Ashutosh Sharma. On Thu

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-04 Thread Ashutosh Sharma
s on our review comments. -- With Regards, Ashutosh Sharma. On Fri, Mar 4, 2022 at 4:59 PM Nitin Jadhav wrote: > > Thanks for reviewing. > > > 6) How about shutdown and end-of-recovery checkpoint? Are you planning > > to have an ereport_startup_progress mechanism as 0002? > >

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-04 Thread Ashutosh Sharma
gards, Ashutosh Sharma. On Fri, Mar 4, 2022 at 3:54 PM Bharath Rupireddy wrote: > > On Thu, Mar 3, 2022 at 10:05 PM Robert Haas wrote: > > > > On Fri, Feb 25, 2022 at 6:03 AM Bharath Rupireddy > > wrote: > > > Added a new function that returns the first and last valid WA

Re: Synchronizing slots from primary to standby

2022-03-09 Thread Ashutosh Sharma
andby goes down, the logical subscribers will stop receiving new changes from the primary as per the design of this patch OR if standby lags behind the primary for whatever reason, it will have a direct impact on logical subscribers as well. -- With Regards, Ashutosh Sharma. On Sat, Feb 19, 2022

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-03-09 Thread Ashutosh Sharma
ld show matching data but the way it is shown doesn't need to be the same. In fact it is not in most of the cases. > I have taken care of the rest of the comments in v5 patch for which > there was clarity. > Thank you very much. Will take a look at it later. -- With Regards, Ashutosh Sharma.

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-10 Thread Ashutosh Sharma
og each copied block. Otherwise, if FILE_COPY I think we need to mention the default strategy in the documentation page. -- With Regards, Ashutosh Sharma. On Thu, Mar 10, 2022 at 4:32 PM Dilip Kumar wrote: > > On Wed, Mar 9, 2022 at 6:44 PM Robert Haas wrote: > > > > > Right, inf

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-10 Thread Ashutosh Sharma
Thanks Dilip for working on the review comments. I'll take a look at the new version of patch and let you know my comments, if any. -- With Regards, Ashutosh Sharma. On Thu, Mar 10, 2022 at 8:38 PM Dilip Kumar wrote: > > On Thu, Mar 10, 2022 at 7:22 PM Ashutosh Sharma wrote: >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-10 Thread Ashutosh Sharma
do we have any plans to call this function directly bypassing read_relmap_file for any upcoming patch? -- With Regards, Ashutosh Sharma.

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-11 Thread Ashutosh Sharma
ointer so we assume the default end pointer is end-of-WAL. > but this doesn't > pg_get_wal_records_info('0/1000000', '0/100'); > > ERROR: WAL start LSN must be less than end LSN > I think this behaviour is fine. We cannot have the same start and end lsn pointers. > And the following shows no records. > pg_get_wal_records_info('0/100', '0/101'); > pg_get_wal_records_info('0/100', '0/128'); > I think we should be erroring out here saying - couldn't find any valid WAL record between given start and end lsn because there exists no valid wal records between the specified start and end lsn pointers. -- With Regards, Ashutosh Sharma.

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-11 Thread Ashutosh Sharma
show a record when there' no record in the > specfied LSN range. But I don't think there's no usefulness of the > behavior. > So, do you want the pg_get_wal_record_info function to be removed as we can use pg_get_wal_records_info() to achieve what it does? -- With Regards, Ashutosh Sharma.

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-11 Thread Ashutosh Sharma
s and other cosmetic stuff. -- With Regards, Ashutosh Sharma. On Fri, Mar 11, 2022 at 3:51 PM Dilip Kumar wrote: > > On Fri, Mar 11, 2022 at 11:52 AM Dilip Kumar wrote: > > > > On Thu, Mar 10, 2022 at 10:18 PM Robert Haas wrote: > > > > > > On Thu, Mar 10, 2022

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-11 Thread Ashutosh Sharma
at it will display all the output columns. Also you may shorten the gap between start and end lsn to reduce the output size. == Any reason for not specifying author name in the .sgml file. Do you want me to add my name to the author? :) Ashutosh Sharma ashu.coe...@gmail.com -- With Rega

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-11 Thread Ashutosh Sharma
On Fri, Mar 11, 2022 at 8:21 PM Robert Haas wrote: > > On Fri, Mar 11, 2022 at 12:15 AM Ashutosh Sharma > wrote: > > Looks better, but why do you want to pass elevel to the > > load_relmap_file()? Are we calling this function from somewhere other > > than read_relmap

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-15 Thread Ashutosh Sharma
rctbid, Oid srcdbid, char *srcpath); I think we can shorten these function names to probably ScanSourceDBPgClassRel(), ScanSourceDBPgClassTuple() and likewise? -- With Regards, Ashutosh Sharma. On Tue, Mar 15, 2022 at 3:24 PM Dilip Kumar wrote: > > On Mon, Mar 14, 2022 at 10:04 PM Robert Haas

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-15 Thread Ashutosh Sharma
On Tue, Mar 15, 2022 at 10:17 PM Robert Haas wrote: > > On Tue, Mar 15, 2022 at 12:30 PM Ashutosh Sharma > wrote: > > > > Few comments on the latest patch: > > > > - /* We need to construct the pathname for this database */ > > -

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-16 Thread Ashutosh Sharma
record info? AFAIU, end lsn is the lsn pointer where you need to stop reading the WAL data. If that is true, then there exists no valid WAL record between the start and end lsn in this particular case. -- With Regards, Ashutosh Sharma. On Wed, Mar 16, 2022 at 7:56 PM Stephen Frost wrote: > >

MINUS SIGN (U+2212) in EUC-JP encoding is mapped to FULLWIDTH HYPHEN-MINUS (U+FF0D) in UTF-8

2020-10-29 Thread Ashutosh Sharma
'sjis'); convert - \x817c (1 row) Please note that the byte sequence (81-7c) in SJIS represents MINUS SIGN in SJIS which means the MINUS SIGN in UTF8 got converted to the MINUS SIGN in SJIS and that is what we expect. Isn't it? -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: MINUS SIGN (U+2212) in EUC-JP encoding is mapped to FULLWIDTH HYPHEN-MINUS (U+FF0D) in UTF-8

2020-10-29 Thread Ashutosh Sharma
On Fri, Oct 30, 2020 at 8:49 AM Kyotaro Horiguchi wrote: > > Hello. > > At Fri, 30 Oct 2020 06:13:53 +0530, Ashutosh Sharma > wrote in > > Hi All, > > > > Today while working on some other task related to database encoding, I > > noticed that the MINUS SI

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-03 Thread Ashutosh Sharma
ow. -- Have you intentionally skipped pg_internal.init file from being copied to the target database? -- With Regards, Ashutosh Sharma. On Thu, Dec 2, 2021 at 7:20 PM Dilip Kumar wrote: > On Wed, Dec 1, 2021 at 6:04 PM Dilip Kumar wrote: > > > Thanks a lot for testing this. Fro

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-05 Thread Ashutosh Sharma
On Fri, Dec 3, 2021 at 8:28 PM Dilip Kumar wrote: > On Fri, Dec 3, 2021 at 7:38 PM Ashutosh Sharma > wrote: > > > > I see that this patch is reducing the database creation time by almost > 3-4 times provided that the template database has some user data in it. > Howe

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-05 Thread Ashutosh Sharma
sistence == RELPERSISTENCE_PERMANENT || copying_initfork); -- With Regards, Ashutosh Sharma. On Mon, Dec 6, 2021 at 9:12 AM Ashutosh Sharma wrote: > On Fri, Dec 3, 2021 at 8:28 PM Dilip Kumar wrote: > >> On Fri, Dec 3, 2021 at 7:38 PM Ashutosh Sharma >> wrote: >> > >> > I see that

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-06 Thread Ashutosh Sharma
Regards, Ashutosh Sharma. On Mon, Dec 6, 2021 at 9:59 AM Dilip Kumar wrote: > On Mon, Dec 6, 2021 at 9:17 AM Ashutosh Sharma > wrote: > > > > Here are few more review comments: > > Thanks for reviewing it. > > > 1) It seems that we are not freeing the

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-06 Thread Ashutosh Sharma
Thanks Robert for sharing your thoughts. On Mon, Dec 6, 2021 at 11:16 PM Robert Haas wrote: > On Mon, Dec 6, 2021 at 9:23 AM Ashutosh Sharma > wrote: > > One last point - If we try to clone a huge database, as expected CREATE > DATABASE emits a lot of WALs, causing a lot

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-08 Thread Ashutosh Sharma
d patch and let me know if it looks fine or not? Neha, can you please re-run the test-cases with the attached patch. Thanks, -- With Regards, Ashutosh Sharma. On Thu, Dec 9, 2021 at 8:43 AM Neha Sharma wrote: > > > > On Thu, Dec 9, 2021 at 4:26 AM Greg Nancarrow wrote: > >>

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-09 Thread Ashutosh Sharma
On Thu, Dec 9, 2021 at 7:23 PM Neha Sharma wrote: > On Thu, Dec 9, 2021 at 11:12 AM Ashutosh Sharma > wrote: > >> Hi, >> >> The issue here is that we are trying to create a table that exists inside >> a non-default tablespace when doing ALTER DATABASE.

Re: WIP: WAL prefetch (another approach)

2021-12-10 Thread Ashutosh Sharma
Hi Thomas, I am unable to apply these new set of patches on HEAD. Can you please share the rebased patch or if you have any work branch can you please point it out, I will refer to it for the changes. -- With Regards, Ashutosh sharma. On Tue, Nov 23, 2021 at 3:44 PM Thomas Munro wrote: >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-12 Thread Ashutosh Sharma
dstrnode.spcNode = srcrnode.spcNode;; There is an extra semicolon here. -- With Regards, Ashutosh Sharma. On Sun, Dec 12, 2021 at 1:39 PM Dilip Kumar wrote: > On Fri, Dec 10, 2021 at 7:39 AM Ashutosh Sharma > wrote: > >> > >> Logfile Snippet: > >> 2021-12-09 17:

Re: Multi-Column List Partitioning

2021-12-13 Thread Ashutosh Sharma
the source. please check. -- With Regards, Ashutosh Sharma. On Mon, Dec 6, 2021 at 7:27 PM Nitin Jadhav wrote: > Thank you for reviewing the patch. > > > partbounds.c: In function ‘get_qual_for_list.isra.18’: > > partbounds.c:4284:29: warning: ‘boundinfo’ may be used uninit

Re: Add sub-transaction overflow status in pg_stat_activity

2021-12-14 Thread Ashutosh Sharma
* The number of subtransactions in the current session exceeded the cached +* subtransaction limit. +*/ + bool subxact_overflowed; All the variables inside this LocalPgBackendStatus structure are prefixed with "backend" word. Can we do the same for the newly added variables as

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-20 Thread Ashutosh Sharma
dbname, tblspcname), errhint("You must move them back to the database's default tablespace before using this command."))); } Also, if I run the checkpoint explicitly before executing the above alter database statement, this error doesn'

Re: Multi-Column List Partitioning

2021-12-20 Thread Ashutosh Sharma
On Mon, Dec 20, 2021 at 7:04 PM Amit Langote wrote: > Hi, > > On Mon, Dec 13, 2021 at 11:37 PM Ashutosh Sharma > wrote: > > > > Hi, > > > > Is this okay? > > > > postgres=# CREATE TABLE t1 (a int, b int) PARTITION BY LIST ( a, a, a ); >

Re: Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."

2021-12-21 Thread Ashutosh Sharma
t segment later */ register_unlink_segment(rnode, forkNum, 0 /* first seg */ ); } == Is it okay to share the same tablespace (infact relfile) between the primary and standby server? Perhaps NO. -- With Regards, Ashutosh Sharma. On Tue, Dec 21, 2021 at 4:47 PM Dilip Kumar wrote: > Wh

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-22 Thread Ashutosh Sharma
Hi Dilip, On Tue, Dec 21, 2021 at 11:10 AM Ashutosh Sharma wrote: > I am getting the below error when running the same test-case that Neha > shared in her previous email. > > ERROR: 55000: some relations of database "test1" are already in > tablespace "tab1"

Re: Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."

2021-12-22 Thread Ashutosh Sharma
On Wed, Dec 22, 2021 at 7:20 AM Dilip Kumar wrote: > On Wed, 22 Dec 2021 at 12:28 AM, Ashutosh Sharma > wrote: > >> >> Is it okay to share the same tablespace (infact relfile) between the >> primary and standby server? Perhaps NO. >> > >> Oops

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-22 Thread Ashutosh Sharma
On Wed, Dec 22, 2021 at 2:44 PM Dilip Kumar wrote: > On Tue, Dec 21, 2021 at 11:10 AM Ashutosh Sharma > wrote: > > > > I am getting the below error when running the same test-case that Neha > shared in her previous email. > > > > ERROR: 55000: some relations o

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-12-22 Thread Ashutosh Sharma
On Wed, Dec 22, 2021 at 5:06 PM Dilip Kumar wrote: > > On Wed, Dec 22, 2021 at 4:26 PM Ashutosh Sharma wrote: > > >> Basically, ALTER TABLE SET TABLESPACE, will register the > >> SYNC_UNLINK_REQUEST for the table files w.r.t the old tablespace, but > >> tho

Re: Parallel copy

2020-07-22 Thread Ashutosh Sharma
t; > > On Fri, Jul 17, 2020 at 7:18 PM Ashutosh Sharma > wrote: > > > > > > Some review comments (mostly) from the leader side code changes: > > > > > > 3) Should we allow Parallel Copy when the insert method is > CIM_MULTI_CONDITIONAL? > > &g

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-07-24 Thread Ashutosh Sharma
know your feedback. Thank you. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Thu, Jul 16, 2020 at 9:44 PM Robert Haas wrote: > On Thu, Jul 16, 2020 at 10:00 AM Robert Haas > wrote: > > I see your point, though: the tuple has to be able to survive >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-07-26 Thread Ashutosh Sharma
had unlogged versions of these functions. But I do not > recall exact rationale.. > Also, I'd be happy if we had something like "Restore this tuple iff this > does not break unique constraint". To do so we need to sort tids by > xmin\xmax, to revive most rec

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-07-28 Thread Ashutosh Sharma
rupted data in a table so that the operations like vacuum, pg_dump/restore succeeds. It's users responsibility to identify the corrupted data and pass its tid to either of these functions as per the requirement. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-07-28 Thread Ashutosh Sharma
ns contain the word "Force" in it. > 2. extension comment pg_surgery.control "extension to perform surgery > the damaged heap table" -> "extension to perform surgery on the > damaged heap table" > Okay, will fix that typo. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-07-31 Thread Ashutosh Sharma
On Wed, Jul 29, 2020 at 9:58 AM Ashutosh Sharma wrote: > > > I think we should let VACUUM do that. >> Sometimes VACUUM will not get to these pages, because they are marked All >> Frozen. >> An possibly some tuples will get stale on this page again >> > >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-07-31 Thread Ashutosh Sharma
Attached is the new version of patch that addresses the comments from Andrey and Beena. On Wed, Jul 29, 2020 at 10:07 AM Ashutosh Sharma wrote: > Hi Beena, > > Thanks for the review. > > 1. We would be marking buffer dirty and writing wal even if we have >> not done any c

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-03 Thread Ashutosh Sharma
y or a WAL won't be written for it. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-03 Thread Ashutosh Sharma
On Mon, Aug 3, 2020 at 7:06 PM Robert Haas wrote: > > On Mon, Aug 3, 2020 at 5:05 AM Ashutosh Sharma wrote: > > Could you please explain this point once more in detail? I am not quite > > able to understand under what circumstances a buffer would be modified, but > >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-05 Thread Ashutosh Sharma
Hi Robert, Thanks for the review. Please find my comments inline: On Sat, Aug 1, 2020 at 12:18 AM Robert Haas wrote: > > On Fri, Jul 31, 2020 at 8:52 AM Ashutosh Sharma wrote: > > Attached is the new version of patch that addresses the comments from > > Andrey and Been

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-05 Thread Ashutosh Sharma
On Thu, Aug 6, 2020 at 1:04 AM Robert Haas wrote: > > On Mon, Aug 3, 2020 at 12:13 PM Ashutosh Sharma wrote: > > If the above path is taken that means none of the items in the page > > got changed. > > Oops. I didn't realize that, sorry. Maybe it would be a little mo

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-05 Thread Ashutosh Sharma
On Thu, Aug 6, 2020 at 1:29 AM Robert Haas wrote: > > On Wed, Aug 5, 2020 at 9:42 AM Ashutosh Sharma wrote: > > Yeah, it's being tested on the main table, not on a toast table. I've > > removed this test-case and also restricted direct access to the toast > >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-06 Thread Ashutosh Sharma
Hello Masahiko-san, Thanks for looking into the patch. Please find my comments inline below: On Thu, Aug 6, 2020 at 1:42 PM Masahiko Sawada wrote: > > On Wed, 5 Aug 2020 at 22:42, Ashutosh Sharma wrote: > > Please check the attached patch for the changes. > > I also look

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-06 Thread Ashutosh Sharma
. 3) Declare some of the variables such as buf, page inside of the do loop instead of declaring them at the top of heap_force_common function. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Thu, Aug 6, 2020 at 2:49 PM Masahiko Sawada wrote: > > On Thu, 6 A

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-06 Thread Ashutosh Sharma
On Thu, Aug 6, 2020 at 9:19 PM Robert Haas wrote: > > On Thu, Aug 6, 2020 at 2:11 AM Ashutosh Sharma wrote: > > Okay, If you want I can remove the restriction on a toast table, but, > > then that means a user can directly remove the data chunks from the > > toast table w

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-07 Thread Ashutosh Sharma
functions on the same tuple repeatedly and see the behaviour. ... -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Thu, Aug 6, 2020 at 2:25 PM Rajkumar Raghuwanshi wrote: > > I have been doing some user-level testing of this feature, apart from sanity >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-11 Thread Ashutosh Sharma
On Fri, Aug 7, 2020 at 9:21 PM Robert Haas wrote: > > On Thu, Aug 6, 2020 at 9:23 AM Ashutosh Sharma wrote: > There's a certain inconsistency to these messages: > > rhaas=# create table foo (a int); > CREATE TABLE > rhaas=# insert into foo values (1); >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-11 Thread Ashutosh Sharma
On Tue, Aug 11, 2020 at 7:33 PM Robert Haas wrote: > > On Tue, Aug 11, 2020 at 3:39 AM Ashutosh Sharma wrote: > > The other two messages for reporting unused items and dead items > > remain the same. Hence, with above change, we would be reporting the > > following 4

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-12 Thread Ashutosh Sharma
Thanks Robert for the review. Please find my comments inline below: On Fri, Aug 7, 2020 at 9:21 PM Robert Haas wrote: > > On Thu, Aug 6, 2020 at 9:23 AM Ashutosh Sharma wrote: > > Attached v4 patch fixes the latest comments from Robert and Masahiko-san. > > Compiler warning:

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-13 Thread Ashutosh Sharma
ering if we should be doing similar changes in other contrib modules (like pgrowlocks, pageinspect and pgstattuple) as well? -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-16 Thread Ashutosh Sharma
evived by > heap_force_freeze(), its ctid still has special values: > MovedPartitionsOffsetNumber and MovedPartitionsBlockNumber. I don't > see a problem yet caused by a visible tuple having the special ctid > value, but it might be worth considering either to reset ctid value as > well or to not freezing already-deleted tuple. > For this as well, the answer remains the same as above. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-18 Thread Ashutosh Sharma
ly(&tuple) || params->index_cleanup == VACOPT_TERNARY_DISABLED) nkeep += 1; else tupgone = true; /* we can delete the tuple */ .. .. } So, the point is, would HeapTupleIs

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-18 Thread Ashutosh Sharma
Attached is the new version of patch that addresses the comments from Asim Praveen and Masahiko-san. It also improves the documentation to some extent. On Tue, Aug 18, 2020 at 1:46 PM Ashutosh Sharma wrote: > > Hello Masahiko-san, > > I've spent some more time trying to under

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-18 Thread Ashutosh Sharma
On Tue, Aug 18, 2020 at 9:44 PM Alvaro Herrera wrote: > > On 2020-Aug-17, Ashutosh Sharma wrote: > > > > + if (heap_force_opt == HEAP_FORCE_KILL) > > > + ItemIdSetDead(itemid); > > > > > > I think that if the page is an a

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-18 Thread Ashutosh Sharma
On Wed, Aug 19, 2020 at 9:27 AM Masahiko Sawada wrote: > > On Mon, 17 Aug 2020 at 15:05, Ashutosh Sharma wrote: > > > > > pg_force_freeze() can revival a tuple that is already deleted but not > > > vacuumed yet. Therefore, the user might need to reindex indexes a

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-19 Thread Ashutosh Sharma
On Wed, Aug 19, 2020 at 3:55 PM Masahiko Sawada wrote: > > On Wed, 19 Aug 2020 at 15:09, Ashutosh Sharma wrote: > > > > On Wed, Aug 19, 2020 at 9:27 AM Masahiko Sawada > > wrote: > > > > > > On Mon, 17 Aug 2020 at 15:05, Ashutosh Sharma > >

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-19 Thread Ashutosh Sharma
On Thu, Aug 20, 2020 at 11:04 AM Masahiko Sawada wrote: > > On Wed, 19 Aug 2020 at 20:45, Ashutosh Sharma wrote: > > > > On Wed, Aug 19, 2020 at 3:55 PM Masahiko Sawada > > wrote: > > > > > > On Wed, 19 Aug 2020 at 15:09, Ashutosh Sharma > > &g

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-21 Thread Ashutosh Sharma
know for any other concern. Thanks, -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Thu, Aug 20, 2020 at 11:43 AM Ashutosh Sharma wrote: > > On Thu, Aug 20, 2020 at 11:04 AM Masahiko Sawada > wrote: > > > > On Wed, 19 Aug 2020 at 20:45, A

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-24 Thread Ashutosh Sharma
corruption as soon as > > the XID counter advances sufficiently. > > +1. > This has been taken care of in the latest v7 patch. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-24 Thread Ashutosh Sharma
On Mon, Aug 24, 2020 at 11:00 PM Andrey M. Borodin wrote: > > Hi! > > > 21 авг. 2020 г., в 18:24, Ashutosh Sharma > > написал(а): > > > > Please find the updated patch with the following new changes: > > Do you have plans to support pg_surgery as externa

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-25 Thread Ashutosh Sharma
Hi Masahiko-san, Thank you for the review. Please check my comments inline below: On Tue, Aug 25, 2020 at 1:39 PM Masahiko Sawada wrote: > > On Fri, 21 Aug 2020 at 22:25, Ashutosh Sharma wrote: > > > > Hi Masahiko-san, > > > > Please find the updated patch

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-25 Thread Ashutosh Sharma
code changes in heap_force_freeze to identify the deleted tuples and maybe skip such tuples. So, yeah, I will do the code changes for handling this and remove the note added in the documentation. Thank you Robert and Masahiko-san for pointing this out. [1] - https://www.postgresql.org/mess

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-26 Thread Ashutosh Sharma
ULE_PATHNAME', 'heap_force_freeze' > +LANGUAGE C STRICT; > > I think these functions should be PARALLEL UNSAFE. > By default the functions are marked PARALLEL UNSAFE, so I think there is nothing to do here. Attached patch with above changes. This patch also takes care of all

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-26 Thread Ashutosh Sharma
On Wed, Aug 26, 2020 at 9:19 PM Robert Haas wrote: > > On Wed, Aug 26, 2020 at 7:36 AM Ashutosh Sharma wrote: > > Removed this note from the documentation and added a note saying: "The > > user needs to ensure that they do not operate pg_force_freeze function > >

Re: Help needed configuring postgreSQL with xml support

2020-08-27 Thread Ashutosh Sharma
In addition to what Thomas said, I would also recommend you to refer to the description of --with-libxml command line option provided in the postgres installation-procedure page - [1]. [1] - https://www.postgresql.org/docs/12/install-procedure.html -- With Regards, Ashutosh Sharma

Should we replace the checks for access method OID with handler OID?

2020-08-27 Thread Ashutosh Sharma
be doing similar changes in these contrib modules also. Thoughts? [1] - https://www.postgresql.org/message-id/1D56CEFD-E195-4E6B-B870-3383E3E8C65E%40vmware.com -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: Please help for error ( file is required for XML support )

2020-08-27 Thread Ashutosh Sharma
se types of questions are not for hackers mailing-list. These are configuration related issues and should probably be raised in pgsql-general mailing list (pgsql-general.postgresql.org). -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-27 Thread Ashutosh Sharma
On Fri, Aug 28, 2020 at 1:44 AM Robert Haas wrote: > > On Wed, Aug 26, 2020 at 10:26 PM Ashutosh Sharma > wrote: > > Okay, point noted. > > I spent some time today working on this patch. I'm fairly happy with > it now and intend to commit it if nobody sees a

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-27 Thread Ashutosh Sharma
or function find_tids_one_page should state the > requirement that the tids array must be sorted. > Okay, will add a comment for this. > > The heap_force_common function contains multiple ereport(NOTICE,...) within a > critical section. I don't think that is normal practice. Can those reports > be buffered until after the critical section is exited? You also have a > CHECK_FOR_INTERRUPTS within the critical section. > This has been fixed in the v9 patch. > [1] https://commitfest.postgresql.org/29/2700/ > — Thanks for adding a commitfest entry for this. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: Should we replace the checks for access method OID with handler OID?

2020-08-27 Thread Ashutosh Sharma
On Thu, Aug 27, 2020 at 9:21 PM Robert Haas wrote: > > On Thu, Aug 27, 2020 at 5:37 AM Ashutosh Sharma wrote: > > While reviewing the patch for pg_surgery contrib module - [1], Asim > > Praveen suggested that it would be better to replace the check for > > access met

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-08-28 Thread Ashutosh Sharma
tids_one_page should state the requirement that the tids array must be sorted. > > > > Okay, will add a comment for this. > Added a comment for this in the attached patch. Please have a look into the attached patch for the changes and let me know for any other concern

Re: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display

2021-11-12 Thread Ashutosh Sharma
a running server, but they would not be written to the > server log. > That's a good idea. How about also adding some GUC(s) to the log archive, recovery related log messages just like we have for checkpoints, autovacuum etc? Maybe something like log_archive, log_recovery etc. -- With Regards, Ashutosh Sharma.

Re: Zedstore - compressed in-core columnar storage

2019-09-17 Thread Ashutosh Sharma
On Thu, Aug 29, 2019 at 5:39 PM Heikki Linnakangas wrote: > > On 29/08/2019 14:30, Ashutosh Sharma wrote: > > > > On Wed, Aug 28, 2019 at 5:30 AM Alexandra Wang > <mailto:lew...@pivotal.io>> wrote: > > > > You are correct that we currently go throu

Re: Support for CALL statement in ecpg

2019-09-17 Thread Ashutosh Sharma
On Tue, Sep 17, 2019 at 1:06 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > I don't find this patch in any commit fest. Seems like a good addition. > Thanks for the consideration. Will add an entry for it in the commit fest. -- With Regards, Ashutosh Sh

Re: Zedstore - compressed in-core columnar storage

2019-09-18 Thread Ashutosh Sharma
On Thu, Sep 19, 2019 at 8:10 AM Alexandra Wang wrote: > > On Tue, Sep 17, 2019 at 4:15 AM Ashutosh Sharma wrote: >> >> create table t1(a int, b int) using zedstore; >> insert into t1 select i, i+10 from generate_series(1, 100) i; >> postgres=# update t1 set

Re: Zedstore - compressed in-core columnar storage

2019-09-19 Thread Ashutosh Sharma
On Thu, Sep 19, 2019 at 11:35 AM Ashutosh Sharma wrote: > > On Thu, Sep 19, 2019 at 8:10 AM Alexandra Wang wrote: > > > > On Tue, Sep 17, 2019 at 4:15 AM Ashutosh Sharma > > wrote: > >> > >> create table t1(a int, b int) using zedstore; > >> i

Re: Zedstore - compressed in-core columnar storage

2019-09-19 Thread Ashutosh Sharma
g data in attbuffers. We can simply reuse it for next set of data to be updated. > [1] https://github.com/l-wang/postgres-1/tree/zedstore-fix-memory-issues -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: Usage of the system truststore for SSL certificate validation

2019-09-19 Thread Ashutosh Sharma
This certainly looks like a good addition to me that can be implemented on both client and server side. It is always good to have a common location where the list of all the certificates from various CA's can be placed for validation. -- With Regards, Ashutosh Sharma EnterpriseDB

Re: Zedstore - compressed in-core columnar storage

2019-09-25 Thread Ashutosh Sharma
Hi Alexandra, On Tue, Sep 17, 2019 at 4:45 PM Ashutosh Sharma wrote: > > On Thu, Aug 29, 2019 at 5:39 PM Heikki Linnakangas wrote: > > > > On 29/08/2019 14:30, Ashutosh Sharma wrote: > > > > > > On Wed, Aug 28, 2019 at 5:30 AM Alexandra Wang > > <

Re: Skip recovery/standby signal files in pg_basebackup

2019-09-30 Thread Ashutosh Sharma
dby.signal file. > Patch is attached. > -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com

Re: Zedstore - compressed in-core columnar storage

2019-09-30 Thread Ashutosh Sharma
On Fri, Sep 27, 2019 at 3:09 PM Alexandra Wang wrote: > > Hi Ashutosh, > > Sorry I indeed missed your question, thanks for the reminder! > > On Wed, Sep 25, 2019 at 4:10 AM Ashutosh Sharma wrote: >> >> > Further, the UPDATE operation on zedstore table is very slo

Re: pgbench - allow to create partitioned tables

2019-10-03 Thread Ashutosh Sharma
s a bug or you guys feel that it isn't and can be ignored? Please let me know your thoughts on this. Thank you. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On Thu, Oct 3, 2019 at 10:30 AM Fabien COELHO wrote: > > > >> Thanks, attached is a pa

Re: pgbench - allow to create partitioned tables

2019-10-03 Thread Ashutosh Sharma
we should possibly replace atoi with strtol function call for better error handling. It handles the erroneous inputs better than atoi. > There is/was a current patch/discussion to improve integer parsing, which > could address this. > It seems like you are trying to point out the follow

Re: refactoring - share str2*int64 functions

2019-10-04 Thread Ashutosh Sharma
handling. Isn't that already being done in pg_strtoint32_check function itself. For e.g. in refint.c the function call to pg_strtoint32_check is followed by a if condition that checks for an error which I assume shouldn't be there as it is already being done by pg_strtoint32_check. -- With

Re: refactoring - share str2*int64 functions

2019-10-04 Thread Ashutosh Sharma
On Fri, Oct 4, 2019 at 8:58 PM Andres Freund wrote: > > Hi, > > On 2019-10-04 14:27:44 +0530, Ashutosh Sharma wrote: > > Is there any specific reason for hard coding the *base* of a number > > representing the string in strtouint64(). I understand that currently > &g

Re: Change atoi to strtol in same place

2019-10-04 Thread Ashutosh Sharma
d. So, is it possible to have the start value as 1024 instead of 1 ? Further, I encountered one syntax error (INT_MAX undeclared) as the header file "limits.h" has not been included in postgres_fe.h or option.h -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com On F

Re: dropping column prevented due to inherited index

2019-10-06 Thread Ashutosh Sharma
table itself otherwise the toast table deletion would fail. But, the problem I see here is that currently we do not have any entry in pg_attribute table that would tell us about the dependency of child column on parent. -- With Regards, Ashutosh Sharma EnterpriseDB:http://www.enterprisedb.com O

  1   2   3   4   >