Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-12 Thread Greg Nancarrow
On Mon, Oct 12, 2020 at 5:36 PM Amit Kapila wrote: > > > > > I have not thought about this yet but I don't understand your > > proposal. How will you set it only for the scope of Gather (Merge)? > > The execution of the Gather node will be interleaved with the Insert > > node, basically, you fetch

Re: Wrong statistics for size of XLOG_SWITCH during pg_waldump.

2020-10-12 Thread movead...@highgo.ca
Thanks for reply. >If you wish to add more information about a XLOG_SWITCH record, I >don't think that changing the signature of XLogDumpRecordLen() is >adapted because the record length of this record is defined as >Horiguchi-san mentioned upthread, and the meaning of junk_len is >confusing here

Re: Transactions involving multiple postgres foreign servers, take 2

2020-10-12 Thread Masahiko Sawada
On Mon, 12 Oct 2020 at 11:08, tsunakawa.ta...@fujitsu.com wrote: > > From: Masahiko Sawada > > I also doubt how useful the per-foreign-server timeout setting you > > mentioned before. For example, suppose the transaction involves with > > three foreign servers that have different timeout setting,

RE: Transactions involving multiple postgres foreign servers, take 2

2020-10-12 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > I was thinking to have a GUC timeout parameter like statement_timeout. > The backend waits for the setting value when resolving foreign > transactions. Me too. > But this idea seems different. FDW can set its timeout > via a transaction timeout API, is that right? I'm n

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Julien Rouhaud
On Wed, Oct 7, 2020 at 9:53 AM Bruce Momjian wrote: > > On Wed, Oct 7, 2020 at 10:42:49AM +0900, Michael Paquier wrote: > > On Tue, Oct 06, 2020 at 09:22:29AM -0400, Bruce Momjian wrote: > > > I propose moving the pg_stat_statements queryid hash code into the > > > server (with a version number),

Re: Wired if-statement in gen_partprune_steps_internal

2020-10-12 Thread Amit Langote
Hi, On Thu, Oct 8, 2020 at 6:56 PM Andy Fan wrote: > > Hi: > > I found the following code in gen_partprune_steps_internal, which > looks the if-statement to be always true since list_length(results) > 1; > I added an Assert(step_ids != NIL) and all the test cases passed. > if the if-statement

Re: Remove some unnecessary if-condition

2020-10-12 Thread Ashutosh Bapat
On Fri, Oct 9, 2020 at 6:29 AM Hou, Zhijie wrote: > > Hi > > I found some likely unnecessary if-condition in code. > > 1. Some check in else branch seems unnecessary. > > In (/src/backend/replication/logical/reorderbuffer.c) > ① @@ -4068,7 +4068,7 @@ ReorderBufferToastAppendChunk(ReorderBuffer *rb

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-12 Thread Amit Kapila
On Mon, Oct 12, 2020 at 12:38 PM Greg Nancarrow wrote: > > On Mon, Oct 12, 2020 at 5:36 PM Amit Kapila wrote: > > > > > > > > I have not thought about this yet but I don't understand your > > > proposal. How will you set it only for the scope of Gather (Merge)? > > > The execution of the Gather n

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
Hi, thank you for the awesome feature. As it may have been discussed, I think the 'name' column in pg_stat_replication_slots is more consistent with the column name and data type matched to the pg_replication_slots catalog. The attached patch changes the name and data type of the 'name' column t

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-10-12 Thread k.jami...@fujitsu.com
On Friday, October 9, 2020 11:12 AM, Horiguchi-san wrote: > I have some comments on the latest patch. Thank you for the feedback! I've attached the latest patches. > visibilitymap_prepare_truncate(Relation rel, BlockNumber nheapblocks) { > BlockNumber newnblocks; > + boolcached; >

Re: [HACKERS] Runtime Partition Pruning

2020-10-12 Thread Ashutosh Bapat
On Mon, Oct 12, 2020 at 7:59 AM Andy Fan wrote: > > Sorry for the late reply! Suppose we have partition defined like this: > p > - p1 > - p2 > > When you talk about "the statistics from the partitioned table", do you > mean the statistics from p or p1/p2? I just confirmed there is no statistic

Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables

2020-10-12 Thread Dilip Kumar
On Fri, Oct 9, 2020 at 2:34 PM Amit Kapila wrote: > > On Fri, Oct 9, 2020 at 2:19 PM Simon Riggs wrote: > > > > On Fri, 9 Oct 2020 at 04:10, Amit Kapila wrote: > > > > > > On Thu, Oct 8, 2020 at 2:34 PM Simon Riggs wrote: > > > > > > > > On Thu, 8 Oct 2020 at 09:47, Dilip Kumar wrote: > > > >

Re: Add primary keys to system catalogs

2020-10-12 Thread John Naylor
On Tue, Oct 6, 2020 at 4:16 PM Andres Freund wrote: > True, we don't create new ones that often. Still think that distributing > such setup over fewer places is good. And it's not like there's only a > handful of pkeys to start with. To me it makes more sense to add a > DECLARE_PRIMARY_KEY in ind

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Mon, Oct 12, 2020 at 10:59 AM Masahiko Sawada wrote: > > On Thu, 8 Oct 2020 at 17:59, Amit Kapila wrote: > > > > On Thu, Oct 8, 2020 at 1:55 PM Masahiko Sawada > > wrote: > > > > > > On Thu, 8 Oct 2020 at 14:10, Amit Kapila wrote: > > > > > > > > > > > > We can write if we want but there are

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-12 Thread Andrey Borodin
> 8 окт. 2020 г., в 04:48, Peter Geoghegan написал(а): > > On Tue, Jun 30, 2020 at 5:03 PM Peter Geoghegan wrote: >> Attached is a POC patch that teaches nbtree to delete old duplicate >> versions from unique indexes. The optimization targets non-HOT >> duplicate version bloat. Although the p

Re: Improved Cost Calculation for IndexOnlyScan

2020-10-12 Thread Hamid Akhtar
On Tue, Sep 29, 2020 at 2:57 PM Heikki Linnakangas wrote: > On 29/09/2020 11:49, Hamid Akhtar wrote: > > So, not actually random replacement here, rather a change with > > baserel->allvisfrac taken into consideration (as given below): > > > > index_random_page_cost = Min(spc_seq_page_cost +

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-10-12 Thread Amit Kapila
On Mon, Oct 12, 2020 at 3:08 PM k.jami...@fujitsu.com wrote: > > Hmm. When I repeated the performance measurement for non-recovery, > I got almost similar execution results for both master and patched. > > Execution Time (in seconds) > | s_b | master | patched | %reg | > |---||

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Mon, 12 Oct 2020 at 18:29, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: > > Hi, thank you for the awesome feature. > Thank you for reporting! > As it may have been discussed, I think the 'name' column in > pg_stat_replication_slots is more consistent with the column name and data > ty

Re: partition routing layering in nodeModifyTable.c

2020-10-12 Thread Heikki Linnakangas
On 09/10/2020 11:01, Amit Langote wrote: On Thu, Oct 8, 2020 at 9:35 PM Amit Langote wrote: On Wed, Oct 7, 2020 at 9:07 PM Heikki Linnakangas wrote: On 07/10/2020 12:50, Amit Langote wrote: I have thought about something like this before. An idea I had is to make es_result_relations array i

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Mon, Oct 12, 2020 at 11:52 AM Masahiko Sawada wrote: > > On Thu, 8 Oct 2020 at 22:57, Amit Kapila wrote: > > > > On Thu, Oct 8, 2020 at 7:46 AM Masahiko Sawada > > wrote: > > > > I have rebased the stream stats patch and made minor modifications. I > > haven't done a detailed review but one t

Re: speed up unicode normalization quick check

2020-10-12 Thread Michael Paquier
On Mon, Oct 12, 2020 at 05:46:16AM -0400, John Naylor wrote: > Hmm, I hadn't actually, but now that you mention it, that looks worth > optimizing that as well, since there are multiple callers that search that > table -- thanks for the reminder. The attached patch was easy to whip up, > being simil

Re: speed up unicode normalization quick check

2020-10-12 Thread Michael Paquier
On Mon, Oct 12, 2020 at 03:39:51PM +0900, Masahiko Sawada wrote: > Yes, this patch resolves the problem. Okay, applied then. -- Michael signature.asc Description: PGP signature

Re: Parallel INSERT (INTO ... SELECT ...)

2020-10-12 Thread Thomas Munro
On Mon, Oct 12, 2020 at 6:35 PM Greg Nancarrow wrote: > On Mon, Oct 12, 2020 at 2:11 PM Thomas Munro wrote: > > Yeah, I think we should go ahead and fix that up front. Here's a > > version with a commit message. > > I've checked it and tested it, and it looks fine to me. > Also, it seems to alig

RE: Remove some unnecessary if-condition

2020-10-12 Thread Hou, Zhijie
> To me it looks good to clean up the conditions as you have done in the patch. > Please add this to commitfest so that it's not forgotten. I have verified > the code and indeed the conditions you are removing are unnecessary. So > the patch can be marked as CFP right away. Thank you for reviewing

Re: Batching page logging during B-tree build

2020-10-12 Thread Dmitry Dolgov
> On Fri, Oct 09, 2020 at 11:08:42PM +0500, Andrey Borodin wrote: > > > 23 сент. 2020 г., в 23:19, Peter Geoghegan написал(а): > > > > On Fri, Sep 18, 2020 at 8:39 AM Andrey M. Borodin > > wrote: > >> Here is PoC with porting that same routine to B-tree. It allows to build > >> B-trees ~10% fas

Re: archive status ".ready" files may be created too early

2020-10-12 Thread Heikki Linnakangas
On 07/07/2020 12:02, matsumura@fujitsu.com wrote: At Monday, July 6, 2020 05:13:40 +, "Kyotaro Horiguchi " wrote in after WAL buffer is filled up to the requested position. So when it crosses segment boundary we know the all past corss segment-boundary records are stable. That means al

Re: [PATCH] Add features to pg_stat_statements

2020-10-12 Thread Yuki Seino
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed The patch applies cleanly and looks fine to me. It's a small

Re: [HACKERS] Runtime Partition Pruning

2020-10-12 Thread Andy Fan
On Mon, Oct 12, 2020 at 5:48 PM Ashutosh Bapat wrote: > On Mon, Oct 12, 2020 at 7:59 AM Andy Fan wrote: > > > > > Sorry for the late reply! Suppose we have partition defined like this: > > p > > - p1 > > - p2 > > > > When you talk about "the statistics from the partitioned table", do you > > me

Re: BUG #15858: could not stat file - over 4GB

2020-10-12 Thread Juan José Santamaría Flecha
On Mon, Oct 12, 2020 at 5:27 AM Tom Lane wrote: > Michael Paquier writes: > > Why are we forcing errno=ENOENT here? Wouldn't it be correct to use > > _dosmaperr(GetLastError()) to get the correct errno? > > Fair question. Juan, was there some good reason not to look at > GetLastError() in this

Re: partition routing layering in nodeModifyTable.c

2020-10-12 Thread Amit Langote
On Mon, Oct 12, 2020 at 8:12 PM Heikki Linnakangas wrote: > On 09/10/2020 11:01, Amit Langote wrote: > > On Thu, Oct 8, 2020 at 9:35 PM Amit Langote wrote: > >> On Wed, Oct 7, 2020 at 9:07 PM Heikki Linnakangas wrote: > >>> On 07/10/2020 12:50, Amit Langote wrote: > I have thought about som

Re: [HACKERS] Custom compression methods

2020-10-12 Thread Tomas Vondra
On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote: ... I have worked on this patch, so as discussed now I am maintaining the preserved compression methods using dependency. Still PRESERVE ALL syntax is not supported, I will work on that part. Cool, I'll take a look. What's your

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Bruce Momjian
On Mon, Oct 12, 2020 at 04:20:05PM +0800, Julien Rouhaud wrote: > But there are many people that aren't happy with the current hashing > approach. If we're going to move the computation in core, shouldn't > we listen to their complaints and let them pay some probably quite > high overhead to base

broken logic of simple_eval_resowner after CALL and COMMIT inside procedure

2020-10-12 Thread Pavel Stehule
Hi Inline handler creates simple_eval_resowner (without parent). Inside plpgsql_estate_setup this value is assigned to estate->simple_eval_resowner <-->if (simple_eval_resowner) <--><-->estate->simple_eval_resowner = simple_eval_resowner; <-->else <--><-->estate->simple_eval_resowner = shared_si

Re: broken logic of simple_eval_resowner after CALL and COMMIT inside procedure

2020-10-12 Thread Pavel Stehule
po 12. 10. 2020 v 16:15 odesílatel Pavel Stehule napsal: > Hi > > Inline handler creates simple_eval_resowner (without parent). > > Inside plpgsql_estate_setup this value is assigned to > estate->simple_eval_resowner > > <-->if (simple_eval_resowner) > <--><-->estate->simple_eval_resowner = simpl

RE: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Shinoda, Noriyoshi (PN Japan A&PS Delivery)
Sawada-san, Thank you your comments. The attached patch reflects the comment. I also made a fix for the regression test. Regards, Noriyoshi Shinoda -Original Message- From: Masahiko Sawada [mailto:masahiko.saw...@2ndquadrant.com] Sent: Monday, October 12, 2020 8:12 PM To: Shinoda, Noriy

Re: BUG #15858: could not stat file - over 4GB

2020-10-12 Thread Tom Lane
=?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?= writes: > On Mon, Oct 12, 2020 at 5:27 AM Tom Lane wrote: >> Michael Paquier writes: >>> Why are we forcing errno=ENOENT here? Wouldn't it be correct to use >>> _dosmaperr(GetLastError()) to get the correct errno? >> Fair question. Juan, was

Bizarre coding in recovery target GUC management

2020-10-12 Thread Tom Lane
Would someone explain to me why assign_recovery_target_lsn and related GUC assign hooks throw errors, rather than doing so in the associated check hooks? An assign hook is not supposed to throw an error. Full stop, no exceptions. We wouldn't bother to separate those hooks otherwise.

Re: partition routing layering in nodeModifyTable.c

2020-10-12 Thread Heikki Linnakangas
On 12/10/2020 16:47, Amit Langote wrote: On Mon, Oct 12, 2020 at 8:12 PM Heikki Linnakangas wrote: 1. We have many different cleanup/close routines now: ExecCloseResultRelations, ExecCloseRangeTableRelations, and ExecCleanUpTriggerState. Do we need them all? It seems to me that we could merge E

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Robert Haas
On Mon, Oct 12, 2020 at 10:14 AM Bruce Momjian wrote: > On Mon, Oct 12, 2020 at 04:20:05PM +0800, Julien Rouhaud wrote: > > But there are many people that aren't happy with the current hashing > > approach. If we're going to move the computation in core, shouldn't > > we listen to their complaint

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-10-12 Thread Justin Pryzby
On Fri, Oct 09, 2020 at 07:42:51PM -0400, Bruce Momjian wrote: > On Fri, Oct 9, 2020 at 02:23:10PM -0500, Justin Pryzby wrote: > > In my local branch, I had revised this comment to say: > > > > + * Note, v8.4 has no tablespace_suffix, which is fine so long as the > > version we > > + * being upg

Re: libpq debug log

2020-10-12 Thread Robert Haas
On Wed, Sep 16, 2020 at 4:41 PM Alvaro Herrera wrote: > 2020-09-16 13:27:29.072 -03 < ErrorResponse 117 S "ERROR" V "ERROR" C > "42704" M "no existe el slot de replicación «foobar»" F "slot.c" L "408" R > "ReplicationSlotAcquireInternal" ^@ Ooh, that looks really nice. The ^@ at the end seems

Re: [PATCH] Add `truncate` option to subscription commands

2020-10-12 Thread David Christensen
> On Oct 11, 2020, at 10:00 PM, Amit Kapila wrote: > > On Mon, Oct 12, 2020 at 3:44 AM David Christensen wrote: >> >> >> On Oct 11, 2020, at 1:14 PM, Euler Taveira >> wrote: >> >>  >> On Fri, 9 Oct 2020 at 15:54, David Christensen wrote: >>> >>> >>> Enclosed find a patch to add a “trunc

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Tom Lane
Robert Haas writes: > I don't really understand how a version number helps. It's not like > there is going to be a v2 that is in all ways better than v1. If there > are different algorithms here, they are going to be customized for > different needs. Yeah, I agree --- a version number is the wron

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Bruce Momjian
On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > Robert Haas writes: > > I don't really understand how a version number helps. It's not like > > there is going to be a v2 that is in all ways better than v1. If there > > are different algorithms here, they are going to be customized for

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Tom Lane
Bruce Momjian writes: > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: >> Yeah, I agree --- a version number is the wrong way to think about this. > The version number was to invalidate _all_ query hashes if the > algorithm is slightly modified, rather than invalidating just some of >

Re: SEARCH and CYCLE clauses

2020-10-12 Thread Robert Haas
On Fri, May 22, 2020 at 5:25 AM Peter Eisentraut wrote: > This is something we need to think about. If we want to check at parse > time whether the two values are not the same (and perhaps not null), > then we either need to restrict the generality of what we can specify, > or we need to be prepa

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2020-10-12 Thread Bruce Momjian
On Mon, Oct 12, 2020 at 04:07:30PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, Oct 12, 2020 at 02:26:15PM -0400, Tom Lane wrote: > >> Yeah, I agree --- a version number is the wrong way to think about this. > > > The version number was to invalidate _all_ query hashes if the > > al

Re: Deleting older versions in unique indexes to avoid page splits

2020-10-12 Thread Peter Geoghegan
On Mon, Oct 12, 2020 at 3:47 AM Andrey Borodin wrote: > The idea looks very interesting. > It resembles GiST microvacuum: GiST tries to vacuum single page before split. AFAICT the GiST microvacuum mechanism is based on the one in nbtree, which is based on setting LP_DEAD bits when index scans fin

[patch] [doc] Further note required activity aspect of automatic checkpoint and archving

2020-10-12 Thread David G. Johnston
Hackers, Over in general [1] Robert Inder griped about the not-so-recent change to our automatic checkpointing, and thus archiving, behavior where non-activity results in nothing happening. In looking over the documentation I felt a few changes could be made to increase the chance that a reader l

[patch] [doc] Clarify that signal functions have no feedback

2020-10-12 Thread David G. Johnston
Hackers, Over in Bug# 16652 [1] Christoph failed to recognize the fact that signal sending functions are inherently one-way just as signals are. It seems worth heading off this situation in the future by making it clear how signals behave and, in the specific case of pg_reload_conf, that the impo

Re: Assertion failure with barriers in parallel hash join

2020-10-12 Thread Melanie Plageman
On Thu, Oct 1, 2020 at 8:08 PM Thomas Munro wrote: > On Tue, Sep 29, 2020 at 9:12 PM Thomas Munro > wrote: > > On Tue, Sep 29, 2020 at 7:11 PM Michael Paquier > wrote: > > > #2 0x009027d2 in ExceptionalCondition > > > (conditionName=conditionName@entry=0xa80846 "!barrier->static_party"

Re: Assertion failure with barriers in parallel hash join

2020-10-12 Thread Thomas Munro
On Tue, Oct 13, 2020 at 12:15 PM Melanie Plageman wrote: > On Thu, Oct 1, 2020 at 8:08 PM Thomas Munro wrote: >> On Tue, Sep 29, 2020 at 9:12 PM Thomas Munro wrote: >> Here's a throw-away patch to add some sleeps that trigger the problem, >> and a first draft fix. I'll do some more testing of t

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Mon, 12 Oct 2020 at 19:24, Amit Kapila wrote: > > On Mon, Oct 12, 2020 at 10:59 AM Masahiko Sawada > wrote: > > > > On Thu, 8 Oct 2020 at 17:59, Amit Kapila wrote: > > > > > > On Thu, Oct 8, 2020 at 1:55 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, 8 Oct 2020 at 14:10, Amit Kapila

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
(Please avoid top-posting) On Mon, 12 Oct 2020 at 23:45, Shinoda, Noriyoshi (PN Japan A&PS Delivery) wrote: > > Sawada-san, Thank you your comments. > > The attached patch reflects the comment. > I also made a fix for the regression test. > > Regards, > Noriyoshi Shinoda > > -Original Message

Re: BUG #15858: could not stat file - over 4GB

2020-10-12 Thread Michael Paquier
On Mon, Oct 12, 2020 at 11:13:38AM -0400, Tom Lane wrote: > Juan José Santamaría Flecha wrote: >> Uhm, a good question indeed, forcing errno serves no purpose there. > > OK, changed. Thanks! -- Michael signature.asc Description: PGP signature

Re: Transactions involving multiple postgres foreign servers, take 2

2020-10-12 Thread Kyotaro Horiguchi
At Fri, 9 Oct 2020 21:45:57 +0900, Masahiko Sawada wrote in > On Fri, 9 Oct 2020 at 14:55, Kyotaro Horiguchi > wrote: > > > > At Fri, 9 Oct 2020 02:33:37 +, "tsunakawa.ta...@fujitsu.com" > > wrote in > > > From: Masahiko Sawada > > > > What about temporary network failures? I think ther

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-10-12 Thread tsunakawa.ta...@fujitsu.com
From: Jamison, Kirk/ジャミソン カーク (1) > Alright. I also removed nTotalBlocks in v24-0003 patch. > > for (i = 0; i < nforks; i++) > { > if (nForkBlocks[i] != InvalidBlockNumber && > nBlocksToInvalidate < BUF_DROP_FULL_SCAN_THRESHOLD) > { > Optimization loop > } > else

Re: Add a description to the documentation that toast_tuple_target affects "Main"

2020-10-12 Thread Kasahara Tatsuhito
Hi, On Fri, Oct 9, 2020 at 5:44 PM Shinya Okano wrote: > > Hi, > > Regarding the toast_tuple_target parameter of CREATE TABLE, the > documentation says that it only affects External or Extended, but it > actually affects the compression of Main as well. > > The attached patch modifies the documen

Re: Transactions involving multiple postgres foreign servers, take 2

2020-10-12 Thread Masahiko Sawada
On Tue, 13 Oct 2020 at 10:00, Kyotaro Horiguchi wrote: > > At Fri, 9 Oct 2020 21:45:57 +0900, Masahiko Sawada > wrote in > > On Fri, 9 Oct 2020 at 14:55, Kyotaro Horiguchi > > wrote: > > > > > > At Fri, 9 Oct 2020 02:33:37 +, "tsunakawa.ta...@fujitsu.com" > > > wrote in > > > > From: Mas

Re: New statistics for tuning WAL buffer size

2020-10-12 Thread Masahiro Ikeda
On 2020-10-06 15:57, Masahiro Ikeda wrote: Hi, I think it's better to add other WAL statistics to the pg_stat_wal view. I'm thinking to add the following statistics. Please let me know your thoughts. 1. Basic wal statistics * wal_records: Total number of WAL records generated * wal_fpi: To

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 4:54 AM Masahiko Sawada wrote: > > Attached the updated version patch. Please review it. > I have pushed this but it failed in one of the BF. See https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2020-10-13%2003%3A07%3A25 The failure is shown below and I

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 9:02 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 4:54 AM Masahiko Sawada > wrote: > > > > Attached the updated version patch. Please review it. > > > > I have pushed this but it failed in one of the BF. See > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fl

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 9:11 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 9:02 AM Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 4:54 AM Masahiko Sawada > > wrote: > > > > > > Attached the updated version patch. Please review it. > > > > > > > I have pushed this but it failed in one of

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
Amit Kapila writes: > I have pushed this but it failed in one of the BF. See > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2020-10-13%2003%3A07%3A25 > The failure is shown below and I am analyzing it. See, if you can > provide any insights. It's not very clear what spill_c

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: > > Amit Kapila writes: > > I have pushed this but it failed in one of the BF. See > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2020-10-13%2003%3A07%3A25 > > The failure is shown below and I am analyzing it. See, if you ca

Re: partition routing layering in nodeModifyTable.c

2020-10-12 Thread Amit Langote
On Tue, Oct 13, 2020 at 1:57 AM Heikki Linnakangas wrote: > On 12/10/2020 16:47, Amit Langote wrote: > > On Mon, Oct 12, 2020 at 8:12 PM Heikki Linnakangas wrote: > >> 1. We have many different cleanup/close routines now: > >> ExecCloseResultRelations, ExecCloseRangeTableRelations, and > >> ExecC

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
Amit Kapila writes: > On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: >> It's not very clear what spill_count actually counts (and the >> documentation sure does nothing to clarify that), but if it has anything >> to do with WAL volume, the explanation might be that florican is 32-bit. >> All the

Re: [HACKERS] Custom compression methods

2020-10-12 Thread Dilip Kumar
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra wrote: > > On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote: > > > >> ... > > > >I have worked on this patch, so as discussed now I am maintaining the > >preserved compression methods using dependency. Still PRESERVE ALL > >syntax is not sup

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > Amit Kapila writes: > > On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: > >> It's not very clear what spill_count actually counts (and the > >> documentation sure does nothing to clarify that), but if it has anything > >> to do with WAL volume

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
I wrote: > prairiedog just failed in not-quite-the-same way, which reinforces the > idea that this test is dependent on MAXALIGN, which determines physical > tuple size. (I just checked the buildfarm, and the four active members > that report MAXALIGN 4 during configure are florican, lapwing, locu

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 10:33 AM Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 10:21 AM Tom Lane wrote: > > > > Amit Kapila writes: > > > On Tue, Oct 13, 2020 at 9:25 AM Tom Lane wrote: > > >> It's not very clear what spill_count actually counts (and the > > >> documentation sure does nothing

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Tom Lane
Amit Kapila writes: >> It is possible that MAXALIGN stuff is playing a role here and or the >> background transaction stuff. I think if we go with the idea of >> testing spill_txns and spill_count being positive then the results >> will be stable. I'll write a patch for that. Here's our first fai

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > Amit Kapila writes: > >> It is possible that MAXALIGN stuff is playing a role here and or the > >> background transaction stuff. I think if we go with the idea of > >> testing spill_txns and spill_count being positive then the results > >> will

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > > > Amit Kapila writes: > > >> It is possible that MAXALIGN stuff is playing a role here and or the > > >> background transaction stuff. I think if we go with the idea of > > >> testing spil

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Amit Kapila
On Tue, Oct 13, 2020 at 11:49 AM Masahiko Sawada wrote: > > On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > > > On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > > > > > Amit Kapila writes: > > > >> It is possible that MAXALIGN stuff is playing a role here and or the > > > >> background

Re: Resetting spilled txn statistics in pg_stat_replication

2020-10-12 Thread Masahiko Sawada
On Tue, 13 Oct 2020 at 15:27, Amit Kapila wrote: > > On Tue, Oct 13, 2020 at 11:49 AM Masahiko Sawada > wrote: > > > > On Tue, 13 Oct 2020 at 14:53, Amit Kapila wrote: > > > > > > On Tue, Oct 13, 2020 at 11:05 AM Tom Lane wrote: > > > > > > > > Amit Kapila writes: > > > > >> It is possible tha