Re: Restore replication settings when modifying a field type

2019-10-27 Thread Kyotaro Horiguchi
isting subscribers. I'm not sure about potential subscribers.. Anyway I think it is a problem that replica identity setting is dropped silently. Perhaps a message something like "REPLICA IDENTITY setting is lost, please redefine after confirmation of compatibility with subscribers." is needed. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Remove one use of IDENT_USERNAME_MAX

2019-10-27 Thread Kyotaro Horiguchi
etpw*() functions. I think we can believe check_usermap() never calls them but I suppose that some comments needed.. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [DOC] Fix for the missing pg_stat_progress_cluster view phase column value

2019-10-27 Thread Kyotaro Horiguchi
er why some of them are described as "CLUSTER is" and others are "The command is".. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: segmentation fault when cassert enabled

2019-10-28 Thread Kyotaro Horiguchi
ructs were changed and I didn't do "make clean". Rarely I needed "make distclean". (Yeah, I didn't ususally turn on --enable-depend..) regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [BUG] standby node can not provide service even it replays all log files

2019-10-28 Thread Kyotaro Horiguchi
At Thu, 24 Oct 2019 17:37:52 +0800 (CST), Thunder wrote in > Thanks for replay.I feel confused about snapshot. > > At 2019-10-23 11:51:19, "Kyotaro Horiguchi" wrote: > >Hello. > > > >At Tue, 22 Oct 2019 20:42:21 +0800 (CST), Thunder wrote > >i

Re: [BUG] standby node can not provide service even it replays all log files

2019-10-28 Thread Kyotaro Horiguchi
Mmm.. At Tue, 29 Oct 2019 13:57:19 +0900 (JST), Kyotaro Horiguchi wrote in > So, somehow we need to complete the KnownAssignedTransactionIds even > if there's any subxid-overflown transactions. As mentioned upthread, > I think we have at least the following choices. >

Re: Problem with synchronous replication

2019-10-29 Thread Kyotaro Horiguchi
is in transaction commit so it is in a HOLD_INTERRUPTS section. ProcessInterrupt does not respond to cancel/die interrupts thus the ereport should return. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Make StringInfo available to frontend code.

2019-10-29 Thread Kyotaro Horiguchi
eing said it doesn't matter if it is tentative and the minimal elog.h for frontend comes soon. > /* It's possible we could use a different value for this in frontend code */ > #define MaxAllocSize ((Size) 0x3fff) /* 1 gigabyte - 1 */ The same symbol is defined for frontend in psprint.c. Isn't it better merge them and place it in postgres_fe.h? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Make StringInfo available to frontend code.

2019-10-29 Thread Kyotaro Horiguchi
Hello. At Tue, 29 Oct 2019 19:06:38 -0700, Andres Freund wrote in > Hi, > > On 2019-10-30 10:58:59 +0900, Kyotaro Horiguchi wrote: > > At Tue, 29 Oct 2019 17:10:01 -0700, Andres Freund > > wrote in > > > Hi, > > > > > > This patch, in a sl

Re: Problem with synchronous replication

2019-10-29 Thread Kyotaro Horiguchi
At Wed, 30 Oct 2019 10:45:11 +0900, Michael Paquier wrote in > On Tue, Oct 29, 2019 at 07:50:01PM +0900, Kyotaro Horiguchi wrote: > > At Fri, 25 Oct 2019 15:18:34 +0800, "Dongming Liu" > > wrote in > >> I recently discovered two possible bugs about syn

Re: Problem with synchronous replication

2019-10-30 Thread Kyotaro Horiguchi
ached(&(MyProc->syncRepLinks))) > { > LWLockAcquire(SyncRepLock, LW_EXCLUSIVE); > if (!SHMQueueIsDetached(&(MyProc->syncRepLinks))) > SHMQueueDelete(&(MyProc->syncRepLinks)); > LWLockRelease(SyncRepLock); > } regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Keep compiler silence (clang 10, implicit conversion from 'long' to 'double' )

2019-11-04 Thread Kyotaro Horiguchi
-((double) PG_INT64_MIN)) # I didn't noticed the existing bit above. Agreed. it is equivalent to the trick AFAICS thus no need to add another one to warry with. > (or s/double/float8/ ?) Maybe. > c.h is probably a reasonable place, seeing that we define the constants > there. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [proposal] recovery_target "latest"

2019-11-04 Thread Kyotaro Horiguchi
whether it should it be left as it is now, or > not. > > Another open question is what to do with recovery_target_inclusive if > recovery_target = "latest" is used. Anyway inclusiveness doesn't affect to "immediate". If we had the "latest" option, it would behave the same way. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Refactor parse analysis of EXECUTE command

2019-11-05 Thread Kyotaro Horiguchi
ements when reusing a stored plan). This requires some Isn't this related to the current structure? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-06 Thread Kyotaro Horiguchi
Thank you for looking this. At Tue, 5 Nov 2019 16:16:14 -0500, Robert Haas wrote in > On Fri, Oct 25, 2019 at 9:21 AM Kyotaro Horiguchi > wrote: > > This is the fixed verison v22. > First, I'd like to restate my understanding of the problem just to see .. > Second

Re: Should we make scary sounding, but actually routine, errors less scary?

2019-11-06 Thread Kyotaro Horiguchi
covery always starts from the latest REDO point. Maybe MinRecoveryPoint is better for the use. > I'm sure there are more types of messages in this category, these are > just the ones I could immediately recall from memory as having scared > actual users unnecessarily. I often see inquiries on "FATAL: the database system is starting up". It is actually FATAL for backends internally but it is also overly scary for users. > I don't plan on fixing these immediately myself, even if we were to > agree on something, so if anybody is interested in helping... regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Using multiple extended statistics for estimates

2019-11-06 Thread Kyotaro Horiguchi
The loop runs through all clauses every time. I agree that that is better than using a copy of the clauses to avoid to step on already estimated clauses, but maybe we need an Assertion that the listidx is not a part of estimatedclauses to make sure no clauses are not estimated twice. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [proposal] recovery_target "latest"

2019-11-06 Thread Kyotaro Horiguchi
lect this behavior. In the first place, latest (or anything it could be named as) is defined as the explit label for the default behavior. Thus the latest should work as if nothing is set to recovery_target* following the definition. That might seems somewhat strange but I think at least it is harmless. recovery_target=immediate + r_t_action=shutdown for a standby works as commanded. Do we need to inhibit that, too? > > The other way around, as I see it, is to define RECOVERY_TARGET_LATEST > > as something more complex, for example, the latest possible endptr in > > latest WAL segment. But it is tricky, because WAL archive may keeps > > growing as recovery is progressing or, in case of standby, master > > keeps sending more and more WAL. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Keep compiler silence (clang 10, implicit conversion from 'long' to 'double' )

2019-11-06 Thread Kyotaro Horiguchi
low (v5). + /* Range check */ + if (unlikely(!FLOAT8_FITS_IN_INT32(num)) || isnan(num)) If compiler doesn't any fancy, num is fed to an arithmetic before checking if it is NaN. That seems have a chance of exception. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [PATCH][DOC] Fix for PREPARE TRANSACTION doc and postgres_fdw message.

2019-11-06 Thread Kyotaro Horiguchi
data on foreign tables using postgres_fdw" Or I think different style is OK here since the message is not of core but of an extension. "postgres_fdw doesn't support PREPARE of a transaction that has modified data on foreign tables" That could be shorter or simpler, of course. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [PATCH][DOC] Fix for PREPARE TRANSACTION doc and postgres_fdw message.

2019-11-07 Thread Kyotaro Horiguchi
a local table instead of foreign tabel at the previous trial. Sorry for the mistake and thank you for pointing it. So my fixed proposals are: "cannot PREPARE a transaction that has operated on foreign tables using postgres_fdw" Or "postgres_fdw doesn't support PREPARE of a transaction that has accessed foreign tables" -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [PATCH][DOC] Fix for PREPARE TRANSACTION doc and postgres_fdw message.

2019-11-07 Thread Kyotaro Horiguchi
oreign tables using postgres_fdw" instead of "postgres_fdw foreign tables". And the other point is using different message from temporary tables. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [PATCH][DOC] Fix for PREPARE TRANSACTION doc and postgres_fdw message.

2019-11-07 Thread Kyotaro Horiguchi
At Thu, 07 Nov 2019 17:27:47 +0900 (JST), Kyotaro Horiguchi wrote in > "modified" is my mistake as in the just posted mail. But the most > significant point in the previous mail is using "foreign tables using > postgres_fdw" instead of "postgres_fdw foreign

Re: Add SQL function to show total block numbers in the relation

2019-11-07 Thread Kyotaro Horiguchi
block size" and "relation size" or even with "table size". (FWIW, I would even do the same for the new function if any...) If they need it so frequently, a user-defined function is easily made up. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [proposal] recovery_target "latest"

2019-11-07 Thread Kyotaro Horiguchi
At Thu, 7 Nov 2019 12:22:28 +0300, Grigory Smolkin wrote in > > On 11/7/19 8:36 AM, Kyotaro Horiguchi wrote: > > At Thu, 7 Nov 2019 02:28:39 +0300, Grigory Smolkin > > wrote in > >> On 11/6/19 1:55 PM, Grigory Smolkin wrote: > >>> On 11/6/19 12:56 PM, Fu

Re: Why overhead of SPI is so large?

2019-11-07 Thread Kyotaro Horiguchi
these > cases. > > I think so this code can be marked as ready for commit I have some comments on this. expr_needs_snapshot checks out some of the node already checked out in exec_simple_check_plan but not all. However I don't see the criteria of the choice. I might be too worrying, but maybe we should write the function in white-listed way, that is, expr_needs_snapshot returns false only if the whole tree consists of only the node types known to the function. And any unknown node makes the function return true immediately. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: pg_waldump and PREPARE

2019-11-07 Thread Kyotaro Horiguchi
efinition. typedef struct xl_xact_prepare { uint32 magic; ... TimestampTz origin_timestamp; + /* +* This record has multiple trailing data sections with variable +* length. See twophase.c for the details. +*/ } xl_xact_prepare; Then, teach xlog_redo to resolve th

Re: pg_waldump and PREPARE

2019-11-07 Thread Kyotaro Horiguchi
I missed the funtion. xl_xact_prepare is used in *two* places in # front end. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

PHJ file leak.

2019-11-11 Thread Kyotaro Horiguchi
from ExecParallelHashJoinOuterGetTuple(). It seems to me that ExecHashTableDestroy is forgeting to release shared tuplestore accessors. Please find the attached. regards. -- Kyotaro Horiguchi NTT Open Source Software Center diff --git a/src/backend/executor/nodeHash.c b/src/backend/executor

Re: PHJ file leak.

2019-11-11 Thread Kyotaro Horiguchi
At Mon, 11 Nov 2019 17:24:45 -0500, Tom Lane wrote in > Thomas Munro writes: > > On Tue, Nov 12, 2019 at 1:24 AM Kyotaro Horiguchi > > wrote: > >> Hello. While looking a patch, I found that PHJ sometimes complains for > >> file leaks if accompanied by LIMIT. &

Re: PHJ file leak.

2019-11-11 Thread Kyotaro Horiguchi
o, is that one wrongly placed? The previous patch would be wrong. The root cause is a open batch so the right thing to be done at scan end is ExecHashTableDeatchBatch. And the real issue here seems to be in ExecutePlan, not in PHJ. regards -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Coding in WalSndWaitForWal

2019-11-11 Thread Kyotaro Horiguchi
GetFlushRecPtr() before entering the loop. > > > Also, what's up with those useless returns? > > Yes, let's rip them out. It seems to me that the fast-path seems intentional. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [BUG FIX] Uninitialized var fargtypes used.

2019-11-11 Thread Kyotaro Horiguchi
At Tue, 12 Nov 2019 12:31:41 +0900, Michael Paquier wrote in > On Mon, Nov 11, 2019 at 06:28:47PM +, Ranier Vilela wrote: > > Can anyone check this bug fix? > > > > +++ event_trigger.c Mon Nov 11 13:52:35 2019 > > @@ -171,7 +171,7 @@ > > HeapTuple tuple; > > Oid

Re: MarkBufferDirtyHint() and LSN update

2019-11-12 Thread Kyotaro Horiguchi
egardless whether the LSN is valid or not. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: PHJ file leak.

2019-11-12 Thread Kyotaro Horiguchi
At Wed, 13 Nov 2019 09:48:19 +1300, Thomas Munro wrote in > On Tue, Nov 12, 2019 at 5:03 PM Thomas Munro wrote: > > On Tue, Nov 12, 2019 at 4:23 PM Thomas Munro wrote: > > > On Tue, Nov 12, 2019 at 4:20 PM Kyotaro Horiguchi > > > wrote: > > > > The

Re: Why overhead of SPI is so large?

2019-11-12 Thread Kyotaro Horiguchi
in-the-pg_catalog-schema test. > IsCatalogNameaspace is doing just simple comparison without any > catalog lookups: > > bool > IsCatalogNamespace(Oid namespaceId) > { >     return namespaceId == PG_CATALOG_NAMESPACE; > } > > I may agree that it "proves nothing" if extension can put stuff in > pg_catalog. > I can replace it with comparison with FirstGenbkiObjectId. > But all this makes sense only if using contain_mutable_function() is > not acceptable. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [PATCH][BUG_FIX] Potential null pointer dereferencing.

2019-11-12 Thread Kyotaro Horiguchi
2 more possible cases, envolving ExecClearTuple: > nodeFunctionscan.c and nodeWindowAgg.c regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [proposal] recovery_target "latest"

2019-11-12 Thread Kyotaro Horiguchi
t launch another wal receiver because we cleard StandbyMode just before. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [proposal] recovery_target "latest"

2019-11-12 Thread Kyotaro Horiguchi
#x27;t need to count failures to do that. It is enough that the function have two more exit point. When streaming timeout fires, and when we found that streaming is not set up when archive/wal failed. In my opinion, it is better that we have less dependency to global variables in such deep leve

Re: Coding in WalSndWaitForWal

2019-11-13 Thread Kyotaro Horiguchi
ces RecentFlushPtr is enough. However, I'm not confident taht removing the (intended) fast path impacts perforance significantly. So I don't object to remove it. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Invisible PROMPT2

2019-11-13 Thread Kyotaro Horiguchi
b%c%w] ' \set PROMPT2 '%w ' where %w in PROMPT2 is replaced by a whitespace with the same length to the output of %w[..%w] part in PROMPT1. > > We already have "%:PROMPT1:" but that expands to the literal value of > > prompt1, not to the value that prompt1 would expand to: > > Yeah, that's not so great for this usage. I guess "expand variables" > could be a separate useful feature (and patch) all by itself... +1. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: MarkBufferDirtyHint() and LSN update

2019-11-14 Thread Kyotaro Horiguchi
N is set is in the same buffer-ex-lock section with XLogInsert.. but not sure. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Coding in WalSndWaitForWal

2019-11-14 Thread Kyotaro Horiguchi
r the first part, but I'm not sure for the second. But it should be avoided if it can be happen. # the walreader's callback structure makes such things less clear :p I remember that there was a fixed bug that logical replication code fails to send a reply for a longer time than timeout

Re: Protect syscache from bloating with negative cache entries

2019-11-19 Thread Kyotaro Horiguchi
h: indirect SearchCatCacheInternal CatCache_Pattern_1.patch: indirect SearchCatCacheN regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From 245e88e1b43df74273fbaa1b22f4f64621ffe9d5 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 14 Nov 2019 19:24:36 +0900 Subject: [PATCH 1/2]

Re: initdb SegFault

2019-11-19 Thread Kyotaro Horiguchi
handling path would > > be worthwhile for OOM paths. Doing only a write() or such to print an > > error message. > > Perhaps. You wouldn't get any translation --- but then, gettext is > probably going to fail anyway under such conditions. I think we should refrain from translating in the cases. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-19 Thread Kyotaro Horiguchi
7 ms 316: 3074 ms 1788 ms 562: 3306 ms 1245 ms 1000: 3440 ms 2182 ms 1778: 5064 ms 6464 ms # WAL's slope becomes steep 3162: 8675 ms 8165 ms I don't think the result of 100 processes is meaningful, so excluding the result a candidate for wal_skip_threshold can be

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-20 Thread Kyotaro Horiguchi
patches into one. Split patches are good when one could > reasonably choose to push a subset, but that didn't apply here. I wouldn't > push a GUC implementation without its documentation. Since the tests fail > without the main bug fix, I wouldn't push tests separately. > > By the way, based on the comment at zheap_prepare_insert(), I expect zheap > will exempt itself from skipping WAL. It may stop calling RelationNeedsWAL() > and instead test for RELPERSISTENCE_PERMANENT. -- Kyotaro Horiguchi NTT Open Source Software Center

an OID >= 8000 in master

2019-11-20 Thread Kyotaro Horiguchi
Hello. I happened to find that the commit 71dcd74 added the function "network_sortsupport" with OID = 8190. Is it right? Otherwise we should we move it to, say, 4035. (I understand that OID [8000, ] are development-use.) regards. -- Kyotaro Horiguchi NTT Open Source Software Ce

Re: Internal function returning pg_statistic

2019-11-20 Thread Kyotaro Horiguchi
t; > CREATE OR REPLACE FUNCTION > > pg_gtt_statistic_for_relation(relid oid) returns setof pg_statistic > > LANGUAGE INTERNAL STRICT > > AS 'pg_gtt_statistic_by_relation'; > > > > And if I try to declare it as returning record and explicitly cast it to > > pg_statistic, then reported error is "cannot cast type record to > > pg_statistic". > > > > So the only possible way I found is to create extension and define > > function in this extension. > > I wonder if there is some better solution? > > > > Thanks in advance, regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: an OID >= 8000 in master

2019-11-20 Thread Kyotaro Horiguchi
tive OID(s) available starting here) Thoughts? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: an OID >= 8000 in master

2019-11-20 Thread Kyotaro Horiguchi
cript. Is the "point" is what the name suggests? The tool is, for developers, a means of finding OIDs *usable for their project*. It doesn't seem appropriate to show OIDs that developers are supposed to refrain from using. In my proposal the tool still shows all unused OIDs as the name

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-20 Thread Kyotaro Horiguchi
the chance of causing performance regressions. (One could, > however, work around the problem by raising wal_skip_threshold.) Kyotaro, if > you agree, could you modify v24nm to implement that? Seems reasonable. Please wait a minite. regards. -- Kyotaro Horiguchi NTT Open Source

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-20 Thread Kyotaro Horiguchi
Wow.. This is embarrassing.. *^^*. At Thu, 21 Nov 2019 16:01:07 +0900 (JST), Kyotaro Horiguchi wrote in > I should have replied this first. > > At Sun, 17 Nov 2019 20:54:34 -0800, Noah Misch wrote in > > On Tue, Nov 05, 2019 at 02:53:35PM -0800, Noah Misch wrote: > > &

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-21 Thread Kyotaro Horiguchi
At Thu, 21 Nov 2019 16:01:07 +0900 (JST), Kyotaro Horiguchi wrote in > > For deleting relfilenodes, smgrDoPendingDeletes() collects a list for > > smgrdounlinkall() to pass to DropRelFileNodesAllBuffers(), which is > > sophisticated about optimizing the shared buffers sca

Re: pause recovery if pitr target not reached

2019-11-21 Thread Kyotaro Horiguchi
if server promotes without stopping as told by target_action variables, it is a sign that something's wrong. But if server pauses before reaching target, operators may overlook the message if they don't know of the behavior. And if server poses in the case, I think there's nothing to do. So +1 for FATAL. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: an OID >= 8000 in master

2019-11-21 Thread Kyotaro Horiguchi
sable for their project*. It doesn't seem > > appropriate to show OIDs that developers are supposed to refrain from > > using. In my proposal the tool still shows all unused OIDs as the name > > suggests when some option specified. > > The existing output seems perfectly

Re: Why overhead of SPI is so large?

2019-11-21 Thread Kyotaro Horiguchi
uot; here. It seems to me that every line in a plpgsql function is regarded as a "query" for stable function, even if the function is called in another "query". In short, we need it, I think. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Why overhead of SPI is so large?

2019-11-22 Thread Kyotaro Horiguchi
At Fri, 22 Nov 2019 08:08:24 +0100, Pavel Stehule wrote in > pá 22. 11. 2019 v 7:33 odesílatel Kyotaro Horiguchi > napsal: > > > At Fri, 22 Nov 2019 06:15:25 +0100, Pavel Stehule > > wrote in > > > čt 21. 11. 2019 v 20:44 odesílatel Tom Lane napsal: >

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-24 Thread Kyotaro Horiguchi
At Sat, 23 Nov 2019 16:21:36 -0500, Noah Misch wrote in > On Wed, Nov 20, 2019 at 03:05:46PM +0900, Kyotaro Horiguchi wrote: > > By the way, before finalize this, I'd like to share the result of a > > brief benchmarking. > > What non-default settings did you use? Plea

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-26 Thread Kyotaro Horiguchi
At Sun, 24 Nov 2019 22:08:39 -0500, Noah Misch wrote in > On Mon, Nov 25, 2019 at 11:08:54AM +0900, Kyotaro Horiguchi wrote: > > At Sat, 23 Nov 2019 16:21:36 -0500, Noah Misch wrote in > > > I noticed an additional defect: > > > > > > BEGIN;

Re: ERROR: attribute number 6 exceeds number of columns 5

2019-11-26 Thread Kyotaro Horiguchi
uot;no such column" type of error, not a run-time > cross-check failure. Something like this works? ALTER TABLE gtest25 ADD COLUMN x int, ADD COLUMN y int GENERATED ALWAYS AS (x * 4) STORED; ERROR: column "x" does not exist DETAIL: An expression cannot reference columns

Re: Remove page-read callback from XLogReaderState.

2019-11-26 Thread Kyotaro Horiguchi
At Thu, 24 Oct 2019 14:51:01 +0900 (JST), Kyotaro Horiguchi wrote in > Rebased. 0dc8ead463 hit this. Rebased. regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From d9007aee88f7400b0f03ced1b80584964a1b0b79 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Thu, 5 Se

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-28 Thread Kyotaro Horiguchi
At Tue, 26 Nov 2019 21:37:52 +0900 (JST), Kyotaro Horiguchi Is is not fully checked. I didn't merged and mesured performance yet, > but I post the status-quo patch for now. It was actually inconsistency caused by swap_relation_files. 1. rd_createSubid of relcache for r2 is not tu

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-11-28 Thread Kyotaro Horiguchi
seems to be about 2500 pages. Seeing this, I became to be worry that the optimization might give far smaller advantage than expected. Putting aside that, it seems to me that the default value for the threshold would be 500-1000, same as the previous benchmark showed. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Remove page-read callback from XLogReaderState.

2019-11-28 Thread Kyotaro Horiguchi
At Wed, 27 Nov 2019 01:11:40 -0300, Alvaro Herrera wrote in > On 2019-Nov-27, Kyotaro Horiguchi wrote: > > > At Thu, 24 Oct 2019 14:51:01 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > Rebased. > > > > 0dc8ead463 hit this. Rebased. > > Pleas

Re: Remove page-read callback from XLogReaderState.

2019-11-29 Thread Kyotaro Horiguchi
At Thu, 28 Nov 2019 21:37:03 +0900 (JST), Kyotaro Horiguchi wrote in > > > 0dc8ead463 hit this. Rebased. > > > > Please review the pg_waldump.c hunks in 0001; they revert recent changes. > > Ughhh! I'l check it. Thank you for noticing!! Fixed that, re-rebase

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-12-03 Thread Kyotaro Horiguchi
Hello. At Thu, 28 Nov 2019 17:23:19 -0500, Noah Misch wrote in > On Thu, Nov 28, 2019 at 09:35:08PM +0900, Kyotaro Horiguchi wrote: > > I measured the performance with the latest patch set. > > > > > 1. Determine $DDL_COUNT, a number of DDL transactions that take about

Re: Session WAL activity

2019-12-03 Thread Kyotaro Horiguchi
on of the session-stats table, the name should be changed. Backend status is more appropriate than PGPROC. See pgstat.c. Some kind of locking is needed to update the fields on shared segment. (LWLocks for PGPROC and PGSTAT_BEGIN/END_WRITE_ACTIVITY for PgBackendStatus) Knitpickings: The patch contai

Re: Increase footprint of %m and reduce strerror()

2019-12-03 Thread Kyotaro Horiguchi
entical messages of different commands appear having different neighbours in different po files. By the way aren't we going to have ereport on frontend? regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: could not stat promote trigger file leads to shutdown

2019-12-04 Thread Kyotaro Horiguchi
ces, including directories, files, commands can face that kind of problem. For example a bogus value for archive_command doesn't preveint server from starting. I understand that the reason is that we don't have a reliable means to check-up the command before we actually execute it, but serv

Re: Session WAL activity

2019-12-04 Thread Kyotaro Horiguchi
Hi. At Wed, 4 Dec 2019 16:40:27 +0300, Konstantin Knizhnik wrote in > > > On 04.12.2019 8:33, Kyotaro Horiguchi wrote: > > It seems to be useful to me. We also might want statistics of other > > session IOs. In that case the table name would be > > "pg_stat_s

Re: Increase footprint of %m and reduce strerror()

2019-12-04 Thread Kyotaro Horiguchi
At Thu, 5 Dec 2019 11:36:48 +0900, Michael Paquier wrote in > On Wed, Dec 04, 2019 at 03:32:11PM +0900, Kyotaro Horiguchi wrote: > > It sounds good to me. Message unification (including printf) needs > > somehow treating trailing new lines, though. About translation > >

Re: Append with naive multiplexing of FDWs

2019-12-04 Thread Kyotaro Horiguchi
any queries can do any meaningful work until all > shards have giving their full results. There's my pending (somewhat stale) patch, which allows to run local scans while waiting for remote servers. https://www.postgresql.org/message-id/20180515.202945.69332784.horiguchi.kyot...@lab.ntt.co.jp I (or we) wanted to introduce the asynchronous node mechanism as the basis of async-capable postgres_fdw. The reason why it is stopping is that we are seeing and I am waiting the executor change that makes executor push-up style, on which the async-node mechanism will be constructed. If that won't happen shortly, I'd like to continue that work.. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Increase footprint of %m and reduce strerror()

2019-12-04 Thread Kyotaro Horiguchi
(Just to clarifying the last mail..) At Thu, 05 Dec 2019 12:06:54 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 5 Dec 2019 11:36:48 +0900, Michael Paquier > wrote in > > On Wed, Dec 04, 2019 at 03:32:11PM +0900, Kyotaro Horiguchi wrote: > > > It sounds good to me.

Re: Append with naive multiplexing of FDWs

2019-12-06 Thread Kyotaro Horiguchi
d it) and.. will re-run on the current development environment. (And re-check the code.). regards. -- Kyotaro Horiguchi NTT Open Source Software Center >From 9f5dc3720ddade94cd66713f4aa79da575b09e31 Mon Sep 17 00:00:00 2001 From: Kyotaro Horiguchi Date: Mon, 22 May 2017 12:42:58 +0900 Subject: [P

Re: [Proposal] Level4 Warnings show many shadow vars

2019-12-08 Thread Kyotaro Horiguchi
arameter name; it conveys > >nothing much, and the "X" prefix is already overused in that area of > >the code. Perhaps "pRedoRecPtr" would be a better choice? Or maybe > >make the local variables be all-lower-case "redorecptr", which would > >fit well in context in places like > pRedoRecPtr, It's perfect for me. Anyway I strongly object to the name 'pRedoRecPtr', which suggests as if it is a C-pointer to some variable. (And I believe we use Hungarian notation only if we don't have a better way...) LatestRedoRecPtr looks better to me. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-12-09 Thread Kyotaro Horiguchi
n. (I suspect that the if condition in mdclose should be an assertion..) > On Tue, Dec 03, 2019 at 08:51:46PM +0900, Kyotaro Horiguchi wrote: > > At Thu, 28 Nov 2019 17:23:19 -0500, Noah Misch wrote in > > > On Thu, Nov 28, 2019 at 09:35:08PM +0900, Kyotaro Horiguchi wrote: >

Re: xact_start for walsender & logical decoding not updated

2019-12-09 Thread Kyotaro Horiguchi
nsactions. I suppose we should do them a unified way. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [proposal] recovery_target "latest"

2019-12-09 Thread Kyotaro Horiguchi
meAvailable handles failure of the current source thus source transition happens only there. The second half just reports failure to the first half. > Both issues are fixed in the new patch version. > Any review and thoughts on the matters would be much appreciated. > > > > > > I think The doc needs to exiplain on the difference between default > > and latest. > Sure, I will work on it. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Session WAL activity

2019-12-10 Thread Kyotaro Horiguchi
> information: > backend with such PID actually never sleep on this event. I saw a case where an entry with very old xact_start_timestamp suddenly popped up in pg_stat_activity but I haven't found the cause.. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Start Walreceiver completely before shut down it on standby server.

2019-12-10 Thread Kyotaro Horiguchi
x, it doesn't seem to cause start process to rewind WAL to that LSN. Even if that happens, it leads to no better than a broken database. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: non-exclusive backup cleanup is mildly broken

2019-12-11 Thread Kyotaro Horiguchi
egistration of the callback. Actually before_shmem_exit_list gets bloat with multiple do_pg_abort_backup entries through repeated rounds of non-exclusive backups. As the result, if one ends a session while a non-exclusive backup is active after closing the previous non-exclusive backup, do_pg_abort_backup aborts for assertion failure. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Append with naive multiplexing of FDWs

2019-12-12 Thread Kyotaro Horiguchi
pf0 21311570+26.4% pf1 19261189+38.3% ptf0 20011557+22.2% ptf1 19031193+37.4% FWIW, attached are the test script. gentblr2.sql: Table creation script. testrun.sh : Benchmarking script. regards. -- Kyotaro Horiguchi NTT Open Source Software Center SELECT :scale

Re: archive status ".ready" files may be created too early

2019-12-12 Thread Kyotaro Horiguchi
ages of working on this problem, but I > thought it would be worth reporting to the community early on in case > anyone has any thoughts on or past experiences with this issue. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: xact_start for walsender & logical decoding not updated

2019-12-12 Thread Kyotaro Horiguchi
At Fri, 13 Dec 2019 13:05:41 +0800, Craig Ringer wrote in > On Wed, 11 Dec 2019 at 02:08, Alvaro Herrera > wrote: > > > On 2019-Dec-10, Tomas Vondra wrote: > > > > > On Tue, Dec 10, 2019 at 09:42:17AM +0900, Kyotaro Horiguchi wrote: > > > > At Tue

Re: non-exclusive backup cleanup is mildly broken

2019-12-15 Thread Kyotaro Horiguchi
stead. Since pg_stop_backup_v2 is the only caller to the function in the whole server code, making cancel_before_shmem_exit a bit wiser (and slower) cannot hurt anyone. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: non-exclusive backup cleanup is mildly broken

2019-12-16 Thread Kyotaro Horiguchi
At Tue, 17 Dec 2019 11:46:03 +0900, Fujii Masao wrote in > On Tue, Dec 17, 2019 at 4:19 AM Robert Haas wrote: > > > > On Sun, Dec 15, 2019 at 8:44 PM Kyotaro Horiguchi > > wrote: > > > However I don't object to the restriction, couldn't we allow the

Re: non-exclusive backup cleanup is mildly broken

2019-12-16 Thread Kyotaro Horiguchi
At Tue, 17 Dec 2019 15:11:55 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 17 Dec 2019 11:46:03 +0900, Fujii Masao wrote > in > PREPARE. The attached does that and changes the if condition of > cancel_before_shmem_exit into assertion. The patch can cause removal of a w

Re: archive status ".ready" files may be created too early

2019-12-17 Thread Kyotaro Horiguchi
Uggg. I must apologyze for the last bogus comment. At Fri, 13 Dec 2019 21:24:36 +, "Bossart, Nathan" wrote in > On 12/12/19, 8:08 PM, "Kyotaro Horiguchi" wrote: > > As the result the patch doesn't seem to save anything than setting up > > and op

Re: archive status ".ready" files may be created too early

2019-12-17 Thread Kyotaro Horiguchi
Thank you Alvaro for the comment (on my comment). At Fri, 13 Dec 2019 18:33:44 -0300, Alvaro Herrera wrote in > On 2019-Dec-13, Kyotaro Horiguchi wrote: > > > At Thu, 12 Dec 2019 22:50:20 +, "Bossart, Nathan" > > wrote in > > > > The crux of the

Re: error context for vacuum to include block number

2019-12-17 Thread Kyotaro Horiguchi
on't like :) lazy_vacuum_heap_index() seems confusing to me. I read the name as Michael did before looking the above explanation. lazy_vacuum_heap_and_index() is clearer to me. lazy_vacuum_heap_with_index() could also work but I'm not sure it's further better. I see some function n

Re: non-exclusive backup cleanup is mildly broken

2019-12-17 Thread Kyotaro Horiguchi
;without knowing whether you'd actually registered a handler or not", but for the case "without knowing whether someone could have registered a handler or not after the registration I made". But there is not so much difference and I don't insist on that because of the reason you mentioned as above. Thanks for elaborating. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: archive status ".ready" files may be created too early

2019-12-18 Thread Kyotaro Horiguchi
t happen. No? The trouble happens after the synced file is archived. If the last record in the arcvhied segment was the first half of a continuation record and crash before writing the last half, crash recovery stops just before the first half and different record can be overwitten. As the result the archived version of the segment becomes rotten. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: [HACKERS] pg_shmem_allocations view

2019-12-18 Thread Kyotaro Horiguchi
hashes but are recognized as holes shown by the above statement. So the current output of the view is wrong in that sense. I think (1) should be resolved before adding the view. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Clean up some old cruft related to Windows

2019-12-18 Thread Kyotaro Horiguchi
around line 117. - installation.sgml is mentioning NT/2000/XP as platforms that don't support adduser/su, command. - "of Windows 2000 or later" is found at installation.sgml:2467 regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: Fetching timeline during recovery

2019-12-19 Thread Kyotaro Horiguchi
the original one without arg. I end up with this solution > because I was worried about adding five more funcs really close to some > existing one. Right. It is a restriction of polymorphic functions. It is in the same relation with pg_stop_backup() and pg_stop_backup(true). > Fifth one is more consistent with what we already have. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-12-20 Thread Kyotaro Horiguchi
may emit incorrect result if it made a catalog change during the current transaction being decoded? If so, this is not a feature but a bug fix. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: mdclose() does not cope w/ FileClose() failure

2019-12-23 Thread Kyotaro Horiguchi
y are comparable each other, or we don't want the array gets resized frequently, we might need to prevent repalloc from happening on every segment increase, too. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: archive status ".ready" files may be created too early

2019-12-23 Thread Kyotaro Horiguchi
At Sat, 21 Dec 2019 01:18:24 +, "Bossart, Nathan" wrote in > On 12/18/19, 8:34 AM, "Bossart, Nathan" wrote: > > On 12/17/19, 2:26 AM, "Kyotaro Horiguchi" wrote: > >> If so, we could amend also that case by marking the last segment as >

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-12-23 Thread Kyotaro Horiguchi
At Tue, 10 Dec 2019 16:59:25 +0900 (JST), Kyotaro Horiguchi wrote in > shared_buffers=1GB/wal_buffers=16MB(defalut). pgbench -s 20 uses 256MB > of storage so all of them can be loaded on shared memory. > > The attached graph shows larger benefit in TPS drop and latency > increa

<    5   6   7   8   9   10   11   12   13   14   >