Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2020-09-06 Thread Amit Kapila
On Sat, Sep 5, 2020 at 8:55 PM Dilip Kumar wrote: > > > I have reviewed the changes and looks fine to me. > Thanks, I have pushed the last patch. Let's wait for a day or so to see the buildfarm reports and then we can probably close this CF entry. I am aware that we have one patch related to stat

Re: A micro-optimisation for walkdir()

2020-09-06 Thread Thomas Munro
On Mon, Sep 7, 2020 at 10:36 AM Tom Lane wrote: > Thomas Munro writes: > > Excellent. I'd like to commit these soon, unless someone has a better > > idea for how to name file_utils_febe.c. > > As long as it's in src/port or src/common, isn't it implicit that > it's probably FE/BE common code? >

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-06 Thread Amit Kapila
On Thu, Jul 23, 2020 at 11:46 AM Masahiko Sawada wrote: > > I've updated the patch so that the stats collector ignores the > 'update' message if the slot stats array is already full. > This patch needs a rebase. I don't see this patch in the CF app. I hope you are still interested in working on t

Re: clarify "rewritten" in pg_checksums docs

2020-09-06 Thread Michael Paquier
On Wed, Sep 02, 2020 at 05:26:16PM +0900, Michael Paquier wrote: > Using rewritten still sounds more adapted to me, as we still write the > thing with chunks of size BLCKSZ. No objections with the addition of > "in-place" for that sentence. Any extra opinions? Seeing no objections, I have applie

Re: SQL-standard function body

2020-09-06 Thread Peter Eisentraut
Some conflicts have emerged, so here is an updated patch. I have implemented/fixed the inlining of set-returning functions written in the new style, which was previously marked TODO in the patch. On 2020-08-28 07:33, Peter Eisentraut wrote: On 2020-06-30 19:49, Peter Eisentraut wrote: This

Re: Auto-vectorization speeds up multiplication of large-precision numerics

2020-09-06 Thread Tom Lane
I wrote: > I made some cosmetic changes to this and committed it. BTW, poking at this further, it seems that the patch only really works for gcc. clang accepts the -ftree-vectorize switch, but looking at the generated asm shows that it does nothing useful. Which is odd, because clang does do loop

Re: [HACKERS] logical decoding of two-phase transactions

2020-09-06 Thread Ajin Cherian
Hello, Trying to revive this patch which attempts to support logical decoding of two phase transactions. I've rebased and polished Nikhil's patch on the current HEAD. Some of the logic in the previous patchset has already been committed as part of large-in-progress transaction commits, like the ha

Re: proposal: possibility to read dumped table's name from file

2020-09-06 Thread Pavel Stehule
pá 4. 9. 2020 v 5:21 odesílatel Pavel Stehule napsal: > > > pá 4. 9. 2020 v 2:15 odesílatel Tom Lane napsal: > >> Alvaro Herrera writes: >> > So, Tom added a coding pattern for doing this in commit 8f8154a503c7, >> > which is ostensibly also to be used in pg_regress [1] -- maybe it'd be >> > us

Re: v13: CLUSTER segv with wal_level=minimal and parallel index creation

2020-09-06 Thread Kyotaro Horiguchi
At Mon, 07 Sep 2020 00:33:03 -0400, Tom Lane wrote in > Justin Pryzby writes: > > This is easily reproduced, at least on pg_attribute Thanks for the repro. > Yeah, duplicated here. The assertion (not SIGSEGV) is in code > added by c6b92041d, so it seems clearly an open issue for v13. > > I d

[doc] minor wording improvement in a couple of places

2020-09-06 Thread Ian Barwick
Hi On these pages: https://www.postgresql.org/docs/current/fdw-callbacks.html https://www.postgresql.org/docs/current/tablesample-method.html we have the phrase: "..., which see for additional details." which strikes me as a bit odd. Suggested phrasing: "...; see this file for additi

Re: v13: CLUSTER segv with wal_level=minimal and parallel index creation

2020-09-06 Thread Tom Lane
Justin Pryzby writes: > This is easily reproduced, at least on pg_attribute Yeah, duplicated here. The assertion (not SIGSEGV) is in code added by c6b92041d, so it seems clearly an open issue for v13. I did not check to see what happens in a non-assert build. regards, t

Re: [PATCH] Initial progress reporting for COPY command

2020-09-06 Thread Michael Paquier
On Thu, Jul 30, 2020 at 08:51:36AM +0900, Fujii Masao wrote: > Yeah, since I read this message, I was thinking that new patch will be > posted. But, Josef, if the situation was changed, could you correct me? > Anyway the patch seems not to be applied cleanly, so at least it needs to > be updated to

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-09-06 Thread Pavel Stehule
po 7. 9. 2020 v 1:46 odesílatel Tom Lane napsal: > Peter Eisentraut writes: > > Here is a new patch series version. > > I have created a new internal function for converting integers to > > numeric, to make the implementation a bit more elegant and compact. > > I reviewed the 0002 patch, finding

Re: v13: CLUSTER segv with wal_level=minimal and parallel index creation

2020-09-06 Thread Justin Pryzby
This is easily reproduced, at least on pg_attribute [pryzbyj@localhost ~]$ /usr/pgsql-13/bin/initdb -D pgsql13.dat [pryzbyj@localhost ~]$ /usr/pgsql-13/bin/postgres -D pgsql13.dat -c logging_collector=off -c port=5678 -c unix_socket_directories=/tmp -c wal-level=minimal -c max_wal_senders=0& [pr

Re: Collation versioning

2020-09-06 Thread Michael Paquier
On Fri, Aug 14, 2020 at 11:02:35AM +0200, Julien Rouhaud wrote: > On Fri, Aug 14, 2020 at 04:37:46PM +0900, Michael Paquier wrote: >> + /* >> +* XXX For deterministic transaction, se should only track the >> version >> +* if the AM relies on a stable ordering. >> +*/ >

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-06 Thread Noah Misch
On Mon, Sep 07, 2020 at 11:06:12AM +0900, Michael Paquier wrote: > On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > > Rebased the second patch. The first patch did not need a rebase. > > It looks like a new rebase is needed, the CF bot is complaining here. http://cfbot.cputube.org/p

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-06 Thread Michael Paquier
On Sun, Sep 06, 2020 at 09:00:02PM +, Zidenberg, Tsahi wrote: > These results show only about 6% performance increase in simple-update, and > very close > performance in other results, most of which could be attributed to benchmark > result jitter. > These results from "well behaved" benchmar

Re: archive status ".ready" files may be created too early

2020-09-06 Thread Michael Paquier
On Wed, Jul 22, 2020 at 02:53:49AM +, matsumura@fujitsu.com wrote: > Then, Record-A may be invalidated by crash-recovery and overwritten by new > WAL record. > The segment-X is not same as the archived one. Please note that the latest patch fails to apply per the CF bot, so a rebase would

Re: vacuum verbose: show pages marked allvisible/frozen/hintbits

2020-09-06 Thread Michael Paquier
On Mon, Jun 15, 2020 at 01:30:58PM +0900, Masahiko Sawada wrote: > For 0002 patch, how users will be able to make any meaning out of how > many hint bits were updated by vacuum? The patch has not been updated for the last three months, though it looks kind of interesting to have more stats for fro

Re: Let people set host(no)ssl settings from initdb

2020-09-06 Thread Michael Paquier
On Thu, Jul 02, 2020 at 04:02:21PM +0200, Daniel Gustafsson wrote: > The CF Patch Tester consider this patch to be malformed and is unable to apply > and test it. Can you please submit a rebased version? I have looked at the patch of this thread, and I doubt that it is a good idea to put more bur

Re: Remove page-read callback from XLogReaderState.

2020-09-06 Thread Michael Paquier
On Fri, Jul 17, 2020 at 02:14:44PM +0900, Takashi Menjo wrote: > I applied your v15 patchset to master > ed2c7f65bd9f15f8f7cd21ad61602f983b1e72e9. Here are three feedback points > for you: And the CF bot complains as well here. Horiguchi-san, this patch is waiting on author for a couple of weeks

v13: CLUSTER segv with wal_level=minimal and parallel index creation

2020-09-06 Thread Justin Pryzby
Following a bulk load, a CLUSTER command run by a maintenance script crashed. This is currently reproducible on that instance, so please suggest if I can provide more info. < 2020-09-06 15:44:16.369 MDT >LOG: background worker "parallel worker" (PID 2576) was terminated by signal 6: Aborted < 2

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-06 Thread Michael Paquier
On Fri, Aug 21, 2020 at 03:25:29PM +0900, Masahiko Sawada wrote: > Thank you for letting me know. I've attached the latest version patch set. This needs a rebase. Patch 0002 is conflicting with some of the recent changes done in syncrep.c and procarray.c, at least. -- Michael signature.asc Des

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-09-06 Thread Michael Paquier
On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > Rebased the second patch. The first patch did not need a rebase. It looks like a new rebase is needed, the CF bot is complaining here. -- Michael signature.asc Description: PGP signature

Re: shared-memory based stats collector

2020-09-06 Thread Michael Paquier
On Thu, Sep 03, 2020 at 01:16:59PM -0400, Stephen Frost wrote: > Shouldn't this: > > a) be back-patched, as the other change was 0001 is just a piece of refactoring, so I see no strong argument in favor of a backpatch, IMHO. > b) also include a change to have the stats collector (which I realize

Re: 回复:how to create index concurrently on partitioned table

2020-09-06 Thread Michael Paquier
On Fri, Sep 04, 2020 at 09:51:13AM +0900, Michael Paquier wrote: > Glad to hear that, please take the time you need. Attached is a rebased version to address the various conflicts after 844c05a. -- Michael diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h index 3129b684f6

Re: Auto-vectorization speeds up multiplication of large-precision numerics

2020-09-06 Thread Tom Lane
Amit Khandekar writes: > I did as above. Attached is the v2 patch. I made some cosmetic changes to this and committed it. AFAICT, there's no measurable performance change to the "numeric" regression test, but I got a solid 45% speedup on "numeric_big", so it's clearly a win for wider arithmetic

Re: New statistics for tuning WAL buffer size

2020-09-06 Thread Masahiro Ikeda
Thanks for the review and advice! On 2020-09-03 16:05, Fujii Masao wrote: On 2020/09/02 18:56, Masahiro Ikeda wrote: +/* -- + * Backend types + * -- You seem to forget to add "*/" into the above comment. This issue could cause the following compiler warning. ../../src/include/p

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-06 Thread Masahiro Ikeda
On 2020-09-03 23:08, Masahiko Sawada wrote: On Fri, 28 Aug 2020 at 17:50, Masahiro Ikeda wrote: > I think there is a case we can't check orphaned foreign > prepared transaction in pg_foreign_xacts view on the new standby > server. > It confuses users and database administrators. > > If the pri

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2020-09-06 Thread Tom Lane
Peter Eisentraut writes: > Here is a new patch series version. > I have created a new internal function for converting integers to > numeric, to make the implementation a bit more elegant and compact. I reviewed the 0002 patch, finding one bug (in int8_sum) and a few more calls of int8_numeric t

Re: A micro-optimisation for walkdir()

2020-09-06 Thread Tom Lane
Thomas Munro writes: > Excellent. I'd like to commit these soon, unless someone has a better > idea for how to name file_utils_febe.c. As long as it's in src/port or src/common, isn't it implicit that it's probably FE/BE common code? I think it'd make more sense to insert all this stuff into fi

Re: A micro-optimisation for walkdir()

2020-09-06 Thread Thomas Munro
On Sun, Sep 6, 2020 at 5:23 AM Juan José Santamaría Flecha wrote: > On Sat, Sep 5, 2020 at 2:13 AM Andres Freund wrote: >> > However, it looks like we might be missing a further opportunity >> > here... Doesn't Windows already give us the flags we need in the >> > dwFileAttributes member of the

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-06 Thread Zidenberg, Tsahi
Hello! First, I apologize for taking so long to answer. This e-mail regretfully got lost in my inbox. On 24/07/2020, 4:17, "Andres Freund" wrote: > What does "not significantly affected" exactly mean? Could you post the > raw numbers? The following tests show benchmark behavior on m6

Re: [bug+patch] Inserting DEFAULT into generated columns from VALUES RTE

2020-09-06 Thread Tom Lane
Mikhail Titov writes: > Previously submitted patch got somehow trailing spaces mangled on the > way out. This is an attempt to use application/octet-stream MIME instead > of text/x-patch to preserve those for regression tests. I took a quick look through this. I agree with the general idea of de

Re: Disk-based hash aggregate's cost model

2020-09-06 Thread Tomas Vondra
Hi, I've tested the costing changes on the simplified TPC-H query, on two different machines, and it seems like a clear improvement. This is using the same cost/duration measure, which I think is pretty neat way to look at this. Sure, it's imperfect (depends on which cost and durations you actua

Re: v13: show extended stats target in \d

2020-09-06 Thread Justin Pryzby
On Sun, Sep 06, 2020 at 01:06:05PM -0700, Peter Geoghegan wrote: > On Tue, Sep 1, 2020 at 2:08 PM Alvaro Herrera > wrote: > > It does need some separator. Maybe a comma is sufficient, but I'm not > > sure: that will fail when we add cross-relation stats, because the > > FROM clause will have mor

Re: v13: show extended stats target in \d

2020-09-06 Thread Peter Geoghegan
On Tue, Sep 1, 2020 at 2:08 PM Alvaro Herrera wrote: > It does need some separator. Maybe a comma is sufficient, but I'm not > sure: that will fail when we add cross-relation stats, because the > FROM clause will have more relations and possibly have commas too. How about a line break? That seem

Re: Improving connection scalability: GetSnapshotData()

2020-09-06 Thread Andres Freund
Hi, On 2020-09-06 14:05:35 +0300, Konstantin Knizhnik wrote: > On 04.09.2020 21:53, Andres Freund wrote: > > > > > May be it is because of more complex architecture of my server? > > Think we'll need profiles to know... > > This is "perf top" of pgebch -c 100 -j 100 -M prepared -S > >   12.16% 

Re: Improving connection scalability: GetSnapshotData()

2020-09-06 Thread Andres Freund
Hi, On 2020-09-05 16:58:31 +0300, Konstantin Knizhnik wrote: > On 04.09.2020 21:53, Andres Freund wrote: > > > > I also used huge_pages=on / configured them on the OS level. Otherwise > > TLB misses will be a significant factor. > > As far as I understand there should not be no any TLB misses be

Re: Yet another fast GiST build (typo)

2020-09-06 Thread Andrey M. Borodin
> 6 сент. 2020 г., в 18:26, Heikki Linnakangas написал(а): > > On 05/09/2020 14:53, Andrey M. Borodin wrote: >> Thanks for ideas, Heikki. Please see v13 with proposed changes. > > Thanks, that was quick! > >> But I've found out that logging page-by-page slows down GiST build by >> approximate

Re: [PATCH] - Provide robust alternatives for replace_string

2020-09-06 Thread Tom Lane
I wrote: > This version looks committable to me, though perhaps someone has > further thoughts? I looked through this again and pushed it. regards, tom lane

Re: file_fdw vs relative paths

2020-09-06 Thread Magnus Hagander
On Mon, Aug 31, 2020 at 5:03 PM Bruce Momjian wrote: > On Mon, Aug 31, 2020 at 01:16:05PM +0200, Magnus Hagander wrote: > > Bruce, I've applied and backpatched your docs patch for this. > > > > Gah, and of course right after doing that, I remembered I wanted to get a > > second change in :) T

Re: Append with naive multiplexing of FDWs

2020-09-06 Thread Etsuro Fujita
On Tue, Sep 1, 2020 at 9:45 AM Kyotaro Horiguchi wrote: > At Mon, 31 Aug 2020 19:10:39 +0900, Etsuro Fujita > wrote in > > On Mon, Aug 31, 2020 at 6:20 PM Etsuro Fujita > > wrote: > > > * I know your patch is a POC one, but one concern about it (and > > > Horiguchi-san's patch set) is concurre

Re: moving aggregate bad error message

2020-09-06 Thread Tom Lane
Jeff Janes writes: > Looking at the code, it seems we are testing rettype != finaltype, but > reporting aggmTransType and aggTransType. Why aren't we reporting what we > are testing? Silly thinko, apparently. Your fix looks right, will push. regards, tom lane

moving aggregate bad error message

2020-09-06 Thread Jeff Janes
I was wondering if I could just add minvfunc, and have the rest of the m* functions be assumed to be the same as their non-moving counterparts. Apparently the answer is 'no'. But in the process, I found a bad error message. When omitting mfinalfunc when there is a finalfunc, I get the error: "ER

Re: Dubious code in pg_rewind's process_target_file()

2020-09-06 Thread Tom Lane
Heikki Linnakangas writes: > On 06/09/2020 18:06, Tom Lane wrote: >> I propose to remove the lstat() in the back branches, but not touch >> HEAD so as not to cause extra merge effort for your patch. > Thanks! Feel free to push it to HEAD, too, the merge conflict will be > trivial to resolve. OK

Re: Dubious code in pg_rewind's process_target_file()

2020-09-06 Thread Heikki Linnakangas
On 06/09/2020 18:06, Tom Lane wrote: Heikki Linnakangas writes: On 05/09/2020 21:18, Tom Lane wrote: Or actually, maybe we should just drop the lstat call altogether? Agreed, the lstat() doesn't do anything interesting. This is refactored away by the patches discussed at https://www.postgre

Re: Dubious code in pg_rewind's process_target_file()

2020-09-06 Thread Tom Lane
Heikki Linnakangas writes: > On 05/09/2020 21:18, Tom Lane wrote: >> Or actually, maybe we should just drop the lstat call altogether? > Agreed, the lstat() doesn't do anything interesting. > This is refactored away by the patches discussed at > https://www.postgresql.org/message-id/f155aab5-132

Re: Missing "Up" navigation link between parts and doc root?

2020-09-06 Thread Peter Eisentraut
On 2020-08-25 21:48, Bruce Momjian wrote: On Sat, Jul 4, 2020 at 08:47:53AM +0200, Fabien COELHO wrote: Hello Peter, The original stylesheets explicitly go out of their way to do it that way. We can easily fix that by removing that special case. See attached patch. That patch only fixes it

Re: Dubious code in pg_rewind's process_target_file()

2020-09-06 Thread Heikki Linnakangas
On 05/09/2020 21:18, Tom Lane wrote: I wrote: It looks to me like we could replace "exists = false" with "return", rather than uselessly constructing a FILE_ACTION_REMOVE entry for a file we've already proven is not there. Or actually, maybe we should just drop the lstat call altogether? AFAIC

Re: Yet another fast GiST build (typo)

2020-09-06 Thread Heikki Linnakangas
On 05/09/2020 14:53, Andrey M. Borodin wrote: Thanks for ideas, Heikki. Please see v13 with proposed changes. Thanks, that was quick! But I've found out that logging page-by-page slows down GiST build by approximately 15% (when CPU constrained). Though In think that this is IO-wise. Hmm, any

Re: Optimising compactify_tuples()

2020-09-06 Thread David Rowley
On Thu, 20 Aug 2020 at 11:28, Thomas Munro wrote: > I fixed up the copyright messages, and removed some stray bits of > build scripting relating to the Perl-generated file. Added to > commitfest. I'm starting to look at this. So far I've only just done a quick performance test on it. With the w

Re: Improving connection scalability: GetSnapshotData()

2020-09-06 Thread Konstantin Knizhnik
On 04.09.2020 21:53, Andres Freund wrote: May be it is because of more complex architecture of my server? Think we'll need profiles to know... This is "perf top" of pgebch -c 100 -j 100 -M prepared -S   12.16%  postgres   [.] PinBuffer   11.92%  postgres