Re: Improvements in Copy From

2020-09-10 Thread Kyotaro Horiguchi
At Thu, 10 Sep 2020 21:55:27 +0300, Surafel Temesgen wrote in > On Thu, Sep 10, 2020 at 1:17 PM vignesh C wrote: > > > > > > > > > We have a patch for column matching feature [1] that may need a header > > line to be further processed. Even without that I think it is preferable to > > process

Corner-case bug in pg_rewind

2020-09-10 Thread Ian Barwick
Hi Take the following cluster with: - node1 (initial primary) - node2 (standby) - node3 (standby) Following activity takes place (greatly simplified from a real-world situation): 1. node1 is shut down. 2. node3 is promoted 3. node2 is attached to node3. 4. node1 is attached to node3 5. nod

Re: Yet another fast GiST build

2020-09-10 Thread Andrey M. Borodin
> 10 сент. 2020 г., в 17:43, Heikki Linnakangas написал(а): > > One more patch version attached. I fixed some memory leaks, and fixed the > abbreviation on 32-bit systems, and a bunch of small comment changes and such. > > - Heikki > The patch looks fine to me. On my machine GiST for point

Re: Optimising compactify_tuples()

2020-09-10 Thread Thomas Munro
On Fri, Sep 11, 2020 at 3:53 AM David Rowley wrote: > That gets my benchmark down to 60.8 seconds, so 2.2 seconds better than v4b. . o O ( I wonder if there are opportunities to squeeze some more out of this with __builtin_prefetch... ) > I've attached v6b and an updated chart showing the result

Re: Optimising compactify_tuples()

2020-09-10 Thread Thomas Munro
On Fri, Sep 11, 2020 at 1:45 AM David Rowley wrote: > On Thu, 10 Sep 2020 at 10:40, Thomas Munro wrote: > > I wonder if we could also identify a range at the high end that is > > already correctly sorted and maximally compacted so it doesn't even > > need to be copied out. > > I've experimented q

Re: Fix for parallel BTree initialization bug

2020-09-10 Thread Hunter, James
Nice repro, thanks! -- James Hunter, Amazon Web Services (AWS) On 9/10/20 7:37 PM, Justin Pryzby wrote: Against all odds, I was able to reproduce this. begin; CREATE TABLE t AS SELECT generate_series(1,99)i; ALTER TABLE t SET (parallel_workers=2, autovacuum_enabled=off); CREATE INDEX ON t(i

RE: Implement UNLOGGED clause for COPY FROM

2020-09-10 Thread tsunakawa.ta...@fujitsu.com
From: Peter Smith On Thu, Sep 10, 2020 at 7:16 PM tsunakawa.ta...@fujitsu.com > wrote: > > ALTER TABLE takes long time proportional to the amount of existing data, > while wal_level = none doesn't. > > Right, but if wal_level=none is considered OK for that table with > existing data, then why no

Re: New statistics for tuning WAL buffer size

2020-09-10 Thread Fujii Masao
On 2020/09/11 12:17, Kyotaro Horiguchi wrote: Hello. At Wed, 09 Sep 2020 13:57:37 +0900, Masahiro Ikeda wrote in I checked what function calls XLogBackgroundFlush() which calls AdvanceXLInsertBuffer() to increment m_wal_buffers_full. I found that WalSndWaitForWal() calls it, so I added it

Re: New statistics for tuning WAL buffer size

2020-09-10 Thread Kyotaro Horiguchi
Hello. At Wed, 09 Sep 2020 13:57:37 +0900, Masahiro Ikeda wrote in > I checked what function calls XLogBackgroundFlush() which calls > AdvanceXLInsertBuffer() to increment m_wal_buffers_full. > > I found that WalSndWaitForWal() calls it, so I added it. > Is it better to move it in WalSndLoop()

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-10 Thread Fujii Masao
On 2020/09/11 0:37, Masahiko Sawada wrote: On Tue, 8 Sep 2020 at 13:00, tsunakawa.ta...@fujitsu.com wrote: From: Amit Kapila I intend to say that the global-visibility work can impact this in a major way and we have analyzed that to some extent during a discussion on the other thread. So,

Re: Inconsistency in determining the timestamp of the db statfile.

2020-09-10 Thread Amit Kapila
On Thu, Sep 10, 2020 at 6:42 PM Alvaro Herrera wrote: > > On 2020-Sep-10, Amit Kapila wrote: > > > On Thu, Sep 10, 2020 at 1:03 PM Magnus Hagander wrote: > > > The comments already say what you said in the second suggestion:"The > > caller must rely on timestamp stored in *ts iff the function ret

Re: SIGQUIT handling, redux

2020-09-10 Thread Tom Lane
I wrote: > I'll take a closer look at the idea of using _exit(1) tomorrow, > but I'd be pretty hesitant to back-patch that. Here's a patch for that; it passes light testing, including forcing the backend through the SIGTERM and timeout code paths. There's not much to it except for changing the si

Re: Fix for parallel BTree initialization bug

2020-09-10 Thread Justin Pryzby
Against all odds, I was able to reproduce this. begin; CREATE TABLE t AS SELECT generate_series(1,99)i; ALTER TABLE t SET (parallel_workers=2, autovacuum_enabled=off); CREATE INDEX ON t(i); commit; SET parallel_leader_participation=off; SET min_parallel_table_scan_size=0; SET enable_bitmapsc

Re: Implement UNLOGGED clause for COPY FROM

2020-09-10 Thread Peter Smith
On Thu, Sep 10, 2020 at 7:16 PM tsunakawa.ta...@fujitsu.com wrote: > ALTER TABLE takes long time proportional to the amount of existing data, > while wal_level = none doesn't. Right, but if wal_level=none is considered OK for that table with existing data, then why not just create the table UNL

Re: logtape.c stats don't account for unused "prefetched" block numbers

2020-09-10 Thread Peter Geoghegan
On Tue, Sep 8, 2020 at 11:28 PM Jeff Davis wrote: > Preallocation showed significant gains for HashAgg, and BufFile doesn't > support sparse writes. So, for HashAgg, it seems like we should just > update the comment and consider it the price of using BufFile. > For Sort, we can just disable preal

Re: Logical Replication - detail message with names of missing columns

2020-09-10 Thread Kyotaro Horiguchi
Thanks for the revised version. At Tue, 8 Sep 2020 22:36:17 +0530, Bharath Rupireddy wrote in > Thanks for the comments. Attaching the v3 patch. > > On Tue, Sep 8, 2020 at 8:19 PM Alvaro Herrera > wrote: > > > > This looks a bit fiddly. Would it be less cumbersome to use > > quote_identifie

Re: PG 13 release notes, first draft

2020-09-10 Thread Tom Lane
Justin Pryzby writes: > Rebasing onto 3965de54e718600a4703233936e56a3202caf73f, I'm left with: Sorry, I hadn't seen that you submitted more updates. I pushed these with minor additional wordsmithing. regards, tom lane

Re: Print logical WAL message content

2020-09-10 Thread Alvaro Herrera
On 2020-Aug-19, Ashutosh Bapat wrote: > I like this format as well. Done. > > PFA the patch attached with your comments addressed. Pushed, thanks! -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: PG 13 release notes, first draft

2020-09-10 Thread Justin Pryzby
On Mon, Sep 07, 2020 at 08:40:26AM -0500, Justin Pryzby wrote: Rebasing onto 3965de54e718600a4703233936e56a3202caf73f, I'm left with: diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 8fffc6fe0a..69d143e10c 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-09-10 Thread Alvaro Herrera
On 2020-Aug-31, Hao Wu wrote: > I tested the patch provided by Alvaro. It seems not well in REPEATABLE READ. > -- the tuples from tpart_2 are gone. > gpadmin=*# select * from tpart; > i | j > + > 10 | 10 > 50 | 50 > (2 rows) Interesting example, thanks. It seems this can be fixed wi

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread John Naylor
Ok, here's an attempt at a somewhat more natural test, to see what happens after bulk updates and deletes, followed by more inserts. The short version is that multi-minmax is resilient to a change that causes a 4x degradation for simple minmax. shared_buffers = 1GB random_page_cost = 1.1 effectiv

Re: PG 13 release notes, first draft

2020-09-10 Thread Tom Lane
"Jonathan S. Katz" writes: > One thing that did not make it through was this: > - 2020-XX-XX, CURRENT AS OF 2020-08-09 > + 2020-09-24, CURRENT AS OF 2020-09-09 Yeah, that's a placeholder to recall how far back to look for additional changes to the relnotes, so unless you already read the git his

Re: SIGQUIT handling, redux

2020-09-10 Thread Tom Lane
Robert Haas writes: > On Thu, Sep 10, 2020 at 12:56 PM Tom Lane wrote: >> Also, man that CHECK_FOR_INTERRUPTS() looks like trouble. >> Could we take that out? > Maybe I'm missing something, but why wouldn't that be a horrible idea? > We do not want to have long waits where we refuse to respond t

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread Alvaro Herrera
On 2020-Sep-10, Tomas Vondra wrote: > I've spent a bit of time experimenting with this. My idea was to allow > keeping an "expanded" version of the summary somewhere. As the addValue > function only receives BrinValues I guess one option would be to just > add bv_mem_values field. Or do you have a

Re: Collation versioning

2020-09-10 Thread Julien Rouhaud
On Thu, Sep 10, 2020 at 6:52 PM Peter Eisentraut wrote: > > On 2020-09-08 21:33, Christoph Berg wrote: > > IOW, I think we should aim at simply tracking the version, and leave > > it to the admin (with the help of supplied SQL queries) to either > > rebuild indexes or waive them. > > It's certainl

Probable documentation errors or improvements

2020-09-10 Thread Yaroslav
Disclaimer: I'm not a native speaker, so not sure those are actually incorrect, and can't offer non-trivial suggestions. General ones: . "boolean" vs "Boolean" --- usage seems to be inconsistent, even in the same context. . Transaction isolation levels are sometimes called "transaction isolation

Re: Improvements in Copy From

2020-09-10 Thread Surafel Temesgen
On Thu, Sep 10, 2020 at 1:17 PM vignesh C wrote: > > > > > We have a patch for column matching feature [1] that may need a header > line to be further processed. Even without that I think it is preferable to > process the header line for nothing than adding those checks to the loop, > performance

Re: PG 13 release notes, first draft

2020-09-10 Thread Jonathan S. Katz
On 9/10/20 1:14 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> Attached is a proposal for the "major enhancements" section. I borrowed >> from the press release[1] but tried to stay true to the release notes >> format and listed out the enhancements that way. > > Pushed with some very minor

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread Tomas Vondra
On Wed, Sep 09, 2020 at 10:26:00PM +0200, Tomas Vondra wrote: On Wed, Sep 09, 2020 at 04:53:30PM -0300, Alvaro Herrera wrote: On 2020-Sep-09, Tomas Vondra wrote: There are some minor optimizations possible - for example I noticed we call minmax_multi_get_strategy_procinfo often because it happ

Re: SIGQUIT handling, redux

2020-09-10 Thread Robert Haas
On Thu, Sep 10, 2020 at 12:56 PM Tom Lane wrote: > Also, man that CHECK_FOR_INTERRUPTS() looks like trouble. > Could we take that out? Maybe I'm missing something, but why wouldn't that be a horrible idea? We do not want to have long waits where we refuse to respond to interrupts. -- Robert Haa

Re: Fix for parallel BTree initialization bug

2020-09-10 Thread Jameson, Hunter 'James'
Answers inline below: On 9/10/20, 4:58 AM, "Amit Kapila" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. On Tue, Sep 8, 2020 at 11:55 PM Jameson, Hunter 'J

Re: Online checksums verification in the backend

2020-09-10 Thread Julien Rouhaud
On Thu, Sep 10, 2020 at 09:47:23AM +0200, Julien Rouhaud wrote: > On Wed, Sep 09, 2020 at 03:41:30PM +0200, Julien Rouhaud wrote: > > On Wed, Sep 09, 2020 at 11:25:29AM +0200, Julien Rouhaud wrote: > > > > > > I'll do some becnhmarking and see if I can get some figures, but it'll > > > probably >

Re: Fix for parallel BTree initialization bug

2020-09-10 Thread Jameson, Hunter 'James'
Answers inline below, sorry for the formatting-- am still trying to get corporate email to work nicely with this mailing list, thanks. On 9/9/20, 9:22 PM, "Justin Pryzby" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-10 Thread Justin Pryzby
On Thu, Sep 10, 2020 at 11:21:02AM -0400, Robert Haas wrote: > On Fri, Aug 28, 2020 at 5:55 AM Ashutosh Sharma wrote: > > Please have a look into the attached patch for the changes and let me know > > for any other concerns. Thank you. > > I have committed this version. Thanks ; I marked it as

Re: PG 13 release notes, first draft

2020-09-10 Thread Tom Lane
"Jonathan S. Katz" writes: > Attached is a proposal for the "major enhancements" section. I borrowed > from the press release[1] but tried to stay true to the release notes > format and listed out the enhancements that way. Pushed with some very minor wording tweaks. rega

Re: PostgreSQL 13 Release Timeline

2020-09-10 Thread Jonathan S. Katz
Hi, On 9/2/20 2:13 PM, Jonathan S. Katz wrote: > * PostgreSQL 13 Release Candidate 1 (RC1) will be released on September > 17, 2020. > > * In absence of any critical issues, PostgreSQL 13 will become generally > available on September 24, 2020. > > The aim is to have all outstanding open items

Re: SIGQUIT handling, redux

2020-09-10 Thread Tom Lane
Robert Haas writes: > On Wed, Sep 9, 2020 at 10:07 PM Tom Lane wrote: >> bgworker_die (SIGTERM) >> Calls ereport(FATAL). This is surely not any safer than, say, >> quickdie(). No, it's worse, because at least that won't try >> to go out via proc_exit(). > I think bgworker_die() is pretty much

Re: Collation versioning

2020-09-10 Thread Peter Eisentraut
On 2020-09-08 21:33, Christoph Berg wrote: IOW, I think we should aim at simply tracking the version, and leave it to the admin (with the help of supplied SQL queries) to either rebuild indexes or waive them. It's certainly safer to track dependency for all indexes and then carefully create ex

Re: New statistics for tuning WAL buffer size

2020-09-10 Thread Fujii Masao
On 2020/09/09 13:57, Masahiro Ikeda wrote: On 2020-09-07 16:19, Fujii Masao wrote: On 2020/09/07 9:58, Masahiro Ikeda wrote: Thanks for the review and advice! On 2020-09-03 16:05, Fujii Masao wrote: On 2020/09/02 18:56, Masahiro Ikeda wrote: +/* -- + * Backend types + * --

Re: Allow CURRENT_ROLE in GRANTED BY

2020-09-10 Thread Peter Eisentraut
On 2020-09-07 12:02, Asif Rehman wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed The patch applies c

Re: Optimising compactify_tuples()

2020-09-10 Thread David Rowley
On Fri, 11 Sep 2020 at 01:45, David Rowley wrote: > I've attached v4b (b is for backwards since the traditional backwards > tuple order is maintained). v4b seems to be able to run my benchmark > in 63 seconds. I did 10 runs today of yesterday's v3 patch and got an > average of 72.8 seconds, so qui

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-10 Thread Masahiko Sawada
On Tue, 8 Sep 2020 at 13:00, tsunakawa.ta...@fujitsu.com wrote: > > From: Amit Kapila > > I intend to say that the global-visibility work can impact this in a > > major way and we have analyzed that to some extent during a discussion > > on the other thread. So, I think without having a complete

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-09-10 Thread Robert Haas
On Fri, Aug 28, 2020 at 5:55 AM Ashutosh Sharma wrote: > Please have a look into the attached patch for the changes and let me know > for any other concerns. Thank you. I have committed this version. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: SIGQUIT handling, redux

2020-09-10 Thread Robert Haas
On Wed, Sep 9, 2020 at 10:07 PM Tom Lane wrote: > bgworker_die (SIGTERM) > > Calls ereport(FATAL). This is surely not any safer than, say, > quickdie(). No, it's worse, because at least that won't try > to go out via proc_exit(). I think bgworker_die() is pretty much a terrible idea. Every back

Re: BUG #15858: could not stat file - over 4GB

2020-09-10 Thread Greg Steiner
I assigned myself as a reviewer for this patch, as I hit this bug today and had to perform a workaround. I have never reviewed a patch before but will try to update within the next 5 days. I intend on performing "Implements Feature" reviewing.

Re: Global snapshots

2020-09-10 Thread Alexey Kondratov
On 2020-09-09 20:29, Fujii Masao wrote: On 2020/09/09 2:00, Alexey Kondratov wrote: According to the Sawada-san's v25 0002 the logic is pretty much the same there: +2. Pre-Commit phase (1st phase of two-phase commit) +3. Commit locally +Once we've prepared all of them, commit the transactio

Re: [PATCH] Missing links between system catalog documentation pages

2020-09-10 Thread Peter Eisentraut
On 2020-09-03 17:40, Dagfinn Ilmari Mannsåker wrote: I just noticed that both this and the command link patch modified the same sentence about CREATE DATABASE and pg_database, so those changes seem to have been lost in the merge. Attached is a follow-up patch that adds them both. I think in th

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-09-10 Thread Peter Eisentraut
On 2020-09-06 02:24, David Rowley wrote: I would add DEBUG1 back into the conditional, like if (__builtin_constant_p(elevel) && ((elevel) >= ERROR || (elevel) <= DEBUG1) ? \ hmm, but surely we don't want to move all code that's in the same branch as an elog(DEBUG1) call into a cold area. Yea

Re: Missing "Up" navigation link between parts and doc root?

2020-09-10 Thread Peter Eisentraut
On 2020-09-08 21:10, Bruce Momjian wrote: On Sun, Sep 6, 2020 at 04:59:11PM +0200, Peter Eisentraut wrote: I have made the analogous changes to the footer as well and committed this. I see this only applied to master. Shouldn't this be backpatched? I wasn't planning to. It's not a bug fix

Re: Range checks of pg_test_fsync --secs-per-test and pg_test_timing --duration

2020-09-10 Thread Peter Eisentraut
On 2020-09-10 09:59, Michael Paquier wrote: I notice that the error checking you introduce is different from the checks for pgbench -t and -T (the latter having no errno checks). I'm not sure which is correct, but it's perhaps worth making them the same. pgbench currently uses atoi() to parse t

Re: [Patch] Add missing libraries to Libs.private of libpq.pc

2020-09-10 Thread Peter Eisentraut
On 2020-09-04 22:07, Peter Eisentraut wrote: On 2020-07-10 21:47, Peter Eisentraut wrote: On 2020-04-08 11:38, Sandro Mani wrote: The following patch, which we added to build mingw-postgresql on Fedora, adds some missing libraries to Libs.private of libpq.pc, discovered when attempting to stati

Re: Boundary value check in lazy_tid_reaped()

2020-09-10 Thread Masahiko Sawada
On Tue, 1 Sep 2020 at 08:01, Peter Geoghegan wrote: > > On Mon, Aug 31, 2020 at 1:56 PM Peter Geoghegan wrote: > > I wonder if Roaring bitmaps would work well for this: > > > > https://arxiv.org/abs/1709.07821 > > Alternatively, perhaps we could use a negative cache of heap blocks > that have no

Re: Optimising compactify_tuples()

2020-09-10 Thread David Rowley
On Thu, 10 Sep 2020 at 10:40, Thomas Munro wrote: > > I wonder if we could also identify a range at the high end that is > already correctly sorted and maximally compacted so it doesn't even > need to be copied out. I've experimented quite a bit with this patch today. I think I've tested every id

Re: Inconsistency in determining the timestamp of the db statfile.

2020-09-10 Thread Alvaro Herrera
On 2020-Sep-10, Amit Kapila wrote: > On Thu, Sep 10, 2020 at 1:03 PM Magnus Hagander wrote: > The comments already say what you said in the second suggestion:"The > caller must rely on timestamp stored in *ts iff the function returns > true.". Read iff "as if and only if" I think "must" should

Re: Logical Replication - detail message with names of missing columns

2020-09-10 Thread Bharath Rupireddy
Added this to the commitfest - https://commitfest.postgresql.org/30/2727/ With Regards, Bharath Rupireddy. EnterpriseDB: http://www.enterprisedb.com

Re: Yet another fast GiST build

2020-09-10 Thread Heikki Linnakangas
On 09/09/2020 19:50, Andrey M. Borodin wrote: 9 сент. 2020 г., в 20:39, Heikki Linnakangas написал(а): On 09/09/2020 15:20, Darafei "Komяpa" Praliaskouski wrote: On Wed, Sep 9, 2020 at 3:09 PM Heikki Linnakangas wrote: Come to think of it, the point z-order comparator could benefit a lot fro

Re: Avoid useless retrieval of defaults and check constraints in pg_dump -a

2020-09-10 Thread Daniel Gustafsson
> On 12 Jul 2020, at 07:48, Julien Rouhaud wrote: > Currently, getTableAttrs() always retrieves info about columns defaults and > check constraints, while this will never be used if --data-only option if > used. > This seems like a waste of resources, so here's a patch to skip those parts > when

Re: Get memory contexts of an arbitrary backend process

2020-09-10 Thread Kasahara Tatsuhito
Hi, On Thu, Sep 10, 2020 at 8:53 PM torikoshia wrote: > > On 2020-09-04 21:46, Tomas Vondra wrote: > > On Fri, Sep 04, 2020 at 11:47:30AM +0900, Kasahara Tatsuhito wrote: > >> On Fri, Sep 4, 2020 at 2:40 AM Tom Lane wrote: > >>> Kasahara Tatsuhito writes: > >>> > Yes, but it's not only for futu

Re: Proposal of new PostgreSQL Extension - PGSpiderExt

2020-09-10 Thread Daniel Gustafsson
> On 10 Sep 2020, at 07:54, Taiga KATAYAMA wrote: > We hope to be incorporated this extension into PostgreSQL as one of > contrib module, and would like to try to propose to Commit Fest. > Could you kindly advise me and share your opinion? FWIW I would like to see fewer modules in contrib rather

Re: Windows regress fails (latest HEAD)

2020-09-10 Thread Ranier Vilela
Em sex., 26 de jun. de 2020 às 08:21, Ranier Vilela escreveu: > Em qui., 11 de jun. de 2020 às 10:28, Ranier Vilela > escreveu: > >> Em qui., 11 de jun. de 2020 às 10:01, Andrew Dunstan < >> andrew.duns...@2ndquadrant.com> escreveu: >> >>> >>> On 6/11/20 8:52 AM, Ranier Vilela wrote: >>> > Hi, >

Re: Fix for parallel BTree initialization bug

2020-09-10 Thread Amit Kapila
On Tue, Sep 8, 2020 at 11:55 PM Jameson, Hunter 'James' wrote: > > Hi, I ran across a small (but annoying) bug in initializing parallel BTree > scans, which causes the parallel-scan state machine to get confused. The fix > is one line; the description is a bit longer— > > > > Before, function _b

Re: Get memory contexts of an arbitrary backend process

2020-09-10 Thread torikoshia
On 2020-09-04 21:46, Tomas Vondra wrote: On Fri, Sep 04, 2020 at 11:47:30AM +0900, Kasahara Tatsuhito wrote: On Fri, Sep 4, 2020 at 2:40 AM Tom Lane wrote: Kasahara Tatsuhito writes: > Yes, but it's not only for future expansion, but also for the > usability and the stability of this feature.

Re: Two fsync related performance issues?

2020-09-10 Thread Thomas Munro
On Thu, Sep 3, 2020 at 12:09 PM Thomas Munro wrote: > On Tue, May 12, 2020 at 12:43 PM Paul Guo wrote: > > RecreateTwoPhaseFile(gxact->xid, buf, len); > I hadn't previously focused on this second part of your email. I > think the fsync() call in RecreateTwoPhaseFile() might be a candidate >

Re: Improvements in Copy From

2020-09-10 Thread vignesh C
On Wed, Sep 9, 2020 at 12:24 PM Peter Smith wrote: > > My basic understanding of first part of your patch is that by > adjusting the "minread" it now allows it to loop multiple times > internally within the CopyGetData rather than calling CopyLoadRawBuf > for every N lines. There doesn't seem to b

Re: Yet another fast GiST build

2020-09-10 Thread Pavel Borisov
> > Interesting to see also the size of index, it should be several times less. > > I've tested this above in CF thread and got ordered GiST index ~1.7 times smaller than non-ordered one for single column of real points. -- Best regards, Pavel Borisov Postgres Professional: http://postgrespro.c

Re: Yet another fast GiST build

2020-09-10 Thread Oleg Bartunov
On Mon, Sep 7, 2020 at 7:50 PM Heikki Linnakangas wrote: > > On 07/09/2020 13:59, Pavel Borisov wrote: > I suppose there is a big jump in integer value (whether signed or > unsigned) as you cross from positive to negative floats, and then the > sort order is reversed. I have no ide

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-10 Thread Fujii Masao
On 2020/09/10 10:13, tsunakawa.ta...@fujitsu.com wrote: Alexey-san, Sawada-san, cc: Fujii-san, From: Fujii Masao But if we implement 2PC as the improvement on FDW independently from PostgreSQL sharding, I think that it's necessary to support other FDW. And this is our direction, isn't it?

Re: Global snapshots

2020-09-10 Thread Fujii Masao
On 2020/09/10 18:01, tsunakawa.ta...@fujitsu.com wrote: From: Fujii Masao But I'm concerned about that it's really hard to say there is no patent risk around that. I'm not sure who can judge there is no patent risk, in the community. Maybe no one? Anyway, I was thinking that Google Spanner,

Re: Bug in logical decoding of in-progress transactions

2020-09-10 Thread Dilip Kumar
On Thu, Sep 10, 2020 at 2:48 PM Amit Kapila wrote: > On Thu, Sep 10, 2020 at 12:00 PM Dilip Kumar > wrote: > > > > On Thu, Sep 10, 2020 at 11:53 AM Dilip Kumar > wrote: > >> > >>> > > >>> > I have written a test case to reproduce the same. > > > > Can we write an isolation test for this scenari

Re: Improvements in Copy From

2020-09-10 Thread vignesh C
On Mon, Sep 7, 2020 at 1:19 PM Surafel Temesgen wrote: > > > Hi Vignesh > > On Wed, Jul 1, 2020 at 3:46 PM vignesh C wrote: >> >> Hi, >> >> While reviewing copy from I identified few improvements for copy from >> that can be done : >> a) copy from stdin copies lesser amount of data to buffer eve

Re: SIGQUIT handling, redux

2020-09-10 Thread Kyotaro Horiguchi
At Wed, 09 Sep 2020 10:46:28 -0400, Tom Lane wrote in > > I also think we should > > back-patch these changes, as the commit mentioned in Horiguchi-san's > > patch for pgarch_exit() was. > > Agreed; I'll go make it happen. Thank you for committing this! regards. -- Kyotaro Horiguchi NTT Open

Re: v13: CLUSTER segv with wal_level=minimal and parallel index creation

2020-09-10 Thread Kyotaro Horiguchi
At Wed, 09 Sep 2020 14:40:37 +0900 (JST), Kyotaro Horiguchi wrote in > Sure. relNode is filled with zeros so RelationInitPhysicalAddr() works > correctly for RelationBuildDesc(). > > Both changes look good to me. Thank you for committing this! regards. -- Kyotaro Horiguchi NTT Open Source So

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2020-09-10 Thread Andrey V. Lepikhov
On 9/9/20 5:51 PM, Amit Langote wrote: On Wed, Sep 9, 2020 at 6:42 PM Alexey Kondratov wrote: On 2020-09-09 11:45, Andrey V. Lepikhov wrote: This does not seem very convenient and will lead to errors in the future. So, I agree with Amit. And InitResultRelInfo() may set ri_usesMultiInsert to

Re: Evaluate expression at planning time for two more cases

2020-09-10 Thread Surafel Temesgen
On Tue, Sep 8, 2020 at 12:59 PM Surafel Temesgen wrote: > Hi Tom > > On Tue, Sep 8, 2020 at 4:46 AM Tom Lane wrote: > > >> The "check_null_side" code you're proposing seems really horrid. >> For one thing, it seems quite out of place for eval_const_expressions >> to be doing that. For another,

Re: PATCH: Attempt to make dbsize a bit more consistent

2020-09-10 Thread Daniel Gustafsson
>> So what do you think of the patch? > > I would suggest to keep the original logic unless there is a bug. IIUC, the premise of this path submission is that this codepath is in fact buggy as it may lead to incorrect results for non-heap relations? Since we have introduced the table AM api I sup

Re: Strange behavior with polygon and NaN

2020-09-10 Thread Kyotaro Horiguchi
Hello, Georgios. At Mon, 07 Sep 2020 12:46:50 +, gkokola...@pm.me wrote in > ‐‐‐ Original Message ‐‐‐ > On Thursday, 27 August 2020 14:24, Kyotaro Horiguchi > wrote: > > > At Wed, 26 Aug 2020 08:18:49 -0400, Tom Lane t...@sss.pgh.pa.us wrote in > > > > > Kyotaro Horiguchi horikyota

Re: autovac issue with large number of tables

2020-09-10 Thread Kasahara Tatsuhito
Therefore, we expect this patch [1] to be committed for its original purpose, as well as to improve autovacuum from v14 onwards.Hi, On Fri, Sep 4, 2020 at 7:50 PM Kasahara Tatsuhito wrote: > > Hi, > > On Wed, Sep 2, 2020 at 2:10 AM Kasahara Tatsuhito > wrote: > > > I wonder if we could have tabl

Re: Bug in logical decoding of in-progress transactions

2020-09-10 Thread Amit Kapila
On Thu, Sep 10, 2020 at 12:00 PM Dilip Kumar wrote: > > On Thu, Sep 10, 2020 at 11:53 AM Dilip Kumar wrote: >> >>> > >>> > I have written a test case to reproduce the same. > Can we write an isolation test for this scenario? See some similar tests in contrib/test_decoding/specs. If that is possi

RE: Implement UNLOGGED clause for COPY FROM

2020-09-10 Thread tsunakawa.ta...@fujitsu.com
From: Peter Smith > Earlier, Osumi-san was rejecting the idea of using ALTER TABLE tbl SET > UNLOGGED on basis that it is too time consuming for large data to > switch the table modes [1]. > Doesn't wal_level=none essentially just behave as if every table was > UNLOGGED; not just the ones we are

Re: Minor cleanup of partbounds.c

2020-09-10 Thread Etsuro Fujita
On Thu, Sep 10, 2020 at 2:05 AM Alvaro Herrera wrote: > On 2020-Sep-09, Etsuro Fujita wrote: > > Here is a patch for minor cleanup of the partbounds.c changes made by > > commit c8434d64c: 1) removes a useless assignment (in normal builds) > > LGTM. > > > and 2) improves comments a little. > > No

RE: Global snapshots

2020-09-10 Thread tsunakawa.ta...@fujitsu.com
From: Fujii Masao > But I'm concerned about that it's really hard to say there is no patent risk > around that. I'm not sure who can judge there is no patent risk, > in the community. Maybe no one? Anyway, I was thinking that Google Spanner, > YugabyteDB, etc use the global transaction approach ba

Re: Global snapshots

2020-09-10 Thread Fujii Masao
On 2020/09/10 10:38, tsunakawa.ta...@fujitsu.com wrote: Hi Andrey san, From: Andrey V. Lepikhov > > From: tsunakawa.ta...@fujitsu.com While Clock-SI seems to be considered the best promising for global Could you take a look at this patent? I'm afraid this is the Clock-SI for MVCC. Micro

Re: Range checks of pg_test_fsync --secs-per-test and pg_test_timing --duration

2020-09-10 Thread Michael Paquier
On Mon, Sep 07, 2020 at 10:06:57AM +0200, Peter Eisentraut wrote: > However, I see that in the case of pg_test_fsync you end up in alarm(0), > which does something different, so it's okay in that case to disallow it. Yep. > I notice that the error checking you introduce is different from the chec

Re: Online checksums verification in the backend

2020-09-10 Thread Julien Rouhaud
On Wed, Sep 09, 2020 at 03:41:30PM +0200, Julien Rouhaud wrote: > On Wed, Sep 09, 2020 at 11:25:29AM +0200, Julien Rouhaud wrote: > > > > I'll do some becnhmarking and see if I can get some figures, but it'll > > probably > > take some time. In the meantime I'm attaching v11 of the patch that sh

Re: Inconsistency in determining the timestamp of the db statfile.

2020-09-10 Thread Amit Kapila
On Thu, Sep 10, 2020 at 1:03 PM Magnus Hagander wrote: > > On Thu, Sep 10, 2020 at 9:05 AM Amit Kapila wrote: >> >> On Thu, Sep 10, 2020 at 11:52 AM Masahiko Sawada >> wrote: >> > >> > >> > Regarding the v2 patch, I think we should return false in the >> > following case too: >> > >> >

Re: Proposals for making it easier to write correct bgworkers

2020-09-10 Thread Magnus Hagander
On Thu, Sep 10, 2020 at 5:02 AM Craig Ringer wrote: > Hi all > > As I've gained experience working on background workers, it's become > increasingly clear that they're a bit too different to normal backends for > many nontrivial uses. > a lot of proposals I agree with. PROPOSED GENERALISED W

Re: Inconsistency in determining the timestamp of the db statfile.

2020-09-10 Thread Magnus Hagander
On Thu, Sep 10, 2020 at 9:05 AM Amit Kapila wrote: > On Thu, Sep 10, 2020 at 11:52 AM Masahiko Sawada > wrote: > > > > > > Regarding the v2 patch, I think we should return false in the > > following case too: > > > > default: > > ereport(pgStatRunningInCollector ? LOG

Re: v13: show extended stats target in \d

2020-09-10 Thread Georgios
‐‐‐ Original Message ‐‐‐ On Thursday, 10 September 2020 00:36, Justin Pryzby wrote: > On Tue, Sep 01, 2020 at 12:35:19PM +, Georgios Kokolatos wrote: > > > I will humbly disagree with the current review. I shall refrain from > > changing the status though because I do not have ve

Re: Inconsistent Japanese name order in v13 contributors list

2020-09-10 Thread Etsuro Fujita
On Wed, Sep 9, 2020 at 9:16 PM Peter Eisentraut wrote: > On 2020-09-09 07:15, Etsuro Fujita wrote: > > Attached is a patch to standardize Japanese names as given-name-first > > in the v13 contributors list as before. > > Given existing practice, this patch looks okay. I've applied the patch. Tha

Re: [PATCH] audo-detect and use -moutline-atomics compilation flag for aarch64

2020-09-10 Thread Zidenberg, Tsahi
On 08/09/2020, 1:01, "Tom Lane" wrote: > I wonder what version of gcc you intend this for. AFAICS, older > gcc versions lack this flag at all, while newer ones have it on > by default. (previously sent private reply, sorry) The moutline-atomics flag showed substantial enough impro

Re: Inconsistency in determining the timestamp of the db statfile.

2020-09-10 Thread Amit Kapila
On Thu, Sep 10, 2020 at 11:52 AM Masahiko Sawada wrote: > > > Regarding the v2 patch, I think we should return false in the > following case too: > > default: > ereport(pgStatRunningInCollector ? LOG : WARNING, > (errmsg("corrupted statistics fil