Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 9:11 AM Greg Nancarrow wrote: > > > > What if this > > > ends up being invoked from inside C code? > > > > > > > I think it shouldn't be a problem unless one is trying to do something > > like insert into foreign key table. So, probably we can have an Assert > > to catch it

Re: Logical replication from PG v13 and below to PG v14 (devel version) is not working.

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 1:53 PM Dilip Kumar wrote: > > On Fri, Sep 25, 2020 at 8:12 AM Amit Kapila wrote: > > > > > > No that won't happen because we send this option to the server > > (publisher in this case) only when version is >=14 and user has > > specified this option. See the below check i

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 1:49 PM k.jami...@fujitsu.com wrote: > > Hi. > > > I'll send performance measurement results in the next email. Thanks a lot > > for > > the reviews! > > Below are the performance measurement results. > I was only able to use low-spec machine: > CPU 4v, Memory 8GB, RHEL, x

Re: [Patch] Optimize dropping of relation buffers using dlist

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 2:25 PM tsunakawa.ta...@fujitsu.com wrote: > > From: Amit Kapila > > No, during recovery also we need to be careful. We need to ensure that > > we use cached value during recovery and cached value is always > > up-to-date. We can't rely on lseek and I have provided some sc

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-25 Thread Julien Rouhaud
On Sat, Sep 26, 2020 at 8:17 AM Andy Fan wrote: > > As for the testing with cache considered, I found how to estimate cache hit > ratio is hard or how to control a hit ratio to test is hard. Recently I am > thinking > a method that we can get a page_reads, shared_buffer_hit from pg_kernel > and t

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Amit Kapila
On Sat, Sep 26, 2020 at 11:00 AM Bharath Rupireddy wrote: > > On Fri, Sep 25, 2020 at 9:23 PM Greg Nancarrow wrote: > > > > On Fri, Sep 25, 2020 at 10:17 PM Amit Kapila > > wrote: > > > > > > > Again, there's a fundamental difference in the Parallel Insert case. > > Right at the top of ExecuteP

Re: VACUUM PARALLEL option vs. max_parallel_maintenance_workers

2020-09-25 Thread Amit Kapila
On Tue, Sep 22, 2020 at 12:50 PM David Rowley wrote: > > On Mon, 21 Sep 2020 at 19:15, Peter Eisentraut > wrote: > > > > On 2020-09-21 05:48, Amit Kapila wrote: > > > What according to you should be the behavior here and how will it be > > > better than current? > > > > I think if I write VACUUM

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Bharath Rupireddy
On Fri, Sep 25, 2020 at 9:23 PM Greg Nancarrow wrote: > > On Fri, Sep 25, 2020 at 10:17 PM Amit Kapila wrote: > > > > Again, there's a fundamental difference in the Parallel Insert case. > Right at the top of ExecutePlan it calls EnterParallelMode(). > For ParallelCopy(), there is no such problem

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 2:31 PM Bharath Rupireddy wrote: > > On Tue, Sep 22, 2020 at 10:26 AM Greg Nancarrow wrote: > > > > For cases where it can't be allowed (e.g. INSERT into a table with > > foreign keys, or INSERT INTO ... SELECT ... ON CONFLICT ... DO UPDATE > > ...") it at least allows par

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 9:23 PM Greg Nancarrow wrote: > > On Fri, Sep 25, 2020 at 10:17 PM Amit Kapila > As it turns out, I think I have solved the commandId issue (and almost > the xid issue) by realising that both the xid and cid are ALREADY > being included as part of the serialized transactio

Re: a potential size overflow issue

2020-09-25 Thread Tom Lane
David Zhang writes: > "InitBufTable" is the function used to initialize the buffer lookup > table for buffer manager. With the memory size increasing nowadays, > there is a potential overflow issue for the parameter "int size" used by > "InitBufTable". This function is invoked in freelist.c as

Re: Optimize memory allocation code

2020-09-25 Thread Merlin Moncure
On Fri, Sep 25, 2020 at 7:32 PM Li Japin wrote: > > > > > On Sep 26, 2020, at 8:09 AM, Julien Rouhaud wrote: > > > > Hi, > > > > On Sat, Sep 26, 2020 at 12:14 AM Li Japin wrote: > >> > >> Hi, hackers! > >> > >> I find the palloc0() is similar to the palloc(), we can use palloc() > >> inside pal

Re: Optimize memory allocation code

2020-09-25 Thread Li Japin
> On Sep 26, 2020, at 8:09 AM, Julien Rouhaud wrote: > > Hi, > > On Sat, Sep 26, 2020 at 12:14 AM Li Japin wrote: >> >> Hi, hackers! >> >> I find the palloc0() is similar to the palloc(), we can use palloc() inside >> palloc0() >> to allocate space, thereby I think we can reduce duplicati

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-25 Thread Andy Fan
On Fri, Sep 25, 2020 at 5:15 PM Ashutosh Bapat wrote: > On Tue, Sep 22, 2020 at 10:57 AM Andy Fan > wrote: > > > > > > My tools set the random_page_cost to 8.6, but based on the fio data, it > should be > > set to 12.3 on the same hardware. and I do see the better plan as well > with 12.3. > >

Re: Optimize memory allocation code

2020-09-25 Thread Julien Rouhaud
Hi, On Sat, Sep 26, 2020 at 12:14 AM Li Japin wrote: > > Hi, hackers! > > I find the palloc0() is similar to the palloc(), we can use palloc() inside > palloc0() > to allocate space, thereby I think we can reduce duplication of code. The code is duplicated on purpose. There's a comment at the

a potential size overflow issue

2020-09-25 Thread David Zhang
Hi hackers, "InitBufTable" is the function used to initialize the buffer lookup table for buffer manager. With the memory size increasing nowadays, there is a potential overflow issue for the parameter "int size" used by "InitBufTable". This function is invoked in freelist.c as below:     Ini

What does pg_stat_get_xact_function_self_time count exactly?

2020-09-25 Thread Chapman Flack
Hi, Is the self-time of a tracked function (as reported by pg_stat_get_xact_function_self_time) its total time minus - the time of other tracked functions it calls? - the time of other tracked or untracked functions it calls? - the time of other tracked/untracked functions or SPI functions it c

Re: Libpq support to connect to standby server as priority

2020-09-25 Thread Tom Lane
Greg Nancarrow writes: > [ v19-0001-Enhance-libpq-target_session_attrs-and-add-target_se.patch ] I started to look through this, and I find that I'm really pretty disappointed in the direction the patch has gone of late. I think there is no defensible reason for the choices that have been made t

Re: pg_upgrade: fail early if a tablespace dir already exists for new cluster version

2020-09-25 Thread Justin Pryzby
Added at https://commitfest.postgresql.org/30/2739/ >From 831246aa6d6837b2b0da7c96ad2f44ffd6cd3951 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 24 Sep 2020 19:49:40 -0500 Subject: [PATCH v2] pg_upgrade --check to avoid tablespace failure mode --- src/bin/pg_upgrade/check.c | 37 ++

Re: history file on replica and double switchover

2020-09-25 Thread David Zhang
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed "make installcheck-world" test was performed on branch "REL_1

Re: extension patch of CREATE OR REPLACE TRIGGER

2020-09-25 Thread Tom Lane
"osumi.takami...@fujitsu.com" writes: > [ CREATE_OR_REPLACE_TRIGGER_v11.patch ] I took a quick look through this. I think there's still work left to do. * I'm concerned by the fact that there doesn't seem to be any defense against somebody replacing a foreign-key trigger with something that doe

Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2

2020-09-25 Thread John Scalia
FIPS only specifies which algorithms are approved for use on it. For instance, MD-5 is NOT approved at all under FIPS. I would say any algorithm should produce the same result regardless of where it is run. BTW, on Redhat servers, the first algorithm listed for use with SSH is MD-5. This causes

Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2

2020-09-25 Thread Bruce Momjian
On Fri, Sep 25, 2020 at 03:38:22PM -0400, John Scalia wrote: > Bruce, > > In my experience, any client is permitted to connect to FIPS140-2 compliant > server. I set this up when I worked at SSA, at management’s request. My question is whether the hash output would match if using different code

Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2

2020-09-25 Thread John Scalia
Bruce, In my experience, any client is permitted to connect to FIPS140-2 compliant server. I set this up when I worked at SSA, at management’s request. — Jay Sent from my iPad > On Sep 25, 2020, at 3:13 PM, Bruce Momjian wrote: > > On Fri, Sep 25, 2020 at 03:56:53PM +0900, Michael Paquier w

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Andres Freund
Hi, On 2020-09-25 14:11:46 -0400, Tom Lane wrote: > Christoph Berg writes: > > Ok, but given that Debian is currently targeting 22 architectures, I doubt > > the PostgreSQL buildfarm covers all of them with the extra JIT option, so I > > should probably make sure to do that here when running te

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Andres Freund
Hi, On 2020-09-25 19:05:52 +0200, Christoph Berg wrote: > Am 25. September 2020 18:42:04 MESZ schrieb Andres Freund > >> * jit is not exercised enough by "make installcheck" > > > >So far we've exercised more widely it by setting up machines that use > >it > >for all queries (by setting the confi

Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2

2020-09-25 Thread Bruce Momjian
On Fri, Sep 25, 2020 at 03:56:53PM +0900, Michael Paquier wrote: > On Fri, Sep 25, 2020 at 01:36:44AM -0400, Tom Lane wrote: > > Peter Eisentraut writes: > >> However, again, the SCRAM > >> implementation would already appear to fail that requirement because it > >> uses a custom HMAC implementa

Re: history file on replica and double switchover

2020-09-25 Thread Fujii Masao
On 2020/09/26 2:58, Grigory Smolkin wrote: Fujii Masao, David Zhang, Anastasia Lubennikova, many thanks to you for efforts with this patch! Can I mark it as ready for committer? Ok, but I attached the updated version of the patch. It's helpful if you review that. In the latest patch, I cha

Re: Asynchronous Append on postgres_fdw nodes.

2020-09-25 Thread Andrey Lepikhov
Your AsyncAppend doesn't switch to another source if the data in current leader is available: /* * The request for the next node cannot be sent before the leader * responds. Finish the current leader if possible. */ if (PQisBusy(leader_state->s.conn)) { int rc = WaitLatchOrSocket(NULL, WL_S

Re: Problem of ko.po on branch REL9_5_STABLE

2020-09-25 Thread Alvaro Herrera
Hi, On 2020-Sep-25, Yang, Rong wrote: > When I checked the encoding of the Po files, I noticed that > src/bin/pg_config/po/ko.po on REL9_5_STABLE branch seemed a little different. > The ‘Content-Type’ of this file and file’s encoding are different from > those under other modules, a

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Tom Lane
Christoph Berg writes: > Ok, but given that Debian is currently targeting 22 architectures, I doubt > the PostgreSQL buildfarm covers all of them with the extra JIT option, so I > should probably make sure to do that here when running tests. +1. I rather doubt our farm is running this type of

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Ranier Vilela
Em sex., 25 de set. de 2020 às 14:36, Ranier Vilela escreveu: > Em sex., 25 de set. de 2020 às 11:30, Christoph Berg > escreveu: > >> Re: Tom Lane >> > > Tom> It's hardly surprising that datumCopy would segfault when given >> a >> > > Tom> null "value" and told it is pass-by-reference. However

Re: history file on replica and double switchover

2020-09-25 Thread Grigory Smolkin
Fujii Masao, David Zhang, Anastasia Lubennikova, many thanks to you for efforts with this patch! Can I mark it as ready for committer? On 9/25/20 10:07 AM, Fujii Masao wrote: On 2020/09/25 13:00, David Zhang wrote: On 2020-09-24 5:00 p.m., Fujii Masao wrote: On 2020/09/25 8:15, David Zhan

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Ranier Vilela
Em sex., 25 de set. de 2020 às 11:30, Christoph Berg escreveu: > Re: Tom Lane > > > Tom> It's hardly surprising that datumCopy would segfault when given a > > > Tom> null "value" and told it is pass-by-reference. However, to get to > > > Tom> the datumCopy call, we must have passed the MemoryC

Re: New statistics for tuning WAL buffer size

2020-09-25 Thread Fujii Masao
On 2020/09/25 12:06, Masahiro Ikeda wrote: On 2020-09-18 11:11, Kyotaro Horiguchi wrote: At Fri, 18 Sep 2020 09:40:11 +0900, Masahiro Ikeda wrote in Thanks. I confirmed that it causes HOT pruning or killing of dead index tuple if DecodeCommit() is called. As you said, DecodeCommit() may ac

Re: "cert" + clientcert=verify-ca in pg_hba.conf?

2020-09-25 Thread Bruce Momjian
On Thu, Sep 24, 2020 at 09:59:50PM -0400, Tom Lane wrote: > Kyotaro Horiguchi writes: > > Thank you Bruce, Michael. This is a rebased version. > > I really strongly object to all the encoded data in this patch. > One cannot read it, one cannot even easily figure out how long > it is until the tes

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Christoph Berg
Am 25. September 2020 18:42:04 MESZ schrieb Andres Freund >> * jit is not exercised enough by "make installcheck" > >So far we've exercised more widely it by setting up machines that use >it >for all queries (by setting the config option). I'm doubtful it's worth >doing differently. Ok, but given

Re: [HACKERS] [PATCH] Generic type subscripting

2020-09-25 Thread Pavel Stehule
ne 20. 9. 2020 v 17:46 odesílatel Dmitry Dolgov <9erthali...@gmail.com> napsal: > > On Fri, Sep 18, 2020 at 07:23:11PM +0200, Pavel Stehule wrote: > > > > > In this way (returning an error on a negative indices bigger than the > > > number of elements) functionality for assigning via subscripting

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Andres Freund
Hi, On 2020-09-25 17:29:07 +0200, Christoph Berg wrote: > I guess that suggests two things: > * jit is not ready for prime time on s390x and I should disable it I don't know how good LLVMs support for s390x JITing is, and given that it's unrealistic for people to get access to s390x... > * jit

Re: Load TIME fields - proposed performance improvement

2020-09-25 Thread Tom Lane
Peter Smith writes: > The patch has been re-implemented based on previous advice. > Please see attached. Hm, so: 1. The caching behavior really needs to account for the possibility of the timezone setting being changed intra-transaction. That's not very likely, perhaps, but we can't deliver a w

Optimize memory allocation code

2020-09-25 Thread Li Japin
Hi, hackers! I find the palloc0() is similar to the palloc(), we can use palloc() inside palloc0() to allocate space, thereby I think we can reduce duplication of code. Best regards! -- Japin Li 0001-Optimize-memory-allocation-code.patch Description: 0001-Optimize-memory-allocation-code.pat

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Greg Nancarrow
On Fri, Sep 25, 2020 at 10:17 PM Amit Kapila wrote: > But we can tighten the condition in GetCurrentCommandId() such that it > Asserts for parallel worker only when currentCommandIdUsed is not set > before start of parallel operation. I also find these changes in the > callers of GetCurrentCommand

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Christoph Berg
Re: To Tom Lane > I poked around with the SET in the offending tests, and the crash is > only present if `set jit_above_cost = 0;` is present. Removing that > makes it pass. Removing work_mem or enable_hashagg does not make a > difference. llvm version is 10.0.1. I put jit_above_cost=0 into postgr

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Christoph Berg
I poked around with the SET in the offending tests, and the crash is only present if `set jit_above_cost = 0;` is present. Removing that makes it pass. Removing work_mem or enable_hashagg does not make a difference. llvm version is 10.0.1. Test file: -- -- Compare results between plans using sor

Re: gs_group_1 crashing on 13beta2/s390x

2020-09-25 Thread Christoph Berg
Re: Tom Lane > > Tom> It's hardly surprising that datumCopy would segfault when given a > > Tom> null "value" and told it is pass-by-reference. However, to get to > > Tom> the datumCopy call, we must have passed the MemoryContextContains > > Tom> check on that very same pointer value, and that

Re: Probable documentation errors or improvements

2020-09-25 Thread Justin Pryzby
Split one patch about text search, added another one (sequences), added some info to commit messages, and added here. https://commitfest.postgresql.org/30/2744/ -- Justin >From adf050ac6cc7d0905fc1613dce1a04f76a892609 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 22 Sep 2020 21:40:17 -

Re: Dumping/restoring fails on inherited generated column

2020-09-25 Thread Tom Lane
Peter Eisentraut writes: > The proposed patches will cause the last statement to be omitted, but > that still won't recreate the original state. The problem is that there > is no command to make a column generated afterwards, like the SET > DEFAULT command, so we can't dump it like this. Righ

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-25 Thread Masahiko Sawada
On Fri, 25 Sep 2020 at 18:21, tsunakawa.ta...@fujitsu.com wrote: > > From: Masahiko Sawada > > I don't think it's always possible to avoid raising errors in advance. > > Considering how postgres_fdw can implement your idea, I think > > postgres_fdw would need PG_TRY() and PG_CATCH() for its conne

Re: Dumping/restoring fails on inherited generated column

2020-09-25 Thread Daniel Gustafsson
> On 25 Sep 2020, at 15:07, Peter Eisentraut > wrote: > We could probably fix this by having ALTER TABLE ONLY / DROP EXPRESSION > update the attlocal column of direct children to true, to make the catalog > state look like something that can be restored. However, that's a fair > amount of co

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Bharath Rupireddy
On Fri, Sep 25, 2020 at 5:47 PM Amit Kapila wrote: > > > > > At least in the case of Parallel INSERT, the leader for the Parallel > > INSERT gets a new xid (GetCurrentFullTransactionId) and it is passed > > through and assigned to each of the workers during their > > initialization (so they are as

Re: Dumping/restoring fails on inherited generated column

2020-09-25 Thread Peter Eisentraut
I have been analyzing this issue again. We have a few candidate patches that do very similar things for avoiding dumping the generation expression of table gtest1_1. We can figure out later which one of these we like best. But there is another issue lurking nearby. The table hierarchy of gt

Re: Feature improvement for FETCH tab completion

2020-09-25 Thread Fujii Masao
On 2020/09/25 17:21, btnakamichin wrote: 2020-09-25 15:38 に Fujii Masao さんは書きました: On 2020/09/25 14:24, btnakamichin wrote: Hello! I’d like to improve the FETCH tab completion. The FETCH tab completion . Therefore, this patch fixes the problem. Previous function completed one of FORWARD, BA

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-25 Thread Bharath Rupireddy
On Fri, Sep 25, 2020 at 3:21 PM Fujii Masao wrote: > > I think that we can simplify the code by merging the connection-retry > code into them, like the attached very WIP patch (based on yours) does. > +1. > > + else > + ereport(ERROR, > + (errc

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Amit Kapila
On Fri, Sep 25, 2020 at 10:02 AM Greg Nancarrow wrote: > > Hi Andres, > > On Thu, Sep 24, 2020 at 12:21 PM Andres Freund wrote: > > > > > >> @@ -116,7 +117,7 @@ toast_save_datum(Relation rel, Datum value, > >> TupleDesc toasttupDesc; > >> Datum t_values[3]; > >>

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-09-25 Thread legrand legrand
oups, sorry so +1 for this fix Regards PAscal -- Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Greg Nancarrow
On Fri, Sep 25, 2020 at 7:01 PM Bharath Rupireddy wrote: > I have few points (inspired from parallel copy feature work) to mention: > > 1. What if the target table is a foreign table or partitioned table? > 2. What happens if the target table has triggers(before statement, > after statement, befor

Re: BUG #16419: wrong parsing BC year in to_date() function

2020-09-25 Thread Asif Rehman
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested Patch looks good to me. The new status of this patch is: Ready f

Re: Resetting spilled txn statistics in pg_stat_replication

2020-09-25 Thread Amit Kapila
On Thu, Sep 24, 2020 at 5:44 PM Amit Kapila wrote: > > On Sat, Sep 19, 2020 at 1:48 PM Amit Kapila wrote: > > > > On Tue, Sep 8, 2020 at 7:02 PM Amit Kapila wrote: > > > > > > On Tue, Sep 8, 2020 at 7:53 AM Masahiko Sawada > > > wrote: > > > > I have fixed these review comments in the attached

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-09-25 Thread Fujii Masao
On 2020/09/25 19:04, legrand legrand wrote: Hi, isn't this already fixed in pg14 https://www.postgresql.org/message-id/e1k0mzg-0002vn...@gemulon.postgresql.org ? IIUC that commit handled CREATE TABLE AS, SELECT INTO, CREATE MATERIALIZED VIEW and FETCH commands, but not REFRESH MATERIALIZED

Re: Online checksums verification in the backend

2020-09-25 Thread Julien Rouhaud
On Wed, Sep 16, 2020 at 11:46 AM Michael Paquier wrote: > > On Fri, Sep 11, 2020 at 09:49:16AM +0200, Julien Rouhaud wrote: > > Thanks! > > I got some numbers out of my pocket, using the following base > configuration: > [...] > > Within parenthesis is the amount of time taken by pg_relation_check

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-09-25 Thread legrand legrand
Hi, isn't this already fixed in pg14 https://www.postgresql.org/message-id/e1k0mzg-0002vn...@gemulon.postgresql.org ? Regards PAscal -- Sent from: https://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

Re: Retry Cached Remote Connections for postgres_fdw in case remote backend gets killed/goes away

2020-09-25 Thread Fujii Masao
On 2020/09/25 13:56, Bharath Rupireddy wrote: On Wed, Sep 23, 2020 at 8:19 PM Fujii Masao wrote: Please let me know if okay with the above agreed points, I will work on the new patch. Yes, please work on the patch! Thanks! I may revisit the above points later, though ;) Thanks, attac

Re: [patch] Concurrent table reindex per-index progress reporting

2020-09-25 Thread Matthias van de Meent
On Fri, 25 Sep 2020 at 08:44, Michael Paquier wrote: > > On Thu, Sep 24, 2020 at 09:19:18PM +0200, Matthias van de Meent wrote: > > While working on a PG12-instance I noticed that the progress reporting of > > concurrent index creation for non-index relations fails to update the > > index/relation

Re: FETCH FIRST clause PERCENT option

2020-09-25 Thread Surafel Temesgen
Hi Michael On Thu, Sep 24, 2020 at 6:58 AM Michael Paquier wrote: > On Mon, Aug 10, 2020 at 01:23:44PM +0300, Surafel Temesgen wrote: > > I also Implement PERCENT WITH TIES option. patch is attached > > i don't start a new tread because the patches share common code > > This fails to apply per th

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-25 Thread k.jami...@fujitsu.com
On Friday, September 25, 2020 6:02 PM, Tsunakawa-san wrote: > From: Jamison, Kirk/ジャミソン カーク > > [Results] > > Recovery/Failover performance (in seconds). 3 trial runs. > > > > | shared_buffers | master | patch | %reg| > > ||||-| > > | 128MB |

RE: Transactions involving multiple postgres foreign servers, take 2

2020-09-25 Thread tsunakawa.ta...@fujitsu.com
From: Masahiko Sawada > I don't think it's always possible to avoid raising errors in advance. > Considering how postgres_fdw can implement your idea, I think > postgres_fdw would need PG_TRY() and PG_CATCH() for its connection > management. It has a connection cache in the local memory using HTAB

Re: Dynamic gathering the values for seq_page_cost/xxx_cost

2020-09-25 Thread Ashutosh Bapat
On Tue, Sep 22, 2020 at 10:57 AM Andy Fan wrote: > > > My tools set the random_page_cost to 8.6, but based on the fio data, it > should be > set to 12.3 on the same hardware. and I do see the better plan as well with > 12.3. > Looks too smooth to believe it is true.. > > The attached result_fio

Re: Parallel INSERT (INTO ... SELECT ...)

2020-09-25 Thread Bharath Rupireddy
On Tue, Sep 22, 2020 at 10:26 AM Greg Nancarrow wrote: > > For cases where it can't be allowed (e.g. INSERT into a table with > foreign keys, or INSERT INTO ... SELECT ... ON CONFLICT ... DO UPDATE > ...") it at least allows parallelism of the SELECT part. > Thanks Greg for the patch. I have few

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-25 Thread tsunakawa.ta...@fujitsu.com
From: Jamison, Kirk/ジャミソン カーク > [Results] > Recovery/Failover performance (in seconds). 3 trial runs. > > | shared_buffers | master | patch | %reg| > ||||-| > | 128MB | 32.406 | 33.785 | 4.08% | > | 1GB| 36.188 | 32.747 | -10.51%

enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-09-25 Thread Katsuragi Yuta
Hi, pg_stat_statements tracks the number of rows processed by some utility commands. But, currently, it does not track the number of rows processed by REFRESH MATERIALIZED VIEW. Attached patch enables pg_stat_statements to track processed rows by REFRESH MATERIALIZED VIEW. Regards, Katsuragi Yu

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-25 Thread tsunakawa.ta...@fujitsu.com
From: Amit Kapila > No, during recovery also we need to be careful. We need to ensure that > we use cached value during recovery and cached value is always > up-to-date. We can't rely on lseek and I have provided some scenario > up thread [1] where such behavior can cause problem and then see the

AppendStringInfoChar instead of appendStringInfoString

2020-09-25 Thread Hou, Zhijie
Hi In (/src/backend/replication/backup_manifest.c) I found the following code: appendStringInfoString(&buf, "\n"); appendStringInfoString(&buf, "\""); Since only one bit string is appended here, I think it will be better to call appendStringInfoChar. Best reagrds, houzj 00

Re: Transactions involving multiple postgres foreign servers, take 2

2020-09-25 Thread Masahiko Sawada
On Thu, 24 Sep 2020 at 17:23, tsunakawa.ta...@fujitsu.com wrote: > > From: Masahiko Sawada > > So with your idea, I think we require FDW developers to not call > > ereport(ERROR) as much as possible. If they need to use a function > > including palloc, lappend etc that could call ereport(ERROR),

Re: Logical replication from PG v13 and below to PG v14 (devel version) is not working.

2020-09-25 Thread Dilip Kumar
On Fri, Sep 25, 2020 at 8:12 AM Amit Kapila wrote: > > On Thu, Sep 24, 2020 at 6:33 PM Ashutosh Sharma wrote: > > > > Hi Amit, > > > > > Here, I think instead of using MySubscription->stream, we should use > > > server/walrecv version number as we used at one place in tablesync.c. > > > > Should

Re: Feature improvement for FETCH tab completion

2020-09-25 Thread btnakamichin
2020-09-25 15:38 に Fujii Masao さんは書きました: On 2020/09/25 14:24, btnakamichin wrote: Hello! I’d like to improve the FETCH tab completion. The FETCH tab completion . Therefore, this patch fixes the problem. Previous function completed one of FORWARD, BACKWARD, RELATIVE, ABSOLUTE, but now it comp

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-09-25 Thread k.jami...@fujitsu.com
Hi. > I'll send performance measurement results in the next email. Thanks a lot for > the reviews! Below are the performance measurement results. I was only able to use low-spec machine: CPU 4v, Memory 8GB, RHEL, xfs filesystem. [Failover/Recovery Test] 1. (Master) Create table (ex. 10,000 tabl

[PATCH] Runtime control of CLOBBER_CACHE_ALWAYS

2020-09-25 Thread Craig Ringer
Hi all While working on extensions I've often wanted to enable cache clobbering for a targeted piece of code, without paying the price of running it for all backends during postgres startup and any initial setup tasks that are required. So here's a patch that, when CLOBBER_CACHE_ALWAYS or CLOBBER

Re: Get memory contexts of an arbitrary backend process

2020-09-25 Thread torikoshia
Hi, Thanks for all your comments, I updated the patch. On Tue, Sep 1, 2020 at 12:03 AM Kasahara Tatsuhito wrote: - How about managing the status of signal send/receive and dump operations on a shared hash or others ? Sending and receiving signals, dumping memory information, and referencin

Re: Handing off SLRU fsyncs to the checkpointer

2020-09-25 Thread Thomas Munro
On Fri, Sep 25, 2020 at 12:53 PM Thomas Munro wrote: > Here's a new version. The final thing I'm contemplating before > pushing this is whether there may be hidden magical dependencies in > the order of operations in CheckPointGuts(), which I've changed > around. Andres, any comments? I nagged

Re: history file on replica and double switchover

2020-09-25 Thread Fujii Masao
On 2020/09/25 13:00, David Zhang wrote: On 2020-09-24 5:00 p.m., Fujii Masao wrote: On 2020/09/25 8:15, David Zhang wrote: Hi, My understanding is that the "archiver" won't even start if "archive_mode = on" has been set on a "replica". Therefore, either (XLogArchiveMode == ARCHIVE_MODE_AL