Re: wrong relkind error messages

2021-06-24 Thread Peter Eisentraut
On 13.04.20 15:54, Peter Eisentraut wrote: I'm not a fan of error messages like     relation "%s" is not a table, foreign table, or materialized view It doesn't tell me what's wrong, it only tells me what else could have worked.  It's also tedious to maintain and the number of combinations g

Re: Added schema level support for publication.

2021-06-24 Thread Greg Nancarrow
On Thu, Jun 24, 2021 at 4:41 PM tanghy.f...@fujitsu.com wrote: > > I applied your V7* patch and complied it. The following warnings came out, > please take a look. > I encountered the following warnings when building with the v7 patches applied: (looks like missing #include "catalog/objectaddres

Re: Pipeline mode and PQpipelineSync()

2021-06-24 Thread Boris Kolpackov
Boris Kolpackov writes: > What's strange here is that the first PQgetResult() call (marked with ???) > returns NULL instead of result for INSERT #1 as in the first call sequence. I've hit another similar case except now an unexpected NULL result is returned in the middle of PGRES_PIPELINE_ABORTE

Add tests for UNBOUNDED syntax ambiguity

2021-06-24 Thread Peter Eisentraut
As many will be aware, there is a syntactic ambiguity in the SQL standard regarding the keyword UNBOUNDED. Since UNBOUNDED is a non-reserved word, it could be the name of a function parameter and be used as an expression. There is a grammar hack to resolve such cases as the keyword. I broug

Re: Pipeline mode and PQpipelineSync()

2021-06-24 Thread Boris Kolpackov
Alvaro Herrera writes: > Curious -- I just noticed that the server understands a message 'H' that > requests a flush of the server buffer. However, libpq has no way to > generate that message as far as I can see. I think you could use that > to request results from the pipeline, without the syn

Re: Pipeline mode and PQpipelineSync()

2021-06-24 Thread Boris Kolpackov
Alvaro Herrera writes: > Subject: [PATCH] Clarify that pipeline sync is mandatory > > --- > doc/src/sgml/libpq.sgml | 6 -- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml > index 441cc0da3a..0217f8d8c7 100644 > --- a/doc

Re: locking [user] catalog tables vs 2pc vs logical rep

2021-06-24 Thread Amit Kapila
On Mon, Jun 21, 2021 at 8:48 AM osumi.takami...@fujitsu.com wrote: > > On Sunday, June 20, 2021 9:50 PM I wrote: > > On Sunday, June 20, 2021 3:23 PM Amit Kapila > > wrote: > > > On Sun, Jun 20, 2021 at 9:28 AM osumi.takami...@fujitsu.com > > > wrote: > > > > * doc/src/sgml/logicaldecoding.sgml

Re: Doc patch for Logical Replication Message Formats (PG14)

2021-06-24 Thread Amit Kapila
On Mon, Jun 21, 2021 at 4:41 PM Brar Piening wrote: > > Amit Kapila schrieb: > > On Mon, Jun 21, 2021 at 4:13 PM Brar Piening wrote: > >> Amit Kapila wrote: > >> After looking at the docs once again I have another minor amendment (new > >> patch attached). > >> > > +The value of t

Using each rel as both outer and inner for JOIN_ANTI

2021-06-24 Thread Richard Guo
Hi hackers, We may have anti-joins in several cases. Sublinks of 'NOT EXISTS' may be pulled up as anti-joins. Left joins whose join quals are strict for any nullable var that is forced null by higher qual levels will also be reduced to anti-joins. So anti-joins are very commonly used in practice.

Re: SSL/TLS instead of SSL in docs

2021-06-24 Thread Daniel Gustafsson
> On 22 Jun 2021, at 06:37, Michael Paquier wrote: > > On Mon, Jun 21, 2021 at 01:23:56PM +0200, Daniel Gustafsson wrote: >> On 18 Jun 2021, at 07:37, Michael Paquier wrote: >>> It looks inconsistent to me to point to the libpq documentation to get >>> the details about SNI. Wouldn't is be bett

Weird use of parentheses in the manual

2021-06-24 Thread Guillaume Lelarge
Hey, I was translating logicaldecoding.sgml, and I saw this sentence: There are multiple required streaming callbacks (stream_start_cb, stream_stop_cb, stream_abort_cb, stream_commit_cb and stream_change_cb) and two optional callbacks (stream_message_cb) and (stream_truncate_cb). The two last se

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-06-24 Thread Pavel Borisov
> > Have you found a case that the v2 patch breaks? > > The v2 patch does follow the analysis in the beginning of the > discussion, which identified that in setting up parallel workers, a > "later transaction snapshot" was taken than the one actually used in > the statement execution, and that's wh

Re: pgbench logging broken by time logic changes

2021-06-24 Thread Andrew Dunstan
On 6/24/21 2:46 AM, Fabien COELHO wrote: > > Bonjour Michaël, > >> Using grep() with "$re" results in all the fields matching.  Using on >> the contrary "/$re/" in grep(), like list_files(), would only match >> the first one, which is correct. > > Ok, good catch. Perl is kind of a strange languag

Re: Using indexUnchanged with nbtree

2021-06-24 Thread Simon Riggs
On Wed, Jun 23, 2021 at 5:42 PM Peter Geoghegan wrote: > > On Wed, Jun 23, 2021 at 9:31 AM Simon Riggs > wrote: > > You're right that skipping the check might also skip killing a prior > > row version, but it doesn't prevent later scans from killing them, so > > there is no correctness aspect to

RE: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread k.jami...@fujitsu.com
Hi Sawada-san, I also tried to play a bit with the latest patches similar to Ikeda-san, and with foreign 2PC parameter enabled/required. > > >> b. about performance bottleneck (just share my simple benchmark > > >> results) > > >> > > >> The resolver process can be performance bottleneck easily a

[Patch] change the default value of shared_buffers in postgresql.conf.sample

2021-06-24 Thread zhangj...@fujitsu.com
Hi all In PostgreSQL 14, The default value of shared_buffers is 128MB, but in postgresql.conf.sample, the default value of shared_buffers is 32MB. I think the following changes should be made. File: postgresql\src\backend\utils\misc\ postgresql.conf.sample #shared_buffers = 32MB => #shared_bu

Re: Weird use of parentheses in the manual

2021-06-24 Thread Amit Kapila
On Thu, Jun 24, 2021 at 5:27 PM Guillaume Lelarge wrote: > > Really tiny patch attached to fix this if it really is wrong, and anyone > cares enough to fix it :) > LGTM. I'll take care of this tomorrow unless someone else has any suggestions in this regard. -- With Regards, Amit Kapila.

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-06-24 Thread Pavel Borisov
By the way in the initial discussion on parallel infrastructure https://www.postgresql.org/message-id/20150208002027.GH9201%40alap3.anarazel.de I've seen a proposal to set the worker's PGXACT->xmin to be the minimum of the top transaction id and the snapshots. -- Best regards, Pavel Borisov Post

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiko Sawada
On Sat, Jun 12, 2021 at 1:25 AM Fujii Masao wrote: > > > > On 2021/05/11 13:37, Masahiko Sawada wrote: > > I've attached the updated patches that incorporated comments from > > Zhihong and Ikeda-san. > > Thanks for updating the patches! > > I'm still reading these patches, but I'd like to share so

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiko Sawada
On Thu, Jun 24, 2021 at 9:46 PM k.jami...@fujitsu.com wrote: > > Hi Sawada-san, > > I also tried to play a bit with the latest patches similar to Ikeda-san, > and with foreign 2PC parameter enabled/required. Thank you for testing the patch! > > > > >> b. about performance bottleneck (just share

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-24 Thread Heikki Linnakangas
On 24/06/2021 12:50, Richard Guo wrote: Hi hackers, We may have anti-joins in several cases. Sublinks of 'NOT EXISTS' may be pulled up as anti-joins. Left joins whose join quals are strict for any nullable var that is forced null by higher qual levels will also be reduced to anti-joins. So anti-

Re: Add tests for UNBOUNDED syntax ambiguity

2021-06-24 Thread Heikki Linnakangas
On 24/06/2021 12:01, Peter Eisentraut wrote: As many will be aware, there is a syntactic ambiguity in the SQL standard regarding the keyword UNBOUNDED. Since UNBOUNDED is a non-reserved word, it could be the name of a function parameter and be used as an expression. There is a grammar hack to r

Relaxing COPY FREEZE restrictions

2021-06-24 Thread Simon Riggs
COPY FREEZE throws ERRORs if you use it inappropriately. This makes it very hard to use in practice. ERRORs should be replaced by NOTICEs, or just silence. That treats FREEZE as an optional performance tweak, which is more appropriate for its use case. In particular, the need to throw ERRORs lead

pgbench using COPY FREEZE

2021-06-24 Thread Simon Riggs
pgbench -i should use COPY FREEZE, patch attached. -- Simon Riggshttp://www.EnterpriseDB.com/ pgbench_copy_freeze.v1.patch Description: Binary data

Re: Using each rel as both outer and inner for JOIN_ANTI

2021-06-24 Thread Tom Lane
Heikki Linnakangas writes: > On 24/06/2021 12:50, Richard Guo wrote: >> I believe if we use the smaller table 'foo' as inner side for this >> query, we would have a cheaper plan. > How would that work? I think you could make it work for the hash-join case by extending the existing mechanism for

Re: Deadlock risk while inserting directly into partition?

2021-06-24 Thread Simon Riggs
On Thu, Jun 24, 2021 at 1:45 AM Tom Lane wrote: > > David Rowley writes: > > ... What I mean is that as we add > > more and more fixes to improve performance of partitioning, that there > > comes a point where the ability to directly reference partitions is a > > hindrance rather than something t

Re: a path towards replacing GEQO with something better

2021-06-24 Thread John Naylor
On Mon, Jun 14, 2021 at 12:10 PM Tomas Vondra wrote: > >> I haven't read the [Kossmann & Stocker, 2000] paper yet, but the > >> [Neumann, 2018] paper seems to build on it, and it seems to work with > >> much larger subtrees of the join tree than k=5. > > > > Right, in particular it builds on "IDP

Re: [Patch] change the default value of shared_buffers in postgresql.conf.sample

2021-06-24 Thread Tom Lane
"zhangj...@fujitsu.com" writes: > In PostgreSQL 14, The default value of shared_buffers is 128MB, but in > postgresql.conf.sample, the default value of shared_buffers is 32MB. > I think the following changes should be made. > File: postgresql\src\backend\utils\misc\ postgresql.conf.sample > #sha

Re: Pipeline mode and PQpipelineSync()

2021-06-24 Thread Alvaro Herrera
On 2021-Jun-23, Boris Kolpackov wrote: > If, however, I execute it by checking for results before sending the > next INSERT, I get the following call sequence: > > PQsendQueryPrepared() # INSERT #1 > PQflush() > PQsendQueryPrepared() # INSERT #2 > PQflush() > ... > PQsendQueryPrepared() # INSERT

Re: [Patch] change the default value of shared_buffers in postgresql.conf.sample

2021-06-24 Thread Daniel Gustafsson
> On 24 Jun 2021, at 17:49, Tom Lane wrote: > ..if we were going to do anything I think that adjusting both initdb.c and > guc.c to use 128MB might be the most appropriate thing. Ensuring consistency doesn't seem like a bad thing in itself, even if it in practice won't make much difference. --

Re: Assertion failure in HEAD and 13 after calling COMMIT in a stored proc

2021-06-24 Thread Ranier Vilela
Em qua., 23 de jun. de 2021 às 21:45, Greg Nancarrow escreveu: > On Wed, Jun 23, 2021 at 11:01 PM Tom Lane wrote: > > > > > > The code is correct as-is; the proposed change would result in taking > > more snapshots than needed. Perhaps the comment needs revision, since > > you both misread it.

Re: Assertion failure in HEAD and 13 after calling COMMIT in a stored proc

2021-06-24 Thread Ranier Vilela
Em qua., 23 de jun. de 2021 às 21:51, Tom Lane escreveu: > Greg Nancarrow writes: > > On Wed, Jun 23, 2021 at 11:01 PM Tom Lane wrote: > >> The comment is written in terms of "when can we > >> skip taking a snapshot", while the test in the code is written for > >> the inverse condition "when do

Re: Iterating on IndexTuple attributes and nbtree page-level dynamic prefix truncation

2021-06-24 Thread Matthias van de Meent
On Thu, 17 Jun 2021 at 17:14, Matthias van de Meent wrote: > > I'll try to > benchmark the patches in this patchset (both 0001, and 0001+0002) in > the upcoming weekend. Somewhat delayed, benchmark results are attached. These are based on 7 iterations of the attached benchmark script ('scratch.sq

Re: subscription/t/010_truncate.pl failure on desmoxytes in REL_13_STABLE

2021-06-24 Thread Tom Lane
Amit Kapila writes: > On Tue, Jun 22, 2021 at 6:54 PM Tom Lane wrote: >> BTW, the reason that the walsender is still showing its "query" as >> "SELECT pg_config..." is that pre-v14 versions don't update the >> reported query for replication commands, only plain-SQL commands. >> I recall that we f

Re: pgbench using COPY FREEZE

2021-06-24 Thread Fabien COELHO
Hello Simon, Indeed. There is already a "ready" patch in the queue, see: https://commitfest.postgresql.org/33/3034/ -- Fabien.

Re: Added schema level support for publication.

2021-06-24 Thread vignesh C
On Thu, Jun 24, 2021 at 2:12 PM Greg Nancarrow wrote: > > On Thu, Jun 24, 2021 at 4:41 PM tanghy.f...@fujitsu.com > wrote: > > > > I applied your V7* patch and complied it. The following warnings came out, please take a look. > > > > I encountered the following warnings when building with the v7

Re: Added schema level support for publication.

2021-06-24 Thread vignesh C
On Thu, Jun 24, 2021 at 12:10 PM tanghy.f...@fujitsu.com wrote: > > Hi > > I applied your V7* patch and complied it. The following warnings came out, > please take a look. > > >pg_publication.c:688:22: warning: ‘tables’ may be used uninitialized in this > >function [-Wmaybe-uninitialized] > >

Re: Doc chapter for Hash Indexes

2021-06-24 Thread Bruce Momjian
aOn Wed, Jun 23, 2021 at 12:56:51PM +0100, Simon Riggs wrote: > On Wed, Jun 23, 2021 at 5:12 AM Amit Kapila wrote: > > > > On Tue, Jun 22, 2021 at 2:31 PM Simon Riggs > > wrote: > > > > > > I attach both clean and compare versions. > > > > > > > Do we want to hold this work for PG15 or commit in

Re: pgbench logging broken by time logic changes

2021-06-24 Thread Michael Paquier
On Thu, Jun 24, 2021 at 08:03:27AM -0400, Andrew Dunstan wrote: > On 6/24/21 2:46 AM, Fabien COELHO wrote: >>> Using grep() with "$re" results in all the fields matching.  Using on >>> the contrary "/$re/" in grep(), like list_files(), would only match >>> the first one, which is correct. >> >> Ok,

Re: Pipeline mode and PQpipelineSync()

2021-06-24 Thread Alvaro Herrera
On 2021-Jun-23, Boris Kolpackov wrote: > If, however, I execute it by checking for results before sending the > next INSERT, I get the following call sequence: > > PQsendQueryPrepared() # INSERT #1 > PQflush() > PQsendQueryPrepared() # INSERT #2 > PQflush() > ... > PQsendQueryPrepared() # INSERT

Re: seawasp failing, maybe in glibc allocator

2021-06-24 Thread Thomas Munro
On Mon, Jun 21, 2021 at 11:57 AM Tom Lane wrote: > If that's an accurate characterization of the tradeoff, I have little > difficulty in voting for #2. A crash is strictly worse than a memory > leak. Besides which, I've heard little indication that they might > revert. Agreed. On Mon, Jun 21,

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiko Sawada
On Thu, Jun 24, 2021 at 10:11 PM Masahiko Sawada wrote: > > On Sat, Jun 12, 2021 at 1:25 AM Fujii Masao > wrote: > > > > > > > > (5) > > +2. Pre-Commit phase (1st phase of two-phase commit) > > +we record the corresponding WAL indicating that the foreign server is > > involved > > +with the cur

Re: cleanup temporary files after crash

2021-06-24 Thread Michael Paquier
On Wed, Jun 23, 2021 at 08:46:12AM +0900, Michael Paquier wrote: > Looks adapted to me. Applied as it was still on the stack. Please note that GUC_NOT_IN_SAMPLE was missing. -- Michael signature.asc Description: PGP signature

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiro Ikeda
On 2021/06/24 22:27, Masahiko Sawada wrote: > On Thu, Jun 24, 2021 at 9:46 PM k.jami...@fujitsu.com > wrote: >> >> Hi Sawada-san, >> >> I also tried to play a bit with the latest patches similar to Ikeda-san, >> and with foreign 2PC parameter enabled/required. > > Thank you for testing the pat

Re: Deadlock risk while inserting directly into partition?

2021-06-24 Thread David Rowley
On Thu, 24 Jun 2021 at 12:32, David Rowley wrote: > The overhead of taking these locks is pretty significant for > partitioned tables with lots of partitions where only 1 of them > survives run-time partition pruning. That's really terrible for > people that want to PREPARE queries and just look

Re: Transactions involving multiple postgres foreign servers, take 2

2021-06-24 Thread Masahiro Ikeda
On 2021/06/24 22:11, Masahiko Sawada wrote: > On Sat, Jun 12, 2021 at 1:25 AM Fujii Masao > wrote: >> On 2021/05/11 13:37, Masahiko Sawada wrote: >> So currently FdwXactInsertEntry() calls XLogInsert() and XLogFlush() for >> XLOG_FDWXACT_INSERT WAL record. Additionally we should also wait ther

Re: Using indexUnchanged with nbtree

2021-06-24 Thread Peter Geoghegan
On Thu, Jun 24, 2021 at 5:39 AM Simon Riggs wrote: > This case occurs when we are doing non-HOT UPDATEs. That command is > searched, so the scan will already have touched the heap and almost > certainly the index also, setting any LP_DEAD bits already in the most > frequent case. But it won't, be

Re: PG 14 release notes, first draft

2021-06-24 Thread Bruce Momjian
On Wed, Jun 23, 2021 at 07:45:53AM -0700, Peter Geoghegan wrote: > On Wed, Jun 23, 2021 at 5:50 AM Simon Riggs > wrote: > > I just noticed that these commits are missing, yet are very important > > new features: > > d9d076222f5b94a8 > > f9900df5f9 > > c98763bf51bf > > > > These are important enoug

Re: strange case of "if ((a & b))"

2021-06-24 Thread Justin Pryzby
On Wed, Apr 28, 2021 at 02:40:09PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > These look strange to me - the inner parens don't do anything. > > I wouldn't write it with 2x parens for the same reason I wouldn't write it > > with > > 8x parens. > > Agreed, but shouldn't we just drop the e

Re: SSL/TLS instead of SSL in docs

2021-06-24 Thread Michael Paquier
On Thu, Jun 24, 2021 at 01:53:47PM +0200, Daniel Gustafsson wrote: > The attached v3 does it this way. Thanks. Mostly what was on message upthread. Applied this one. > Yes, there are a few but not too many. Whenever the protocol is refererred to > and not the concept of an encrypted connection

Re: Doc chapter for Hash Indexes

2021-06-24 Thread Amit Kapila
On Fri, Jun 25, 2021 at 1:29 AM Bruce Momjian wrote: > > aOn Wed, Jun 23, 2021 at 12:56:51PM +0100, Simon Riggs wrote: > > On Wed, Jun 23, 2021 at 5:12 AM Amit Kapila wrote: > > > > > > On Tue, Jun 22, 2021 at 2:31 PM Simon Riggs > > > wrote: > > > > > > > > I attach both clean and compare versi

Re: Decoding speculative insert with toast leaks memory

2021-06-24 Thread Amit Kapila
On Thu, Jun 24, 2021 at 11:04 AM Michael Paquier wrote: > > On Thu, Jun 24, 2021 at 12:25:15AM -0400, Tom Lane wrote: > > Amit Kapila writes: > >> Okay, I'll change this in back branches and HEAD to keep the code > >> consistent, or do you think it is better to retain the order in HEAD > >> as it

PG Docs for ALTER SUBSCRIPTION REFRESH PUBLICATION - copy_data option

2021-06-24 Thread Peter Smith
Hi. (For brevity, in this mail I refer to "ALTER SUBSCRIPTION sub REFRESH PUBLICATION" as "ASRP") -- The PG Docs for ASRP WITH (copy_data = true), says "(Previously subscribed tables are not copied.)" [1]. I thought this rule meant that only tables which got added by ALTER PUBLICATION pubname A

Re: subscription/t/010_truncate.pl failure on desmoxytes in REL_13_STABLE

2021-06-24 Thread Amit Kapila
On Thu, Jun 24, 2021 at 11:25 PM Tom Lane wrote: > > Amit Kapila writes: > > On Tue, Jun 22, 2021 at 6:54 PM Tom Lane wrote: > >> BTW, the reason that the walsender is still showing its "query" as > >> "SELECT pg_config..." is that pre-v14 versions don't update the > >> reported query for replic

Adding more PGDLLIMPORTs

2021-06-24 Thread Craig Ringer
Hi all As a result of some recent work on Windows I have a list of PGDLLIMPORTs I'd like to add to existing exported globals. All affected variables are already extern, so this doesn't expose any new API not already available to non-Windows extensions. I've split the patch up for clarity: * v1-

Re: Remove redundant variable pageSize in gistinitpage

2021-06-24 Thread Peter Eisentraut
On 26.04.21 05:12, Bharath Rupireddy wrote: In gistinitpage, pageSize variable looks redundant, instead we could just pass BLCKSZ. This will be consistent with its peers BloomInitPage, brin_page_init and SpGistInitPage. Attaching a small patch. Thoughts? Committed. This code was new in this fo

Allow streaming the changes after speculative aborts.

2021-06-24 Thread Amit Kapila
Till now, we didn't allow to stream the changes in logical replication till we receive speculative confirm or the next DML change record after speculative inserts. The reason was that we never use to process speculative aborts but after commit 4daa140a2f it is possible to process them so we can all

Re: Weird use of parentheses in the manual

2021-06-24 Thread Amit Kapila
On Thu, Jun 24, 2021 at 6:38 PM Amit Kapila wrote: > > On Thu, Jun 24, 2021 at 5:27 PM Guillaume Lelarge > wrote: > > > > Really tiny patch attached to fix this if it really is wrong, and anyone > > cares enough to fix it :) > > > > LGTM. I'll take care of this tomorrow unless someone else has a