Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-12-06 Thread Amit Kapila
On Fri, Dec 6, 2019 at 5:00 PM Amit Khandekar wrote: > > On Fri, 6 Dec 2019 at 15:40, Amit Kapila wrote: > > > > > > Few comments: > > -- > > > > 1. > > + /* Now that the state fields are initialized, it is safe to return it. */ > > + *iter_state = state; > > + > > /* alloca

Re: smgr vs DropRelFileNodeBuffers() vs filesystem state vs no critical section

2019-12-06 Thread Amit Kapila
On Sat, Dec 7, 2019 at 5:42 AM Andres Freund wrote: > > Tom, I seem to recall a recent thread of yours discussing a different > approach to truncation. I wonder if there's some intersection with > that. But unfortunately my search somehow has come up with nothing so > far - do you remember enough

Re: what is the purpose to use 3 function to compare cost add_path/set_cheapest/get_cheapest_fractional_path

2019-12-06 Thread Amit Kapila
On Fri, Dec 6, 2019 at 11:26 AM Andy Fan wrote: > > Hello Hackers: >I'm reading the code of optimizer and get confused about the 3 functions. > add_path/set_cheapest/get_cheapest_fractional_path > > > add_(partial_)path: > For every relations, optimizer will build path for it and add then ca

Re: Windows buildfarm members vs. new async-notify isolation test

2019-12-06 Thread Amit Kapila
On Sat, Dec 7, 2019 at 5:01 AM Tom Lane wrote: > > Andrew Dunstan writes: > > On 12/5/19 4:37 AM, Amit Kapila wrote: > >> IIUC, this means that commit (step l2commit) is finishing before the > >> notify signal is reached that session. If so, can we at least confirm > >> that by adding something

Re: [Proposal] Level4 Warnings show many shadow vars

2019-12-06 Thread Mark Dilger
On 12/6/19 3:18 PM, Ranier Vilela wrote: Robert Haas wrote: Maybe you want to post a few examples. It's hard to discuss in the abstract. I am working on the patch. I think this is a great example. I do not know if it is better to rename the local parameter, or if it should be renamed the gl

Re: ssl passphrase callback

2019-12-06 Thread Andrew Dunstan
On 12/6/19 6:20 PM, Tom Lane wrote: > Andrew Dunstan writes: >> I've just been looking at that. load_external_function() doesn't >> actually do anything V1-ish with the value, it just looks up the symbol >> using dlsym and returns it cast to a PGFunction. Is there any reason I >> can't just use

smgr vs DropRelFileNodeBuffers() vs filesystem state vs no critical section

2019-12-06 Thread Andres Freund
Hi, [1] made me notice these issues. The issues here are mostly independent, but it's still worthwhile to read that thread - in particular because my proposed solution for the problem is possibly somewhat related to this issue. And, if we were to go for my more extreme proposal below, the fix for

Re: Windows buildfarm members vs. new async-notify isolation test

2019-12-06 Thread Tom Lane
Andrew Dunstan writes: > On 12/5/19 4:37 AM, Amit Kapila wrote: >> IIUC, this means that commit (step l2commit) is finishing before the >> notify signal is reached that session. If so, can we at least confirm >> that by adding something like select pg_sleep(1) in that step? So, >> l2commit will

Re: ssl passphrase callback

2019-12-06 Thread Tom Lane
Andrew Dunstan writes: > I've just been looking at that. load_external_function() doesn't > actually do anything V1-ish with the value, it just looks up the symbol > using dlsym and returns it cast to a PGFunction. Is there any reason I > can't just use that and cast it again to the callback funct

RE: [Proposal] Level4 Warnings show many shadow vars

2019-12-06 Thread Ranier Vilela
Robert Haas wrote: >Maybe you want to post a few examples. It's hard to discuss in the abstract. I am working on the patch. I think this is a great example. I do not know if it is better to rename the local parameter, or if it should be renamed the global variable. line: 68 var char **synchronous

Re: 'Invalid lp' during heap_xlog_delete

2019-12-06 Thread Andres Freund
Hi, On 2019-11-08 12:46:51 -0800, Daniel Wood wrote: > Page on disk has empty lp 1 > * Insert into page lp 1 > > checkpoint START.Redo eventually starts here. > ** Delete all rows on page. It's worthwhile to note that this part cannot happen without full page writes disabled. By dint of a ch

Re: ssl passphrase callback

2019-12-06 Thread Andrew Dunstan
On 11/15/19 8:59 AM, Andrew Dunstan wrote: > On 11/14/19 3:21 PM, Alvaro Herrera wrote: >> On 2019-Nov-14, Andrew Dunstan wrote: >> >>> I guess this would work. There would have to be a deal of code to load >>> the library and lookup the symbol. Do we really think it's worth it? >>> Leveraging sh

Re: Windows buildfarm members vs. new async-notify isolation test

2019-12-06 Thread Andrew Dunstan
On 12/5/19 4:37 AM, Amit Kapila wrote: > On Wed, Dec 4, 2019 at 9:51 PM Andrew Dunstan > wrote: >> On Wed, Dec 4, 2019 at 12:12 AM Tom Lane wrote: >>> Amit Kapila writes: On Tue, Dec 3, 2019 at 10:10 PM Tom Lane wrote: > Hmm ... just looking at the code again, could it be that there'

Re: log bind parameter values on error

2019-12-06 Thread Alvaro Herrera
On 2019-Dec-06, Pavel Stehule wrote: > pá 6. 12. 2019 v 18:57 odesílatel Alvaro Herrera > napsal: > > (It seems a bit weird that if I assign NULL to :two pgbench stores the > > empty string, instead of the NULL that I get as in the above which is > > what happens when the variable is not defined

Re: log bind parameter values on error

2019-12-06 Thread Alvaro Herrera
I added some tests to the pgbench suite in the attached. I also finally put it the business to limit the length of parameters reported. I'd probably drop testlibpq5.c, since it seems a bit pointless now ... -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24

Re: Frontend/Backend Protocol: SSL / GSS Protocol Negotiation Problem

2019-12-06 Thread Jakob Egger
> On 6. Dec 2019, at 16:45, Andrew Gierth wrote: > >> "Jakob" == Jakob Egger writes: > > Jakob> But this also needs to be fixed on the client side as well, > Jakob> otherwise affected clients can't connect to older servers > Jakob> anymore. > > There's a workaround, which is to set PGGSS

Re: Using XLogFileNameP in critical section

2019-12-06 Thread Andrew Dunstan
On 12/3/19 11:24 AM, Tom Lane wrote: > I wrote: >> Also, buildfarm member drongo is not happy: >> postgres.def : error LNK2001: unresolved external symbol XLogFileNameP >> [C:\prog\bf\root\HEAD\pgsql.build\postgres.vcxproj] >> Release/postgres/postgres.lib : fatal error LNK1120: 1 unresolved ext

Re: log bind parameter values on error

2019-12-06 Thread Pavel Stehule
pá 6. 12. 2019 v 18:57 odesílatel Alvaro Herrera napsal: > On 2019-Dec-05, Tom Lane wrote: > > > Possibly a workable compromise is to emit the info as an error > > context line, appending it to whatever context exists today. > > The result might look like, say, > > > > ERROR: whatever > > CONTEX

Re: log bind parameter values on error

2019-12-06 Thread Alvaro Herrera
On 2019-Dec-05, Tom Lane wrote: > Possibly a workable compromise is to emit the info as an error > context line, appending it to whatever context exists today. > The result might look like, say, > > ERROR: whatever > CONTEXT: SQL function "foo" > extended query with parameters x, y, ... This is

Re: Change atoi to strtol in same place

2019-12-06 Thread Joe Nelson
I see this patch has been moved to the next commitfest. What's the next step, does it need another review? -- Joe Nelson https://begriffs.com

Re: Why JIT speed improvement is so modest?

2019-12-06 Thread Andres Freund
Hi, On 2019-12-06 19:52:15 +0300, Konstantin Knizhnik wrote: > On 06.12.2019 18:53, Robert Haas wrote: > > On Thu, Nov 28, 2019 at 2:08 AM Konstantin Knizhnik > > wrote: > > > calls float4_accum for each row of T, the same query in VOPS will call > > > vops_float4_avg_accumulate for each tile whi

Re: Why JIT speed improvement is so modest?

2019-12-06 Thread Konstantin Knizhnik
On 06.12.2019 19:52, Konstantin Knizhnik wrote: On 06.12.2019 18:53, Robert Haas wrote: On Thu, Nov 28, 2019 at 2:08 AM Konstantin Knizhnik wrote: calls float4_accum for each row of T, the same query in VOPS will call vops_float4_avg_accumulate for each tile which contains 64 elements. So

Re: global / super barriers (for checksums)

2019-12-06 Thread Andres Freund
Hi, Thanks for the updated version! On 2019-12-02 13:06:24 -0500, Robert Haas wrote: > On Sun, Nov 17, 2019 at 8:38 AM Magnus Hagander wrote: > > I'm definitely happy to work with it, but I did not and do not feel I have > > the skills for doing the "proper review" needed for it. So I am also v

Re: Why JIT speed improvement is so modest?

2019-12-06 Thread Konstantin Knizhnik
On 06.12.2019 18:53, Robert Haas wrote: On Thu, Nov 28, 2019 at 2:08 AM Konstantin Knizhnik wrote: calls float4_accum for each row of T, the same query in VOPS will call vops_float4_avg_accumulate for each tile which contains 64 elements. So vops_float4_avg_accumulate is called 64 times less

Re: error context for vacuum to include block number

2019-12-06 Thread Justin Pryzby
Find attached updated patch: . Use structure to include relation name. . Split into a separate patch rename of "StringInfoData buf". 2019-11-27 20:04:53.640 CST [14244] ERROR: canceling statement due to statement timeout 2019-11-27 20:04:53.640 CST [14244] CONTEXT: block 2314 of relation t 20

Re: Why JIT speed improvement is so modest?

2019-12-06 Thread Robert Haas
On Thu, Nov 28, 2019 at 2:08 AM Konstantin Knizhnik wrote: > calls float4_accum for each row of T, the same query in VOPS will call > vops_float4_avg_accumulate for each tile which contains 64 elements. > So vops_float4_avg_accumulate is called 64 times less than float4_accum. > And inside it cont

Re: [Proposal] Level4 Warnings show many shadow vars

2019-12-06 Thread Robert Haas
On Fri, Dec 6, 2019 at 7:59 AM Ranier Vilela wrote: > What did the original author want, use the global variable or not use it by > overriding the name. > If it was to use the global variable, it will affect the behavior of the > function, I believe. Well, you haven't provided any examples, so

Re: Frontend/Backend Protocol: SSL / GSS Protocol Negotiation Problem

2019-12-06 Thread Andrew Gierth
> "Jakob" == Jakob Egger writes: Jakob> But this also needs to be fixed on the client side as well, Jakob> otherwise affected clients can't connect to older servers Jakob> anymore. There's a workaround, which is to set PGGSSENCMODE=disable on the client. It would be far better to avoid c

[PATCH] Finally split StdRdOptions into HeapOptions and ToastOptions

2019-12-06 Thread Nikolay Shaplov
In the thread https://www.postgresql.org/message-id/2620882.s52SJui4ql@x200m I've suggested to split one big StdRdOption that is used for options storage into into Options structures individual for each relkind and each relam That patch have been split into smaller parts, most of them were alrea

Online checksums verification in the backend

2019-12-06 Thread Julien Rouhaud
Hi, This topic was discussed several times, with the most recent discussions found at [1] and [2]. Based on those discussions, my understanding is that the current approach in BASE_BACKUP has too many drawbacks and we should instead do this check in the backend. I've been working using such appr

Re: Protocol problem with GSSAPI encryption?

2019-12-06 Thread Jakob Egger
> On 4. Dec 2019, at 06:24, Stephen Frost wrote: > > Greetings, > > * Andrew Gierth (and...@tao11.riddles.org.uk) wrote: >>> "Peter" == Peter Eisentraut writes: >> It seems to me that this is a bug in ProcessStartupPacket, which should accept both GSS or SSL negotiation requests

Re: Frontend/Backend Protocol: SSL / GSS Protocol Negotiation Problem

2019-12-06 Thread Jakob Egger
> On 6. Dec 2019, at 15:08, Michael Paquier wrote: > > On Fri, Dec 06, 2019 at 02:25:46PM +0100, Jakob Egger wrote: >> I've received a bug report from a PostgreSQL user that psql 12.1 >> failed to connect to a PostgreSQL 12.1 server, with the following >> error message: >> >> psql: error: cou

Re: Frontend/Backend Protocol: SSL / GSS Protocol Negotiation Problem

2019-12-06 Thread Michael Paquier
On Fri, Dec 06, 2019 at 02:25:46PM +0100, Jakob Egger wrote: > I've received a bug report from a PostgreSQL user that psql 12.1 > failed to connect to a PostgreSQL 12.1 server, with the following > error message: > > psql: error: could not connect to server: FATAL: unsupported > frontend protocol

Re: backup manifests

2019-12-06 Thread Robert Haas
On Fri, Dec 6, 2019 at 1:35 AM Rushabh Lathia wrote: > There is one review comment from Jeevan Chalke, which still pending > to address is: > >> 4. >> Why we need a "File" at the start of each entry as we are adding files only? >> I wonder if we also need to provide a tablespace name and directory

Re: [PATCH] print help from psql when user tries to run pg_restore, pg_dump etc

2019-12-06 Thread Fabien COELHO
Hello Craig, New users frequently attempt to run PostgreSQL's command line utilities from the psql prompt. Alas, that is true. I also have the reverse, i.e. SQL commands fed to bash, which does not like it much. They tend to be confused when this appears to do absolutely nothing:

Re: 64 bit transaction id

2019-12-06 Thread Robert Haas
On Thu, Nov 7, 2019 at 10:28 AM Bruce Momjian wrote: > The above is a very good summary of the constraints that have led to our > current handling of XID wraparound. If we are concerned about excessive > vacuum freeze overhead, why is the default autovacuum_freeze_max_age = > 2 so low? T

Frontend/Backend Protocol: SSL / GSS Protocol Negotiation Problem

2019-12-06 Thread Jakob Egger
Hi! I've received a bug report from a PostgreSQL user that psql 12.1 failed to connect to a PostgreSQL 12.1 server, with the following error message: psql: error: could not connect to server: FATAL: unsupported frontend protocol 1234.5679: server supports 2.0 to 3.0 After inspecting a TCP dump

RE: [Proposal] Level4 Warnings show many shadow vars

2019-12-06 Thread Ranier Vilela
De: Mark Dilger Enviado: quinta-feira, 5 de dezembro de 2019 21:06 >I suggested increasing the default warnings in an email some time ago, >to which Tom made reasonable objections. You might want to take a >look at his comments, and consider if you can overcome the concerns >he had: I understand

Re: segmentation fault when cassert enabled

2019-12-06 Thread Amit Kapila
On Mon, Nov 25, 2019 at 8:25 PM Jehan-Guillaume de Rorthais wrote: > > On Wed, 6 Nov 2019 14:34:38 +0100 > Peter Eisentraut wrote: > > > On 2019-11-05 17:29, Jehan-Guillaume de Rorthais wrote: > > > My best bet so far is that logicalrep_relmap_invalidate_cb is not called > > > after the DDL on th

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-12-06 Thread Amit Khandekar
On Fri, 6 Dec 2019 at 15:40, Amit Kapila wrote: > > On Thu, Dec 5, 2019 at 4:20 PM Amit Kapila wrote: > > > > On Tue, Dec 3, 2019 at 11:10 AM Amit Khandekar > > wrote: > > > > > > > > > Done as stated above; attached v3 patch. I have verified that the file > > > handles do get closed in PG_CATC

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-12-06 Thread Amit Kapila
On Thu, Dec 5, 2019 at 4:20 PM Amit Kapila wrote: > > On Tue, Dec 3, 2019 at 11:10 AM Amit Khandekar wrote: > > > > > > Done as stated above; attached v3 patch. I have verified that the file > > handles do get closed in PG_CATCH block via > > ReorderBufferIterTXNFinish(). > > > > I couldn't repro

Re: Rework manipulation and structure of attribute mappings

2019-12-06 Thread Amit Langote
Thanks for the updated patch. On Wed, Dec 4, 2019 at 5:03 PM Michael Paquier wrote: > On Mon, Nov 25, 2019 at 05:55:50PM +0900, Amit Langote wrote: > > Actually, I was just suggesting that we create a new function > > convert_tuples_by_position_map() and put the logic that compares the > > two Tu

Re: Session WAL activity

2019-12-06 Thread Konstantin Knizhnik
On 06.12.2019 4:57, Michael Paquier wrote: On Thu, Dec 05, 2019 at 12:23:40PM +0300, Konstantin Knizhnik wrote: Concerning keeping PGPROC size as small as possible, I agree that it is reasonable argument. But even now it is very large (816 bytes) and adding extra 8 bytes will increase it on l

Re: Removal of support for OpenSSL 0.9.8 and 1.0.0

2019-12-06 Thread Daniel Gustafsson
> On 6 Dec 2019, at 02:33, Michael Paquier wrote: > Another argument in favor of dropping 1.0.0 and 0.9.8 is that > it is a pain to check an OpenSSL patch across that many versions, > multiplied by the number of Postgres branches in need of patching :) That is indeed a very good argument. cheer

Re: Append with naive multiplexing of FDWs

2019-12-06 Thread Kyotaro Horiguchi
At Fri, 6 Dec 2019 10:03:44 +1300, Thomas Munro wrote in > On Fri, Dec 6, 2019 at 9:20 AM Robert Haas wrote: > > I don't know whether this patch has that kind of problem. If it > > doesn't, I would consider that a promising sign. > > I'll look into that. If there is a measurable impact, I sus