Re: poc - possibility to write window function in PL languages

2021-01-20 Thread Pavel Stehule
Hi so 16. 1. 2021 v 0:09 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > [ plpgsql-window-functions-20210104.patch.gz ] > > I spent some time looking at this patch. It would certainly be > appealing to have some ability to write custom window functions > without descending into C; but

[bug?] EXPLAIN outputs 0 for rows and width in cost estimate for update nodes

2021-01-20 Thread tsunakawa.ta...@fujitsu.com
Hello, While I'm investigating problems with parallel DML on another thread, I encountered a fishy behavior of EXPLAIN on HEAD. Is this a bug? As follows, the rows and width values of Update node is 0. These were 1 and 10 respectively in versions 9.4.26 and 10.12 at hand. postgres=# creat

Re: [bug?] EXPLAIN outputs 0 for rows and width in cost estimate for update nodes

2021-01-20 Thread Thomas Munro
On Wed, Jan 20, 2021 at 9:12 PM tsunakawa.ta...@fujitsu.com wrote: > This behavior may possibly be considered as an intended behavior for the > reason that Update/Insert/Delete nodes don't output rows without RETURNING. > Is this a bug or a correct behavior? Hi Tsunakawa-san, This was a chang

RE: [bug?] EXPLAIN outputs 0 for rows and width in cost estimate for update nodes

2021-01-20 Thread tsunakawa.ta...@fujitsu.com
Hi Thomas-san, From: Thomas Munro > This was a change made deliberately. Do you see a problem? Thank you, I was surprised at your very quick response. I just wanted to confirm I can believe EXPLAIN output. Then the problem is the sample output in the manual. The fix is attached. Regards

Re: Wrong usage of RelationNeedsWAL

2021-01-20 Thread Kyotaro Horiguchi
At Tue, 19 Jan 2021 01:31:52 -0800, Noah Misch wrote in > On Tue, Jan 19, 2021 at 01:48:31PM +0900, Kyotaro Horiguchi wrote: > > I understand that you are suggesting that at least > > TransactionIdLimitedForOldSnapshots should follow not only relation > > persistence but RelationNeedsWAL, based o

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Wed, Jan 20, 2021 at 11:53 AM Fujii Masao wrote: > So, furthermore, we can use hash_search() to find the target cached > connection, instead of using hash_seq_search(), when the server name > is given. This would simplify the code a bit more? Of course, > hash_seq_search() is necessary when clo

Re: [bug?] EXPLAIN outputs 0 for rows and width in cost estimate for update nodes

2021-01-20 Thread Laurenz Albe
On Wed, 2021-01-20 at 08:35 +, tsunakawa.ta...@fujitsu.com wrote: > > This was a change made deliberately. Do you see a problem? > > Thank you, I was surprised at your very quick response. > I just wanted to confirm I can believe EXPLAIN output. > Then the problem is the sample output in the

Re: Rethinking plpgsql's assignment implementation

2021-01-20 Thread Pavel Stehule
út 19. 1. 2021 v 19:21 odesílatel Pavel Stehule napsal: > Hi > > Now, I am testing subscribing on the jsonb feature, and I found one issue, > that is not supported by parser. > > When the target is scalar, then all is ok. But we can have a plpgsql array > of jsonb values. > > postgres=# do $$ > d

Re: should INSERT SELECT use a BulkInsertState?

2021-01-20 Thread Justin Pryzby
On Sat, Dec 05, 2020 at 01:59:41PM -0600, Justin Pryzby wrote: > On Thu, Dec 03, 2020 at 10:59:34AM +0530, Bharath Rupireddy wrote: > > On Wed, Dec 2, 2020 at 10:24 PM Justin Pryzby wrote: > > > > > > One loose end in this patch is how to check for volatile default > > > expressions. > > > > I t

Re: pg_stat_statements oddity with track = all

2021-01-20 Thread Julien Rouhaud
Hi, On Tue, Jan 19, 2021 at 4:55 PM Masahiro Ikeda wrote: > > Thanks for making the patch to add "toplevel" column in > pg_stat_statements. > This is a review comment. Thanks a lot for the thorough review! > I tested the "update" command can work. > postgres=# ALTER EXTENSION pg_stat_statements

Re: [bug?] EXPLAIN outputs 0 for rows and width in cost estimate for update nodes

2021-01-20 Thread Thomas Munro
On Wed, Jan 20, 2021 at 9:42 PM Laurenz Albe wrote: > On Wed, 2021-01-20 at 08:35 +, tsunakawa.ta...@fujitsu.com wrote: > > > This was a change made deliberately. Do you see a problem? > > > > Thank you, I was surprised at your very quick response. > > I just wanted to confirm I can believe E

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/20 17:41, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 11:53 AM Fujii Masao wrote: So, furthermore, we can use hash_search() to find the target cached connection, instead of using hash_seq_search(), when the server name is given. This would simplify the code a bit more? Of cou

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-20 Thread Greg Nancarrow
On Fri, Jan 8, 2021 at 8:16 PM Amit Kapila wrote: > > > - if (pcxt->nworkers_launched > 0) > + if (pcxt->nworkers_launched > 0 && !(isParallelModifyLeader && > !isParallelModifyWithReturning)) > { > > I think this check could be simplified to if (pcxt->nworkers_launched > > 0 && isParallelModify

Re: a misbehavior of partition row movement (?)

2021-01-20 Thread Amit Langote
On Wed, Jan 20, 2021 at 4:13 PM Peter Eisentraut wrote: > On 2021-01-08 09:54, Amit Langote wrote: > >>> I don't quite recall if the decision to implement it like this was > >>> based on assuming that this is what users would like to see happen in > >>> this case or the perceived difficulty of imp

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Bharath Rupireddy
On Wed, Jan 20, 2021 at 3:24 PM Fujii Masao wrote: > > Keeping above in mind, I feel we can do hash_seq_search(), as we do > > currently, even when the server name is given as input. This way, we > > don't need to bother much on the above points. > > > > Thoughts? > > Thanks for explaining this! Y

Re: pg_stat_statements oddity with track = all

2021-01-20 Thread Masahiro Ikeda
On 2021-01-20 18:14, Julien Rouhaud wrote: On Tue, Jan 19, 2021 at 4:55 PM Masahiro Ikeda wrote: I tested the "update" command can work. postgres=# ALTER EXTENSION pg_stat_statements UPDATE TO '1.10'; Although the "toplevel" column of all queries which already stored is 'false', we have to dec

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Michael Paquier
On Mon, Jan 18, 2021 at 02:37:57AM -0600, Justin Pryzby wrote: > Attached. I will re-review these myself tomorrow. I have begun looking at 0001 and 0002... +/* + * This is mostly duplicating ATExecSetTableSpaceNoStorage, + * which should maybe be factored out to a library function. + */ Wouldn't

Re: Stronger safeguard for archive recovery not to miss data

2021-01-20 Thread Laurenz Albe
On Wed, 2021-01-20 at 13:10 +0900, Fujii Masao wrote: > +errhint("Run recovery again from a new base > backup taken after setting wal_level higher than minimal"))); > > Isn't it impossible to do this in normal archive recovery case? In that case, > since the server

Re: TOAST condition for column size

2021-01-20 Thread torikoshia
On 2021-01-19 19:32, Amit Kapila wrote: On Mon, Jan 18, 2021 at 7:53 PM torikoshia Because no benefit is to be expected by compressing it. The size will be mostly the same. Also, even if we somehow try to fit this data via toast, I think reading speed will be slower because for all such columns a

Re: Added schema level support for publication.

2021-01-20 Thread Rahila Syed
Hi Vignesh, > I have handled the above scenario(drop schema should automatically > remove the schema entry from publication schema relation) & addition > of tests in the new v2 patch attached. > Thoughts? > Please see some initial comments: 1. I think there should be more tests to show that the

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-20 Thread Fujii Masao
On 2021/01/20 19:17, Bharath Rupireddy wrote: On Wed, Jan 20, 2021 at 3:24 PM Fujii Masao wrote: Keeping above in mind, I feel we can do hash_seq_search(), as we do currently, even when the server name is given as input. This way, we don't need to bother much on the above points. Thoughts?

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-20 Thread Amit Kapila
On Wed, Jan 20, 2021 at 10:58 AM Peter Geoghegan wrote: > > On Tue, Jan 19, 2021 at 7:54 PM Amit Kapila wrote: > > The worst cases could be (a) when there is just one such duplicate > > (indexval logically unchanged) on the page and that happens to be the > > last item and others are new insertio

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-20 Thread Amit Kapila
On Wed, Jan 20, 2021 at 7:03 PM Amit Kapila wrote: > > On Wed, Jan 20, 2021 at 10:58 AM Peter Geoghegan wrote: > > > > On Tue, Jan 19, 2021 at 7:54 PM Amit Kapila wrote: > > > The worst cases could be (a) when there is just one such duplicate > > > (indexval logically unchanged) on the page and

Re: Printing backtrace of postgres processes

2021-01-20 Thread vignesh C
On Wed, Jan 20, 2021 at 2:52 AM Tom Lane wrote: > > Robert Haas writes: > > On Tue, Jan 19, 2021 at 12:50 PM Tom Lane wrote: > >> I think it's got security hazards as well. If we restricted the > >> feature to cause a trace of only one process at a time, and required > >> that process to be log

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-20 Thread Amit Kapila
On Wed, Jan 20, 2021 at 10:50 AM Andres Freund wrote: > > Hi, > > On 2021-01-20 09:24:35 +0530, Amit Kapila wrote: > > I feel extending the deletion mechanism based on the number of LP_DEAD > > items sounds more favorable than giving preference to duplicate > > items. Sure, it will give equally go

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-20 Thread Amit Kapila
On Wed, Jan 20, 2021 at 3:27 PM Greg Nancarrow wrote: > > On Fri, Jan 8, 2021 at 8:16 PM Amit Kapila wrote: > > > > > > I'm not sure if the problem of missing targetlist should be handled here > > > (BTW, > > > NIL is the constant for an empty list, not NULL). Obviously this is a > > > consequen

Re: Discarding DISCARD ALL

2021-01-20 Thread James Coleman
I hope to do further review of the patch later this week, but I wanted to at least comment on this piece: On Wed, Jan 20, 2021 at 2:48 AM Peter Eisentraut wrote: > > On 2020-12-23 15:33, Simon Riggs wrote: > > Poolers such as pgbouncer would then be able to connect transaction > > mode pools by s

Re: Discarding DISCARD ALL

2021-01-20 Thread Simon Riggs
On Wed, 20 Jan 2021 at 14:21, James Coleman wrote: > An alternative approach that occurred to me while typing this reply: a > setting in Postgres that would disallow setting session level GUCs > (i.e., enforce `SET LOCAL` transaction level usage instead) would > remove a large chunk of our need t

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-20 Thread Dian M Fay
On Tue Jan 19, 2021 at 1:42 PM EST, Pavel Stehule wrote: > Hi > > I found minor issues. > > Doc - missing tag > > and three whitespaces issues > > see attached patch > > Following sentence is hard to read due long nested example > > If the > + path contradicts structure of modified jsonb for any >

Re: Wrong usage of RelationNeedsWAL

2021-01-20 Thread Kyotaro Horiguchi
At Wed, 20 Jan 2021 17:34:44 +0900 (JST), Kyotaro Horiguchi wrote in > Anyway, it seems actually dangerous that cause pruning on wal-skipped > relation. > > > with your patch versions. Could you try implementing both test procedures > > in > > src/test/modules/snapshot_too_old? There's no ne

Re: pg_stat_statements oddity with track = all

2021-01-20 Thread Masahiko Sawada
On Wed, Jan 20, 2021 at 6:15 PM Julien Rouhaud wrote: > > Hi, > > On Tue, Jan 19, 2021 at 4:55 PM Masahiro Ikeda > wrote: > > > > Thanks for making the patch to add "toplevel" column in > > pg_stat_statements. > > This is a review comment. > > Thanks a lot for the thorough review! > > > I tested

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alvaro Herrera
On 2021-Jan-20, Michael Paquier wrote: > +/* > + * This is mostly duplicating ATExecSetTableSpaceNoStorage, > + * which should maybe be factored out to a library function. > + */ > Wouldn't it be better to do first the refactoring of 0002 and then > 0001 so as REINDEX can use the new routine, inst

Re: Discarding DISCARD ALL

2021-01-20 Thread James Coleman
On Wed, Jan 20, 2021 at 9:58 AM Simon Riggs wrote: > > On Wed, 20 Jan 2021 at 14:21, James Coleman wrote: > > > An alternative approach that occurred to me while typing this reply: a > > setting in Postgres that would disallow setting session level GUCs > > (i.e., enforce `SET LOCAL` transaction

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alvaro Herrera
On 2021-Jan-20, Alvaro Herrera wrote: > On 2021-Jan-20, Michael Paquier wrote: > > > +/* > > + * This is mostly duplicating ATExecSetTableSpaceNoStorage, > > + * which should maybe be factored out to a library function. > > + */ > > Wouldn't it be better to do first the refactoring of 0002 and th

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-20 Thread Dmitry Dolgov
> On Tue Jan 19, 2021 at 1:42 PM EST, Pavel Stehule wrote: > > I found minor issues. > > Doc - missing tag > > and three whitespaces issues > > see attached patch Thanks, I need to remember to not skipp doc building for testing process even for such small changes. Hope now I didn't forget anything

Getting column names/types from select query?

2021-01-20 Thread Wesley Aptekar-Cassels
Hi all, I am interested in figuring out how to get the names and types of the columns from an arbitrary query. Essentially, I want to be able to take a query like: CREATE TABLE foo( bar bigserial, baz varchar(256) ); SELECT * FROM foo WHERE bar = 42; and figure out programmatically tha

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-20 Thread Dian M Fay
On Wed Jan 20, 2021 at 11:22 AM EST, Dmitry Dolgov wrote: > > On Tue Jan 19, 2021 at 1:42 PM EST, Pavel Stehule wrote: > > > > I found minor issues. > > > > Doc - missing tag > > > > and three whitespaces issues > > > > see attached patch > > Thanks, I need to remember to not skipp doc building for

Re: Getting column names/types from select query?

2021-01-20 Thread Tom Lane
"Wesley Aptekar-Cassels" writes: > I am interested in figuring out how to get the names and types of the > columns from an arbitrary query. Where do you need this information? Usually the easiest way is to prepare (plan) the query and then extract metadata, for instance PQprepare and PQdescribeP

Re: Odd, intermittent failure in contrib/pageinspect

2021-01-20 Thread Tom Lane
Michael Paquier writes: > On Tue, Jan 19, 2021 at 05:03:49PM -0500, Tom Lane wrote: >> In short I propose the attached patch, which also gets rid of >> that duplicate query. > Agreed, +1. Pushed, thanks for looking at it. regards, tom lane

strange error reporting

2021-01-20 Thread Robert Haas
I just made the mistake of trying to run pgbench without first running createdb and got this: pgbench: error: connection to database "" failed: could not connect to socket "/tmp/.s.PGSQL.5432": FATAL: database "rhaas" does not exist This looks pretty bogus because (1) I was not attempting to con

Jsonpath ** vs lax mode

2021-01-20 Thread Alexander Korotkov
Hi! We have a bug report which says that jsonpath ** operator behaves strangely in the lax mode [1]. Naturally, the result of this query looks counter-intuitive. # select jsonb_path_query_array('[{"a": 1, "b": [{"a": 2}]}]', 'lax $.**.a'); jsonb_path_query_array [1, 1,

Re: strange error reporting

2021-01-20 Thread Tom Lane
Robert Haas writes: > I just made the mistake of trying to run pgbench without first running > createdb and got this: > pgbench: error: connection to database "" failed: could not connect to > socket "/tmp/.s.PGSQL.5432": FATAL: database "rhaas" does not exist > This looks pretty bogus because

Re: Phrase search vs. multi-lexeme tokens

2021-01-20 Thread Alexander Korotkov
On Thu, Jan 7, 2021 at 6:36 AM Alexander Korotkov wrote: > > > I read your patch over quickly and it seems like a reasonable > > approach (but sadly underdocumented). Can we extend the idea > > to fix the to_tsquery case? > > Sure, I'll provide a revised patch. The next version of the patch is a

Re: strange error reporting

2021-01-20 Thread Robert Haas
On Wed, Jan 20, 2021 at 12:19 PM Tom Lane wrote: > Robert Haas writes: > > I just made the mistake of trying to run pgbench without first running > > createdb and got this: > > > pgbench: error: connection to database "" failed: could not connect to > > socket "/tmp/.s.PGSQL.5432": FATAL: databa

Re: Getting column names/types from select query?

2021-01-20 Thread Wesley Aptekar-Cassels
> Where do you need this information? I'm writing some code that takes a given query, and generates type-safe bindings for it, so people can write SQL queries and get structs (or vectors of structs) out the other end. So I'm pretty flexible about where I get it, given that it'll be part of my b

Re: strange error reporting

2021-01-20 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 20, 2021 at 12:19 PM Tom Lane wrote: >> But the 'could not connect to socket' part is a consequence of my >> recent fiddling with libpq's connection failure reporting, see >> 52a10224e. We could discuss exactly how that ought to be spelled, >> but the idea is to

Re: strange error reporting

2021-01-20 Thread Robert Haas
On Wed, Jan 20, 2021 at 12:47 PM Tom Lane wrote: > Fair. One possibility, which'd take a few more cycles in libpq but > likely not anything significant, is to replace "could not connect to ..." > with "while connecting to ..." once we're past the connect() per se. Yeah. I think this is kind of a

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alexey Kondratov
On 2021-01-20 18:54, Alvaro Herrera wrote: On 2021-Jan-20, Alvaro Herrera wrote: On 2021-Jan-20, Michael Paquier wrote: > +/* > + * This is mostly duplicating ATExecSetTableSpaceNoStorage, > + * which should maybe be factored out to a library function. > + */ > Wouldn't it be better to do firs

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alexey Kondratov
On 2021-01-20 21:08, Alexey Kondratov wrote: On 2021-01-20 18:54, Alvaro Herrera wrote: On 2021-Jan-20, Alvaro Herrera wrote: On 2021-Jan-20, Michael Paquier wrote: > +/* > + * This is mostly duplicating ATExecSetTableSpaceNoStorage, > + * which should maybe be factored out to a library funct

Re: Jsonpath ** vs lax mode

2021-01-20 Thread Alvaro Herrera
On 2021-Jan-20, Alexander Korotkov wrote: > My proposal is to make everything after the ** operator use strict mode > (patch attached). I think this shouldn't be backpatched, just applied to > the v14. Other suggestions? I think changing the mode midway through the operation is strange. What d

Re: strange error reporting

2021-01-20 Thread Alvaro Herrera
On 2021-Jan-20, Robert Haas wrote: > On Wed, Jan 20, 2021 at 12:19 PM Tom Lane wrote: > > Robert Haas writes: > > > I just made the mistake of trying to run pgbench without first running > > > createdb and got this: > > > > > pgbench: error: connection to database "" failed: could not connect to

Re: strange error reporting

2021-01-20 Thread Robert Haas
On Wed, Jan 20, 2021 at 1:25 PM Alvaro Herrera wrote: > That's because pgbench reports the input argument dbname, but since you > didn't specify anything, then PQconnectdbParams() uses the libpq > behavior. I think we'd have to use PQdb() instead. I figured it was something like that. I don't kn

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Alvaro Herrera
On 2021-Jan-20, Alexey Kondratov wrote: > On 2021-01-20 21:08, Alexey Kondratov wrote: > > > > I did a refactoring of ATExecSetTableSpaceNoStorage() in the 0001. New > > function SetRelTablesapce() is placed into the tablecmds.c. Following > > 0002 gets use of it. Is it close to what you and Mich

Re: strange error reporting

2021-01-20 Thread Alvaro Herrera
On 2021-Jan-20, Robert Haas wrote: > On Wed, Jan 20, 2021 at 1:25 PM Alvaro Herrera > wrote: > > That's because pgbench reports the input argument dbname, but since you > > didn't specify anything, then PQconnectdbParams() uses the libpq > > behavior. I think we'd have to use PQdb() instead. >

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-20 Thread Peter Geoghegan
On Wed, Jan 20, 2021 at 5:33 AM Amit Kapila wrote: > > Victor independently came up with a benchmark that ran over several > > hours, with cleanup consistently held back by ~5 minutes by a long > > running transaction: > > > > AFAICS, the long-running transaction used in the test is below: > SELEC

Re: strange error reporting

2021-01-20 Thread Tom Lane
Alvaro Herrera writes: > On 2021-Jan-20, Robert Haas wrote: >> I figured it was something like that. I don't know whether the right >> thing is to use something like PQdb() to get the correct database >> name, or whether we should go with Tom's suggestion and omit that >> detail altogether, but I

catalogs.sgml documentation ambiguity

2021-01-20 Thread Joel Jacobson
Some catalog tables have references to pg_attribute.attnum. In the documentation, it only says "(references pg_attribute.attnum)" but not which oid column to include in the two-column "foreign key". This would not be a problem if there would only be one reference to pg_class.oid, but some catalo

Re: Allow matching whole DN from a client certificate

2021-01-20 Thread Jacob Champion
On Mon, 2021-01-18 at 11:23 +0100, Daniel Gustafsson wrote: > + /* use commas instead of slashes */ > + X509_NAME_print_ex(bio, x509name, 0, XN_FLAG_SEP_COMMA_PLUS); > I don't have strong opinions on whether we shold use slashes or commas, but I > think it needs to be do

Re: [HACKERS] [PATCH] Generic type subscripting

2021-01-20 Thread Dmitry Dolgov
> On Wed, Jan 20, 2021 at 11:34:16AM -0500, Dian M Fay wrote: > > Thanks, I need to remember to not skipp doc building for testing process > > even for such small changes. Hope now I didn't forget anything. > > > > > On Wed, Jan 20, 2021 at 09:58:43AM -0500, Dian M Fay wrote: > > > > > Here's a ful

Re: Calculation of relids (pull_varnos result) for PlaceHolderVars

2021-01-20 Thread Tom Lane
I wrote: > ... > 2. pull_varnos() is not passed the planner "root" data structure, > so it can't get at the PlaceHolderInfo list. We can change its > API of course, but that propagates to dozens of places. > ... > The 0001 patch attached goes ahead and makes those API changes. > I think this is pe

Re: strange error reporting

2021-01-20 Thread Robert Haas
On Wed, Jan 20, 2021 at 1:54 PM Tom Lane wrote: > Alvaro Herrera writes: > > On 2021-Jan-20, Robert Haas wrote: > >> I figured it was something like that. I don't know whether the right > >> thing is to use something like PQdb() to get the correct database > >> name, or whether we should go with

Re: poc - possibility to write window function in PL languages

2021-01-20 Thread Tom Lane
Pavel Stehule writes: > The second question is work with partition context value. This should be > only one value, and of only one but of any type per function. In this case > we cannot use GET statements. I had an idea of enhancing declaration. Some > like > DECLARE > pcx PARTITION CONTEXT (in

Re: poc - possibility to write window function in PL languages

2021-01-20 Thread Pavel Stehule
st 20. 1. 2021 v 21:07 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > The second question is work with partition context value. This should be > > only one value, and of only one but of any type per function. In this > case > > we cannot use GET statements. I had an idea of enhancing de

Re: poc - possibility to write window function in PL languages

2021-01-20 Thread Tom Lane
Pavel Stehule writes: > st 20. 1. 2021 v 21:07 odesílatel Tom Lane napsal: >> Uh, what? I don't understand what this "partition context" is. > It was my name for an access to window partition local memory - > WinGetPartitionLocalMemory Ah. > We need some interface for this cache I'm not conv

Re: poc - possibility to write window function in PL languages

2021-01-20 Thread Pavel Stehule
st 20. 1. 2021 v 21:32 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > st 20. 1. 2021 v 21:07 odesílatel Tom Lane napsal: > >> Uh, what? I don't understand what this "partition context" is. > > > It was my name for an access to window partition local memory - > > WinGetPartitionLocalMe

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-20 Thread Peter Geoghegan
On Wed, Jan 20, 2021 at 10:53 AM Peter Geoghegan wrote: > This patch is unusual in that you really need to think about emergent > behaviors to understand it. That is certainly a difficult thing to do, > and it's understandable that even an expert might not grok it without > considering it carefull

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-20 Thread James Hilliard
On Tue, Jan 19, 2021 at 6:37 PM Tom Lane wrote: > > James Hilliard writes: > > Actually, this looks path looks wrong in general, the value for > > "xcrun --sdk macosx --show-sdk-path" should take precedence over > > "xcrun --show-sdk-path" as the latter may be used for IOS potentially. > > What i

Re: list of extended statistics on psql

2021-01-20 Thread Tomas Vondra
On 1/20/21 7:41 AM, Tatsuro Yamada wrote: Hi Tomas, On 2021/01/20 11:35, Tatsuro Yamada wrote: Apologies for all the extra work - I haven't realized this flaw when pushing for showing more stuff :-( Don't worry about it. We didn't notice the problem even when viewed by multiple people on

Re: pg_upgrade fails with non-standard ACL

2021-01-20 Thread Anastasia Lubennikova
On 03.01.2021 14:29, Noah Misch wrote: On Thu, Jun 11, 2020 at 07:58:43PM +0300, Anastasia Lubennikova wrote: On 08.06.2020 19:31, Alvaro Herrera wrote: I'm thinking what's a good way to have a test that's committable. Maybe it would work to add a TAP test to pg_upgrade that runs initdb, does

Re: ResourceOwner refactoring

2021-01-20 Thread Heikki Linnakangas
On 19/01/2021 11:09, Heikki Linnakangas wrote: On 18/01/2021 18:11, Robert Haas wrote: On Mon, Jan 18, 2021 at 11:11 AM Robert Haas wrote: On Mon, Jan 18, 2021 at 10:19 AM Heikki Linnakangas wrote: On 18/01/2021 16:34, Alvaro Herrera wrote: So according to your performance benchmark, we're

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
OK, pushed after a little bit of additional polishing (mostly comments). Thanks everyone! -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-20 Thread Tom Lane
James Hilliard writes: > On Tue, Jan 19, 2021 at 6:37 PM Tom Lane wrote: >> I've found no direct means to control the >> SDK path at all, but so far it appears that "xcrun --show-sdk-path" >> agrees with the compiler's default -isysroot path as seen in the >> compiler's -v output. I suspect that

Re: Allow matching whole DN from a client certificate

2021-01-20 Thread Jacob Champion
On Wed, 2021-01-20 at 19:07 +, Jacob Champion wrote: > I think you'll want to be careful to specify the format as much as > possible, both to make sure that other backend TLS implementations can > actually use the same escaping system and to ensure that user regexes > don't suddenly start match

Re: [PATCH 1/1] Fix detection of pwritev support for OSX.

2021-01-20 Thread James Hilliard
On Wed, Jan 20, 2021 at 4:07 PM Tom Lane wrote: > > James Hilliard writes: > > On Tue, Jan 19, 2021 at 6:37 PM Tom Lane wrote: > >> I've found no direct means to control the > >> SDK path at all, but so far it appears that "xcrun --show-sdk-path" > >> agrees with the compiler's default -isysroot

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
Hmm, seems that florican doesn't like this :-( https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2021-01-20%2023%3A08%3A15 It's a i386 machine running FreeBSD, so not sure what exactly it's picky about. But when I tried running this under valgrind, I get some strange failures

Re: list of extended statistics on psql

2021-01-20 Thread Tatsuro Yamada
Hi Tomas and hackers, On 2021/01/21 7:00, Tomas Vondra wrote: I created patches and my test results on PG10, 11, 12, and 14 are fine.    0001: - Fix query to use pg_statistic_ext only - Replace statuses "required" and "built" with "defined" - Remove the size columns - Fix do

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tom Lane
Tomas Vondra writes: > OK, pushed after a little bit of additional polishing (mostly comments). > Thanks everyone! florican reports this is seriously broken on 32-bit hardware: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2021-01-20%2023%3A08%3A15 First guess is incorrect

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Zhihong Yu
Hi, The assignment to resultRelInfo is done when junk_filter_needed is true: if (junk_filter_needed) { resultRelInfo = mtstate->resultRelInfo; Should the code for determining batch size access mtstate->resultRelInfo directly ? diff --git a/src/backend/executor/nodeMod

Heap's backwards scan scans the incorrect pages with heap_setscanlimits()

2021-01-20 Thread David Rowley
Hackers, It looks like both heapgettup() and heapgettup_pagemode() are coded incorrectly when setting the page to start the scan on for a backwards scan when heap_setscanlimits() has been used. It looks like the code was not updated during 7516f5259. The current code is: /* start from last page

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
On 1/21/21 12:52 AM, Tomas Vondra wrote: Hmm, seems that florican doesn't like this :-( https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2021-01-20%2023%3A08%3A15 It's a i386 machine running FreeBSD, so not sure what exactly it's picky about. But when I tried running this

Re: Printing LSN made easy

2021-01-20 Thread Kyotaro Horiguchi
At Wed, 20 Jan 2021 16:40:59 +0900, Michael Paquier wrote in > On Wed, Jan 20, 2021 at 07:25:37AM +0100, Peter Eisentraut wrote: > > It looks like we are not getting any consensus on this approach. One > > reduced version I would consider is just the second part, so you'd write > > something li

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
On 1/21/21 12:59 AM, Tom Lane wrote: Tomas Vondra writes: OK, pushed after a little bit of additional polishing (mostly comments). Thanks everyone! florican reports this is seriously broken on 32-bit hardware: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=florican&dt=2021-01-20%202

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
On 1/21/21 1:17 AM, Zhihong Yu wrote: Hi, The assignment to resultRelInfo is done when junk_filter_needed is true:         if (junk_filter_needed)         {             resultRelInfo = mtstate->resultRelInfo; Should the code for determining batch size access mtstate->resultRelInfo direct

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Zhihong Yu
Hi, Tomas: In my opinion, my patch is a little better. Suppose one of the conditions in the if block changes in between the start of loop and the end of the loop: * Determine if the FDW supports batch insert and determine the batch * size (a FDW may support batching, but it may be disabl

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
On 1/21/21 2:02 AM, Zhihong Yu wrote: Hi, Tomas: In my opinion, my patch is a little better. Suppose one of the conditions in the if block changes in between the start of loop and the end of the loop:      * Determine if the FDW supports batch insert and determine the batch      * size (a

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Zhihong Yu
Hi, Do we need to consider how this part of code inside ExecInitModifyTable() would evolve ? I think placing the compound condition toward the end of ExecInitModifyTable() is reasonable because it checks the latest information. Regards On Wed, Jan 20, 2021 at 5:11 PM Tomas Vondra wrote: > > >

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tom Lane
Tomas Vondra writes: > I may be wrong, but the most likely explanation seems to be this is due > to the junk filter initialization, which simply moves past the end of > the mtstate->resultRelInfo array. resultRelInfo is certainly pointing at garbage at that point. > It kinda seems the GetForei

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Amit Langote
On Thu, Jan 21, 2021 at 9:56 AM Tomas Vondra wrote: > On 1/21/21 1:17 AM, Zhihong Yu wrote: > > Hi, > > The assignment to resultRelInfo is done when junk_filter_needed is true: > > > > if (junk_filter_needed) > > { > > resultRelInfo = mtstate->resultRelInfo; > > > >

Re: strange error reporting

2021-01-20 Thread Tom Lane
Robert Haas writes: >>> Maybe it would be better if it said: >>> connection to database at socket "/tmp/.s.PGSQL.5432" failed: FATAL: >>> database "rhaas" does not exist >> I'd be inclined to spell it "connection to server at ... failed", >> but that sort of wording is surely also possible. > "c

Re: Printing backtrace of postgres processes

2021-01-20 Thread Craig Ringer
On Wed, 20 Jan 2021 at 01:31, Robert Haas wrote: > On Sat, Jan 16, 2021 at 3:21 PM Tom Lane wrote: > > I'd argue that backtraces for those processes aren't really essential, > > and indeed that trying to make the syslogger report its own backtrace > > is damn dangerous. > > I agree. Ideally I'd

RE: POC: postgres_fdw insert batching

2021-01-20 Thread tsunakawa.ta...@fujitsu.com
From: Zhihong Yu > Do we need to consider how this part of code inside ExecInitModifyTable() > would evolve ? > I think placing the compound condition toward the end of > ExecInitModifyTable() is reasonable because it checks the latest information. +1 for Zaihong-san's idea. But instead of re

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2021-01-20 Thread Michael Paquier
On Wed, Jan 20, 2021 at 03:34:39PM -0300, Alvaro Herrera wrote: > On 2021-Jan-20, Alexey Kondratov wrote: >> Ugh, forgot to attach the patches. Here they are. > > Yeah, looks reasonable. Patch 0002 still has a whole set of issues as I pointed out a couple of hours ago, but if we agree on 0001 as

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
On 1/21/21 2:24 AM, Amit Langote wrote: On Thu, Jan 21, 2021 at 9:56 AM Tomas Vondra wrote: On 1/21/21 1:17 AM, Zhihong Yu wrote: Hi, The assignment to resultRelInfo is done when junk_filter_needed is true: if (junk_filter_needed) { resultRelInfo = mtstate-

Re: Printing backtrace of postgres processes

2021-01-20 Thread Craig Ringer
On Wed, 20 Jan 2021 at 05:23, Tom Lane wrote: > > Recursion is scary, but it should (I think) not be possible if this > is driven off CHECK_FOR_INTERRUPTS. There will certainly be none > of those in libbacktrace. > We can also hold interrupts for the call, and it might be wise to do so. One po

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Tomas Vondra
On 1/21/21 2:22 AM, Tom Lane wrote: Tomas Vondra writes: I may be wrong, but the most likely explanation seems to be this is due to the junk filter initialization, which simply moves past the end of the mtstate->resultRelInfo array. resultRelInfo is certainly pointing at garbage at that po

RE: Parallel INSERT (INTO ... SELECT ...)

2021-01-20 Thread Hou, Zhijie
> > Thanks for the feedback. > Posting an updated set of patches. Changes are based on feedback, as detailed > below: Hi It seems there are some previous comments[1][2] not addressed in current patch. Just to make sure it's not missed. [1] https://www.postgresql.org/message-id/77e1c06ffb2240838e

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Zhihong Yu
Hi, Takayuki-san: My first name is Zhihong. You can call me Ted if you want to save some typing :-) Cheers On Wed, Jan 20, 2021 at 5:37 PM tsunakawa.ta...@fujitsu.com < tsunakawa.ta...@fujitsu.com> wrote: > From: Zhihong Yu > > > Do we need to consider how this part of code inside > ExecInitMo

RE: POC: postgres_fdw insert batching

2021-01-20 Thread tsunakawa.ta...@fujitsu.com
From: Tomas Vondra > Right. But I think Tom is right this should initialize ri_BatchSize for all > the > resultRelInfo elements, not just the first one. Per the attached patch, which > resolves the issue both on x86_64 and armv7l for me. I think Your patch is perfect in the sense that it's ready

Re: POC: postgres_fdw insert batching

2021-01-20 Thread Amit Langote
On Thu, Jan 21, 2021 at 10:42 AM Tomas Vondra wrote: > On 1/21/21 2:24 AM, Amit Langote wrote: > > On Thu, Jan 21, 2021 at 9:56 AM Tomas Vondra > > wrote: > >> On 1/21/21 1:17 AM, Zhihong Yu wrote: > >>> Hi, > >>> The assignment to resultRelInfo is done when junk_filter_needed is true: > >>> > >>

Re: Printing backtrace of postgres processes

2021-01-20 Thread Tom Lane
Craig Ringer writes: > On Wed, 20 Jan 2021 at 05:23, Tom Lane wrote: >> BTW, it also looks like the patch is doing nothing to prevent the >> backtrace from being sent to the connected client. > I don't see a good reason to send a bt to a client. Even though these > backtraces won't be analysing

  1   2   >