Re: logical replication empty transactions

2021-04-14 Thread Ajin Cherian
On Thu, Apr 15, 2021 at 1:29 PM Ajin Cherian wrote: > > I've rebased the patch and made changes so that the patch supports > "streaming in-progress transactions" and handling of logical decoding > messages (transactional and non-transactional). > I see that this patch not only makes sure that emp

Re: Replication slot stats misgivings

2021-04-14 Thread Amit Kapila
On Wed, Apr 14, 2021 at 5:52 PM vignesh C wrote: > I have made minor changes to the 0001 and 0002 patches. Attached is the combined patch for them, I think we can push them as one patch. Changes made are (a) minor editing in comments, (b) changed the condition when to report stats such that unles

Re: Can a child process detect postmaster death when in pg_usleep?

2021-04-14 Thread Bharath Rupireddy
On Thu, Apr 15, 2021 at 5:28 AM Thomas Munro wrote: Thanks a lot for the detailed explanation. > On Thu, Apr 15, 2021 at 2:06 AM Bharath Rupireddy > wrote: > > 1) Is it really harmful to use pg_usleep in a postmaster child process > > as it doesn't let the child process detect postmaster death?

Re: File truncation within PostgresNode::issues_sql_like() wrong on Windows

2021-04-14 Thread Michael Paquier
On Wed, Apr 14, 2021 at 09:26:19PM -0400, Andrew Dunstan wrote: > Well, let me try it on fairywren tomorrow. Since we manage this on the > buildfarm without any use at all of Win32API::File it might not be > necessary in TAP code either, particularly if we're not truncating the file. Thanks. If t

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Peter Geoghegan
On Wed, Apr 14, 2021 at 8:38 PM Andres Freund wrote: > The reason I didn't do further reviews for things in this thread was > that I was trying really hard to get the shared memory stats patch into > a committable shape - there were just not enough hours in the day. I > think it's to be expected t

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Andres Freund
Hi, On 2021-04-14 19:53:29 -0700, Peter Geoghegan wrote: > > Or at least > > tests for it should be added (pg_resetwal + autovacuum_naptime=1s or > > such should make that doable, or even just running a small test with > > lower thresholds). > > You know what else doesn't have test coverage? Any k

Re: logical replication empty transactions

2021-04-14 Thread Ajin Cherian
On Thu, Sep 17, 2020 at 3:29 PM Michael Paquier wrote: > On Wed, Jul 29, 2020 at 08:08:06PM +0530, Rahila Syed wrote: > > The make check passes. > > Since then, the patch is failing to apply, waiting on author and the > thread has died 6 weeks or so ago, so I am marking it as RwF in the > CF. > >

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Peter Geoghegan
On Wed, Apr 14, 2021 at 6:53 PM Andres Freund wrote: > > To a large degree the failsafe is something that is written in the > > hope that it will never be needed. This is unlike most other things, > > and has its own unique risks. > > Among them that the code is not covered by tests and is unlikel

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread James Coleman
On Wed, Apr 14, 2021 at 5:42 PM James Coleman wrote: > > On Mon, Apr 12, 2021 at 8:37 AM Tomas Vondra > wrote: > > > > On 4/12/21 2:24 PM, Luc Vlaming wrote: > > > Hi, > > > > > > When trying to run on master (but afaik also PG-13) TPC-DS queries 94, > > > 95 and 96 on a SF10 I get the error "cou

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Andres Freund
Hi, On 2021-04-14 14:55:36 -0700, Peter Geoghegan wrote: > On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote: > > I'm getting a bit bothered by the speed at which you're pushing fairly > > substantial behavioural for vacuum. In this case without even a warning > > that you're about to do so. >

Re: psql - add SHOW_ALL_RESULTS option

2021-04-14 Thread Tom Lane
Michael Paquier writes: > On Mon, Apr 12, 2021 at 03:33:01PM -0400, Alvaro Herrera wrote: >> I, for one, would prefer to see the feature repaired in this cycle. > If it is possible to get that fixed, I would not mind waiting a bit > more but it would be nice to see some actual proposals. There a

Replacing pg_depend PIN entries with a fixed range check

2021-04-14 Thread Tom Lane
In [1] Andres and I speculated about whether we really need all those PIN entries in pg_depend. Here is a draft patch that gets rid of them. It turns out to be no big problem to replace the PIN entries with an OID range check, because there's a well-defined point in initdb where it wants to pin (

Re: Proposal: Save user's original authenticated identity for logging

2021-04-14 Thread Michael Paquier
On Tue, Apr 13, 2021 at 03:47:21PM +, Jacob Champion wrote: > Looks like the farm has gone green, after some test fixups. Thanks for > all the reviews! You may want to follow this thread as well, as the topic is related to what has been discussed on this thread as there is an impact in a diffe

Re: File truncation within PostgresNode::issues_sql_like() wrong on Windows

2021-04-14 Thread Andrew Dunstan
On 4/14/21 8:10 PM, Michael Paquier wrote: > On Wed, Apr 14, 2021 at 05:10:41PM -0400, Andrew Dunstan wrote: >> That seems rather heavy-handed. The buildfarm's approach is a bit >> different. Essentially it seeks to the previous position of the log file >> before reading contents. Here is its equ

Re: psql - add SHOW_ALL_RESULTS option

2021-04-14 Thread Michael Paquier
On Mon, Apr 12, 2021 at 03:33:01PM -0400, Alvaro Herrera wrote: > Please note that there's no "for now" about it -- if the patch is > reverted, the only way to get it back is to wait for PG15. That's > undesirable. A better approach is to collect all those bugs and get > them fixed. There's plen

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread James Coleman
On Wed, Apr 14, 2021 at 8:21 PM Robert Haas wrote: > > On Wed, Apr 14, 2021 at 5:43 PM James Coleman wrote: > > The query in question is: > > select count(*) > > from store_sales > > ,household_demographics > > ,time_dim, store > > where ss_sold_time_sk =

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread James Coleman
On Wed, Apr 14, 2021 at 8:21 PM Robert Haas wrote: > > On Wed, Apr 14, 2021 at 5:43 PM James Coleman wrote: > > The query in question is: > > select count(*) > > from store_sales > > ,household_demographics > > ,time_dim, store > > where ss_sold_time_sk =

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Andres Freund
Hi, On 2021-04-14 20:08:10 -0400, Robert Haas wrote: > On Wed, Apr 14, 2021 at 5:55 PM Peter Geoghegan wrote: > > On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote: > > > I'm getting a bit bothered by the speed at which you're pushing fairly > > > substantial behavioural for vacuum. In this c

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 8:20 PM James Coleman wrote: > > Hmm, could be. Although, the stack trace at issue doesn't seem to show > > a call to create_incrementalsort_plan(). > > The changes to gather merge path generation made it possible to use > those paths in more cases for both incremental sort

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 5:43 PM James Coleman wrote: > The query in question is: > select count(*) > from store_sales > ,household_demographics > ,time_dim, store > where ss_sold_time_sk = time_dim.t_time_sk > and ss_hdemo_sk = household_demogra

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Peter Geoghegan
On Wed, Apr 14, 2021 at 5:08 PM Robert Haas wrote: > I think this is largely missing the point Andres was making, which is > that you made a significant behavior change after feature freeze > without any real opportunity for discussion. I don't believe that it was a significant behavior change, f

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread James Coleman
On Wed, Apr 14, 2021 at 8:16 PM Robert Haas wrote: > > On Mon, Apr 12, 2021 at 8:37 AM Tomas Vondra > wrote: > > Could be related to incremental sort, which allowed some gather merge > > paths that were impossible before. We had a couple issues related to > > that fixed in November, IIRC. > > Hmm

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Vik Fearing
On 4/15/21 12:18 AM, Mark Dilger wrote: > > >> On Apr 14, 2021, at 2:47 PM, Vik Fearing wrote: >> >> On 4/14/21 7:36 PM, Tom Lane wrote: >>> Mark Dilger writes: > On Apr 13, 2021, at 3:26 PM, Tom Lane wrote: > However I think we may still need an assumption that earthdistance > and

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread Robert Haas
On Mon, Apr 12, 2021 at 8:37 AM Tomas Vondra wrote: > Could be related to incremental sort, which allowed some gather merge > paths that were impossible before. We had a couple issues related to > that fixed in November, IIRC. Hmm, could be. Although, the stack trace at issue doesn't seem to show

Re: File truncation within PostgresNode::issues_sql_like() wrong on Windows

2021-04-14 Thread Michael Paquier
On Wed, Apr 14, 2021 at 05:10:41PM -0400, Andrew Dunstan wrote: > That seems rather heavy-handed. The buildfarm's approach is a bit > different. Essentially it seeks to the previous position of the log file > before reading contents. Here is its equivalent of slurp_file: > > use Fcntl qw(:seek

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 5:55 PM Peter Geoghegan wrote: > On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote: > > I'm getting a bit bothered by the speed at which you're pushing fairly > > substantial behavioural for vacuum. In this case without even a warning > > that you're about to do so. > >

Re: Can a child process detect postmaster death when in pg_usleep?

2021-04-14 Thread Thomas Munro
Hi Bharath, On Thu, Apr 15, 2021 at 2:06 AM Bharath Rupireddy wrote: > 1) Is it really harmful to use pg_usleep in a postmaster child process > as it doesn't let the child process detect postmaster death? Yeah, that's a bad idea. Any long-term waiting (including short waits in a loop) should id

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Mark Dilger
> On Apr 14, 2021, at 2:47 PM, Vik Fearing wrote: > > On 4/14/21 7:36 PM, Tom Lane wrote: >> Mark Dilger writes: On Apr 13, 2021, at 3:26 PM, Tom Lane wrote: However I think we may still need an assumption that earthdistance and cube are in the same schema --- any comments on

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Peter Geoghegan
On Wed, Apr 14, 2021 at 12:33 PM Andres Freund wrote: > I'm getting a bit bothered by the speed at which you're pushing fairly > substantial behavioural for vacuum. In this case without even a warning > that you're about to do so. To a large degree the failsafe is something that is written in the

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Vik Fearing
On 4/14/21 7:36 PM, Tom Lane wrote: > Mark Dilger writes: >>> On Apr 13, 2021, at 3:26 PM, Tom Lane wrote: >>> However I think we may still need an assumption that earthdistance >>> and cube are in the same schema --- any comments on that? > >> This is probably not worth doing, and we are alread

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-14 Thread James Coleman
On Mon, Apr 12, 2021 at 8:37 AM Tomas Vondra wrote: > > On 4/12/21 2:24 PM, Luc Vlaming wrote: > > Hi, > > > > When trying to run on master (but afaik also PG-13) TPC-DS queries 94, > > 95 and 96 on a SF10 I get the error "could not find pathkey item to sort". > > When I disable enable_gathermerge

Re: File truncation within PostgresNode::issues_sql_like() wrong on Windows

2021-04-14 Thread Andrew Dunstan
On 4/14/21 4:13 AM, Michael Paquier wrote: > Hi all, > > As fairywren has proved a couple of days ago, it is not really a good > idea to rely on a file truncation to check for patterns in the logs of > the backend: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-04-07%

Re: Possible typo/unclear comment in joinpath.c

2021-04-14 Thread James Coleman
On Wed, Apr 14, 2021 at 1:27 PM Tom Lane wrote: > > Justin Pryzby writes: > > On Wed, Apr 14, 2021 at 11:36:38AM -0400, James Coleman wrote: > >> In joinpath.c three times we reference "extra_lateral_rels" (with > >> underscores like it's a field), but as far as I can tell that's not a > >> field

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Tom Lane
Andrew Dunstan writes: > On 4/14/21 2:03 PM, Tom Lane wrote: >> This may mean that squeezing these contrib changes into v14 is a lost >> cause. We certainly shouldn't try to do what I suggest above for >> v14; but without it, these changes are just moving the security >> issue to a different plac

Re: Possible typo/unclear comment in joinpath.c

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 2:32 PM Tom Lane wrote: > No, I take that back. There were no references to extra_lateral_rels > after that commit; these comments were added by 45be99f8c, about > six weeks later. The latter was a pretty large patch and had > presumably been under development for quite s

Re: New IndexAM API controlling index vacuum strategies

2021-04-14 Thread Andres Freund
Hi, On 2021-04-13 12:59:03 -0700, Peter Geoghegan wrote: > I agree. Bypassing heap truncation is exactly the kind of thing that > risks adding significant, unpredictable delay at a time when we need > to advance relfrozenxid as quickly as possible. > > I pushed a trivial commit that makes the fai

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Andrew Dunstan
On 4/14/21 2:03 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Apr 14, 2021 at 1:41 PM Tom Lane wrote: >>> Could we hack things so that extension scripts are only allowed to >>> reference objects created (a) by the system, (b) earlier in the >>> same script, or (c) owned by one of the dec

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-14 Thread Bruce Momjian
On Tue, Apr 13, 2021 at 01:30:16PM -0400, Álvaro Herrera wrote: > On 2021-Apr-12, Bruce Momjian wrote: > > > OK, the attached patch renames pg_stat_activity.queryid to 'query_id'. I > > have not changed any of the APIs which existed before this feature was > > added, and are called "queryid" or "q

Re: Possible typo/unclear comment in joinpath.c

2021-04-14 Thread Tom Lane
I wrote: > Justin Pryzby writes: >> It looks like a loose end from >> commit edca44b1525b3d591263d032dc4fe500ea771e0e > Yeah :-(. I'm usually pretty careful about grepping for comment > references as well as code references to a field when I do something > like that, but obviously I missed that

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 14, 2021 at 1:41 PM Tom Lane wrote: >> Could we hack things so that extension scripts are only allowed to >> reference objects created (a) by the system, (b) earlier in the >> same script, or (c) owned by one of the declared prerequisite >> extensions? Seems lik

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 1:41 PM Tom Lane wrote: > Doesn't help that much, because you still have to reference objects > already created by your own extension, so it's hard to see how the > target schema won't need to be in the path. Oh, woops. > Could we hack things so that extension scripts are

Re: Options given both on cmd-line and in the config with different values

2021-04-14 Thread Tom Lane
Honza Horak writes: > I'm trying to understand what is happening in the following bug report: > https://bugzilla.redhat.com/show_bug.cgi?id=1935301 > The upgrade process makes it a bit more difficult, but it seems to boil > down to this problem -- even when pg_ctl gets clear guidance where to >

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 14, 2021 at 10:49 AM Tom Lane wrote: >> The situation of interest is where you are trying to install an extension >> into a schema that also contains malicious objects. We've managed to make >> most of the commands you might use in an extension script secure aga

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Tom Lane
Mark Dilger writes: >> On Apr 13, 2021, at 3:26 PM, Tom Lane wrote: >> However I think we may still need an assumption that earthdistance >> and cube are in the same schema --- any comments on that? > This is probably not worth doing, and we are already past feature > freeze, but adding syntax t

Re: Possible typo/unclear comment in joinpath.c

2021-04-14 Thread Tom Lane
Justin Pryzby writes: > On Wed, Apr 14, 2021 at 11:36:38AM -0400, James Coleman wrote: >> In joinpath.c three times we reference "extra_lateral_rels" (with >> underscores like it's a field), but as far as I can tell that's not a >> field anywhere in the source code, and looking at the code that >>

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 10:49 AM Tom Lane wrote: > The situation of interest is where you are trying to install an extension > into a schema that also contains malicious objects. We've managed to make > most of the commands you might use in an extension script secure against > that situation, and

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Mark Dilger
> On Apr 13, 2021, at 3:26 PM, Tom Lane wrote: > > However I think we may still need an assumption that earthdistance > and cube are in the same schema --- any comments on that? This is probably not worth doing, and we are already past feature freeze, but adding syntax to look up the namespa

Bogus collation version recording in recordMultipleDependencies

2021-04-14 Thread Tom Lane
I noticed some broken-looking logic in recordMultipleDependencies concerning how it records collation versions. It was a bit harder than I expected to demonstrate the bugs, but I eventually succeeded with u8=# create function foo(varchar) returns bool language sql return false; CREATE FUNCTION u8

Re: pg_amcheck contrib application

2021-04-14 Thread Robert Haas
On Mon, Apr 12, 2021 at 11:06 PM Mark Dilger wrote: > It now reports: > > # heap table "postgres"."public"."test", block 0, offset 18, attribute 2: > # toast value 16461 missing chunk 3 with expected size 1996 > # heap table "postgres"."public"."test", block 0, offset 18, attribute 2: > #

Re: jsonb subscripting assignment performance

2021-04-14 Thread Alexander Korotkov
On Wed, Apr 14, 2021 at 10:57 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > sure - there is big room for optimization. But this patch was big enough > > without its optimization. And it was not clean, if I will be committed or > > not (it waited in commitfest application for 4 years). So I ac

[PATCH] expand the units that pg_size_pretty supports on output

2021-04-14 Thread David Christensen
Hi folks, Enclosed is a patch that expands the unit output for pg_size_pretty(numeric) going up to Yottabytes; I reworked the existing numeric output code to account for the larger number of units we're using rather than just adding nesting levels. There are also a few other places that could ben

Re: Possible typo/unclear comment in joinpath.c

2021-04-14 Thread Justin Pryzby
On Wed, Apr 14, 2021 at 11:36:38AM -0400, James Coleman wrote: > In joinpath.c three times we reference "extra_lateral_rels" (with > underscores like it's a field), but as far as I can tell that's not a > field anywhere in the source code, and looking at the code that > follows it seems like it sho

Options given both on cmd-line and in the config with different values

2021-04-14 Thread Honza Horak
Hello hackers, I'm trying to understand what is happening in the following bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1935301 The upgrade process makes it a bit more difficult, but it seems to boil down to this problem -- even when pg_ctl gets clear guidance where to find datadir

Possible typo/unclear comment in joinpath.c

2021-04-14 Thread James Coleman
In joinpath.c three times we reference "extra_lateral_rels" (with underscores like it's a field), but as far as I can tell that's not a field anywhere in the source code, and looking at the code that follows it seems like it should be referencing "lateral_relids" (and the "extra" is really "extra [

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Tom Lane
Robert Haas writes: > On Wed, Apr 14, 2021 at 8:58 AM Noah Misch wrote: >> Once CREATE EXTENSION is over, things are a great deal safer under this >> proposal, as you say. I suspect it makes CREATE EXTENSION more hazardous. >> Today, typical SQL commands in extension creation scripts don't activ

Re: [PATCH] Identify LWLocks in tracepoints

2021-04-14 Thread Robert Haas
On Tue, Apr 13, 2021 at 10:42 PM Craig Ringer wrote: > I'd really love it if a committer could add an explanatory comment or > two in the area though. I'm happy to draft a comment patch if anyone > agrees my suggestion is sensible. The key things I needed to know when > studying the code were: > >

Re: [PATCH] Identify LWLocks in tracepoints

2021-04-14 Thread Robert Haas
On Tue, Apr 13, 2021 at 4:46 PM Andres Freund wrote: > I still don't like the two bytes, fwiw ;). Especially because it's 4 > bytes due to padding right now. I'm not surprised by that disclosure. But I think it's entirely worth it. Making wait states visible in pg_stat_activity isn't the most use

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-04-14 Thread Andrei Zubkov
Hello, Kuroda! On Fri, 2021-04-09 at 00:23 +, kuroda.hay...@fujitsu.com wrote: > I think you are right. > According to [1] we can bump up the version per one PG major version, > and any features are not committed yet for 15. > > [1]: https://www.postgresql.org/message-id/20201202040516.GA437

Re: [BUG] Autovacuum not dynamically decreasing cost_limit and cost_delay

2021-04-14 Thread Mead, Scott
> On Mar 1, 2021, at 8:43 PM, Masahiko Sawada wrote: > > CAUTION: This email originated from outside of the organization. Do not click > links or open attachments unless you can confirm the sender and know the > content is safe. > > > > On Mon, Feb 8, 2021 at 11:49 PM Mead, Scott wrote: >

ANALYZE counts LP_DEAD line pointers as n_dead_tup

2021-04-14 Thread Masahiko Sawada
Hi all, If we create a table with vacuum_index_cleanup = off or execute VACUUM with INDEX_CLEANUP = off, vacuum updates pg_stat_all_tables.n_dead_tup to the number of HEAPTUPLE_RECENTLY_DEAD tuples. Whereas analyze updates it to the sum of the number of HEAPTUPLE_DEAD/RECENTLY_DEAD tuples and LP_D

Can a child process detect postmaster death when in pg_usleep?

2021-04-14 Thread Bharath Rupireddy
Hi, In my dev system(Ubuntu) when the postmaster is killed with SIGKILL, SIGPWR is being sent to its child processes (backends/any other bg process). If a child process is waiting with pg_usleep, it looks like it is not detecting the postmaster's death and it doesn't exit immediately but stays for

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 8:58 AM Noah Misch wrote: > Once CREATE EXTENSION is over, things are a great deal safer under this > proposal, as you say. I suspect it makes CREATE EXTENSION more hazardous. > Today, typical SQL commands in extension creation scripts don't activate > inexact argument typ

Re: sepgsql logging

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 8:42 AM Dave Page wrote: > Attached is a patch to clean this up. It will log denials as such regardless > of whether or not either selinux or sepgsql is in permissive mode. When > either is in permissive mode, it'll add " permissive=1" to the end of the log > messages. e

Re: [PATCH] PREPARE TRANSACTION unexpected behavior with TEMP TABLE

2021-04-14 Thread Robert Haas
On Wed, Apr 14, 2021 at 6:45 AM Himanshu Upadhyaya wrote: > The purpose of this FIX is to mainly focus on getting consistent behavior > with PREPARE TRANSACTION. With the case that I had mentioned > previously, my expectation was either both PREPARE TRANSACTION should fail > or both should succeed

RE: Support tab completion for upper character inputs in psql

2021-04-14 Thread tanghy.f...@fujitsu.com
On Thursday, April 8, 2021 4:14 PM, Peter Eisentraut wrote >Seeing the tests you provided, it's pretty obvious that the current >behavior is insufficient. I think we could probably think of a few more >tests, for example exercising the "If case insensitive matching was >requested initially,

Re: [PATCH] Identify LWLocks in tracepoints

2021-04-14 Thread Peter Eisentraut
On 12.04.21 07:46, Craig Ringer wrote: > To use systemtap semaphores (the _ENABLED macros) you need to run dtrace > -g to generate a probes.o then link that into postgres. > > I don't think we do that. I'll double check soon. We do that.  (It's -G.) Huh. I could've

Re: CTE push down

2021-04-14 Thread Ashutosh Bapat
On Tue, Apr 13, 2021 at 6:58 PM Alexander Pyhalov wrote: > > Hi. > > Currently PostgreSQL supports CTE push down for SELECT statements, but > it is implemented as turning each CTE reference into subquery. > > When CTE is referenced multiple times, we have choice - to materialize > CTE (and disable

Re: Converting contrib SQL functions to new style

2021-04-14 Thread Noah Misch
On Tue, Apr 13, 2021 at 11:11:13PM -0400, Tom Lane wrote: > Noah Misch writes: > > On Tue, Apr 13, 2021 at 06:26:34PM -0400, Tom Lane wrote: > >> Attached are some draft patches to convert almost all of the > >> contrib modules' SQL functions to use SQL-standard function bodies. > >> The point of

Re: sepgsql logging

2021-04-14 Thread Dave Page
Hi On Thu, Apr 1, 2021 at 3:30 PM Dave Page wrote: > > > On Thu, Apr 1, 2021 at 3:23 PM Tom Lane wrote: > >> Andrew Dunstan writes: >> > On 4/1/21 8:32 AM, Dave Page wrote: >> >> It seems to me that sepgsql should also log the denial, but flag that >> >> permissive mode is on. >> >> > +1 for d

Re: Extensions not dumped when --schema is used

2021-04-14 Thread Noah Misch
On Wed, Apr 14, 2021 at 10:38:17AM +0900, Michael Paquier wrote: > On Tue, Apr 13, 2021 at 08:00:34AM -0700, Noah Misch wrote: > > On Tue, Apr 13, 2021 at 02:43:11PM +0900, Michael Paquier wrote: > >>> - If extschema='public', "pg_dump -e plpgsql --schema=public" includes > >>> commands to dump t

Re: Replication slot stats misgivings

2021-04-14 Thread vignesh C
On Wed, Apr 14, 2021 at 12:09 PM Amit Kapila wrote: > > On Tue, Apr 13, 2021 at 1:37 PM vignesh C wrote: > > > > On Mon, Apr 12, 2021 at 7:03 PM Masahiko Sawada > > wrote: > > > > > > > > > The following test for the latest v8 patch seems to show different. > > > total_bytes is 1808 whereas spi

Re: logical replication worker accesses catalogs in error context callback

2021-04-14 Thread Bharath Rupireddy
On Wed, Mar 17, 2021 at 4:52 PM Bharath Rupireddy wrote: > > On Tue, Mar 16, 2021 at 2:21 AM Tom Lane wrote: > > > Thanks for pointing to the changes in the commit message. I corrected > > > them. Attaching v4 patch set, consider it for further review. > > > > I took a quick look at this. I'm qu

Re: Monitoring stats docs inconsistency

2021-04-14 Thread Amit Kapila
On Tue, Apr 13, 2021 at 6:08 PM vignesh C wrote: > > Few of the statistics description in monitoring_stats.sgml doc is not > consistent. Made all the descriptions consistent by including > punctuation marks at the end of each description. > Thoughts? > I think monitoring.sgml uses a similar patte

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-04-14 Thread e . sokolova
Thank you for working on this issue. Your comments helped me make this patch more correct. Lines with "colon" format shouldn't use equal signs, and should use two spaces between fields. Done. Now extra line looks like "Loop min_rows: %.0f max_rows: %.0f total_rows: %.0f" or "Loop min_time:

Re: jsonb subscripting assignment performance

2021-04-14 Thread Joel Jacobson
On Wed, Apr 14, 2021, at 11:07, Oleg Bartunov wrote: > I and Nikita are working on OLTP jsonb > http://www.sai.msu.su/~megera/postgres/talks/jsonb-pgconfonline-2021.pdf > Page 49/55 in the PDF: "UPDATE test_toast SET jb = jsonb_set(jb, {keyN,0}, ?);" Would you get similar improvements if updat

Re: [PATCH] PREPARE TRANSACTION unexpected behavior with TEMP TABLE

2021-04-14 Thread Himanshu Upadhyaya
Hi Robert, Thanks for sharing your thoughts. The purpose of this FIX is to mainly focus on getting consistent behavior with PREPARE TRANSACTION. With the case that I had mentioned previously, my expectation was either both PREPARE TRANSACTION should fail or both should succeed but here second same

Re: Unresolved repliaction hang and stop problem.

2021-04-14 Thread Amit Kapila
On Tue, Apr 13, 2021 at 1:18 PM Krzysztof Kois wrote: > > Hello, > After upgrading the cluster from 10.x to 13.1 we've started getting a problem > describe pgsql-general: > https://www.postgresql.org/message-id/8bf8785c-f47d-245c-b6af-80dc1eed40db%40unitygroup.com > We've noticed similar issue be

Re: Truncate in synchronous logical replication failed

2021-04-14 Thread Amit Kapila
On Tue, Apr 13, 2021 at 8:07 PM Petr Jelinek wrote: > > > On 12 Apr 2021, at 08:58, Amit Kapila wrote: > > > > The problem happens only when we try to fetch IDENTITY_KEY attributes > > because pgoutput uses RelationGetIndexAttrBitmap() to get that > > information which locks the required indexes.

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-04-14 Thread Andrei Zubkov
On Wed, 2021-04-14 at 17:32 +0800, Julien Rouhaud wrote: > > did you enable compute_query_id new parameter?  Hi, Julien! Thank you very much! I've missed it. >

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-04-14 Thread Julien Rouhaud
Le mer. 14 avr. 2021 à 17:22, Andrei Zubkov a écrit : > > But I'm unable to test the patch - it seems that pg_stat_statements is > receiving queryId = 0 for every statements in every hook now and > statements are not tracked at all. > > Am I mistaken somewhere? Maybe you know why this is happenin

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-04-14 Thread Andrei Zubkov
Hi, Kuroda! I've intended to change the pg_stat_statements version with rebasing this patch to the current master branch state. Now this is commit 07e5e66. But I'm unable to test the patch - it seems that pg_stat_statements is receiving queryId = 0 for every statements in every hook now and state

Re: jsonb subscripting assignment performance

2021-04-14 Thread Pavel Stehule
st 14. 4. 2021 v 11:07 odesílatel Oleg Bartunov napsal: > > > On Wed, Apr 14, 2021 at 11:09 AM Pavel Stehule > wrote: > >> >> >> st 14. 4. 2021 v 9:57 odesílatel Dmitry Dolgov <9erthali...@gmail.com> >> napsal: >> >>> > On Wed, Apr 14, 2021 at 09:20:08AM +0200, Pavel Stehule wrote: >>> > st 14.

Re: jsonb subscripting assignment performance

2021-04-14 Thread Oleg Bartunov
On Wed, Apr 14, 2021 at 11:09 AM Pavel Stehule wrote: > > > st 14. 4. 2021 v 9:57 odesílatel Dmitry Dolgov <9erthali...@gmail.com> > napsal: > >> > On Wed, Apr 14, 2021 at 09:20:08AM +0200, Pavel Stehule wrote: >> > st 14. 4. 2021 v 7:39 odesílatel Joel Jacobson >> napsal: >> > >> > > Hi, >> > >

Re: More sepgsql weirdness

2021-04-14 Thread Dave Page
Hi On Tue, Apr 13, 2021 at 6:22 PM Robert Haas wrote: > On Tue, Apr 13, 2021 at 10:33 AM Dave Page wrote: > > On a system with selinux and sepgsql configured, search path resolution > appears to fail if sepgsql is in enforcing mode, but selinux is in > permissive mode (which, as I understand it

RE: Truncate in synchronous logical replication failed

2021-04-14 Thread osumi.takami...@fujitsu.com
On Wednesday, April 14, 2021 11:38 AM Japin Li wrote: > On Tue, 13 Apr 2021 at 21:54, osumi.takami...@fujitsu.com > wrote: > > On Monday, April 12, 2021 3:58 PM Amit Kapila > wrote: > >> On Mon, Apr 12, 2021 at 10:03 AM osumi.takami...@fujitsu.com > >> wrote: > >> > but if we take a measure to

File truncation within PostgresNode::issues_sql_like() wrong on Windows

2021-04-14 Thread Michael Paquier
Hi all, As fairywren has proved a couple of days ago, it is not really a good idea to rely on a file truncation to check for patterns in the logs of the backend: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-04-07%2013%3A29%3A28 Visibly, a logic based on the log file t

Re: jsonb subscripting assignment performance

2021-04-14 Thread Pavel Stehule
st 14. 4. 2021 v 9:57 odesílatel Dmitry Dolgov <9erthali...@gmail.com> napsal: > > On Wed, Apr 14, 2021 at 09:20:08AM +0200, Pavel Stehule wrote: > > st 14. 4. 2021 v 7:39 odesílatel Joel Jacobson > napsal: > > > > > Hi, > > > > > > commit 676887a3 added support for jsonb subscripting. > > > > >

Re: jsonb subscripting assignment performance

2021-04-14 Thread Dmitry Dolgov
> On Wed, Apr 14, 2021 at 09:20:08AM +0200, Pavel Stehule wrote: > st 14. 4. 2021 v 7:39 odesílatel Joel Jacobson napsal: > > > Hi, > > > > commit 676887a3 added support for jsonb subscripting. > > > > Many thanks for working on this. I really like the improved syntax. > > > > I was also hoping fo

Re: jsonb subscripting assignment performance

2021-04-14 Thread Joel Jacobson
On Wed, Apr 14, 2021, at 09:20, Pavel Stehule wrote: > sure - there is big room for optimization. But this patch was big enough > without its optimization. And it was not clean, if I will be committed or not > (it waited in commitfest application for 4 years). So I accepted implemented > behavio

RE: Truncate in synchronous logical replication failed

2021-04-14 Thread osumi.takami...@fujitsu.com
On Tuesday, April 13, 2021 11:38 PM Petr Jelinek wrote: > > On 12 Apr 2021, at 08:58, Amit Kapila wrote: > > On Mon, Apr 12, 2021 at 10:03 AM osumi.takami...@fujitsu.com > > wrote: > >> > >>> I checked the PG-DOC, found it says that “Replication of TRUNCATE > >>> commands is supported”[1], so m

View invoker privileges

2021-04-14 Thread Ivan Ivanov
Hello guys! In Postgres we can create view with view owner privileges only. What’s the reason that there is no option to create view with invoker privileges? Is there any technical or security subtleties related to absence of this feature?

Re: jsonb subscripting assignment performance

2021-04-14 Thread Pavel Stehule
st 14. 4. 2021 v 7:39 odesílatel Joel Jacobson napsal: > Hi, > > commit 676887a3 added support for jsonb subscripting. > > Many thanks for working on this. I really like the improved syntax. > > I was also hoping for some performance benefits, > but my testing shows that > >jsonb_value['exist