Re: proposal - psql - use pager for \watch command

2021-04-20 Thread Pavel Stehule
st 21. 4. 2021 v 8:49 odesílatel Thomas Munro napsal: > On Wed, Apr 21, 2021 at 6:33 PM Pavel Stehule > wrote: > > here is an rebase of Thomas's implementation > > Thanks. I finished up not committing that one for 14 because I wasn't > sure about the way to rebase it on top of 3a513067 (now rev

Re: proposal - psql - use pager for \watch command

2021-04-20 Thread Thomas Munro
On Wed, Apr 21, 2021 at 6:33 PM Pavel Stehule wrote: > here is an rebase of Thomas's implementation Thanks. I finished up not committing that one for 14 because I wasn't sure about the way to rebase it on top of 3a513067 (now reverted); that "restore" stuff seemed a bit weird. Let's try again i

Re: proposal - psql - use pager for \watch command

2021-04-20 Thread Pavel Stehule
Hi čt 8. 4. 2021 v 1:38 odesílatel Thomas Munro napsal: > Here's a rebase, due to a conflict with 3a513067 "psql: Show all query > results by default" which moved a few things around making it harder > to use the pager for the right scope. Lacking time, I came up with > this change to PSQLexecW

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Ondřej Žižka
Hello Satyanarayana, This can be an option for us in our case. But there also needs to be a process how to detect these "stuck commits" and how to invalidate/remove them, because in reality, if the app/user would not see the change in the database, it/he/she will try to insert/delete it again.

Re: Replication slot stats misgivings

2021-04-20 Thread Amit Kapila
On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > > I've attached the patch. In addition to the test Vignesh prepared, I > added one test for the message for creating a slot that checks if the > statistics are initialized after re-creating the same name slot. > I am not sure how much use

Re: wal stats questions

2021-04-20 Thread torikoshia
On 2021-04-16 10:27, Masahiro Ikeda wrote: On 2021/04/13 9:33, Fujii Masao wrote: On 2021/03/30 20:37, Masahiro Ikeda wrote: OK, I added the condition to the fast-return check. I noticed that I misunderstood that the purpose is to avoid expanding a clock check using WAL stats counters. But,

Re: multi-install PostgresNode fails with older postgres versions

2021-04-20 Thread Michael Paquier
On Tue, Apr 20, 2021 at 01:11:59PM -0400, Andrew Dunstan wrote: > Here's the patch for that. Thanks. > + # Accept standard formats, in case caller has handed us the output of a > + # postgres command line tool > + $arg = $1 > + if ($arg =~ m/\(?PostgreSQL\)? (\d+(?:\.\d+)*

Re: Is it worth to optimize VACUUM/ANALYZE by combining duplicate rel instances into single rel instance?

2021-04-20 Thread Bharath Rupireddy
On Wed, Apr 21, 2021 at 8:02 AM Kyotaro Horiguchi wrote: > > Thanks! I think we could avoid extra processing costs for cases like > > VACUUM/ANALYZE foo, foo; when no explicit columns are specified. The > > avoided costs can be lock acquire, relation open, vacuum/analyze, > > relation close, start

Re: Replication slot stats misgivings

2021-04-20 Thread vignesh C
On Wed, Apr 21, 2021 at 9:47 AM Amit Kapila wrote: > > On Wed, Apr 21, 2021 at 9:39 AM vignesh C wrote: > > > > I feel we can change CATALOG_VERSION_NO so that we will get this error > > "The database cluster was initialized with CATALOG_VERSION_NO > > 2021X, but the server was compiled with

Re: Support tab completion for upper character inputs in psql

2021-04-20 Thread Peter Smith
On Wed, Apr 14, 2021 at 11:34 PM tanghy.f...@fujitsu.com wrote: > > On Thursday, April 8, 2021 4:14 PM, Peter Eisentraut > wrote > > >Seeing the tests you provided, it's pretty obvious that the current > >behavior is insufficient. I think we could probably think of a few more > >tests, for exam

Re: Replication slot stats misgivings

2021-04-20 Thread Amit Kapila
On Wed, Apr 21, 2021 at 9:39 AM vignesh C wrote: > > I feel we can change CATALOG_VERSION_NO so that we will get this error > "The database cluster was initialized with CATALOG_VERSION_NO > 2021X, but the server was compiled with CATALOG_VERSION_NO > 2021X." which will prevent the above is

Re: Replication slot stats misgivings

2021-04-20 Thread vignesh C
On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > > On Tue, Apr 20, 2021 at 7:22 PM vignesh C wrote: > > > > On Tue, Apr 20, 2021 at 9:08 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Apr 19, 2021 at 4:48 PM Masahiko Sawada > > > wrote: > > > > > > > > On Mon, Apr 19, 2021 at 2:14

Re: Replication slot stats misgivings

2021-04-20 Thread Amit Kapila
On Tue, Apr 20, 2021 at 7:54 PM Masahiko Sawada wrote: > I have one question: + /* + * Create the replication slot stats hash table if we don't have + * it already. + */ + if (replSlotStats == NULL) { - if (namestrcmp(&replSlotStats[i].slotname, name) == 0) - return i; /* found */ + HASHCTL ha

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-20 Thread Michael Paquier
On Tue, Apr 20, 2021 at 08:44:23PM +0100, Daniel Carter wrote: > The original motivation for investigating this was setting up a web app > which could authenticate to a database server using a Kerberos ticket. Since > the web framework already needs to create a connection string (with database > na

Re: Table refer leak in logical replication

2021-04-20 Thread Michael Paquier
On Wed, Apr 21, 2021 at 11:13:06AM +0900, Amit Langote wrote: > Agree about adding tests along these lines. Will post in a bit. Thanks! -- Michael signature.asc Description: PGP signature

Re: Tiny update to pg_stat_statements documentation

2021-04-20 Thread Michael Paquier
On Wed, Apr 21, 2021 at 09:46:59AM +0800, Julien Rouhaud wrote: > +1, it looks good to me. Cool, thanks for confirming. The top of the docs have IMO enough details about the requirements around compute_query_id and third-part modules, so we are done here. -- Michael signature.asc Description: P

prerequisites of pull_up_sublinks

2021-04-20 Thread Andy Fan
Hi, I'm reading the pull_up_sublinks, and find the below comments. * However, this optimization *only* * works at the top level of WHERE or a JOIN/ON clause, because we cannot * distinguish whether the ANY ought to return FALSE or NULL in cases * involving NULL inputs. Also, in an outer join'

Re: Is it worth to optimize VACUUM/ANALYZE by combining duplicate rel instances into single rel instance?

2021-04-20 Thread Michael Paquier
On Wed, Apr 21, 2021 at 11:32:49AM +0900, Kyotaro Horiguchi wrote: > On the other hand the patch creates a relation list just for this > purpose, which is not needed to run VACUUM/ANALYZE, and VACUUM/ANALYE > works well with duplicates in target relations. Yeah, I don't think either that this is w

RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-20 Thread tsunakawa.ta...@fujitsu.com
From: Tom Lane > [ raised eyebrow... ] I find it very hard to understand why that would > be necessary, or even a good idea. Not least because there's no spare > room there; you'd have to incur a substantial enlargement of the > array to add another flag. But also, that would indeed lock down >

Re: Docs: Move parallel_leader_participation GUC description under relevant category

2021-04-20 Thread Bharath Rupireddy
On Wed, Apr 21, 2021 at 8:00 AM Michael Paquier wrote: > > On Tue, Apr 20, 2021 at 09:16:49PM +0530, Bharath Rupireddy wrote: > > It looks like even though the commit e5253fdc4f that added the > > parallel_leader_participation GUC correctly categorized it as > > RESOURCES_ASYNCHRONOUS parameter in

Re: Is it worth to optimize VACUUM/ANALYZE by combining duplicate rel instances into single rel instance?

2021-04-20 Thread Kyotaro Horiguchi
At Wed, 21 Apr 2021 07:34:40 +0530, Bharath Rupireddy wrote in > On Sat, Apr 10, 2021 at 8:03 PM Tom Lane wrote: > > > > Bharath Rupireddy writes: > > > I'm reading the code for vacuum/analyze and it looks like currently we > > > call vacuum_rel/analyze_rel for each relation specified. Which m

Re: Docs: Move parallel_leader_participation GUC description under relevant category

2021-04-20 Thread Michael Paquier
On Tue, Apr 20, 2021 at 09:16:49PM +0530, Bharath Rupireddy wrote: > It looks like even though the commit e5253fdc4f that added the > parallel_leader_participation GUC correctly categorized it as > RESOURCES_ASYNCHRONOUS parameter in the code, but in the docs it is kept > under irrelevant section i

Re: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-20 Thread Tom Lane
"tsunakawa.ta...@fujitsu.com" writes: > From: Tom Lane >> No. You'd have to be superuser anyway to do that, and we're not in the >> habit of trying to put training wheels on superusers. > Understood. However, we may add the parallel safety member in > fmgr_builtins[] in another thread for par

Re: Table refer leak in logical replication

2021-04-20 Thread Amit Langote
On Wed, Apr 21, 2021 at 9:31 AM Michael Paquier wrote: > On Tue, Apr 20, 2021 at 06:20:03PM +0530, Amit Kapila wrote: > > +1. I think it makes sense to add a test case especially because we > > don't have any existing test in this area. > > Yes, let's add add something into 013_partition.pl within

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-04-20 Thread Justin Pryzby
On Tue, Apr 20, 2021 at 06:16:28PM -0700, Andres Freund wrote: > On 2021-04-20 20:03:13 -0500, Justin Pryzby wrote: > > That's a query over a 2 day period (midnight to midnight+2), so it's not > > hard > > for me to believe it sometimes exceeds 100k cost units > > (jit_inline_above_cost), > > dep

Re: Stale description for pg_basebackup

2021-04-20 Thread Kyotaro Horiguchi
At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi > wrote in > > Hello. > > > > It seems to me that there's a stale description in the documentation > > of pg_basebackup. > > > > https://www.postgresql.org/docs

Re: Is it worth to optimize VACUUM/ANALYZE by combining duplicate rel instances into single rel instance?

2021-04-20 Thread Bharath Rupireddy
On Sat, Apr 10, 2021 at 8:03 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > I'm reading the code for vacuum/analyze and it looks like currently we > > call vacuum_rel/analyze_rel for each relation specified. Which means > > that if a relation is specified more than once, then we simply > >

non-blocking delayChkpt

2021-04-20 Thread Andres Freund
Hi, During commits, and some other places, there's a short phase at which we block checkpoints from starting: /* * Mark ourselves as within our "commit critical section". This * forces any concurrent checkpoint to wait until we've updated

RE: [bug?] Missed parallel safety checks, and wrong parallel safety

2021-04-20 Thread tsunakawa.ta...@fujitsu.com
From: Tom Lane > Bharath Rupireddy writes: > > IMO, the reason for not checking the parallel safety of the support > > functions is that the functions themselves can have whole lot of other > > functions (can be nested as well) which might be quite hard to check > > at the planning time. That is

Re: An omission of automatic completion in tab-complete.c

2021-04-20 Thread Michael Paquier
On Tue, Apr 20, 2021 at 11:35:13AM +0300, Aleksander Alekseev wrote: > I invested some time in checking this patch. It passes make > check-world / make installcheck-world and adds CURRENT_ROLE to the > automatic completion. Thanks Aleksander and Wei. Applied. -- Michael signature.asc Descriptio

Re: Tiny update to pg_stat_statements documentation

2021-04-20 Thread Julien Rouhaud
On Wed, Apr 21, 2021 at 10:27:43AM +0900, Michael Paquier wrote: > On Wed, Apr 21, 2021 at 11:10:36AM +1000, Greg Nancarrow wrote: > > However, I don't think the additional comment is really warranted > > here, as the other typical usage settings are not commented, and all > > settings are explaine

Re: Stale description for pg_basebackup

2021-04-20 Thread Kyotaro Horiguchi
At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi wrote in > Hello. > > It seems to me that there's a stale description in the documentation > of pg_basebackup. > > https://www.postgresql.org/docs/13/app-pgbasebackup.html > > > Note that there are some limitations in taking a backup

Re: Tiny update to pg_stat_statements documentation

2021-04-20 Thread Bharath Rupireddy
On Wed, Apr 21, 2021 at 6:40 AM Greg Nancarrow wrote: > > On Tue, Apr 20, 2021 at 8:06 PM Bharath Rupireddy > wrote: > > > I've attached a patch for this. > > > > +1. How about mentioning something like below? > > > > +compute_query_id = on # when in-core query identifier computation is > > desir

Re: Tiny update to pg_stat_statements documentation

2021-04-20 Thread Michael Paquier
On Wed, Apr 21, 2021 at 11:10:36AM +1000, Greg Nancarrow wrote: > However, I don't think the additional comment is really warranted > here, as the other typical usage settings are not commented, and all > settings are explained in the surrounding documentation. Good catch, Greg. I agree to keep t

Re: `make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Xing GUO
On 4/21/21, Tom Lane wrote: > Xing GUO writes: >> Thank you very much. I'm facing the same problem yesterday. May I >> suggest that document it in the installation guide on MacOS platform? > > It is documented --- see last para under > > https://www.postgresql.org/docs/current/installation-platfo

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-04-20 Thread Andres Freund
Hi, On 2021-04-20 20:03:13 -0500, Justin Pryzby wrote: > That's a query over a 2 day period (midnight to midnight+2), so it's not hard > for me to believe it sometimes exceeds 100k cost units > (jit_inline_above_cost), > depending on where we are in that interval, and on planner statistics. It >

Re: `make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Tom Lane
Xing GUO writes: > Thank you very much. I'm facing the same problem yesterday. May I > suggest that document it in the installation guide on MacOS platform? It is documented --- see last para under https://www.postgresql.org/docs/current/installation-platform-notes.html#INSTALLATION-NOTES-MACOS

Re: Tiny update to pg_stat_statements documentation

2021-04-20 Thread Greg Nancarrow
On Tue, Apr 20, 2021 at 8:06 PM Bharath Rupireddy wrote: > > I've attached a patch for this. > > +1. How about mentioning something like below? > > +compute_query_id = on # when in-core query identifier computation is > desired, otherwise off. > Hmm, I think that comment is perhaps slightly misle

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-04-20 Thread Justin Pryzby
On Tue, Apr 20, 2021 at 05:20:56PM -0700, Andres Freund wrote: > On 2021-04-20 00:58:21 -0500, Justin Pryzby wrote: > > On Tue, Apr 20, 2021 at 12:38:26AM -0500, Justin Pryzby wrote: > > > I don't know if this is related to the other issues, but this seems leaky. > > > > And it explains how the co

Re: `make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Xing GUO
Hi hackers, Thank you very much. I'm facing the same problem yesterday. May I suggest that document it in the installation guide on MacOS platform? On 4/21/21, Andrew Dunstan wrote: > > On 4/20/21 11:02 AM, Tom Lane wrote: >> Aleksander Alekseev writes: >>> While trying to build PostgreSQL from

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-20 Thread Alvaro Herrera
Actually I had a silly bug in the version that attempted to cache a partdesc that omits detached partitions. This one, while not fully baked, seems to work correctly (on top of the previous one). The thing that I don't fully understand is why we have to require to have built the regular one first

Re: Table refer leak in logical replication

2021-04-20 Thread Michael Paquier
On Tue, Apr 20, 2021 at 06:20:03PM +0530, Amit Kapila wrote: > +1. I think it makes sense to add a test case especially because we > don't have any existing test in this area. Yes, let's add add something into 013_partition.pl within both subscriber1 and subscriber2. This will not catch up the r

Re: Privilege boundary between sysadmin and database superuser [Was: Re: pg_amcheck option to install extension]

2021-04-20 Thread Mark Dilger
> On Apr 20, 2021, at 3:19 PM, Tom Lane wrote: > > Mark Dilger writes: >>> On Apr 20, 2021, at 5:54 AM, Robert Haas wrote: >>> On Tue, Apr 20, 2021 at 1:31 AM Mark Dilger >>> wrote: I think you are conflating the concept of an operating system adminstrator with the concept of the

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-04-20 Thread Andres Freund
Hi, On 2021-04-20 00:58:21 -0500, Justin Pryzby wrote: > On Tue, Apr 20, 2021 at 12:38:26AM -0500, Justin Pryzby wrote: > > I don't know if this is related to the other issues, but this seems leaky. > > And it explains how the context use counter can exceed its threshold. > > create or replace f

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-20 Thread Alexey Kondratov
On 2021-04-20 18:03, Tom Lane wrote: Andrew Dunstan writes: On 4/19/21 7:22 PM, Tom Lane wrote: I wonder whether we could get away with just replacing the $use_tcp test with $TestLib::windows_os. It's not really apparent to me why we should care about 127.0.0.not-1 on Unix-oid systems. Yea

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-20 Thread Alvaro Herrera
OK so after mulling this over for a long time, here's a patch for this. It solves the problem by making the partition descriptor no longer be cached if a detached partition is omitted. Any transaction that needs a partition descriptor that excludes detached partitions, will have to recreate the pa

Re: Privilege boundary between sysadmin and database superuser [Was: Re: pg_amcheck option to install extension]

2021-04-20 Thread Tom Lane
Mark Dilger writes: >> On Apr 20, 2021, at 5:54 AM, Robert Haas wrote: >> On Tue, Apr 20, 2021 at 1:31 AM Mark Dilger >> wrote: >>> I think you are conflating the concept of an operating system adminstrator >>> with the concept of the database superuser/owner. >> You should conflate those thin

Privilege boundary between sysadmin and database superuser [Was: Re: pg_amcheck option to install extension]

2021-04-20 Thread Mark Dilger
> On Apr 20, 2021, at 5:54 AM, Robert Haas wrote: > > On Tue, Apr 20, 2021 at 1:31 AM Mark Dilger > wrote: >> I think you are conflating the concept of an operating system adminstrator >> with the concept of the database superuser/owner. > > You should conflate those things, because there's

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 2:29 PM Stefan Keller wrote: > Just for the records: A learned index as no more foreknowledge about > the dataset as other indices. Maybe. ML models are famously prone to over-interpreting training data. In any case I am simply not competent to assess how true this is. >

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Tom Lane
Stefan Keller writes: > I'd give learned indexes at least a change to provide a > proof-of-concept. And I want to learn more about the requirements to > be accepted as a new index (before undergoing month's of code > sprints). There's enough support these days that you can build a new index type

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Stefan Keller
Just for the records: A learned index as no more foreknowledge about the dataset as other indices. I'd give learned indexes at least a change to provide a proof-of-concept. And I want to learn more about the requirements to be accepted as a new index (before undergoing month's of code sprints). A

Re: RFE: Make statistics robust for unplanned events

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 5:00 AM Patrik Novotny wrote: > As far as I've checked, this would have to be implemented. > > My question would be whether there is something that would make this > impossible to implement, and if there isn't, I'd like this to be considered a > feature request. I agree

Re: Synchronous commit behavior during network outage

2021-04-20 Thread SATYANARAYANA NARLAPURAM
One idea here is to make the backend ignore query cancellation/backend termination while waiting for the synchronous commit ACK. This way client never reads the data that was never flushed remotely. The problem with this approach is that your backends get stuck until your commit log record is flush

Re: when the startup process doesn't

2021-04-20 Thread SATYANARAYANA NARLAPURAM
+1 for both log messages and allowing connections. I believe these two complement each other. In the cloud world, we oftentimes want to monitor the progress of the recovery without connecting to the server as the operators don't necessarily have the required permissions to connect and query. Secon

Re: fix old confusing JSON example

2021-04-20 Thread Tom Lane
Justin Pryzby writes: > On Tue, Apr 20, 2021 at 09:07:52PM +0200, Erik Rijkers wrote: >> I just happened to use the website-documentation and noticed that there the >> change is not done: it still has the erroneous line, in the docs of 13 >> (current), and 12; the docs of 14devel are apparently

Re: when the startup process doesn't

2021-04-20 Thread Alvaro Herrera
On 2021-Apr-20, Andres Freund wrote: > On 2021-04-19 13:55:13 -0400, Robert Haas wrote: > > Another possible approach would be to accept connections for > > monitoring purposes even during crash recovery. We can't allow access > > to any database at that point, since the system might not be > > co

Re: when the startup process doesn't

2021-04-20 Thread Andres Freund
Hi, On 2021-04-20 14:56:58 -0400, Tom Lane wrote: > I wonder though whether we really need authentication here. pg_ping > already exposes whether the database is up, to anyone who can reach the > postmaster port at all. Would it be so horrible if the "can't accept > connections" error message in

Re: when the startup process doesn't

2021-04-20 Thread Andres Freund
Hi, On 2021-04-19 13:55:13 -0400, Robert Haas wrote: > Another possible approach would be to accept connections for > monitoring purposes even during crash recovery. We can't allow access > to any database at that point, since the system might not be > consistent, but we could allow something like

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 12:51 PM Jonah H. Harris wrote: >> Maybe I'll be wrong about learned indexes - who knows? But the burden >> of proof is not mine. I prefer to spend my time on things that I am >> reasonably confident will work out well ahead of time. > > > Agreed on all of your takes, Peter

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Jonah H. Harris
On Tue, Apr 20, 2021 at 3:45 PM Peter Geoghegan wrote: > On Tue, Apr 20, 2021 at 12:35 PM Chapman Flack > wrote: > > How would showing that to be true for data structure X be different from > > making a case for data structure X? > > You don't have to understand the theoretical basis of B-Tree i

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 12:35 PM Chapman Flack wrote: > How would showing that to be true for data structure X be different from > making a case for data structure X? You don't have to understand the theoretical basis of B-Tree indexes to see that they work well. In fact, it took at least a decad

Re: fix old confusing JSON example

2021-04-20 Thread Justin Pryzby
On Tue, Apr 20, 2021 at 09:07:52PM +0200, Erik Rijkers wrote: > I just happened to use the website-documentation and noticed that there the > change is not done: it still has the erroneous line, in the docs of 13 > (current), and 12; the docs of 14devel are apparently updated. > > That makes me

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-20 Thread Daniel Carter
Hi Stephen, On 20/04/2021 20:01, Stephen Frost wrote: I'm not necessarily against this, but typically the GSSAPI library provides a way for you to control this using, eg, the KRB5_CCACHE environment variable. Is there some reason why that couldn't be used..? The original motivation for invest

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Chapman Flack
On 04/20/21 15:24, Peter Geoghegan wrote: > data structures that work well don't need anybody to make a case for them. > They simply work well for the task they were designed for. How would showing that to be true for data structure X be different from making a case for data structure X? Regards,

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Peter Geoghegan
On Tue, Apr 20, 2021 at 11:18 AM Andrey Borodin wrote: > BTW take a look into PGM [0]. I'm slowly working on implementing it. > I think it is kind of straightforward to implement it as extension. > I've started from forking B-tree[1]. I've removed support of anything that is > not int4. > Then I

Re: fix old confusing JSON example

2021-04-20 Thread Erik Rijkers
> On 2021.04.16. 10:00 Michael Paquier wrote: > On Sat, Apr 03, 2021 at 02:28:38PM +0200, Erik Rijkers wrote: > > So, that gives information on two operators, and then gives one > > example query for each. Clearly, the second example was meant to > > illustrate a where-clause with the @? operat

Re: when the startup process doesn't

2021-04-20 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > Yeah, being able to pick up on this remotely seems like it'd be quite > > nice. I'm not really thrilled with the idea, but the best I've got > > offhand for this would be a new role that's "pg_recovery_login" where an

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-20 Thread Stephen Frost
Greetings, * Daniel Carter (danielchriscarter+postg...@gmail.com) wrote: > This is a small patch (against master) to allow an application using libpq > with GSSAPI authentication to specify where to fetch the credential cache > from -- it effectively consists of a new field in PQconninfoOptions to

Re: when the startup process doesn't

2021-04-20 Thread Tom Lane
Stephen Frost writes: > Yeah, being able to pick up on this remotely seems like it'd be quite > nice. I'm not really thrilled with the idea, but the best I've got > offhand for this would be a new role that's "pg_recovery_login" where an > admin can GRANT that role to the roles they'd like to be

Re: when the startup process doesn't

2021-04-20 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Magnus Hagander writes: > > On Tue, Apr 20, 2021 at 5:17 PM Jehan-Guillaume de Rorthais > > wrote: > >> Two another options: > >> 1. if this is limited to local access only, outside of the log entries, the > >> status of the startup could be up

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Ondřej Žižka
I am sorry, I forgot mentioned, that in the second situation I added a primary key to the table. Ondrej On 20/04/2021 18:49, Ondřej Žižka wrote: Hello Aleksander, Thank you for the reaction. This was tested on version 13.2. There are also other possible situations with the same setup and s

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Ondřej Žižka
Hello Maksim, I know your post [1]. That thread is why there we performed more tests (see another my email in this thread). We are trying to somehow implement RPO=0 solution using PostgreSQL. Knowing this... Would be possible to build RPO=0 solution with PostgreSQL? Ondrej On 20/04/2021 18:

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Ondřej Žižka
Hello Aleksander, Thank you for the reaction. This was tested on version 13.2. There are also other possible situations with the same setup and similar issue: - When the background process on server fails On postgresql1: tecmint=# select * from a; --> LAN on sync replica

Re: when the startup process doesn't

2021-04-20 Thread Tom Lane
Magnus Hagander writes: > On Tue, Apr 20, 2021 at 5:17 PM Jehan-Guillaume de Rorthais > wrote: >> Two another options: >> 1. if this is limited to local access only, outside of the log entries, the >> status of the startup could be updated in the controldata file as well. This >> would allows to

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Maksim Milyutin
On 20.04.2021 19:38, Tomas Vondra wrote: On 4/20/21 6:23 PM, Aleksander Alekseev wrote: Hi Ondřej, Thanks for the report. It seems to be a clear violation of what is promised in the docs. Although it's unlikely that someone implemented an application which deals with important data and "pres

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Andrey Borodin
> 20 апр. 2021 г., в 22:56, Stefan Keller написал(а): > > Are there any advances in a learned index for PostgreSQL? BTW take a look into PGM [0]. I'm slowly working on implementing it. I think it is kind of straightforward to implement it as extension. I've started from forking B-tree[1]. I'v

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2021-04-20 Thread Stefan Keller
Dear Olegs, dear Nikolay, dear all Allow me to revive this thread: Are there any advances in a learned index for PostgreSQL? Background: I'm trying to benchmark those experimental indices. For this I did some bibliography work (see below). Fun fact: Not only Postgres people love high-proof drink

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Maksim Milyutin
Hi! This is a known issue with synchronous replication [1]. You might inject into unmodified operation some dummy modification to overcome the negative sides of such partially committing without source code patching. On 20.04.2021 19:23, Aleksander Alekseev wrote: Although it's unlikely th

Re: when the startup process doesn't

2021-04-20 Thread Magnus Hagander
On Tue, Apr 20, 2021 at 5:17 PM Jehan-Guillaume de Rorthais wrote: > > On Tue, 20 Apr 2021 15:04:28 +0200 > Magnus Hagander wrote: > [...] > > Yeah, I think we should definitely limit this to local access, one way > > or another. Realistically using pg_hba is going to require catalog > > access,

Re: multi-install PostgresNode fails with older postgres versions

2021-04-20 Thread Andrew Dunstan
On 4/19/21 12:37 PM, Andrew Dunstan wrote: > On 4/19/21 10:43 AM, Mark Dilger wrote: >>> On Apr 19, 2021, at 5:11 AM, Andrew Dunstan wrote: >>> >>> I think therefore I'm inclined for now to do nothing for old version >>> compatibility. >> I agree with waiting until the v15 development cycle. >> >

Re: Problems around compute_query_id

2021-04-20 Thread Bruce Momjian
On Thu, Apr 15, 2021 at 03:43:59PM +0800, Julien Rouhaud wrote: > On Mon, Apr 12, 2021 at 02:56:59PM +0800, Julien Rouhaud wrote: > > I think we should simply document that %Q is not compatible with > > log_statements. > > Hearing no objection I documented that limitation. > > > > > > While maki

Re: pg_amcheck option to install extension

2021-04-20 Thread Robert Haas
On Tue, Apr 20, 2021 at 12:05 PM Tom Lane wrote: > > I think the distinction I would draw is between things we would expect > > to be present in every Postgres installation (e.g. pg_stat_statements, > > auto_explain, postgres_fdw, hstore) and things we don't for one reason > > or another (e.g. pgc

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Tomas Vondra
On 4/20/21 6:23 PM, Aleksander Alekseev wrote: > Hi Ondřej, > > Thanks for the report. It seems to be a clear violation of what is > promised in the docs. Although it's unlikely that someone implemented > an application which deals with important data and "pressed Ctr+C" as > it's done in psql.

Re: PATCH: Add GSSAPI ccache_name option to libpq

2021-04-20 Thread Daniel Carter
Hi Aleksander, On 20/04/2021 11:30, Aleksander Alekseev wrote: Hi Daniel, It's my first go at submitting a patch -- it works as far as I can tell, but I suspect there will probably still be stuff to fix before it's ready to use! You are doing great :) Thanks for the encouragement! There

Re: Synchronous commit behavior during network outage

2021-04-20 Thread Aleksander Alekseev
Hi Ondřej, Thanks for the report. It seems to be a clear violation of what is promised in the docs. Although it's unlikely that someone implemented an application which deals with important data and "pressed Ctr+C" as it's done in psql. So this might be not such a critical issue after all. BTW wha

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-20 Thread Bruce Momjian
On Wed, Apr 14, 2021 at 02:33:26PM -0400, Bruce Momjian wrote: > On Tue, Apr 13, 2021 at 01:30:16PM -0400, Álvaro Herrera wrote: > > On 2021-Apr-12, Bruce Momjian wrote: > > > > > OK, the attached patch renames pg_stat_activity.queryid to 'query_id'. I > > > have not changed any of the APIs which

Re: `make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Andrew Dunstan
On 4/20/21 11:02 AM, Tom Lane wrote: > Aleksander Alekseev writes: >> While trying to build PostgreSQL from source (master branch, 95c3a195) on a >> MacBook I discovered that `make check` fails: > This is the usual symptom of not having disabled SIP :-(. > > If you don't want to do that, do "mak

Re: pg_amcheck option to install extension

2021-04-20 Thread Tom Lane
Andrew Dunstan writes: > On 4/20/21 11:09 AM, Tom Lane wrote: >> Indeed. But I'm down on this idea of inventing src/extensions, >> because then there will constantly be questions about whether FOO >> belongs in contrib/ or src/extensions/. > I think the distinction I would draw is between things

Re: pg_amcheck option to install extension

2021-04-20 Thread Andrew Dunstan
On 4/20/21 11:09 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Actually I think the best balance would be to leave things where they >> are, and move amcheck to src/extensions/ once the next devel cycle >> opens. That way, we avoid the (pretty much pointless) laborious task of >> moving pg_am

Re: `make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Aleksander Alekseev
Hi Tom, Many thanks, running "make install" before "make check" helped. On Tue, Apr 20, 2021 at 6:02 PM Tom Lane wrote: > > Aleksander Alekseev writes: > > While trying to build PostgreSQL from source (master branch, 95c3a195) on a > > MacBook I discovered that `make check` fails: > > This is

Docs: Move parallel_leader_participation GUC description under relevant category

2021-04-20 Thread Bharath Rupireddy
Hi, It looks like even though the commit e5253fdc4f that added the parallel_leader_participation GUC correctly categorized it as RESOURCES_ASYNCHRONOUS parameter in the code, but in the docs it is kept under irrelevant section i.e. "Query Planning/Other Planner Options". This is reported in the bu

Re: "could not find pathkey item to sort" for TPC-DS queries 94-96

2021-04-20 Thread Tom Lane
James Coleman writes: > On Mon, Apr 19, 2021 at 7:10 PM Tom Lane wrote: >> After some more testing, that seems like a good thing to do, >> so here's a v4. > This all looks good to me. Pushed, thanks for reviewing! regards, tom lane

Re: pg_amcheck option to install extension

2021-04-20 Thread Mark Dilger
> On Apr 20, 2021, at 5:54 AM, Robert Haas wrote: > > On Tue, Apr 20, 2021 at 1:31 AM Mark Dilger > wrote: >> I think you are conflating the concept of an operating system adminstrator >> with the concept of the database superuser/owner. > > You should conflate those things, because there's

Re: when the startup process doesn't

2021-04-20 Thread Jehan-Guillaume de Rorthais
On Tue, 20 Apr 2021 15:04:28 +0200 Magnus Hagander wrote: [...] > Yeah, I think we should definitely limit this to local access, one way > or another. Realistically using pg_hba is going to require catalog > access, isn't it? And we can't just go ignore those rows in pg_hba > that for example refe

Re: pg_amcheck option to install extension

2021-04-20 Thread Tom Lane
Alvaro Herrera writes: > Actually I think the best balance would be to leave things where they > are, and move amcheck to src/extensions/ once the next devel cycle > opens. That way, we avoid the (pretty much pointless) laborious task of > moving pg_amcheck to contrib only to move it back on the

Re: pg_amcheck option to install extension

2021-04-20 Thread Andrew Dunstan
On 4/20/21 8:47 AM, Robert Haas wrote: > On Mon, Apr 19, 2021 at 2:55 PM Andrew Dunstan wrote: >> There are at least two other client side programs in contrib. So this >> argument doesn't quite hold water from a consistency POV. > I thought that at first, too. But then I realized that those prog

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-20 Thread Tom Lane
Andrew Dunstan writes: > On 4/19/21 7:22 PM, Tom Lane wrote: >> I wonder whether we could get away with just replacing the $use_tcp >> test with $TestLib::windows_os. It's not really apparent to me >> why we should care about 127.0.0.not-1 on Unix-oid systems. > Yeah > The comment is a bit stran

Re: `make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Tom Lane
Aleksander Alekseev writes: > While trying to build PostgreSQL from source (master branch, 95c3a195) on a > MacBook I discovered that `make check` fails: This is the usual symptom of not having disabled SIP :-(. If you don't want to do that, do "make install" before "make check".

Re: Free port choosing freezes when PostgresNode::use_tcp is used on BSD systems

2021-04-20 Thread Andrew Dunstan
On 4/19/21 7:22 PM, Tom Lane wrote: > Alexey Kondratov writes: >> And this is an absolute true, on BSD-like systems (macOS and FreeBSD >> tested) it hangs on looping through the entire ports range over and over >> when $PostgresNode::use_tcp = 1 is set, since bind fails with: > Hm. > >> That w

`make check` doesn't pass on MacOS Catalina

2021-04-20 Thread Aleksander Alekseev
Hi hackers, While trying to build PostgreSQL from source (master branch, 95c3a195) on a MacBook I discovered that `make check` fails: ``` == removing existing temp instance == == creating temporary instance == == initializing database sy

  1   2   >