Re: [BUG] orphaned function

2021-03-27 Thread Alvaro Herrera
On 2021-Feb-02, Drouvot, Bertrand wrote: > On 12/18/20 12:26 AM, Tom Lane wrote: > > But any of these options lead to the same question: why stop there? > > An approach that would actually be defensible, perhaps, is to incorporate > > this functionality into the dependency mechanism: any time we'

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2021-03-27 Thread Alvaro Herrera
On 2021-Jan-27, Paul Guo wrote: > Here is a git diff against the previous patch. I’ll send out the new > rebased patches after the consensus is reached. Hmm, can you post a rebased set, where the points under discussion are marked in XXX comments explaining what the issue is? This thread is long

Re: WIP: BRIN multi-range indexes

2021-03-27 Thread Alvaro Herrera
On 2021-Mar-26, Tomas Vondra wrote: > Hi, > > I've pushed both the bloom and minmax-multi indexes today. One thing I've been wondering all along is how useful are these BRIN-backed bloom indexes compared to contrib-supplied bloom indexes. My guess is that the BRIN implementation has some advanta

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-03-27 Thread Alvaro Herrera
Looking at 0001+0003, I see it claims GIN support for <<@ and @>>, but actually only the former is implemented fully; the latter is missing a strategy number in ginarrayproc.c and pg_amop.dat, and also src/test/regress/sql/gin.sql does not test it. I suspect ginqueryarrayextract needs to be told a

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2021-03-27 Thread Alvaro Herrera
On 2021-Mar-27, Mark Rofail wrote: > Hello Alvaro, > > Looking at 0001+0003, I see it claims GIN support for <<@ and @>>, but > > actually only the former is implemented fully; the latter is missing a > > strategy number in ginarrayproc.c and pg_amop.dat, and also > > src/test/regress/sql/gin.sql

Re: Idea: Avoid JOINs by using path expressions to follow FKs

2021-03-27 Thread Alvaro Herrera
On 2021-Mar-27, Joel Jacobson wrote: > If there would be multiple foreign keys on a column we try to follow, > the query planner would throw an error forcing the user to use explicit joins > instead. This seems pretty dangerous -- you just have to create one more FK, and suddenly a query that wo

Re: Rename of triggers for partitioned tables

2021-03-29 Thread Alvaro Herrera
On 2021-Mar-29, Arne Roland wrote: > @@ -1475,7 +1467,12 @@ renametrig(RenameStmt *stmt) > tuple = heap_copytuple(tuple); /* need a modifiable copy */ > trigform = (Form_pg_trigger) GETSTRUCT(tuple); > tgoid = trigform->oid; > - > + if (tgpare

Re: Rename of triggers for partitioned tables

2021-03-29 Thread Alvaro Herrera
On 2021-Mar-29, Arne Roland wrote: > Alvaro Herrera wrote: > > I think this is not what we want to do. What you're doing here as I > > understand is traversing the inheritance hierarchy down using the > > trigger name, and then fail if the trigger with that name has a

Re: psql lacking clearerr()

2021-03-29 Thread Alvaro Herrera
On 2021-Mar-25, Kyotaro Horiguchi wrote: > That worked for me:p And the following steps always raises that error. > > postgres=# select 1; (just to let it into history). > postgres=# C-s -> C-p -> C-m -> C-c > postgres=# select 1; > ... > could not print result table: Success Ah, thanks! Indee

Re: psql lacking clearerr()

2021-03-29 Thread Alvaro Herrera
On 2021-Mar-29, Alvaro Herrera wrote: > (And, yes, I'm to remove the %m too, because clearly that was a mistake.) Re-reading the other thread, I think the %m should stay. -- Álvaro Herrera Valdivia, Chile "I think my standards have lowered enough that now I think 'go

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-30 Thread Alvaro Herrera
On 2021-Mar-30, Michael Paquier wrote: > On Tue, Mar 30, 2021 at 03:50:28PM +0900, Michael Paquier wrote: > > The test_*() ones are just wrappers for psql able to use a customized > > connection string. It seems to me that it would make sense to move > > those two into PostgresNode::psql itself a

Re: multi-install PostgresNode fails with older postgres versions

2021-03-30 Thread Alvaro Herrera
On 2021-Mar-30, Mark Dilger wrote: > The problem is clear enough; -N/--nosync was added in 9.3, and > PostgresNode::init is passing -N to initdb unconditionally. I wonder > if during PostgresNode::new a call should be made to pg_config and the > version information grep'd out so that version speci

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-30 Thread Alvaro Herrera
On 2021-Mar-30, Daniel Gustafsson wrote: > +$node->connect_ok($common_connstr . " " . "user=ssltestuser", > > This double concatenation could be a single concat, or just use scalar value > interpolation in the string to make it even more readable. As it isn't using > the same line broken pattern

Re: multi-install PostgresNode fails with older postgres versions

2021-03-30 Thread Alvaro Herrera
On 2021-Mar-30, Mark Dilger wrote: > Once you have a node running, you can query the version using > safe_psql, but that clearly doesn't work soon enough, since we need > the information prior to running initdb. I was thinking something like examining some file in the install dir -- say, include/

Re: multi-install PostgresNode fails with older postgres versions

2021-03-30 Thread Alvaro Herrera
On 2021-Mar-31, Michael Paquier wrote: > There is already TestLib::check_pg_config(). Shouldn't you leverage > that with PG_VERSION_NUM or equivalent? hmm, I wonder if we shouldn't take the stance that it is not TestLib's business to be calling any Pg binaries. So that routine should be moved t

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-31 Thread Alvaro Herrera
On 2021-Mar-31, Bruce Momjian wrote: > On Wed, Mar 31, 2021 at 11:25:32AM +0800, Julien Rouhaud wrote: > > On Thu, Mar 25, 2021 at 10:36:38AM +0800, Julien Rouhaud wrote: > > > On Wed, Mar 24, 2021 at 01:02:00PM -0300, Alvaro Herrera wrote: > > > > I find it odd t

Re: multi-install PostgresNode fails with older postgres versions

2021-03-31 Thread Alvaro Herrera
On 2021-Mar-31, Mark Dilger wrote: > PostgresNode::start() doesn't work for servers older than version 10, > either. If I hack that function to sleep until the postmaster.pid > file exists, it works, but that is really ugly and is just to prove to > myself that it is a timing issue. There were a

Re: libpq debug log

2021-04-02 Thread Alvaro Herrera
On 2021-Apr-02, Kyotaro Horiguchi wrote: > At Fri, 02 Apr 2021 14:40:09 +0900 (JST), Kyotaro Horiguchi > wrote in > > So, the cheapest measure for regression test would be just making the > > So, the cheapest measure for regression test would be just *masking* the > > > length field, while r

Re: libpq debug log

2021-04-02 Thread Alvaro Herrera
On 2021-Apr-02, Alvaro Herrera wrote: > On 2021-Apr-02, Kyotaro Horiguchi wrote: > > > At Fri, 02 Apr 2021 14:40:09 +0900 (JST), Kyotaro Horiguchi > > wrote in > > > > So, the cheapest measure for regression test would be just making the > > > > So

Re: libpq debug log

2021-04-02 Thread Alvaro Herrera
On 2021-Apr-02, Tom Lane wrote: > I wrote: > > I bet what is happening on drongo is that the compiler has generated a > > __FILE__ value that contains backslashes not slashes, and this code > > doesn't know how to shorten those. So maybe instead of lobotomizing > > this test, we should fix that.

Re: simplifying foreign key/RI checks

2021-04-02 Thread Alvaro Herrera
On 2021-Apr-02, Amit Langote wrote: > On Sat, Mar 20, 2021 at 10:21 PM Amit Langote wrote: > > Updated patches attached. Sorry about the delay. > > Rebased over the recent DETACH PARTITION CONCURRENTLY work. > Apparently, ri_ReferencedKeyExists() was using the wrong snapshot. Hmm, I wonder if

Re: libpq debug log

2021-04-02 Thread Alvaro Herrera
On 2021-Apr-02, Tom Lane wrote: > On third thought, maybe we should push your patch too. Although I think > 53aafdb9f is going to fix the platform-specific aspect of this, we are > still going to risk some implementation dependence of the libpq_pipeline > results: > > * Every so often, the numbe

Re: libpq debug log

2021-04-02 Thread Alvaro Herrera
On 2021-Apr-02, Tom Lane wrote: > Alvaro Herrera writes: > > As in the attached patch. > > +1, but the comment could be more specific. Maybe like "In regress mode, > suppress the length of ErrorResponse and NoticeResponse messages --- the F > (file name) field,

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-04-03 Thread Alvaro Herrera
On 2021-Mar-25, David Steele wrote: > On 1/22/21 4:15 AM, Heikki Linnakangas wrote: > > On 21/01/2021 14:38, Jürgen Purtz wrote: > > > This supervisor process is called > > linkend="glossary-postmaster">postmaster and listens at > > > a specified TCP/IP port for incoming connections. Whenever he

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-04-03 Thread Alvaro Herrera
On 2021-Apr-03, Jürgen Purtz wrote: > On 03.04.21 15:39, Alvaro Herrera wrote: > > Yes, there is. AFAICS Heikki committed a small wordsmithing patch -- > > not the large patch with the additional chapter. > > What can i do to move the matter forward? Please post a versio

Re: [PATCH] Implement motd for PostgreSQL

2021-04-03 Thread Alvaro Herrera
On 2021-Apr-03, Joel Jacobson wrote: > I'm actually using it myself in production for something, to display > instructions to users when they login. Yeah, such as "If your CREATE sentences don't work, please run CREATE SCHEMA AUTHORIZATION CURRENT_USER" for systems where the PUBLIC schema has b

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-03 Thread Alvaro Herrera
Thanks for the quick rework. I like this design much better and I think this is pretty close to committable. Here's a rebased copy with some small cleanups (most notably, avoid calling pgstat_propagate_changes when the partition doesn't have a tabstat entry; also, free the lists that are allocate

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-04-04 Thread Alvaro Herrera
On 2021-Apr-04, Jürgen Purtz wrote: > The small patch 'arch-dev.sgml.20210121.diff' contains only some clearing up > concerning the used terminology and its alignments with the glossary. The > patch was rejected by Heikki. This comment is not helpful, because it's not obvious where would I find t

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-04 Thread Alvaro Herrera
On 2021-Apr-04, Tomas Vondra wrote: > 1) I still don't understand why inheritance and declarative partitioning > are treated differently. Seems unnecessary nad surprising, but maybe > there's a good reason? I suppose the rationale is that for inheritance we have always done it that way -- similar

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-04 Thread Alvaro Herrera
On 2021-Apr-04, Tomas Vondra wrote: > In fact, one of the first posts in this threads links to this: > > https://www.postgresql.org/message-id/4823.1262132964%40sss.pgh.pa.us > > i.e. Tom actually proposed doing something like this back in 2009, so > presumably he though it's desirable back then

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-04-05 Thread Alvaro Herrera
On 2021-Apr-05, Jürgen Purtz wrote: > In addition to this chain Erik introduced in November within the same thread > some changes to the chapter "Overview of Query Handling", which subsequently > was expanded by Heikki and me with the sequence of > 'arch-dev.sgml.x.diff' files. This is what I

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-06, Nitin Jadhav wrote: > I have reviewed the code. Here are a few minor comments. > > 1. > +void > +pgstat_report_queryid(uint64 queryId, bool force) > +{ > + volatile PgBackendStatus *beentry = MyBEEntry; > + > + if (!beentry) > + return; > + > + /* > + * if track_activities is disa

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-04, Tomas Vondra wrote: > 1) I still don't understand why inheritance and declarative partitioning > are treated differently. Seems unnecessary nad surprising, but maybe > there's a good reason? I think there is a good reason to treat them the same: pgstat does not have a provision to

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-07, Thomas Munro wrote: > I wonder if it would be better to have the client code access these > values through functions (even if they just access the variables in a > static inline function), to create a bit more separation? Something > like XLogReaderGetWanted(&page_lsn, &bytes_want

Re: Remove page-read callback from XLogReaderState.

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-07, Thomas Munro wrote: > On Wed, Apr 7, 2021 at 11:18 AM Alvaro Herrera > wrote: > > BTRW it's funny that after these patches, "xlogreader" no longer reads > > anything. It's more an "xlog interpreter" -- the piece of code that &g

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-06 Thread Alvaro Herrera
On 2021-Apr-07, yuzuko wrote: > I'm working on fixing the patch according to the comments. > I'll send it as soon as I can. Thanks, I've been giving it a look too. > I've been thinking about traditional inheritance, I realized that we > need additional > handling to support them because unlike d

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > When I'm creating a new node, I'm using the "pgaTester" factory class. It > relies on PATH to check the major version using pg_config, then loads the > appropriate class. >From a code cleanliness point of view, I agree that having separate clas

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Jehan-Guillaume de Rorthais wrote: > Yes, it would be much saner to make PostgresNode the factory class. Plus, some > more logic could be injected there to either auto-detect the version (current > behavior) or eg. use a given path to the binaries as Mark did in its patch. I'm not

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Andrew Dunstan wrote: > Aren't you likely to end up duplicating substantial amounts of code, > though? No — did you look at his code? Each version is child of the one just above, so you only need to override things where behavior changes from one version to the next. > I'm certa

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Mark Dilger wrote: > It's not sufficient to think about postgres versions as "10", "11", > etc. You have to be able to spin up nodes of any build, like "9.0.7". > There are specific versions of postgres with specific bugs that cause > specific problems, and later versions of postg

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Andrew Dunstan wrote: > b) as it stands pgaTester.pm can't be used for multiple versions in a > single program, which is a design goal here - it sets the single class > to invoke in its BEGIN block. At the very least we would need to replace > that with code which would require the

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Mark Dilger wrote: > I was commenting on the design to have the PostgresNode derived > subclass hard-coded to return "10" as the version: > > sub version { return 10 } That seems a minor bug rather than a showstopper design deficiency. I agree that hardcoding the version in t

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Mark Dilger wrote: > It seems we're debating between two designs. In the first, each > PostgresNode function knows about version limitations and has code > like: > > DoSomething() if $self->at_least_version("11") Yeah, I didn't like this approach -- it is quite messy. > a

Re: multi-install PostgresNode fails with older postgres versions

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Andrew Dunstan wrote: > Oh, you want to roll them all up into one file? That could work. It's a > bit frowned on by perl purists, but I've done similar (see PGBuild/SCM.pm). Ah! Yeah, pretty much exactly like that, including the "no critic" flag ... -- Álvaro Herrera

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-07 Thread Alvaro Herrera
OK, I bit the bullet and re-did the logic in the way I had proposed earlier in the thread: do the propagation on the collector's side, by sending only the list of ancestors: the collector can read the tuple change count by itself, to add it to each ancestor. This seems less wasteful. Attached is

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Bruce Momjian wrote: > On Thu, Apr 8, 2021 at 10:38:08AM +0800, Julien Rouhaud wrote: > > Thanks! And I agree with using query_id in the new field names while > > keeping > > queryid for pg_stat_statements to avoid unnecessary query breakage. > > I think we need more feedback

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Alvaro Herrera wrote: > OK, I bit the bullet and re-did the logic in the way I had proposed > earlier in the thread: do the propagation on the collector's side, by > sending only the list of ancestors: the collector can read the tuple > change count by itself,

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-07 Thread Alvaro Herrera
On 2021-Apr-07, Alvaro Herrera wrote: > However, I just noticed there is a huge problem, which is that the new > code in relation_needs_vacanalyze() is doing find_all_inheritors(), and > we don't necessarily have a snapshot that lets us do that. While adding > a snapshot acquisi

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Tom Lane wrote: > Alvaro Herrera writes: > > autovacuum: handle analyze for partitioned tables > > Looks like this has issues under EXEC_BACKEND: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=culicidae&dt=2021-04-08%2005%3A50%3A08 Hmm,

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Tomas Vondra wrote: > On 4/8/21 5:22 AM, Alvaro Herrera wrote: > > However, I just noticed there is a huge problem, which is that the new > > code in relation_needs_vacanalyze() is doing find_all_inheritors(), and > > we don't necessarily have a sna

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Tomas Vondra wrote: > On 4/8/21 5:27 PM, Alvaro Herrera wrote: > > > Same as for any other relation: ANALYZE would set it, after it's done > > scanning the table. We would to make sure that nothing resets it to > > empty, though, and that it doesn&#x

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Bruce Momjian wrote: > pg_stat_activity.queryid is new, but I can imagine cases where you would > join pg_stat_activity to pg_stat_statements to get an estimate of how > long the query will take --- having one using an underscore and another > one not seems odd. OK. So far, you h

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, James Coleman wrote: > I assume proper procedure for the CF entry is to move it into the > current CF and then mark it as committed, however I don't know how (or > don't have permissions?) to move it into the current CF. How does one > go about doing that? > > Here's the entry: ht

Re: SQL-standard function body

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Julien Rouhaud wrote: > On Thu, Apr 08, 2021 at 02:58:02AM -0400, Tom Lane wrote: > > No, because if that were the explanation then we'd be getting no > > buildfarm coverage at all for for pg_stat_statements. Which aside > > from being awful contradicts the results at coverage.po

Re: VACUUM (DISABLE_PAGE_SKIPPING on)

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Simon Riggs wrote: > On Thu, 8 Apr 2021 at 16:58, David Steele wrote: > > It's not clear to me which patch is which, so perhaps move one CF entry > > to next CF and clarify which patch is current? > > Entry: Maximize page freezing > has this patch, perfectly fine, awaiting revie

Re: Binary search in ScalarArrayOpExpr for OR'd constant arrays

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, James Coleman wrote: > On Thu, Apr 8, 2021 at 1:04 PM Alvaro Herrera wrote: > > > > On 2021-Apr-08, James Coleman wrote: > > > > > I assume proper procedure for the CF entry is to move it into the > > > current CF and then mark it as

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Tom Lane wrote: > Yeah. I hit this on another machine that isn't using EXEC_BACKEND, > and I concur it looks more like a race condition. I think the problem > is that autovacuum is calling find_all_inheritors() on a relation it > has no lock on, contrary to that function's API sp

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-08 Thread Alvaro Herrera
h does that. -- Álvaro Herrera39°49'30"S 73°17'W "I dream about dreams about dreams", sang the nightingale under the pale moon (Sandman) >From a54701552f2ba9295aae4fe0fc22c7bac912bf45 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date:

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Zhihong Yu wrote: > Hi, > Within truncate_update_partedrel_stats(), dirty is declared within the loop. > + if (rd_rel->reltuples != 0) > + { > ... > + if (dirty) > > The two if blocks can be merged. The variable dirty can be dropped. Hi, thanks for reviewing. Y

Re: Autovacuum on partitioned table (autoanalyze)

2021-04-08 Thread Alvaro Herrera
rs for individual leaf partitions." -- Álvaro Herrera39°49'30"S 73°17'W >From 37a829ec7b9c46acbbdb02f231288e39d22fcd04 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Thu, 8 Apr 2021 17:53:22 -0400 Subject: [PATCH] document reloptions for partitioned tables --- doc/

Re: Lots of incorrect comments in nodeFuncs.c

2021-04-08 Thread Alvaro Herrera
On 2021-Apr-08, Tom Lane wrote: > Maybe like > > case T_ScalarArrayOpExpr: > /* ScalarArrayOpExpr's result is boolean ... */ > coll = InvalidOid; /* ... so it has no collation */ > break; This is much clearer, yeah.

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-09 Thread Alvaro Herrera
On 2021-Apr-09, Tom Lane wrote: > Could we get this pushed sooner rather than later? The buildfarm > is showing a wide variety of intermittent failures on HEAD, and it's > hard to tell how many of them trace to this one bug. Pushed now, thanks. -- Álvaro Herrera Valdivia, Chile "Digital

Re: psql - add SHOW_ALL_RESULTS option

2021-04-09 Thread Alvaro Herrera
On 2021-Apr-08, Fabien COELHO wrote: > It is definitely a open item. I'm not sure where you want to add it… > possibly the "Pg 14 Open Items" wiki page? I tried but I do not have enough > privileges, if you can do it please proceed. I added an entry in the next CF > in the bugfix section. User "c

Re: libpq debug log

2021-04-09 Thread Alvaro Herrera
On 2021-Apr-02, Tom Lane wrote: > Alvaro Herrera writes: > > On 2021-Apr-02, Tom Lane wrote: > >> +1, but the comment could be more specific. Maybe like "In regress mode, > >> suppress the length of ErrorResponse and NoticeResponse messages --- the F > >&

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-09 Thread Alvaro Herrera
On 2021-Apr-09, Robert Haas wrote: > Does this need to worry about new partitions getting attached to a > partitioned table, or old ones getting detached? (Maybe it does > already, not sure.) Good question. It does not. I suppose you could just let that happen automatically -- I mean, next time

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-09 Thread Alvaro Herrera
On 2021-Apr-09, Justin Pryzby wrote: > One data point: we do DETACH/ATTACH tables during normal operation, before > type-promoting ALTERs, to avoid worst-case disk use, and to avoid locking the > table for a long time. It'd be undesirable (but maybe of no great > consequence) > to trigger an ALT

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-04-09 Thread Alvaro Herrera
On 2021-Mar-25, Amul Sul wrote: > Ok, in the attached patch, I have added the inline function to rel.h, and for > that, I end up including smgr.h to rel.h. I tried to replace all rel->rd_smgr > by RelationGetSmgr() function and removed the RelationOpenSmgr() call from > the nearby to it which I do

Re: pgsql: Add libpq pipeline mode support to pgbench

2021-04-09 Thread Alvaro Herrera
On 2021-Mar-17, Daniel Verite wrote: > Fabien COELHO wrote: > > > For consistency with the existing \if … \endif, ISTM that it could have > > been named \batch … \endbatch or \pipeline … \endpipeline? > > "start" mirrors "end". To me, the analogy with \if-\endif is not > obvious. > Gramma

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-09 Thread Alvaro Herrera
Hello On 2021-Apr-09, Andres Freund wrote: > I assume this is also the likely explanation for / fix for: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2021-04-08%2016%3A03%3A03 > > ==3500389== VALGRINDERROR-BEGIN > ==3500389== Invalid read of size 8 > ==3500389==at 0x4EC

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-11 Thread Alvaro Herrera
On 2021-Mar-31, Tom Lane wrote: > diff -U3 > /home/buildfarm/trilobite/buildroot/HEAD/pgsql.build/src/test/isolation/expected/detach-partition-concurrently-4.out > > /home/buildfarm/trilobite/buildroot/HEAD/pgsql.build/src/test/isolation/output_iso/results/detach-partition-concurrently-4.out >

Re: psql - add SHOW_ALL_RESULTS option

2021-04-12 Thread Alvaro Herrera
On 2021-Apr-12, Bossart, Nathan wrote: > The following patch seems to resolve the issue, although I'll admit I > haven't dug into this too deeply. In any case, +1 for reverting the > patch for now. Please note that there's no "for now" about it -- if the patch is reverted, the only way to get it

Re: Proposal for working on open source with PostgreSQL

2021-04-12 Thread Alvaro Herrera
On 2021-Apr-12, Laurenz Albe wrote: > I couldn't see any detail information about the project in your proposal, > except > that the project is called "plsample". Is there more information somewhere? > > If it is a procedural language as the name suggests, you probably don't have > to modify Pos

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-13 Thread Alvaro Herrera
On 2021-Apr-13, Amit Langote wrote: > Actually it occurred to me this morning that CLOBBER_CACHE_ALWAYS is > what exposed this problem on this animal (not sure if other such > animals did too though). With CLOBBER_CACHE_ALWAYS, a PartitionDesc > will be built afresh on most uses. In this particu

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-04-13 Thread Alvaro Herrera
On 2021-Apr-12, Bruce Momjian wrote: > OK, the attached patch renames pg_stat_activity.queryid to 'query_id'. I > have not changed any of the APIs which existed before this feature was > added, and are called "queryid" or "queryId" --- it is kind of a mess. > I assume I should leave those unchang

Re: expose parallel leader in CSV and log_line_prefix

2020-07-22 Thread Alvaro Herrera
On 2020-Jul-21, Michael Paquier wrote: > On Mon, Jul 20, 2020 at 11:12:31PM -0500, Justin Pryzby wrote: > >> + Process ID of the parallel group leader if this process is involved > >> + in parallel query, or null. For a parallel group leader, this > >> field > >> + is NULL. >

Re: heap_abort_speculative() sets xmin to Invalid* without HEAP_XMIN_INVALID

2020-07-23 Thread Alvaro Herrera
On 2020-Jul-23, Andres Freund wrote: > I think we should change heap_abort_speculative() to set > HEAP_XMIN_INVALID in master. +1 > But we can't really do anything about > existing tuples without it - therefore we will have to forever take care > about encountering that combination :(. > > Perh

Re: [BUG] Error in BRIN summarization

2020-07-27 Thread Alvaro Herrera
On 2020-Jul-27, Anastasia Lubennikova wrote: > Here is the updated version of the fix. > The problem can be reproduced on all supported versions, so I suggest to > backpatch it. > Code slightly changed in v12, so here are two patches: one for versions 9.5 > to 11 and another for versions from 12 t

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Alvaro Herrera
On 2020-Jul-23, Peter Geoghegan wrote: > Attached is v3 of the hash_mem_multiplier patch series, which now has > a preparatory patch that removes hashagg_avoid_disk_plan. I notice you put the prototype for get_hash_mem in nodeHash.h. This would be fine if not for the fact that optimizer needs to

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Alvaro Herrera
On 2020-Jul-27, Peter Geoghegan wrote: > On Mon, Jul 27, 2020 at 10:30 AM Alvaro Herrera > wrote: > > On 2020-Jul-23, Peter Geoghegan wrote: > > I notice you put the prototype for get_hash_mem in nodeHash.h. This > > would be fine if not for the fact that opt

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Alvaro Herrera
On 2020-Jul-27, Peter Geoghegan wrote: > The v4-0001-Remove-hashagg_avoid_disk_plan-GUC.patch changes are > surprisingly complicated. It would be nice if you could take a look at > that aspect (or confirm that it's included in your review). I think you mean "it replaces surprisingly complicated c

Re: [PATCH] - Provide robust alternatives for replace_string

2020-07-31 Thread Alvaro Herrera
What happens if a replacement string happens to be split in the middle by the fgets buffering? I think it'll fail to be replaced. This applies to both versions. In the stringinfo version it seemed to me that using pnstrdup is possible to avoid copying trailing bytes. If you're asking for opinio

Re: [PATCH] - Provide robust alternatives for replace_string

2020-08-03 Thread Alvaro Herrera
On 2020-Aug-03, Asim Praveen wrote: > Thank you Alvaro for reviewing the patch! > > > On 01-Aug-2020, at 7:22 AM, Alvaro Herrera wrote: > > > > What happens if a replacement string happens to be split in the middle > > by the fgets buffering? I think it

ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-03 Thread Alvaro Herrera
x27;s just too problematic a case; you would still need to have AEL on the default partition. I haven't yet experimented with queries running in a standby in tandem with a detach. -- Álvaro Herrera >From 2f9202bf6f4c86d607032d7f04d3b2cee74a9617 Mon Sep 17 00:00:00 2001 From: Alvaro Herr

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-03 Thread Alvaro Herrera
On 2020-Aug-03, Alvaro Herrera wrote: > There was a lot of great discussion which ended up in Robert completing > a much sought implementation of non-blocking ATTACH. DETACH was > discussed too because it was a goal initially, but eventually dropped > from that patch altogether.

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-08-04 Thread Alvaro Herrera
On 2020-Aug-03, Alvaro Herrera wrote: > Why two transactions? The reason is that in order for this to work, we > make a catalog change (mark it detached), and commit so that all > concurrent transactions can see the change. A second transaction waits > for anybody who holds any

Re: [DOC] Document concurrent index builds waiting on each other

2020-08-04 Thread Alvaro Herrera
utable functions?) -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From a691c48ddd5d9ff99e2f17b91777028bd5fbf36b Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Tue, 4 Aug 2020 22:04:57 -0400 Subject: [PATCH] Flag CREATE

Re: [DOC] Document concurrent index builds waiting on each other

2020-08-04 Thread Alvaro Herrera
On 2020-Aug-04, Alvaro Herrera wrote: > diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h > index b20e2ad4f6..43c8ea3e31 100644 > --- a/src/include/storage/proc.h > +++ b/src/include/storage/proc.h > @@ -53,6 +53,8 @@ struct XidCache > #define

Re: [PATCH] - Provide robust alternatives for replace_string

2020-08-05 Thread Alvaro Herrera
On 2020-Aug-05, Asim Praveen wrote: > Please find attached a StringInfo based solution to this problem. It > uses fgetln instead of fgets such that a line is read in full, without > ever splitting it. never heard of fgetln, my system doesn't have a manpage for it, and we don't use it anywhere AF

PROC_IN_ANALYZE stillborn 13 years ago

2020-08-05 Thread Alvaro Herrera
ining & Services >From f96b28defe856e284a44b207c2016c72a48a2ff2 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Wed, 5 Aug 2020 18:57:50 -0400 Subject: [PATCH] Remove PROC_IN_ANALYZE --- src/backend/commands/analyze.c | 13 + src/include/storage/proc.h | 3 +-- src/inc

walsender waiting_for_ping spuriously set

2020-08-06 Thread Alvaro Herrera
lat/blu436-smtp25712b7ef9fc2adeb87c522dc...@phx.gbl -- Álvaro Herrera Valdivia, Chile >From bf5ca106817744c1ba300ecb8c86d230789988e0 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Thu, 6 Aug 2020 14:57:09 -0400 Subject: [PATCH] Fix waiting_for_ping in walsender --- src/backen

Re: PROC_IN_ANALYZE stillborn 13 years ago

2020-08-07 Thread Alvaro Herrera
On 2020-Aug-05, Andres Freund wrote: > I'm mildly against that, because I'd really like to start making use of > the flag. Not so much for cancellations, but to avoid the drastic impact > analyze has on bloat. In OLTP workloads with big tables, and without > disabled cost limiting for analyze (or

Re: walsender waiting_for_ping spuriously set

2020-08-07 Thread Alvaro Herrera
I just noticed that part of this comment I'm modifying: > @@ -1444,17 +1444,13 @@ WalSndWaitForWal(XLogRecPtr loc) >* We only send regular messages to the client for full decoded >* transactions, but a synchronous replication and walsender > shutdown >

Re: walsender waiting_for_ping spuriously set

2020-08-07 Thread Alvaro Herrera
On 2020-Aug-07, Alvaro Herrera wrote: > I'm thinking in keeping the sentences that were added in that commit, > maybe like so: > > > * We only send regular messages to the client for full decoded > > * transactions, but a synchronous

Re: walsender waiting_for_ping spuriously set

2020-08-08 Thread Alvaro Herrera
Pushed. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

massive FPI_FOR_HINT load after promote

2020-08-10 Thread Alvaro Herrera
Last week, James reported to us that after promoting a replica, some seqscan was taking a huge amount of time; on investigation he saw that there was a high rate of FPI_FOR_HINT wal messages by the seqscan. Looking closely at the generated traffic, HEAP_XMIN_COMMITTED was being set on some tuples.

remove spurious CREATE INDEX CONCURRENTLY wait

2020-08-10 Thread Alvaro Herrera
enderás" (Confucio) >From 2596c3033aacceb021463f58b50e2c4eed8a5ab2 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Tue, 4 Aug 2020 22:04:57 -0400 Subject: [PATCH] Flag CREATE INDEX CONCURRENTLY to avoid spurious waiting --- src/backend/commands/indexcmds.c | 13 +++-- src/include/storage/pr

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2020-08-10 Thread Alvaro Herrera
+ James Coleman -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: massive FPI_FOR_HINT load after promote

2020-08-11 Thread Alvaro Herrera
On 2020-Aug-11, Masahiko Sawada wrote: > On Tue, 11 Aug 2020 at 07:56, Alvaro Herrera wrote: > > So if you have some table where tuples gain hint bits in bulk, and > > rarely modify the pages afterwards, and promote before those pages are > > frozen, then you may end up wit

Re: [BUG] Error in BRIN summarization

2020-08-11 Thread Alvaro Herrera
On 2020-Jul-30, Anastasia Lubennikova wrote: > While testing this fix, Alexander Lakhin spotted another problem. I > simplified  the test case to this: Ah, good catch. I think a cleaner way to fix this problem is to just consider the range as not summarized and return NULL from there, as in the

Re: [BUG] Error in BRIN summarization

2020-08-11 Thread Alvaro Herrera
On 2020-Jul-23, Anastasia Lubennikova wrote: > This error is caused by the problem with root_offsets array bounds. It > occurs if a new HOT tuple was inserted after we've collected root_offsets, > and thus we don't have root_offset for tuple's offnum. Concurrent insertions > are possible, because

<    1   2   3   4   5   6   7   8   9   10   >