Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Fujii Masao
On 2021/03/16 11:59, Kyotaro Horiguchi wrote: At Tue, 16 Mar 2021 03:12:54 +0900, Fujii Masao wrote in The wait event WalReceiverWaitStart has been categorized in the type Client. But why? Walreceiver is waiting for startup process to set the lsn and timeline while it is reporting WalReceiv

RE: libpq debug log

2021-03-15 Thread tsunakawa.ta...@fujitsu.com
I've not finished reviewing yet, but there seems to be many mistakes. I'm sending second set of review comments now so you can fix them in parallel. (8) + charid = '\0'; This initialization is not required because id will always be assigned a value shortly. (9) +static int

Re: pg_amcheck contrib application

2021-03-15 Thread Noah Misch
On Mon, Mar 15, 2021 at 02:57:20PM -0400, Tom Lane wrote: > Mark Dilger writes: > > On Mar 15, 2021, at 10:04 AM, Tom Lane wrote: > >> These animals have somewhat weird alignment properties: MAXALIGN is 8 > >> but ALIGNOF_DOUBLE is only 4. I speculate that that is affecting their > >> choices ab

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Dilip Kumar
On Tue, Mar 16, 2021 at 11:18 AM Justin Pryzby wrote: > > I'm a minor contributor now to a couple bits of this patch set, but I can > answer a couple of these points. > > On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote: > > Comments about 0003: > > - why is HIDE_TOAST_COMPRESSION use

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Justin Pryzby
I'm a minor contributor now to a couple bits of this patch set, but I can answer a couple of these points. On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote: > Comments about 0003: > - why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be > comparable to HIDE_TABLEAM? That

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Dilip Kumar
On Tue, Mar 16, 2021 at 4:28 AM Andres Freund wrote: Replying to some of the comments.. > - Is nodeModifyTable.c really the right place for the logic around > CompareCompressionMethodAndDecompress()? And is doing it in every > place that does "user initiated" inserts really the right way? W

Re: Allow batched insert during cross-partition updates

2021-03-15 Thread Amit Langote
Hi Georgios, On Fri, Mar 12, 2021 at 7:59 PM wrote: > On Friday, March 12, 2021 3:45 AM, Amit Langote > wrote: > > On Thu, Mar 11, 2021 at 8:36 PM gkokola...@pm.me wrote: > > > On Thursday, March 11, 2021 9:42 AM, Amit Langote amitlangot...@gmail.com > > > wrote: > > > > What we do support how

RE: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-15 Thread kuroda.hay...@fujitsu.com
Dear Ikeda-san I think the idea is good. I read the patch and other sources, and I found process_startup_packet_die also execute _exit(1). I think they can be combined into one function and moved to interrupt.c, but some important comments might be removed. How do you think? Best Regards, Hay

RE: subscriptionCheck failures

2021-03-15 Thread osumi.takami...@fujitsu.com
Hello On Tuesday, March 16, 2021 12:31 PM Amit Kapila wrote: > On Mon, Mar 15, 2021 at 6:00 PM Thomas Munro > wrote: > > > > Hi, > > > > This seems to be a new low frequency failure, I didn't see it mentioned > already: Oh, this is the test I wrote and included as part of the commit ce0fdbfe #

Re: subscriptionCheck failures

2021-03-15 Thread Amit Kapila
On Mon, Mar 15, 2021 at 6:00 PM Thomas Munro wrote: > > Hi, > > This seems to be a new low frequency failure, I didn't see it mentioned > already: > Thanks for reporting, I'll look into it. -- With Regards, Amit Kapila.

RE: libpq debug log

2021-03-15 Thread tsunakawa.ta...@fujitsu.com
I'm looking at the last file libpq-trace.c. I'll continue the review after lunch. Below are some comments so far. (1) - Enables tracing of the client/server communication to a debugging file stream. + Enables tracing of the client/server communication to a debugging file + str

Re: Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Kyotaro Horiguchi
At Tue, 16 Mar 2021 03:12:54 +0900, Fujii Masao wrote in > The wait event WalReceiverWaitStart has been categorized in the type > Client. > But why? Walreceiver is waiting for startup process to set the lsn and > timeline while it is reporting WalReceiverWaitStart. So its type > should be IPC, >

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:57 AM, Tom Lane wrote: > > Not sure that I believe the theory that this is from bad luck of > concurrent autovacuum timing, though. The fact that we're seeing > this on just those two animals suggests strongly to me that it's > architecture-correlated, instead. I find

Re: shared-memory based stats collector

2021-03-15 Thread Andres Freund
Hi, On 2021-03-13 10:05:21 -0800, Andres Freund wrote: > Cool. I'll give it a try. I have a few questions about the patch: - Why was collect_oids() changed to a different hashtable as part of this change? Seems fairly independent? - What's the point of all those cached_* stuff? There's not a

Re: Consider Parallelism While Planning For REFRESH MATERIALIZED VIEW

2021-03-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 8:25 PM Bharath Rupireddy wrote: > > > The problem with a case like REFRESH MATERIALIZED VIEW is that there's > > > nothing to prevent something that gets run in the course of the query > > > from trying to access the view (and the heavyweight lock won't prevent > > > that,

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

2021-03-15 Thread Justin Pryzby
On Mon, Mar 15, 2021 at 08:30:18PM -0400, Tom Lane wrote: > + /* Pop the error callback */ > + error_context_stack = error_context_stack->previous; > + > /* > * Once all parameters have been received, prepare for > printing them >

Re: Improvements and additions to COPY progress reporting

2021-03-15 Thread Michael Paquier
On Mon, Mar 15, 2021 at 12:43:40PM +0100, Matthias van de Meent wrote: > Hmm, does CFBot not run checkout on windows with crlf line endings? I > had expected it to do as such. This is environment-sensitive, so I am not surprised that Appveyor changes the way newlines are handled there. I could se

Re: Enhance traceability of wal_level changes for backup management

2021-03-15 Thread Laurenz Albe
On Mon, 2021-03-15 at 17:32 -0400, Stephen Frost wrote: > We explicitly document that people can switch the WAL level and restart > to do bulk data loads faster, and there's certainly no shortage of > discussion (including what prompted this thread..) about doing exactly > that. Adding more docume

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

2021-03-15 Thread Tom Lane
Justin Pryzby writes: > On Mon, Mar 15, 2021 at 06:45:49PM -0400, Tom Lane wrote: >> I started to look at this, and immediately began to wonder where is the >> previous discussion you're evidently referring to. Can you dig up an >> archives link? > I think I was referring to this (from the commi

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Peter Geoghegan
On Mon, Mar 15, 2021 at 4:11 PM Andres Freund wrote: > > > I'm not comfortable with this change without adding more safety > > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit > > > and the xid needs to be frozen, we'll either cause errors or > > > corruption. Yes, that's a

make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-15 Thread Masahiro Ikeda
Hi, This thread came from another thread about collecting the WAL stats([1]). Is it better to make the stats collector shutdown without writing the stats files if the immediate shutdown is requested? There was a related discussion([2]) although it's stopped on 12/1/2016. IIUC, the thread's

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Andres Freund
Hi, On 2021-03-15 13:58:02 -0700, Peter Geoghegan wrote: > On Mon, Mar 15, 2021 at 12:58 PM Peter Geoghegan wrote: > > > I'm not comfortable with this change without adding more safety > > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit > > > and the xid needs to be froze

Re: fdatasync performance problem with large number of DB files

2021-03-15 Thread Thomas Munro
On Tue, Mar 16, 2021 at 3:30 AM Paul Guo wrote: > By the way, there is a usual case that we could skip fsync: A fsync-ed > already standby generated by pg_rewind/pg_basebackup. > The state of those standbys are surely not > DB_SHUTDOWNED/DB_SHUTDOWNED_IN_RECOVERY, so the > pgdata directory is fs

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Andres Freund
Hi, On 2021-03-15 12:58:33 -0700, Peter Geoghegan wrote: > On Mon, Mar 15, 2021 at 12:21 PM Andres Freund wrote: > > It's evil sorcery. Fragile sorcery. I think Robert, Tom and me all run > > afoul of edge cases around it in the last few years. > > Right, which is why I thought that I might be m

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-15 Thread Alvaro Herrera
On 2021-Feb-26, Alvaro Herrera wrote: > Hmm, but if we take this approach, then we're still vulnerable to the > problem that somebody can do DETACH CONCURRENTLY and cancel the wait (or > crash the server), then mess up the state before doing DETACH FINALIZE: > when they cancel the wait, the lock w

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Andres Freund
Hi, On 2021-03-15 15:29:05 -0400, Robert Haas wrote: > On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar wrote: > > In the attached patches I have changed this, ... > > OK, so just looking over this patch series, here's what I think: > > - 0001 and 0002 are now somewhat independent of the rest of this

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

2021-03-15 Thread Justin Pryzby
On Mon, Mar 15, 2021 at 06:45:49PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > For example: > > $ python3.5 -c "import pg; db=pg.DB(); q = db.query(\"SET > > log_parameter_max_length_on_error=-1;\"); db.prepare('p', 'SELECT > > \$1::smallint'); db.query_prepared('p',6);" > > 2021-01-0

Re: [PATCH]: Allow errors in parameter values to be reported during the BIND phase itself..

2021-03-15 Thread Tom Lane
Justin Pryzby writes: > For example: > $ python3.5 -c "import pg; db=pg.DB(); q = db.query(\"SET > log_parameter_max_length_on_error=-1;\"); db.prepare('p', 'SELECT > \$1::smallint'); db.query_prepared('p',6);" > 2021-01-03 02:21:04.547 CST [20157] ERROR: value "6" is out of range for

pg_subscription - substream column?

2021-03-15 Thread Peter Smith
I noticed that the PG docs [1] for the catalog pg_subscription doesn't have any mention of the substream column. Accidental omission by commit 4648243 [2] from last year? [1] https://www.postgresql.org/docs/devel/catalog-pg-subscription.html [2] https://github.com/postgres/postgres/commit/4

Re: Calendar support in localization

2021-03-15 Thread Thomas Munro
Hi Surafel, On Tue, Mar 16, 2021 at 3:48 AM Surafel Temesgen wrote: > My country(Ethiopia) is one of the nations that uses different kind of > calendar than what PostgreSQL have so we are deprived from the benefit of > data datatype. We just uses String to store date that limits our application

Re: Nondeterministic collations and the value returned by GROUP BY x

2021-03-15 Thread Tom Lane
Jim Finnerty writes: > PostgreSQL 12 and onward supports nondeterministic collations. For "GROUP > BY x", which value of 'x' will PostgreSQL return in this case? The first > value of x? > The SQL standard (section 8.2) states that the specific value returned is > implementation-defined, but re

Re: Enhance traceability of wal_level changes for backup management

2021-03-15 Thread Stephen Frost
Greetings, * tsunakawa.ta...@fujitsu.com (tsunakawa.ta...@fujitsu.com) wrote: > From: David Steele > > As a backup software author, I don't see this feature as very useful. > > > > The problem is that there are lots of ways for WAL to go missing so > > monitoring the WAL archive for gaps is esse

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Peter Geoghegan
On Mon, Mar 15, 2021 at 12:58 PM Peter Geoghegan wrote: > > I'm not comfortable with this change without adding more safety > > checks. If there's ever a case in which the HEAPTUPLE_DEAD case is hit > > and the xid needs to be frozen, we'll either cause errors or > > corruption. Yes, that's alread

Re: logical replication worker accesses catalogs in error context callback

2021-03-15 Thread Tom Lane
Bharath Rupireddy writes: > Thanks for pointing to the changes in the commit message. I corrected > them. Attaching v4 patch set, consider it for further review. I took a quick look at this. I'm quite worried about the potential performance cost of the v4-0001 patch (the one for fixing slot_stor

Re: Support tab completion for upper character inputs in psql

2021-03-15 Thread Peter Eisentraut
On 09.02.21 15:48, Tang, Haiying wrote: I'm still confused about the APPROPRIATE behavior of tab completion. It seems ALTER table/tablespace SET/RESET is already case-insensitive. For example # alter tablespace dbspace set(e[tab] # alter tablespace dbspace set(effective_io_concurrency # alter

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Peter Geoghegan
On Mon, Mar 15, 2021 at 12:21 PM Andres Freund wrote: > It's evil sorcery. Fragile sorcery. I think Robert, Tom and me all run > afoul of edge cases around it in the last few years. Right, which is why I thought that I might be missing something; why put up with that at all for so long? > > But

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-15 Thread Alvaro Herrera
On 2021-Mar-15, Justin Pryzby wrote: > Are you going to update the assertion ? > > +#if 0 > >

Re: EXPLAIN/EXPLAIN ANALYZE REFRESH MATERIALIZED VIEW

2021-03-15 Thread Tom Lane
[ Sorry for not looking at this thread sooner ] Bharath Rupireddy writes: > Currently, $subject is not allowed. We do plan the mat view query > before every refresh. I propose to show the explain/explain analyze of > the select part of the mat view in case of Refresh Mat View(RMV). TBH, I think

Re: REINDEX backend filtering

2021-03-15 Thread Julien Rouhaud
On Tue, Mar 16, 2021 at 2:32 AM Mark Dilger wrote: > > We do test corrupt relations. We intentionally corrupt the pages within > corrupted heap tables to check that they get reported as corrupt. (See > src/bin/pg_amcheck/t/004_verify_heapam.pl) I disagree. You're testing a modified version o

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:57 AM, Tom Lane wrote: > > Yeah, that could be phrased better. Attaching the 0001 patch submitted earlier, plus 0002 which fixes the confusing corruption message. v6-0001-Turning-off-autovacuum-during-corruption-tests.patch Description: Binary data v6-0002-Fixin

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Robert Haas
On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar wrote: > In the attached patches I have changed this, ... OK, so just looking over this patch series, here's what I think: - 0001 and 0002 are now somewhat independent of the rest of this work, and could be dropped, but I think they're a good idea, so

Re: New IndexAM API controlling index vacuum strategies

2021-03-15 Thread Andres Freund
Hi, On 2021-03-14 19:04:34 -0700, Peter Geoghegan wrote: > Attached is a POC-quality revision of Masahiko's > skip_index_vacuum.patch [1]. There is an improved design for skipping > index vacuuming (improved over the INDEX_CLEANUP stuff from Postgres > 12). I'm particularly interested in your pers

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:57 AM, Tom Lane wrote: > > Do we have a strong enough lock on > the table under examination to be sure that autovacuum couldn't remove > a dead toast entry before we reach it? The main table and the toast table are only locked with AccessShareLock. Each page in the

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-15 Thread Justin Pryzby
Are you going to update the assertion ? +#if 0 Assert((meta == META

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:32 AM, Mark Dilger > wrote: > > If you had a real, not fake, collation provider which actually provided a > collation with an actual version number, stopped the server, changed the > behavior of the collation as well as its version number, started the server, > and

Re: pg_amcheck contrib application

2021-03-15 Thread Tom Lane
Mark Dilger writes: > On Mar 15, 2021, at 10:04 AM, Tom Lane wrote: >> These animals have somewhat weird alignment properties: MAXALIGN is 8 >> but ALIGNOF_DOUBLE is only 4. I speculate that that is affecting their >> choices about whether an out-of-line TOAST value is needed, breaking >> this t

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2021-03-15 Thread Alvaro Herrera
Here's what seems a final version of the patch. I renamed one more function: PQsendPipeline is now PQpipelineSync. I also reworded the docs in a couple of places, added a few tests to the pgbench patch, and made it work. Note the pgbench results in pipeline mode: ./pgbench -r -Mextended -n -f

Re: Tightening up allowed custom GUC names

2021-03-15 Thread Tom Lane
[ getting back to this, after a bit of procrastination ] Andrew Dunstan writes: > On 2/11/21 1:32 PM, Tom Lane wrote: >> Noah Misch writes: >>> On Tue, Feb 09, 2021 at 05:34:37PM -0500, Tom Lane wrote: * A case could be made for tightening things up a lot more, and not allowing anythin

Re: Parser Hook

2021-03-15 Thread Joshua Drake
> > > > Also, I'm not sure that many extensions would really benefit from custom > utility command, as you can already do pretty much anything you want using > SQL > functions. For instance it would be nice for hypopg to be able to support > > CREATE HYPOTHETICAL INDEX ... > > rather than > > SELE

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:11 AM, Mark Dilger > wrote: > > I will submit a patch that turns off autovacuum for the test node shortly. v5-0001-Turning-off-autovacuum-during-corruption-tests.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise P

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:10 AM, Julien Rouhaud wrote: > > On Mon, Mar 15, 2021 at 10:56:50AM -0700, Mark Dilger wrote: >> >> >>> On Mar 15, 2021, at 10:50 AM, Julien Rouhaud wrote: >>> >>> On Mon, Mar 15, 2021 at 10:40:25AM -0700, Mark Dilger wrote: I'm saying that your patch seems to

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Avinash Kumar
Hi, On Mon, Mar 15, 2021 at 1:18 PM Peter Geoghegan wrote: > On Mon, Mar 15, 2021 at 6:56 AM Avinash Kumar > wrote: > > psql:amchecksql.sql:17: DEBUG: leaf block 1043751 of index > "idx_id_mtime" has no first data item > > That one is harmless. > > > And one error as follows. > > > > psql:amch

Type of wait events WalReceiverWaitStart and WalSenderWaitForWAL

2021-03-15 Thread Fujii Masao
The wait event WalReceiverWaitStart has been categorized in the type Client. But why? Walreceiver is waiting for startup process to set the lsn and timeline while it is reporting WalReceiverWaitStart. So its type should be IPC, instead? The wait event WalSenderWaitForWAL has also been categorized

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 10:04 AM, Tom Lane wrote: > > Looks like we're not quite out of the woods, as hornet and tern are > still unhappy: > > # Failed test 'pg_amcheck excluding all corrupt schemas status (got 2 vs > expected 0)' > # at t/003_check.pl line 498. > > # Failed test 'pg_am

Re: REINDEX backend filtering

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 10:56:50AM -0700, Mark Dilger wrote: > > > > On Mar 15, 2021, at 10:50 AM, Julien Rouhaud wrote: > > > > On Mon, Mar 15, 2021 at 10:40:25AM -0700, Mark Dilger wrote: > >> I'm saying that your patch seems to call down to > >> get_collation_actual_version() via get_collat

Re: Different compression methods for FPI

2021-03-15 Thread Justin Pryzby
On Sun, Mar 14, 2021 at 07:31:35PM -0500, Justin Pryzby wrote: > On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote: > > > 13 марта 2021 г., в 06:28, Justin Pryzby > > > написал(а): > > > Updated patch with a minor fix to configure.ac to avoid warnings on OSX. > > > And 2ndary patches

Re: Parser Hook

2021-03-15 Thread Pavel Stehule
po 15. 3. 2021 v 18:54 odesílatel Julien Rouhaud napsal: > On Mon, Mar 15, 2021 at 06:41:36PM +0100, Pavel Stehule wrote: > > po 15. 3. 2021 v 18:18 odesílatel Julien Rouhaud > > napsal: > > > > > On Mon, Mar 15, 2021 at 06:05:52PM +0100, Pavel Stehule wrote: > > > > > > > > Possibility to work

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 10:50 AM, Julien Rouhaud wrote: > > On Mon, Mar 15, 2021 at 10:40:25AM -0700, Mark Dilger wrote: >> I'm saying that your patch seems to call down to >> get_collation_actual_version() via get_collation_version_for_oid() from your >> new function do_check_index_has_outdat

Re: Parser Hook

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 06:41:36PM +0100, Pavel Stehule wrote: > po 15. 3. 2021 v 18:18 odesílatel Julien Rouhaud > napsal: > > > On Mon, Mar 15, 2021 at 06:05:52PM +0100, Pavel Stehule wrote: > > > > > > Possibility to work with a parser is one main reason for forking > > postgres. > > > Lot of

Re: REINDEX backend filtering

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 10:40:25AM -0700, Mark Dilger wrote: > I'm saying that your patch seems to call down to > get_collation_actual_version() via get_collation_version_for_oid() from your > new function do_check_index_has_outdated_collation(), but I'm not seeing how > that gets exercised. It

Re: Parser Hook

2021-03-15 Thread Pavel Stehule
po 15. 3. 2021 v 18:18 odesílatel Julien Rouhaud napsal: > On Mon, Mar 15, 2021 at 06:05:52PM +0100, Pavel Stehule wrote: > > > > Possibility to work with a parser is one main reason for forking > postgres. > > Lot of interestings projects fail on the cost of maintaining their own > fork. > > > >

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 10:34 AM, Julien Rouhaud wrote: > > On Mon, Mar 15, 2021 at 10:13:55AM -0700, Mark Dilger wrote: >> >> >>> On Mar 15, 2021, at 9:52 AM, Julien Rouhaud wrote: >>> >>> But there are also the tests in collate.icu.utf8.out which will fake >>> outdated >>> collations (tha

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2021-03-15 Thread Tom Lane
David Steele writes: > On 12/15/20 9:03 AM, Peter Eisentraut wrote: >> Here is a new patch for this.  This now follows the implementation that >> Tom has suggested:  Leave date_part() alone, add a new set of extract() >> functions, and map the SQL EXTRACT construct to those.  I have basically >

Re: REINDEX backend filtering

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 10:13:55AM -0700, Mark Dilger wrote: > > > > On Mar 15, 2021, at 9:52 AM, Julien Rouhaud wrote: > > > > But there are also the tests in collate.icu.utf8.out which will fake > > outdated > > collations (that's the original tests for the collation tracking patches) > > a

Re: Parser Hook

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 06:05:52PM +0100, Pavel Stehule wrote: > > Possibility to work with a parser is one main reason for forking postgres. > Lot of interestings projects fail on the cost of maintaining their own fork. > > Maybe a good enough possibility is the possibility to inject an own pars

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 9:52 AM, Julien Rouhaud wrote: > > But there are also the tests in collate.icu.utf8.out which will fake outdated > collations (that's the original tests for the collation tracking patches) and > then check that outdated indexes are reindexed with both REINDEX and REINDEX

Re: documentation fix for SET ROLE

2021-03-15 Thread Bossart, Nathan
On 3/15/21, 7:06 AM, "Laurenz Albe" wrote: > On Fri, 2021-03-12 at 21:41 +, Bossart, Nathan wrote: >> On 3/12/21, 11:14 AM, "Joe Conway" wrote: >> > Looking back at the commit history it seems to me that this only works >> > accidentally. Perhaps it would be best to fix RESET ROLE and be done

Re: Parser Hook

2021-03-15 Thread Pavel Stehule
> Also, I'm not sure that many extensions would really benefit from custom > utility command, as you can already do pretty much anything you want using > SQL > functions. For instance it would be nice for hypopg to be able to support > > CREATE HYPOTHETICAL INDEX ... > > rather than > > SELECT hyp

Re: Parser Hook

2021-03-15 Thread Jim Mlodgenski
On Mon, Mar 15, 2021 at 12:58 PM Joel Jacobson wrote: > On Mon, Mar 15, 2021, at 16:48, Jim Mlodgenski wrote: > > The example I have is adding a CREATE JOB command that a scheduler may > use. > > > This CREATE JOB thing sounds interesting. > > Are you working on adding the ability to schedule SQL

Re: pg_amcheck contrib application

2021-03-15 Thread Tom Lane
Looks like we're not quite out of the woods, as hornet and tern are still unhappy: # Failed test 'pg_amcheck excluding all corrupt schemas status (got 2 vs expected 0)' # at t/003_check.pl line 498. # Failed test 'pg_amcheck excluding all corrupt schemas stdout /(?^:^$)/' # at t/003_chec

Re: Parser Hook

2021-03-15 Thread Jim Mlodgenski
On Mon, Mar 15, 2021 at 12:43 PM Julien Rouhaud wrote: > On Mon, Mar 15, 2021 at 11:48:58AM -0400, Jim Mlodgenski wrote: > > > > Going deeper on this, I created another POC as an example. Yes, having a > > hook at the top of the parser does mean an extension needs to copy the > > existing grammar

Re: Parser Hook

2021-03-15 Thread Joel Jacobson
On Mon, Mar 15, 2021, at 16:48, Jim Mlodgenski wrote: > The example I have is adding a CREATE JOB command that a scheduler may use. This CREATE JOB thing sounds interesting. Are you working on adding the ability to schedule SQL-commands to run in the background, similar to cronjob and/or adding

Re: REINDEX backend filtering

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 09:30:43AM -0700, Mark Dilger wrote: > > In the docs, 0001, "Fow now, the only dependency handled currently", > > "Fow now" is misspelled, and "For now" seems redundant when used with > "currently". > > > In the docs, 0002, "For now only dependency on collations are sup

Re: Parser Hook

2021-03-15 Thread Julien Rouhaud
On Mon, Mar 15, 2021 at 11:48:58AM -0400, Jim Mlodgenski wrote: > > Going deeper on this, I created another POC as an example. Yes, having a > hook at the top of the parser does mean an extension needs to copy the > existing grammar and modify it. Without a total redesign of how the grammar > is h

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 14, 2021, at 8:33 PM, Julien Rouhaud wrote: > > In the docs, 0001, "Fow now, the only dependency handled currently", "Fow now" is misspelled, and "For now" seems redundant when used with "currently". In the docs, 0002, "For now only dependency on collations are supported." "de

Re: Confusing behavior of psql's \e

2021-03-15 Thread Laurenz Albe
On Fri, 2021-03-12 at 13:12 -0500, Tom Lane wrote: > I pushed the race-condition-fixing part of this, since that's an > unarguable bug fix and hence seems OK to back-patch. (I added a > check on change of file size, because why not.) Thank you! > Attached is the rest, just to keep the cfbot happ

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Peter Geoghegan
On Mon, Mar 15, 2021 at 6:56 AM Avinash Kumar wrote: > psql:amchecksql.sql:17: DEBUG: leaf block 1043751 of index "idx_id_mtime" > has no first data item That one is harmless. > And one error as follows. > > psql:amchecksql.sql:17: ERROR: down-link lower bound invariant violated for > index

Re: Proposal: Save user's original authenticated identity for logging

2021-03-15 Thread Jacob Champion
On Tue, 2021-03-09 at 19:10 +, Jacob Champion wrote: > And v5 is rebased over this morning's SSL test changes. Rebased again after the SSL test revert (this is the same as v4). --Jacob From 470bf11f4b8feb6c22dc72626f6f3fcb7971ac26 Mon Sep 17 00:00:00 2001 From: Jacob Champion Date: Wed, 3 Feb

Re: Parser Hook

2021-03-15 Thread Jim Mlodgenski
On Mon, Feb 22, 2021 at 3:52 PM Andres Freund wrote: > Hi, > > On 2021-02-22 11:20:54 -0500, Jim Mlodgenski wrote: > > As Jan mentioned in his thread about a pluggable wire protocol [0], AWS > is > > working on a set of extensions for Babelfish. The intention is to not > > necessarily have it as

Re: Improve join selectivity estimation using extended statistics

2021-03-15 Thread Konstantin Knizhnik
On 11.03.2021 03:47, Tomas Vondra wrote: Hi Konstantin, Thanks for working on this! Using extended statistics to improve join cardinality estimates was definitely on my radar, and this patch seems like a good start. I had two basic ideas about how we might improve join estimates: (a) use pe

Re: MultiXact\SLRU buffers configuration

2021-03-15 Thread Gilles Darold
Le 12/03/2021 à 13:44, Andrey Borodin a écrit : > >> 11 марта 2021 г., в 20:50, Gilles Darold написал(а): >> >> >> The patch doesn't apply anymore in master cause of error: patch failed: >> src/backend/utils/init/globals.c:150 >> >> >> >> An other remark about this patch is that it should be ment

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-03-15 Thread Mark Dilger
> On Mar 2, 2021, at 6:08 AM, Pavel Borisov wrote: > > I completely agree that checking uniqueness requires looking at the heap, but > I don't agree that every caller of bt_index_check on an index wants that > particular check to be performed. There are multiple ways in which an index > mi

Calendar support in localization

2021-03-15 Thread Surafel Temesgen
Hi all, My country(Ethiopia) is one of the nations that uses different kind of calendar than what PostgreSQL have so we are deprived from the benefit of data datatype. We just uses String to store date that limits our application quality greatly. The lag became even worst once application and syste

Re: locking [user] catalog tables vs 2pc vs logical rep

2021-03-15 Thread vignesh C
On Tue, Feb 23, 2021 at 3:59 AM Andres Freund wrote: > > Hi, > > The 2pc decoding added in > > commit a271a1b50e9bec07e2ef3a05e38e7285113e4ce6 > Author: Amit Kapila > Date: 2021-01-04 08:34:50 +0530 > > Allow decoding at prepare time in ReorderBuffer. > > has a deadlock danger when used in

Re: Freeze the inserted tuples during CTAS?

2021-03-15 Thread Paul Guo
> to set visibility map bits on materialized views. I'll start a new >thread to discuss that. Thanks. Also I withdrew the patch.

Re: fdatasync performance problem with large number of DB files

2021-03-15 Thread Paul Guo
> On 2021/3/15, 7:34 AM, "Thomas Munro" wrote: >> On Mon, Mar 15, 2021 at 11:52 AM Thomas Munro wrote: >> Time being of the essence, here is the patch I posted last year, this >> time with a GUC and some docs. You can set sync_after_crash to >> "fsync" (default) or "syncfs"

Re: Regression tests vs SERIALIZABLE

2021-03-15 Thread Tom Lane
Thomas Munro writes: > On Mon, Mar 15, 2021 at 5:24 PM Thomas Munro wrote: >> However, since commit 862ef372d6b, there *is* one test that fails if >> you run make installcheck against a cluster running with -c >> default_transaction_isolation=serializable: transaction.sql. Is that >> a mistake?

Re: 2019-03 CF now in progress

2021-03-15 Thread David Steele
On 3/1/21 10:30 AM, David Steele wrote: Hackers, The 2019-03 commitfest is now in progress. It's a big one as usual. Needs review: 213. Waiting on Author: 21. Ready for Committer: 28. Committed: 29. Withdrawn: 3. Total: 294. We are now halfway through the 2021-03 commitfest, though historical

Re: documentation fix for SET ROLE

2021-03-15 Thread Laurenz Albe
On Fri, 2021-03-12 at 21:41 +, Bossart, Nathan wrote: > On 3/12/21, 11:14 AM, "Joe Conway" wrote: > > Looking back at the commit history it seems to me that this only works > > accidentally. Perhaps it would be best to fix RESET ROLE and be done with > > it. > > That seems reasonable to me.

Re: dynamic result sets support in extended query protocol

2021-03-15 Thread David Steele
Hi Peter, On 12/30/20 9:33 AM, Peter Eisentraut wrote: On 2020-10-09 20:46, Andres Freund wrote: Is there really a good reason for forcing the client to issue NextResult, Describe, Execute for each of the dynamic result sets? It's not like there's really a case for allowing the clients to skip

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Avinash Kumar
Hi, On Sun, Mar 14, 2021 at 11:24 PM Peter Geoghegan wrote: > On Sun, Mar 14, 2021 at 6:54 PM Avinash Kumar > wrote: > > Following may be helpful to understand what I meant. > > > > I have renamed the table and index names before adding it here. > > It should be possible to run amcheck on your

RE: Enhance traceability of wal_level changes for backup management

2021-03-15 Thread osumi.takami...@fujitsu.com
On Friday, March 12, 2021 5:04 PM Peter Eisentraut wrote: > On 08.03.21 03:45, osumi.takami...@fujitsu.com wrote: > > OK. The basic idea is to enable backup management tools to recognize > > wal_level drop between*snapshots*. > > When you have a snapshot of the cluster at one time and another one

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Avinash Kumar
Hi Thomas, On Sun, Mar 14, 2021 at 10:01 PM Avinash Kumar wrote: > Hi Thomas, > > On Sun, Mar 14, 2021 at 9:40 PM Thomas Munro > wrote: > >> On Mon, Mar 15, 2021 at 1:29 PM Avinash Kumar >> wrote: >> > Is this expected when replication is happening between PostgreSQL >> databases hosted on dif

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Avinash Kumar
Hi Thomas, On Sun, Mar 14, 2021 at 9:40 PM Thomas Munro wrote: > On Mon, Mar 15, 2021 at 1:29 PM Avinash Kumar > wrote: > > Is this expected when replication is happening between PostgreSQL > databases hosted on different OS versions like Ubuntu 16 and Ubuntu 20 ? > Or, do we think this is some

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 1:29 PM Avinash Kumar wrote: > Is this expected when replication is happening between PostgreSQL databases > hosted on different OS versions like Ubuntu 16 and Ubuntu 20 ? Or, do we > think this is some sort of corruption ? Is this index on a text datatype, and using a c

Re: allow partial union-all and improve parallel subquery costing

2021-03-15 Thread David Steele
Hi Luc, On 12/30/20 8:54 AM, Luc Vlaming wrote: Created a commitfest entry assuming this is the right thing to do so that someone can potentially pick it up during the commitfest. Providing an updated patch based on latest master. Looks like you need another rebase: http://cfbot.cputube.o

Re: Detecting File Damage & Inconsistencies

2021-03-15 Thread David Steele
On 11/18/20 5:23 AM, Simon Riggs wrote: On Wed, 18 Nov 2020 at 06:42, Craig Ringer wrote: On Fri, Nov 13, 2020 at 7:24 PM Simon Riggs wrote: What I'm proposing is an option to add 16 bytes onto each COMMIT record Would it make sense to write this at the time we write a topxid assignment

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-03-15 Thread David Steele
On 12/23/20 2:27 PM, Stephen Frost wrote: * Justin Pryzby (pry...@telsasoft.com) wrote: On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: * I don't think it's okay to change the existing signatures of pg_ls_logdir() et al. Even if you can make an argument that it's not too harmful to a

Re: Strange behavior with polygon and NaN

2021-03-15 Thread David Steele
On 12/21/20 3:30 AM, Kyotaro Horiguchi wrote: At Tue, 01 Dec 2020 10:03:42 -0500, Tom Lane wrote in I think it should be "needs review" now. Conflicted with some commit(s) uncertain to me. Rebased. Tom, Georgios, thoughts on the new patch? Regards, -- -David da...@pgmasters.net

subscriptionCheck failures

2021-03-15 Thread Thomas Munro
Hi, This seems to be a new low frequency failure, I didn't see it mentioned already: # Failed test 'DROP SUBSCRIPTION during error can clean up the slots on the publisher' # at t/004_sync.pl line 171. # got: '1' # expected: '0' https://buildfarm.postgresql.org/cgi-bin/show_log

  1   2   >