Re: Schema variables - new implementation for Postgres 15

2021-09-10 Thread Pavel Stehule
ks run without errors for both assert-disabled > and assert-enabled) > > Please, check, attached patch. I fixed a routine for processing a list of identifiers - now it works with the identifier's node more sensitive. Previous implementation of strVal was more tolerant. Regards Pavel

Re: Teach pg_receivewal to use lz4 compression

2021-09-10 Thread gkokolatos
‐‐‐ Original Message ‐‐‐ On Friday, July 9th, 2021 at 04:49, Michael Paquier wrote: Hi, please find v3 of the patch attached, rebased to the current head. > > Michael Paquier wrote: > > > > * http://www.zlib.org/rfc-gzip.html. > > - - For lz4 compressed segments > */ >

Re: Schema variables - new implementation for Postgres 15

2021-09-10 Thread Erik Rijkers
t-disabled and assert-enabled) Please, check, attached patch. I fixed a routine for processing a list of identifiers - now it works with the identifier's node more sensitive. Previous implementation of strVal was more tolerant. > [schema-variables-20210910.patch] Apply, compile, ma

Re: Schema variables - new implementation for Postgres 15

2021-09-10 Thread Pavel Stehule
check, attached patch. I fixed a routine for processing a list of > > identifiers - now it works with the identifier's node more sensitive. > > Previous implementation of strVal was more tolerant. > > > [schema-variables-20210910.patch] > > Apply, compile, make, &a

The End of the WAL

2021-09-10 Thread Christoph Berg
Re: Heikki Linnakangas > On 09/09/2021 17:25, Tom Lane wrote: > > Having done that, the check in md.c could be reduced to an Assert, > > although there's something to be said for leaving it as-is as an > > extra layer of defense. > > Some operations call smgrextend() directly, like B-tree index cr

Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails

2021-09-10 Thread Etsuro Fujita
On Fri, Sep 10, 2021 at 1:00 AM Tom Lane wrote: > Etsuro Fujita writes: > > Having said that, I think another option for this would be to left the > > code as-is; assume that 1) the foreign var has "COLLATE default”, not > > an unknown collation, when labeled with "COLLATE default”, and 2) > > "C

Re: Skipping logical replication transactions on subscriber side

2021-09-10 Thread Greg Nancarrow
On Fri, Sep 10, 2021 at 12:33 AM Masahiko Sawada wrote: > > Sorry for the late response. I've attached the updated patches that > incorporate all comments unless I missed something. Please review > them. > Here's some review comments for the v13-0001 patch: doc/src/sgml/monitoring.sgml (1) Ther

Re: parallelizing the archiver

2021-09-10 Thread Robert Haas
On Tue, Sep 7, 2021 at 6:36 PM Bossart, Nathan wrote: > Based on previous threads I've seen, I believe many in the community > would like to replace archive_command entirely, but what I'm proposing > here would build on the existing tools. I'm currently thinking of > something a bit like autovacu

RE: Support tab completion for upper character inputs in psql

2021-09-10 Thread tanghy.f...@fujitsu.com
On Tuesday, September 7, 2021 5:25 PM, Peter Eisentraut wrote: >The coding of valid_input_text() seems a bit bulky. I think you can do >the same thing using strspn() without a loop. Thanks, modified in V9 patch. >The name is also not great. It's not like other strings are not "valid". Modi

Re: a misbehavior of partition row movement (?)

2021-09-10 Thread Amit Langote
On Fri, Sep 3, 2021 at 12:23 PM Amit Langote wrote: > Hi Andrew, > > On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan wrote: > > On 7/13/21 8:09 AM, Amit Langote wrote: > > > Unfortunately, I don’t think I’ll have time in this CF to solve some > > > very fundamental issues I found in the patch durin

Re: EXPLAIN(VERBOSE) to CTE with SEARCH BREADTH FIRST fails

2021-09-10 Thread torikoshia
On 2021-09-09 19:03, Peter Eisentraut wrote: On 07.09.21 20:31, Tom Lane wrote: torikoshia writes: While working on [1], we found that EXPLAIN(VERBOSE) to CTE with SEARCH BREADTH FIRST ends up ERROR. Yeah. It's failing here: * We're deparsing a Plan tree so we don't

Re: parallelizing the archiver

2021-09-10 Thread Julien Rouhaud
On Fri, Sep 10, 2021 at 9:13 PM Robert Haas wrote: > > To me, it seems way more beneficial to think about being able to > invoke archive_command with many files at a time instead of just one. > I think for most plausible archive commands that would be way more > efficient than what you propose her

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2021-09-10 Thread Bharath Rupireddy
On Fri, Sep 10, 2021 at 7:21 AM Michael Paquier wrote: > > On Thu, Sep 09, 2021 at 10:49:46PM +, Bossart, Nathan wrote: > > +1 > > A backend approach has the advantage that you can use the proper locks > to make sure that a segment is not recycled or removed by a concurrent > checkpoint, so th

Re: [Patch] ALTER SYSTEM READ ONLY

2021-09-10 Thread Amul Sul
On Thu, Sep 9, 2021 at 11:12 PM Mark Dilger wrote: > > Thank you, for looking at the patch. Please see my reply inline below: > > > On Sep 8, 2021, at 6:44 AM, Amul Sul wrote: > > > > Here is the rebased version. > > v33-0004 > > This patch moves the include of "catalog/pg_control.h" from tran

Re: Increase value of OUTER_VAR

2021-09-10 Thread Aleksander Alekseev
Hi hackers, > > So I'm inclined to propose pushing this and seeing what happens. > > +1 +1. The proposed changes will be beneficial in the long term. They will affect existing extensions. However, the scale of the problem seems to be exaggerated. I can confirm that the patch passes installcheck-

Re: parallelizing the archiver

2021-09-10 Thread Robert Haas
On Fri, Sep 10, 2021 at 10:19 AM Julien Rouhaud wrote: > Those approaches don't really seems mutually exclusive? In both case > you will need to internally track the status of each WAL file and > handle non contiguous file sequences. In case of parallel commands > you only need additional knowle

Re: [Patch] ALTER SYSTEM READ ONLY

2021-09-10 Thread Mark Dilger
> On Sep 10, 2021, at 7:36 AM, Amul Sul wrote: > >> v33-0005 >> >> This patch makes bool XLogInsertAllowed() more complicated than before. The >> result used to depend mostly on the value of LocalXLogInsertAllowed except >> that when that value was negative, the result was determined by >

Re: a misbehavior of partition row movement (?)

2021-09-10 Thread Zhihong Yu
On Fri, Sep 10, 2021 at 7:06 AM Amit Langote wrote: > On Fri, Sep 3, 2021 at 12:23 PM Amit Langote > wrote: > > Hi Andrew, > > > > On Fri, Sep 3, 2021 at 6:19 AM Andrew Dunstan > wrote: > > > On 7/13/21 8:09 AM, Amit Langote wrote: > > > > Unfortunately, I don’t think I’ll have time in this CF

Re: parallelizing the archiver

2021-09-10 Thread Julien Rouhaud
On Fri, Sep 10, 2021 at 11:22 PM Robert Haas wrote: > > Well, I guess I'm not convinced. Perhaps people with more knowledge of > this than I may already know why it's beneficial, but in my experience > commands like 'cp' and 'scp' are usually limited by the speed of I/O, > not the fact that you on

Re: parallelizing the archiver

2021-09-10 Thread Andrey Borodin
> 10 сент. 2021 г., в 19:19, Julien Rouhaud написал(а): > Wouldn't it be better to > have a new archive_mode, e.g. "daemon", and have postgres responsible > to (re)start it, and pass information through the daemon's > stdin/stdout or something like that? We don't even need to introduce new arc

Re: [Patch] ALTER SYSTEM READ ONLY

2021-09-10 Thread Mark Dilger
> On Sep 10, 2021, at 8:42 AM, Mark Dilger wrote: > > Take for example a code stanza from heapam.c: > >if (needwal) >CheckWALPermitted(); > >/* NO EREPORT(ERROR) from here till changes are logged */ >START_CRIT_SECTION(); > > Now, I know that interrupts won't be processe

Re: Toast compression method options

2021-09-10 Thread Jaime Casanova
On Fri, Sep 10, 2021 at 10:54:04AM +0530, Dilip Kumar wrote: > On Fri, 10 Sep 2021 at 10:40 AM, Jaime Casanova < > jcasa...@systemguards.com.ec> wrote: > > > On Mon, Jul 19, 2021 at 01:24:03PM +0530, Dilip Kumar wrote: > > > On Sun, Jul 18, 2021 at 9:15 PM Dilip Kumar > > wrote: > > > > > > > > O

Re: [Patch] ALTER SYSTEM READ ONLY

2021-09-10 Thread Robert Haas
On Fri, Sep 10, 2021 at 12:20 PM Mark Dilger wrote: > A better example may be found in ginmetapage.c: > > needwal = RelationNeedsWAL(indexrel); > if (needwal) > { > CheckWALPermitted(); > computeLeafRecompressWALData(leaf); > } > > /*

Re: parallelizing the archiver

2021-09-10 Thread Bossart, Nathan
On 9/10/21, 8:22 AM, "Robert Haas" wrote: > On Fri, Sep 10, 2021 at 10:19 AM Julien Rouhaud wrote: >> Those approaches don't really seems mutually exclusive? In both case >> you will need to internally track the status of each WAL file and >> handle non contiguous file sequences. In case of par

Re: parallelizing the archiver

2021-09-10 Thread Jacob Champion
On Fri, 2021-09-10 at 23:48 +0800, Julien Rouhaud wrote: > I totally agree that batching as many file as possible in a single > command is probably what's gonna achieve the best performance. But if > the archiver only gets an answer from the archive_command once it > tried to process all of the fi

Re: parallelizing the archiver

2021-09-10 Thread Robert Haas
On Fri, Sep 10, 2021 at 11:49 AM Julien Rouhaud wrote: > I totally agree that batching as many file as possible in a single > command is probably what's gonna achieve the best performance. But if > the archiver only gets an answer from the archive_command once it > tried to process all of the fil

incorrect file name in backend_progress.c header

2021-09-10 Thread Zhihong Yu
Hi, I was looking at backend_progress.c and noticed that the filename and path were wrong in the header. Here is patch which corrects the mistake. Please take a look. Thanks backend_prog-hdr.patch Description: Binary data

Re: [Patch] ALTER SYSTEM READ ONLY

2021-09-10 Thread Mark Dilger
> On Sep 10, 2021, at 9:56 AM, Robert Haas wrote: > > I think the relevant question here is not "could a signal handler > fire?" but "can we hit a CHECK_FOR_INTERRUPTS()?". If the relevant > question is the former, then there's no hope of ever making it work > because there's always a race con

Re: parallelizing the archiver

2021-09-10 Thread Bossart, Nathan
On 9/10/21, 10:12 AM, "Robert Haas" wrote: > If on the other hand you imagine a system that's not very busy, say 1 > WAL file being archived every 10 seconds, then using a batch size of > 30 would very significantly delay removal of old files. However, on > this system, batching probably isn't rea

Re: Numeric x^y for negative x

2021-09-10 Thread Jaime Casanova
On Thu, Sep 02, 2021 at 07:27:09AM +0100, Dean Rasheed wrote: > On Thu, 2 Sept 2021 at 00:39, Jaime Casanova > wrote: > > > > Hi Dean, > > > > It seems you already committed this. But it's still as "Ready for > > committer" in the commitfest app. > > > > Are we waiting for something else or we can

Re: parallelizing the archiver

2021-09-10 Thread Robert Haas
On Fri, Sep 10, 2021 at 1:07 PM Bossart, Nathan wrote: > That being said, I think the discussion about batching is a good one > to have. If the overhead described in your SCP example is > representative of a typical archive_command, then parallelism does > seem a bit silly. I think that's pretty

Re: [Patch] ALTER SYSTEM READ ONLY

2021-09-10 Thread Robert Haas
On Fri, Sep 10, 2021 at 1:16 PM Mark Dilger wrote: > uses "immediately" and "will kill the running transaction" which reenforced > the impression that this mechanism is heavier handed than it is. It's intended to be just as immediate as e.g. pg_cancel_backend() and pg_terminate_backend(), which

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Justin Pryzby
On Fri, Sep 10, 2021 at 10:11:34AM -0700, Zhihong Yu wrote: > Hi, > I was looking at backend_progress.c and noticed that the filename and path > were wrong in the header. > > Here is patch which corrects the mistake. For the record, I don't really like boilerplate, but fixing the boilerplates all

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Zhihong Yu
On Fri, Sep 10, 2021 at 10:56 AM Justin Pryzby wrote: > On Fri, Sep 10, 2021 at 10:11:34AM -0700, Zhihong Yu wrote: > > Hi, > > I was looking at backend_progress.c and noticed that the filename and > path > > were wrong in the header. > > > > Here is patch which corrects the mistake. > > For the

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Zhihong Yu
On Fri, Sep 10, 2021 at 11:07 AM Zhihong Yu wrote: > > > On Fri, Sep 10, 2021 at 10:56 AM Justin Pryzby > wrote: > >> On Fri, Sep 10, 2021 at 10:11:34AM -0700, Zhihong Yu wrote: >> > Hi, >> > I was looking at backend_progress.c and noticed that the filename and >> path >> > were wrong in the hea

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Justin Pryzby
> For the first list, do you want to include the path to the file for > IDENTIFICATION ? > If so, I can prepare a patch covering the files in that list. Since there's so few exceptions to the "rule", I think they should be fixed for consistency. Here's an awk which finds a few more - including th

Re: incorrect file name in backend_progress.c header

2021-09-10 Thread Zhihong Yu
On Fri, Sep 10, 2021 at 11:20 AM Justin Pryzby wrote: > > For the first list, do you want to include the path to the file for > > IDENTIFICATION ? > > If so, I can prepare a patch covering the files in that list. > > Since there's so few exceptions to the "rule", I think they should be > fixed fo

Re: WIP: System Versioned Temporal Table

2021-09-10 Thread Jaime Casanova
On Tue, Aug 10, 2021 at 01:20:14PM +0100, Simon Riggs wrote: > On Wed, 14 Jul 2021 at 12:48, vignesh C wrote: > > > The patch does not apply on Head anymore, could you rebase and post a > > patch. I'm changing the status to "Waiting for Author". > > OK, so I've rebased the patch against current

Re: parallelizing the archiver

2021-09-10 Thread Andrey Borodin
> 10 сент. 2021 г., в 22:18, Bossart, Nathan написал(а): > > I was thinking that archive_batch_size would be the maximum batch > size. If the archiver only finds a single file to archive, that's all > it'd send to the archive command. If it finds more, it'd send up to > archive_batch_size to

Re: extensible options syntax for replication parser?

2021-09-10 Thread Robert Haas
On Thu, Oct 8, 2020 at 10:33 AM Robert Haas wrote: > That patch set includes this patch, and the reason for the behavior > difference turned out to be that I had gotten an if-test that is part > of this patch backwards. Here is v3, fixing that. It is a little > disappointing that this mistake didn

Re: Estimating HugePages Requirements?

2021-09-10 Thread Robert Haas
On Thu, Sep 9, 2021 at 5:53 PM Bossart, Nathan wrote: > For 0001, the biggest thing on my mind at the moment is the name of > the GUC. "huge_pages_required" feels kind of ambiguous. From the > name alone, it could mean either "the number of huge pages required" > or "huge pages are required for

Re: slab allocator performance issues

2021-09-10 Thread Tomas Vondra
Hi, I've been investigating the regressions in some of the benchmark results, together with the generation context benchmarks [1]. Turns out it's pretty difficult to benchmark this, because the results strongly depend on what the backend did before. For example if I run slab_bench_fifo with

Re: ORDER BY pushdowns seem broken in postgres_fdw

2021-09-10 Thread David Zhang
On 2021-09-06 1:16 a.m., Ronan Dunklau wrote: Le vendredi 3 septembre 2021, 22:54:25 CEST David Zhang a écrit : The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, passed Spec compliant: not

Re: Remove_temp_files_after_crash and significant recovery/startup time

2021-09-10 Thread Tom Lane
"McCoy, Shawn" writes: > I noticed that the new parameter remove_temp_files_after_crash is currently > set to a default value of "true" in the version 14 release. It seems this was > discussed in this thread [1], and it doesn't look to me like there's been a > lot of stress testing of this feat

Re: Remove_temp_files_after_crash and significant recovery/startup time

2021-09-10 Thread Tomas Vondra
On 9/10/21 10:58 PM, McCoy, Shawn wrote: I noticed that the new parameter remove_temp_files_after_crash is currently set to a default value of "true" in the version 14 release. It seems this was discussed in this thread [1], and it doesn't look to me like there's been a lot of stress testing of

Re: Remove_temp_files_after_crash and significant recovery/startup time

2021-09-10 Thread Euler Taveira
On Fri, Sep 10, 2021, at 5:58 PM, McCoy, Shawn wrote: > I noticed that the new parameter remove_temp_files_after_crash is currently > set to a default value of "true" in the version 14 release. It seems this was > discussed in this thread [1], and it doesn't look to me like there's been a > lot

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2021-09-10 Thread Melanie Plageman
On Wed, Sep 8, 2021 at 9:28 PM Melanie Plageman wrote: > > On Fri, Aug 13, 2021 at 3:08 AM Andres Freund wrote: > > > > Hi, > > > > On 2021-08-11 16:11:34 -0400, Melanie Plageman wrote: > > > On Tue, Aug 3, 2021 at 2:13 PM Andres Freund wrote: > > > > > Also, I'm unsure how writing the buffer ac

Re: Estimating HugePages Requirements?

2021-09-10 Thread Bossart, Nathan
On 9/10/21, 1:02 PM, "Robert Haas" wrote: > On Thu, Sep 9, 2021 at 5:53 PM Bossart, Nathan wrote: >> I think it might be clearer to >> somehow indicate that the value is essentially the size of the main >> shared memory area in terms of the huge page size, but I'm not sure >> how to do that conci

Re: SQL:2011 application time

2021-09-10 Thread Jaime Casanova
On Mon, Sep 06, 2021 at 12:52:37PM -0700, Paul A Jungwirth wrote: > On Sat, Sep 4, 2021 at 12:56 PM Jaime Casanova > wrote: > > > > patch 01: does apply but doesn't compile, attached the compile errors. > > patch 04: does not apply clean. > > Thanks for taking a look! I've rebased & made it compi

Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

2021-09-10 Thread Fujii Masao
On 2021/07/23 20:07, Ranier Vilela wrote: Em sex., 23 de jul. de 2021 às 07:02, Aleksander Alekseev mailto:aleksan...@timescale.com>> escreveu: Hi hackers, The following review has been posted through the commitfest application: make installcheck-world:  tested, passed

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-09-10 Thread Dilip Kumar
On Wed, Sep 8, 2021 at 9:54 PM Robert Haas wrote: > On Fri, Sep 3, 2021 at 5:54 PM Andres Freund wrote: > > > I think we already have such a code in multiple places where we bypass > the > > > shared buffers for copying the relation > > > e.g. index_copy_data(), heapam_relation_copy_data(). > >

Re: Teach pg_receivewal to use lz4 compression

2021-09-10 Thread Jian Guo
@@ -250,14 +302,18 @@ FindStreamingStart(uint32 *tli) /* * Check that the segment has the right size, if it's supposed to be * completed. For non-compressed segments just check the on-disk size -* and see if it matches a completed

Re: strange case of "if ((a & b))"

2021-09-10 Thread Michael Paquier
On Thu, Sep 09, 2021 at 02:14:50PM +0900, Kyotaro Horiguchi wrote: > Maybe I'm missing something, but I can see several instances of the > "eval-bool ? true : false" pattern after fd0625c7a9 that are not in > the latest 0002. Yep. There are more of these, and I have just looked at some of them as

Re: 2021-09 Commitfest

2021-09-10 Thread Jaime Casanova
On Wed, Sep 01, 2021 at 09:26:33AM -0500, Jaime Casanova wrote: > On Wed, Sep 01, 2021 at 03:10:32PM +0200, Daniel Gustafsson wrote: > > It is now 2021-09-01 Anywhere On Earth so I’ve set the September commitfest > > to > > In Progress and opened the November one for new entries. Jaime Casanova h

Re: Polyphase merge is obsolete

2021-09-10 Thread Jaime Casanova
On Wed, Jul 14, 2021 at 06:04:14PM +0300, Heikki Linnakangas wrote: > On 14/07/2021 15:12, vignesh C wrote: > > On Sat, Jan 23, 2021 at 3:49 AM Heikki Linnakangas wrote: > > > Here's an updated version that fixes one bug: > > > > > > The CFBot was reporting a failure on the FreeBSD system [1]. It