pgbench - refactor init functions with buffers

Hello, While developing pgbench to allow partitioned tabled, I reproduced the string management style used in the corresponding functions, but was pretty unhappy with that kind of pattern: snprintf(buf + strlen(buf), sizeof(buf) - strlen(buf), ...) However adding a feature is not th

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Tue, Oct 22, 2019 at 10:46 AM Amit Kapila wrote: > > On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote: > > > > I have attempted to test the performance of (Stream + Spill) vs > > (Stream + BGW pool) and I can see the similar gain what Alexey had > > shown[1]. > > > > In addition to this, I hav

Re: Questions/Observations related to Gist vacuum

On Tue, Oct 22, 2019 at 10:50 AM Dilip Kumar wrote: > > On Tue, Oct 22, 2019 at 9:10 AM Amit Kapila wrote: > > > > On Fri, Oct 18, 2019 at 4:51 PM Dilip Kumar wrote: > > > > > > I have prepared a first version of the patch. Currently, I am > > > performing an empty page deletion for all the cas

Re: Questions/Observations related to Gist vacuum

On Tue, Oct 22, 2019 at 9:10 AM Amit Kapila wrote: > > On Fri, Oct 18, 2019 at 4:51 PM Dilip Kumar wrote: > > > > I have prepared a first version of the patch. Currently, I am > > performing an empty page deletion for all the cases. > > > > Few comments: > -- > 1. > -/* > - *

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote: > > I have attempted to test the performance of (Stream + Spill) vs > (Stream + BGW pool) and I can see the similar gain what Alexey had > shown[1]. > > In addition to this, I have rebased the latest patchset [2] without > the two-phase logical dec

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote: > > On Mon, Oct 14, 2019 at 3:09 PM Dilip Kumar wrote: > > > > On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra > > wrote: > > > > > > > > > Sure, I wasn't really proposing to adding all stats from that patch, > > > including those related to streami

Re: Questions/Observations related to Gist vacuum

On Fri, Oct 18, 2019 at 4:51 PM Dilip Kumar wrote: > > I have prepared a first version of the patch. Currently, I am > performing an empty page deletion for all the cases. > Few comments: -- 1. -/* - * State kept across vacuum stages. - */ typedef struct { - IndexBulkDelete

Re: dropdb --force

On Mon, Oct 21, 2019 at 2:15 PM Pavel Stehule wrote: > > po 21. 10. 2019 v 10:25 odesílatel Amit Kapila > napsal: >> >> >> Sorry, but I am not able to understand the reason. Are you worried >> about the comments atop CountOtherDBBackends which says it is used in >> Drop Database and related com

Re: Re: SQL/JSON: functions

On 10/21/19 12:44 PM, Nikita Glukhov wrote: v39 patch is based on 5ee96b3e2221d154ffcb719bd2dee1179c53f821 Use the following git command to apply patches: git am ~/Downloads/0001-Jsonpath-support-for-json-v39.patch Thank you. The patch applied fine, with no errors. Is this the type of test

logical replication empty transactions

After setting up logical replication of a slowly changing table using the built in pub/sub facility, I noticed way more network traffic than made sense. Looking into I see that every transaction in that database on the master gets sent to the replica. 99.999+% of them are empty transactions ('B'

Re: jsonb_set() strictness considered harmful to data

Steven Pousty wrote: > On Sun, Oct 20, 2019 at 4:31 PM raf wrote: > > > Steven Pousty wrote: > > > > > I would think though that raising an exception is better than a > > > default behavior which deletes data. > > > > I can't help but feel the need to make the point that > > the function is not

Re: jsonb_set() strictness considered harmful to data

On 10/21/19 12:50 PM, Tomas Vondra wrote: On Mon, Oct 21, 2019 at 08:06:46AM -0700, Adrian Klaver wrote: On 10/20/19 11:07 PM, Tomas Vondra wrote: On Sun, Oct 20, 2019 at 06:51:05PM -0400, Andrew Dunstan wrote: True. And AFAIK catching exceptions is not really possible in some code, e.g. in

Re: jsonb_set() strictness considered harmful to data

On Mon, Oct 21, 2019 at 08:06:46AM -0700, Adrian Klaver wrote: On 10/20/19 11:07 PM, Tomas Vondra wrote: On Sun, Oct 20, 2019 at 06:51:05PM -0400, Andrew Dunstan wrote: True. And AFAIK catching exceptions is not really possible in some code, e.g. in stored procedures (because we can't do sub

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

On 2019-10-21 15:04:36 -0400, Tom Lane wrote: > There is a general solution that works across platforms; it's called > memset() and it's what we're using today. I'm beginning to think that > we should just reject this patch. It's certainly not enough of an > improvement to justify the amount of d

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

Stephen Frost writes: > * Joe Nelson (j...@begriffs.com) wrote: >> If it's being put behind a macro then *stylistically* it shouldn't >> matter whether {} or {0} is chosen, right? In which case {0} would >> be a better choice because it's supported everywhere. > The problem with {0} in the first

Re: intermittent test failure on Windows

Andrew Dunstan writes: > Bowerbird (Visual Studio 2017 / Windows 10 pro) just had a failure on > the pg_ctl test : > > I surmise that what's happening here is that the test is trying to read > current

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

Greetings, * Joe Nelson (j...@begriffs.com) wrote: > > Is it possible to define the macro to be {} where supported and {0} > > where needed? Something like: > > If it's being put behind a macro then *stylistically* it shouldn't > matter whether {} or {0} is chosen, right? In which case {0} would

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

> Is it possible to define the macro to be {} where supported and {0} > where needed? Something like: If it's being put behind a macro then *stylistically* it shouldn't matter whether {} or {0} is chosen, right? In which case {0} would be a better choice because it's supported everywhere.

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

On Mon, 21 Oct 2019 at 11:46, Chapman Flack wrote: > > I would favor {} in a heartbeat if it were standard, because that > sucker is an idiom. > > Failing that, though, I think I still favor the macro, because > question (1) seems less fuzzy than question (2), and on "clear", > the macro wins. >

Re: [BUG] standby node can not provide service even it replays all log files

On Mon, Oct 21, 2019 at 4:13 AM Thunder wrote: > Can we fix this issue like the following patch? > > $git diff src/backend/access/transam/xlog.c > diff --git a/src/backend/access/transam/xlog.c > b/src/backend/access/transam/xlog.c > index 49ae97d4459..0fbdf6fd64a 100644 > --- a/src/backend/acces

Re: Backport "WITH ... AS MATERIALIZED" syntax to <12?

On Sat, Oct 19, 2019 at 10:53 AM Andrew Dunstan wrote: > > FWIW I'm not sure the "we don't want to upgrade application code at the > > same time as the database" is really tenable. > > I'm -1 for exactly this reason. -1 from me, too, also for this reason. I bet if we started looking we'd find man

Re: SQL/JSON: functions

On 21.10.2019 19:00, Andrew Alsup wrote: On 9/27/19 9:42 PM, Nikita Glukhov wrote: Attached 39th version of the patches rebased onto current master. I am unable to cleanly apply this patch. I've tried applying to the current master (4f4061b2dd) I've also tried apply to commit b81a9c2fc5 back

Re: jsonb_set() strictness considered harmful to data

On Sun, Oct 20, 2019 at 4:31 PM raf wrote: > Steven Pousty wrote: > > > I would think though that raising an exception is better than a > > default behavior which deletes data. > > I can't help but feel the need to make the point that > the function is not deleting anything. It is just > returnin

Re: Re: SQL/JSON: functions

On 9/27/19 9:42 PM, Nikita Glukhov wrote: Attached 39th version of the patches rebased onto current master. -- Nikita Glukhov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company I am unable to cleanly apply this patch. I've tried applying to the current master (4f4061

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

On 10/21/19 11:25 AM, Joe Nelson wrote: > we have the > INIT_ALL_ELEMS_ZERO macro to avoid (2). However the more I look at the > code using that macro the less I like it. The {0} initializer is more > idiomatic. If faced with the two questions: 1. which of a or b is more "clear" ? 2. which of a o

Re: jsonb_set() strictness considered harmful to data

On Sun, Oct 20, 2019 at 3:51 PM Andrew Dunstan < andrew.duns...@2ndquadrant.com> wrote: > I'm not arguing against the idea of improving the situation. But I am > arguing against a minimal fix that will not provide much of value to a > careful app developer. i.e. I want to do more to support app de

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

> > I don't understand why this is an issue worth deviating from the > > standard for. > > Because this use and the way the standard is defined in this case is > confusing and could lead later hackers to misunderstand what's going on > and end up creating bugs- The two possible misunderstandings

Re: Declaring a strict function returns not null / eval speed

Hi, On 2019-10-20 10:27:19 -0400, Tom Lane wrote: > "RETURNS NOT NULL", perhaps? That'd have the advantage of not requiring > any new keyword. That could work. > I'm a little bit skeptical of the actual value of adding this additional > level of complexity, but I suppose we can't determine tha

intermittent test failure on Windows

Bowerbird (Visual Studio 2017 / Windows 10 pro) just had a failure on the pg_ctl test : There was a similar failure 17 days ago. I surmise that what's happening here is that the test is trying to

Re: jsonb_set() strictness considered harmful to data

On 10/20/19 11:07 PM, Tomas Vondra wrote: On Sun, Oct 20, 2019 at 06:51:05PM -0400, Andrew Dunstan wrote: True. And AFAIK catching exceptions is not really possible in some code, e.g. in stored procedures (because we can't do subtransactions, so no exception blocks). Can you explain the ab

Re: jsonb_set() strictness considered harmful to data

On 10/21/19 2:07 AM, Tomas Vondra wrote: > On Sun, Oct 20, 2019 at 06:51:05PM -0400, Andrew Dunstan wrote: >> >>> I think the general premise of this thread is that the application >>> developer does not realize that may be necessary, because it's a bit >>> surprising behavior, particularly when

Re: [HACKERS] Arrays of domains

Hi, On 2017-09-29 13:10:35 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 09/28/2017 05:44 PM, Tom Lane wrote: > >> Assuming that that's going to happen for v11, I'm inclined to leave the > >> optimization problem until the dust settles around CaseTestExpr. > >> Does anyone feel that this

Re: configure fails for perl check on CentOS8

On 10/20/19 7:36 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 10/20/19 1:23 PM, Tom Lane wrote: >>> The right way to fix it, likely, is to add CFLAGS_SL while performing this >>> particular autoconf test, as that would replicate the switches used for >>> plperl (and it turns out that addin

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

On Sat, Oct 19, 2019 at 9:14 PM Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > Especially not when the person suggesting to do so isn't > > even doing the leg work to estimate the portability issues. > > I figured it was common knowledge that gcc/clang supported it just

Re: Questions/Observations related to Gist vacuum

On Mon, Oct 21, 2019 at 2:58 PM Andrey Borodin wrote: > > > > > 21 окт. 2019 г., в 11:12, Dilip Kumar написал(а): > > > > On Mon, Oct 21, 2019 at 2:30 PM Andrey Borodin wrote: > >> > >> I've took a look into the patch, and cannot understand one simple thing... > >> We should not call gistvacuum_

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Mon, Oct 21, 2019 at 2:50 PM Amit Kapila wrote: > > On Mon, Oct 21, 2019 at 10:48 AM Dilip Kumar wrote: > > > > On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote: > > > > > 3. > > > @@ -2479,7 +2480,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, > > > ReorderBufferTXN *txn) > > > > > > /

Re: Questions/Observations related to Gist vacuum

> 21 окт. 2019 г., в 11:12, Dilip Kumar написал(а): > > On Mon, Oct 21, 2019 at 2:30 PM Andrey Borodin wrote: >> >> I've took a look into the patch, and cannot understand one simple thing... >> We should not call gistvacuum_delete_empty_pages() for same gist_stats twice. >> Another way once

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

On Mon, Oct 21, 2019 at 10:48 AM Dilip Kumar wrote: > > On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote: > > > 3. > > @@ -2479,7 +2480,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, > > ReorderBufferTXN *txn) > > > > /* update the statistics */ > > rb->spillCount += 1; > > - rb->spillTxn

Re: Questions/Observations related to Gist vacuum

On Mon, Oct 21, 2019 at 2:30 PM Andrey Borodin wrote: > > Hi! > > > 18 окт. 2019 г., в 13:21, Dilip Kumar написал(а): > > > > On Fri, Oct 18, 2019 at 10:55 AM Amit Kapila > > wrote: > >> > >> > >> I think we can do it in general as adding some check for parallel > >> vacuum there will look bit

Re: pglz performance

> 28 сент. 2019 г., в 10:29, Andrey Borodin написал(а): > > I hope to benchmark decompression on Silesian corpus soon. I've done it. And results are quite controversial. Dataset adds 12 payloads to our 5. Payloads have relatively high entropy. In many cases pglz cannot compress them at all,

Re: Questions/Observations related to Gist vacuum

Hi! > 18 окт. 2019 г., в 13:21, Dilip Kumar написал(а): > > On Fri, Oct 18, 2019 at 10:55 AM Amit Kapila wrote: >> >> >> I think we can do it in general as adding some check for parallel >> vacuum there will look bit hackish. > I agree with that point. > It is not clear if we get enough >> be

Re: dropdb --force

po 21. 10. 2019 v 10:25 odesílatel Amit Kapila napsal: > On Mon, Oct 21, 2019 at 12:24 PM Pavel Stehule > wrote: > > > > po 21. 10. 2019 v 8:38 odesílatel Amit Kapila > napsal: > >> > >> > If we don't wait in TerminateOtherDBBackends, then probably there > should be necessary some cycles inside

Re: Obsolete comment in partbounds.c

On Sat, Oct 19, 2019 at 5:56 PM Etsuro Fujita wrote: > On Fri, Oct 18, 2019 at 6:56 PM Alvaro Herrera > wrote: > > On 2019-Oct-18, Etsuro Fujita wrote: > > > While reviewing the partitionwise-join patch, I noticed $Subject，ie, > > > this in create_list_bounds(): > > > > > > /* >

Re: Fix comment in XLogFileInit()

On Mon, Oct 21, 2019 at 12:28 PM Fujii Masao wrote: > > I found that the argument name of XLogFileInit() is wrong in its comment. > Attached is the patch that fixes that typo. > LGTM. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: dropdb --force

On Mon, Oct 21, 2019 at 12:24 PM Pavel Stehule wrote: > > po 21. 10. 2019 v 8:38 odesílatel Amit Kapila > napsal: >> >> > If we don't wait in TerminateOtherDBBackends, then probably there should >> > be necessary some cycles inside CountOtherDBBackends. I think so code like >> > is correct >>

Re:[BUG] standby node can not provide service even it replays all log files

Can we fix this issue like the following patch? $git diff src/backend/access/transam/xlog.c diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 49ae97d4459..0fbdf6fd64a 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@

Incorrect relation locked at beginning of REINDEX CONCURRENTLY

Hi all, While digging into the issues reported lately about REINDEX CONCURRENTLY, I have bumped into the following, independent, issue: /* Now open the relation of the new index, a lock is also needed on it */ newIndexRel = index_open(indexId, ShareUpdateExclusiveLock) In this code path, indexId

[BUG] standby node can not provide service even it replays all log files

Hi hackers, I found this issue when restart standby node and then try to connect it. It return "psql: FATAL: the database system is starting up". The steps to reproduce this issue. 1. Create a session to run uncommit_trans.sql 2. Create the other session to do checkpoint 3. Restart standby no

Re: dropdb --force

Hi > When you say 'without any problem', do you mean to say that the drop > database is successful? In your tests were those sessions idle? If > so, I think we should try with busy sessions. Also, if you write any > script to do these tests, kindly share the same so that others can > also repe

Re: adding partitioned tables to publications

Hi Petr, Thanks for your comments. On Sun, Oct 13, 2019 at 5:01 AM Petr Jelinek wrote: > On 07/10/2019 02:55, Amit Langote wrote: > > One cannot currently add partitioned tables to a publication. > > > > create table p (a int, b int) partition by hash (a); > > create table p1 partition of p for