Re: Avoid stuck of pbgench due to skipped transactions

Hello Yugo-san, For example, when I usee a large rate (-R) for throttling and a small latency limit (-L) values with a duration (-T), pbbench got stuck. $ pgbench -T 5 -R 1 -L 1; Indeed, it does not get out of the catchup loop for a long time because even scheduling takes more time

Re: "an SQL" vs. "a SQL"

On Thu, 10 Jun 2021, 15:35 Alvaro Herrera, wrote: > src/backend/libpq/auth.c:847:* has. If it's an MD5 hash, we must do > MD5 authentication, and if it's a > src/backend/libpq/auth.c:848:* SCRAM secret, we must do SCRAM > authentication. > Not sure whether you were just listing examples

Re: Fdw batch insert error out when set batch_size > 65535

On 2021-Jun-12, Tomas Vondra wrote: > There's one caveat, though - for regular builds the slowdown is pretty > much eliminated. But with valgrind it's still considerably slower. For > postgres_fdw the "make check" used to take ~5 minutes for me, now it > takes >1h. And yes, this is entirely due to

Re: unnesting multirange data types

On Sun, Jun 13, 2021 at 1:16 AM Jonathan S. Katz wrote: > On 6/12/21 5:57 PM, Alexander Korotkov wrote: > > On Sat, Jun 12, 2021 at 2:44 AM Alexander Korotkov > > wrote: > >> ()On Sat, Jun 12, 2021 at 2:30 AM Justin Pryzby > >> wrote: > >>> On Fri, Jun 11, 2021 at 11:37:58PM +0300, Alexander K

Re: Questions about support function and abbreviate

Hi Han, Darafei already provided a good answer to your question, I will add just a few things with the hope of making things more clear for your use case. SortSupport implementation in PostgreSQL allows to make comparisons at binary level in a dedicated region of memory where data can be quickly

Re: unnesting multirange data types

On 6/12/21 5:57 PM, Alexander Korotkov wrote: > On Sat, Jun 12, 2021 at 2:44 AM Alexander Korotkov > wrote: >> ()On Sat, Jun 12, 2021 at 2:30 AM Justin Pryzby wrote: >>> On Fri, Jun 11, 2021 at 11:37:58PM +0300, Alexander Korotkov wrote: On Fri, Jun 11, 2021 at 1:04 AM Justin Pryzby wrote:

Re: unnesting multirange data types

On Sat, Jun 12, 2021 at 2:44 AM Alexander Korotkov wrote: > ()On Sat, Jun 12, 2021 at 2:30 AM Justin Pryzby wrote: > > On Fri, Jun 11, 2021 at 11:37:58PM +0300, Alexander Korotkov wrote: > > > On Fri, Jun 11, 2021 at 1:04 AM Justin Pryzby > > > wrote: > > > > > > > > +{ oid => '1293', descr =>

Re: recovery test failures on hoverfly

Andrew Dunstan writes: > Note, too that the psql and safe_psql methods also pass the query via stdin. Yeah. We need all of these to act the same, IMO. Recall that the previous patches that introduced the undef hack were changing callers of those routines, not poll_query_until.

Re: recovery test failures on hoverfly

On 6/12/21 5:28 PM, Tom Lane wrote: > Andrew Dunstan writes: >> I'm a bit dubious about this. It doesn't seem more robust to insist that >> we pass undef in certain cases. > True, it'd be nicer if that didn't matter; mainly because people > will get it wrong in future. Right, that's what I'm w

Re: Error on pgbench logs

+ while ((next = agg->start_time + agg_interval * INT64CONST(100)) <= now) I can find the similar code to convert "seconds" to "us" using casting like end_time = threads[0].create_time + (int64) 100 * duration; or next_report = last_report + (int64) 100 * progress

Re: Race condition in recovery?

On 6/12/21 1:54 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 6/12/21 1:07 PM, Tom Lane wrote: >>> OK. But it makes me itch a bit that this one wait-for-wal-to-be- >>> processed query looks different from all the other ones. >> I'm happy to bring the other two queries that look like this i

Re: recovery test failures on hoverfly

Andrew Dunstan writes: > I'm a bit dubious about this. It doesn't seem more robust to insist that > we pass undef in certain cases. True, it'd be nicer if that didn't matter; mainly because people will get it wrong in future. > If passing the SQL via stdin is fragile, > as we also found to be th

Re: recovery test failures on hoverfly

On 6/12/21 3:15 PM, Tom Lane wrote: >>> Michael Paquier writes: This is the same problem as c757a3da and 6d41dd0, where we write a query to a pipe but the kill, causing a failure, makes the test fail with a SIGPIPE in IPC::Run as a query is sent down to a pipe. > After checking th

Re: Character expansion with ICU collations

Re: >> Can a CI collation be ordered upper case first, or is this a limitation of ICU? > I don't know the authoritative answer to that, but to me it doesn't make > sense, since the effect of a case-insensitive collation is to throw away > the third-level weights, so there is noth

Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

Hi, On 2021-06-12 10:55:22 -0300, Ranier Vilela wrote: > With the recent changes at procarray.c, I take a look in. > msvc compiler, has some warnings about signed vs unsigned. > 1. Size_t is weird, because all types are int. Not sure why I ended up using size_t here. There are cases where using

Re: recovery test failures on hoverfly

>> Michael Paquier writes: >>> This is the same problem as c757a3da and 6d41dd0, where we write a >>> query to a pipe but the kill, causing a failure, makes the test fail >>> with a SIGPIPE in IPC::Run as a query is sent down to a pipe. After checking the git logs, I realized that this failure is

Avoid stuck of pbgench due to skipped transactions

Hi, I found that pgbench could get stuck when every transaction come to be skipped and the number of transaction is not limitted by -t option. For example, when I usee a large rate (-R) for throttling and a small latency limit (-L) values with a duration (-T), pbbench got stuck. $ pgbench -T 5

Re: logical replication of truncate command with trigger causes Assert

On 2021-Jun-12, Tom Lane wrote: > Amit Kapila writes: > > On Fri, Jun 11, 2021 at 8:56 PM Tom Lane wrote: > >> I was thinking maybe we could mark all these replication protocol > >> violation errors non-translatable. While we don't want to crash on a > >> protocol violation, it shouldn't really

Re: Error on pgbench logs

On Thu, 10 Jun 2021 23:29:30 +0200 (CEST) Fabien COELHO wrote: > > Bonjour Michaël, > > Here is an updated patch. While having a look at Kyotaro-san patch, I > noticed that the aggregate stuff did not print the last aggregate. I think > that it is a side effect of switching the precision from

Re: A new function to wait for the backend exit after termination

On Sat, Jun 12, 2021 at 08:21:39AM -0700, Noah Misch wrote: > On Sat, Jun 12, 2021 at 12:12:12AM -0500, Justin Pryzby wrote: > > Even if it's not removed, the descriptions should be cleaned up. > > > > | src/include/catalog/pg_proc.dat- descr => 'terminate a backend process > > and if timeout is

Re: Error on pgbench logs

On Fri, 11 Jun 2021 16:09:10 +0200 (CEST) Fabien COELHO wrote: > > Bonjour Michaël, > > >> + /* flush remaining stats */ > >> + if (!logged && latency == 0.0) > >> + logAgg(logfile, agg); > > > > You are right, this is missing the final stats. Why the choice

Re: Race condition in recovery?

Andrew Dunstan writes: > On 6/12/21 1:07 PM, Tom Lane wrote: >> OK. But it makes me itch a bit that this one wait-for-wal-to-be- >> processed query looks different from all the other ones. > I'm happy to bring the other two queries that look like this into line > with this one if you like. I se

Re: Failure in subscription test 004_sync.pl

Amit Kapila writes: > On Sat, Jun 12, 2021 at 1:13 PM Michael Paquier wrote: >> wrasse has just failed with what looks like a timing error with a >> replication slot drop: >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=wrasse&dt=2021-06-12%2006%3A16%3A30 > If we want to fix this, we

Re: Race condition in recovery?

On 6/12/21 1:07 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 6/12/21 10:20 AM, Tom Lane wrote: >>> I wonder whether that is a workaround for the poll_query_until bug >>> I proposed to fix at [1]. >> No, it's because I found it annoying and confusing that there was an >> invisible result wh

Re: pg_filenode_relation(0,0) elog

Justin Pryzby writes: > On Fri, Jun 11, 2021 at 11:51:35PM -0400, Tom Lane wrote: >> Likely it should check the reltablespace, too. > I don't think so. The docs say: > https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-DBLOCATION > |For a relation in the database's defa

Re: Race condition in recovery?

Andrew Dunstan writes: > On 6/12/21 10:20 AM, Tom Lane wrote: >> I wonder whether that is a workaround for the poll_query_until bug >> I proposed to fix at [1]. > No, it's because I found it annoying and confusing that there was an > invisible result when last_archived_wal is null. OK. But it m

Re: logical replication of truncate command with trigger causes Assert

Amit Kapila writes: > On Fri, Jun 11, 2021 at 8:56 PM Tom Lane wrote: >> I was thinking maybe we could mark all these replication protocol >> violation errors non-translatable. While we don't want to crash on a >> protocol violation, it shouldn't really be a user-facing case either. > I don't s

[PATCH] check_random_seed: use a boolean not an int..

This confused me for a minute so took the opportunity to clean it up. Since: 2594cf0e8c0440619b1651c5a406d376657c --- src/backend/commands/variable.c | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/backend/commands/variable.c b/src/backend/commands/variabl

Re: Race condition in recovery?

On 6/12/21 10:20 AM, Tom Lane wrote: > Andrew Dunstan writes: >> I have pushed a fix, tested on a replica of fairywren/drongo, > This bit seems a bit random: > > # WAL segment, this is enough to guarantee that the history file was > # archived. > my $archive_wait_query = > - "SELECT '$walfil

SQLSTATE for replication connection failures

So far as I can find, just about everyplace that deals with replication connections has slipshod error reporting. An example from worker.c is LogRepWorkerWalRcvConn = walrcv_connect(MySubscription->conninfo, true, MySubscription->name, &err)

Re: A new function to wait for the backend exit after termination

On Sat, Jun 12, 2021 at 12:12:12AM -0500, Justin Pryzby wrote: > Even if it's not removed, the descriptions should be cleaned up. > > | src/include/catalog/pg_proc.dat- descr => 'terminate a backend process and > if timeout is specified, wait for its exit or until timeout occurs', > => I think d

Re: pg_filenode_relation(0,0) elog

On Fri, Jun 11, 2021 at 11:51:35PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > Per sqlsmith. > > postgres=# SELECT pg_filenode_relation(0,0); > > ERROR: unexpected duplicate for tablespace 0, relfilenode 0 > > Ugh. > > > The usual expectation is that sql callable functions should return

Add proper planner support for ORDER BY / DISTINCT aggregates

A few years ago I wrote a patch to implement the missing aggregate combine functions for array_agg and string_agg [1]. In the end, the patch was rejected due to some concern [2] that if we allow these aggregates to run in parallel then it might mess up the order in which values are being aggregate

Re: Use pg_nextpower2_* in a few more places

On Sat, Jun 12, 2021 at 7:35 AM David Rowley wrote: > On Sun, 13 Jun 2021 at 02:08, Zhihong Yu wrote: > > Maybe add an assertion after the assignment, that newalloc >= > LWLockTrancheNamesAllocated. > > I don't quite see how it would be possible for that to ever fail. I > could understand addin

Re: Use pg_nextpower2_* in a few more places

On Sun, 13 Jun 2021 at 02:08, Zhihong Yu wrote: > Maybe add an assertion after the assignment, that newalloc >= > LWLockTrancheNamesAllocated. I don't quite see how it would be possible for that to ever fail. I could understand adding an Assert() if some logic was outside the function and we w

Re: Race condition in recovery?

Andrew Dunstan writes: > I have pushed a fix, tested on a replica of fairywren/drongo, This bit seems a bit random: # WAL segment, this is enough to guarantee that the history file was # archived. my $archive_wait_query = - "SELECT '$walfile_to_be_archived' <= last_archived_wal FROM pg_stat

Re: Use pg_nextpower2_* in a few more places

On Sat, Jun 12, 2021 at 6:40 AM David Rowley wrote: > Thanks for having a look. > > On Sun, 13 Jun 2021 at 00:50, Zhihong Yu wrote: > > - newalloc = Max(LWLockTrancheNamesAllocated, 8); > > - while (newalloc <= tranche_id) > > - newalloc *= 2; > > + newalloc = pg_next

Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

Hi, With the recent changes at procarray.c, I take a look in. msvc compiler, has some warnings about signed vs unsigned. So. 1. Size_t is weird, because all types are int. 2. Wouldn't it be better to initialize static variables? 3. There are some shadowing parameters. 4. Possible loop beyond num

Re: Use pg_nextpower2_* in a few more places

Thanks for having a look. On Sun, 13 Jun 2021 at 00:50, Zhihong Yu wrote: > - newalloc = Max(LWLockTrancheNamesAllocated, 8); > - while (newalloc <= tranche_id) > - newalloc *= 2; > + newalloc = pg_nextpower2_32(Max(8, tranche_id + 1)); > > Should LWLockTrancheNamesAll

Re: Race condition in recovery?

On 6/12/21 7:31 AM, Andrew Dunstan wrote: > On 6/12/21 3:48 AM, Michael Paquier wrote: >> On Fri, Jun 11, 2021 at 10:46:45AM -0400, Tom Lane wrote: >>> I think jacana uses msys[2?], so this likely indicates a problem >>> in path sanitization for the archive command. Andrew, any advice? >> Err, s

Re: Failure in subscription test 004_sync.pl

On Sat, Jun 12, 2021 at 1:13 PM Michael Paquier wrote: > > wrasse has just failed with what looks like a timing error with a > replication slot drop: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=wrasse&dt=2021-06-12%2006%3A16%3A30 > > Here is the error: > error running SQL: 'psql::1:

Re: Use pg_nextpower2_* in a few more places

On Sat, Jun 12, 2021 at 5:32 AM David Rowley wrote: > Back in f0705bb62, we added pg_nextpower2_32 and pg_nextpower2_64 to > efficiently obtain the next power of 2 of a given number using an > intrinsic function to find the left-most 1 bit. > > In d025cf88b and 02a2e8b44, I added some usages of t

Use pg_nextpower2_* in a few more places

Back in f0705bb62, we added pg_nextpower2_32 and pg_nextpower2_64 to efficiently obtain the next power of 2 of a given number using an intrinsic function to find the left-most 1 bit. In d025cf88b and 02a2e8b44, I added some usages of these new functions but I didn't quite get all of them done. T

Re: Logical replication keepalive flood

On Fri, Jun 11, 2021 at 7:07 AM Kyotaro Horiguchi wrote: > > At Thu, 10 Jun 2021 12:18:00 +0530, Amit Kapila > wrote in > > Good analysis. I think this analysis has shown that walsender is > > sending messages at top speed as soon as they are generated. So, I am > > wondering why there is any ne

Re: Race condition in recovery?

On 6/12/21 3:48 AM, Michael Paquier wrote: > On Fri, Jun 11, 2021 at 10:46:45AM -0400, Tom Lane wrote: >> I think jacana uses msys[2?], so this likely indicates a problem >> in path sanitization for the archive command. Andrew, any advice? > Err, something around TestLib::perl2host()? I'm work

Small clean up in nodeAgg.c

Much of nodeAgg.c does not really know the difference between the aggregate's combine function and the aggregate's transition function. This was done on purpose so that we can keep as much code the same as possible between partial aggregate and finalize aggregate. We can take this a bit further wi

Re: Question about StartLogicalReplication() error path

On Fri, Jun 11, 2021 at 11:52 AM Jeff Davis wrote: > > On Fri, 2021-06-11 at 10:13 +0530, Amit Kapila wrote: > > I think because there is no need to process the WAL that has been > > confirmed by the client. Do you see any problems with this scheme? > > Several: > > * Replication setups are comple

Re: psql - add SHOW_ALL_RESULTS option

Hello Peter, My overly naive trust in non regression test to catch any issues has been largely proven wrong. Three key features do not have a single tests. Sigh. I'll have some time to look at it over next week-end, but not before. I have reverted the patch and moved the commit fest entry t

Re: Add client connection check during the execution of the query

On Fri, Jun 11, 2021 at 9:24 PM Thomas Munro wrote: > On Fri, Apr 30, 2021 at 2:23 PM Thomas Munro > wrote: > > Here's something I wanted to park here to look into for the next > > cycle: it turns out that kqueue's EV_EOF flag also has the right > > semantics for this. That leads to the idea o

Re: Hook for extensible parsing.

On Tue, Jun 08, 2021 at 12:16:48PM +0800, Julien Rouhaud wrote: > On Sun, Jun 06, 2021 at 02:50:19PM +0800, Julien Rouhaud wrote: > > On Sat, May 01, 2021 at 03:24:58PM +0800, Julien Rouhaud wrote: > > > > > > I'm attaching some POC patches that implement this approach to start a > > > discussion.

Re: Race condition in recovery?

On Fri, Jun 11, 2021 at 10:46:45AM -0400, Tom Lane wrote: > I think jacana uses msys[2?], so this likely indicates a problem > in path sanitization for the archive command. Andrew, any advice? Err, something around TestLib::perl2host()? -- Michael signature.asc Description: PGP signature

Failure in subscription test 004_sync.pl

Hi all, wrasse has just failed with what looks like a timing error with a replication slot drop: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=wrasse&dt=2021-06-12%2006%3A16%3A30 Here is the error: error running SQL: 'psql::1: ERROR: could not drop replication slot "tap_sub" on publish

Re: Questions about support function and abbreviate

Hi Darafei, Thanks for your reply. However, I still don't get the full picture of this. Let me make my question more clear. First of all, in the *`gistproc.c `* of Postgres, it shows that the `abbreviate`

Re: Decoding of two-phase xacts missing from CREATE_REPLICATION_SLOT command

On Sat, Jun 12, 2021 at 12:56 AM Jeff Davis wrote: > > On Fri, 2021-06-11 at 15:43 +0530, Amit Kapila wrote: > > The new patches look mostly good apart from the below cosmetic > > issues. > > I think the question is whether we want to do these for PG-14 or > > postpone them till PG-15. I think the

Re: logical replication of truncate command with trigger causes Assert

On Fri, Jun 11, 2021 at 8:56 PM Tom Lane wrote: > > Amit Kapila writes: > > On Fri, Jun 11, 2021 at 12:20 AM Tom Lane wrote: > >> Another thing > >> I'm wondering is how many of these messages really need to be > >> translated. We could use errmsg_internal and avoid burdening the > >> translato