Re: Single transaction in the tablesync worker?

Looking at the V29 style tablesync slot names now they appear like this: WARNING: could not drop tablesync replication slot "pg_16397_sync_16389_6927117142022745645" That is in the order subid + relid + sysid Now that I see it in a message it seems a bit strange with the sysid just tacked onto

RE: libpq debug log

From: Kyotaro Horiguchi > > (45) > This looks like a fusion of PQtrace and PQtraceEX. By the way, the > timestamp flag is needed at log emittion. So we can change the state > anytime. > > PQtrace(conn, of); > PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS); > > PQtraceSetFlags(conn, 0); > I

Re: [POC] Fast COPY FROM command for the table with foreign partitions

On 2/9/21 12:47 PM, tsunakawa.ta...@fujitsu.com wrote: From: Andrey V. Lepikhov I guess you used many hash partitions. Sadly, The current COPY implementation only accumulates either 1,000 rows or 64 KB of input data (very small!) before flushing all CopyMultiInsertBuffers. One CopyMultiInser

Re: libpq debug log

At Mon, 8 Feb 2021 14:57:47 +, "iwata@fujitsu.com" wrote in > I update the patch. > I modified code according to review comments of Tsunakawa san and Horiguchi > san. > > And I fixed some bugs. Thanks for the new version. +typedef enum +{ + MSGDIR_FROM_BACKEND, + MSGDIR_F

Re: libpq debug log

At Tue, 9 Feb 2021 08:10:19 +, "tsunakawa.ta...@fujitsu.com" wrote in > From: Kyotaro Horiguchi > > > (45) > > This looks like a fusion of PQtrace and PQtraceEX. By the way, the > > timestamp flag is needed at log emittion. So we can change the state > > anytime. > > > > PQtrace(conn, of);

RE: [POC] Fast COPY FROM command for the table with foreign partitions

From: Andrey V. Lepikhov > I tried to use 1E4 - 1E8 rows in a tuple buffer. But the results weren't > impressive. I guess that's because the 64 KB threshold came first. > We can use one more GUC instead of a precompiled constant. Yes, agreed. > > Why don't we focus on committing the basic pa

Re: Improvements and additions to COPY progress reporting

po 8. 2. 2021 v 19:35 odesílatel Matthias van de Meent napsal: > > Hi, > > With [0] we got COPY progress reporting. Before the column names of > this newly added view are effectively set in stone with the release of > pg14, I propose the following set of relatively small patches. These > are v2, b

Re: Single transaction in the tablesync worker?

When looking at the DropSubscription code I noticed that there is a small difference between the HEAD code and the V29 code when slot_name = NONE. HEAD does -- if (!slotname) { table_close(rel, NoLock); return; } -- V29 does -- if (!slotname)

Re: ModifyTable overheads in generic plans

On Mon, Jan 25, 2021 at 2:23 PM Amit Langote wrote: > On Tue, Dec 22, 2020 at 5:16 PM Amit Langote wrote: > > On Mon, Dec 7, 2020 at 3:53 PM Amit Langote wrote: > > > On Thu, Nov 12, 2020 at 5:04 PM Amit Langote > > > wrote: > > > > Attached new 0002 which does these adjustments. I went with

Re: [HACKERS] Custom compression methods

On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar wrote: > > On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar wrote: > > > > On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: > > > > > > Even more review comments, still looking mostly at 0001: > > > > > > If there's a reason why parallel_schedule is arrangin

Re: adding wait_start column to pg_locks

On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms. So spinlock is necessary when updating "waitStart" without holding the partition lock? Also GetLockStatusData() needs spinlock

Re: Online checksums patch - once again

(I may have said this before, but) My overall high-level impression of this patch is that it's really cmmplex for a feature that you use maybe once in the lifetime of a cluster. I'm happy to review but I'm not planning to commit this myself. I don't object if some other committer picks this up

Re: Libpq support to connect to standby server as priority

On Mon, Feb 8, 2021 at 8:17 PM vignesh C wrote: > > > > > I think what we want to do is mark default_transaction_read_only as > > GUC_REPORT, instead. That will give a reliable report of what the > > state of its GUC is, and you can combine it with is_hot_standby > > to decide whether the session

Re: libpq compression

On 08.02.2021 22:23, Daniil Zakhlystov wrote: Hi everyone, I’ve been making some experiments with an on-the-fly compression switch lately and have some updates. ... pg_restore of IMDB database test results Chunked compression with only CopyData or DataRow compression (second approach): tim

Re: adding wait_start column to pg_locks

On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms. So spinlock is necessary when updating "waitStart" without holding the partition loc

Re: Support for NSS as a libpq TLS backend

> On 9 Feb 2021, at 07:47, Michael Paquier wrote: > > On Tue, Feb 09, 2021 at 12:08:37AM +0100, Daniel Gustafsson wrote: >> Attached is a new patchset where I've tried to split the patches even further >> to try and separate out changes for easier review. While not a perfect split >> I'm sure, a

Re: Single transaction in the tablesync worker?

On Tue, Feb 9, 2021 at 12:02 PM Peter Smith wrote: > > Here are my feedback comments for the V29 patch. > Thanks. > > 3. > Previously the tablesync origin name format was encapsulated in a > common function. IMO it was cleaner/safer how it was before, instead > of the same "pg_%u_%u" cut/paste a

Re: Single transaction in the tablesync worker?

On Tue, Feb 9, 2021 at 1:37 PM Peter Smith wrote: > > Looking at the V29 style tablesync slot names now they appear like this: > > WARNING: could not drop tablesync replication slot > "pg_16397_sync_16389_6927117142022745645" > That is in the order subid + relid + sysid > > Now that I see it in

Re: adding wait_start column to pg_locks

On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms. So spinlock is necessary when updating "w

Re: pg_replication_origin_drop API potential race condition

On 2021-Feb-09, Amit Kapila wrote: > > IIUC, you are suggesting to use lock for the particular origin instead > > of locking the corresponding catalog table in functions > > pg_replication_origin_advance and replorigin_drop_by_name. Right. > I think it won't be that straightforward as we don't h

Re: libpq debug log

On 2021-Feb-04, tsunakawa.ta...@fujitsu.com wrote: > From: 'Alvaro Herrera' > > > (41) > > > +void > > > +PQtraceEx(PGconn *conn, FILE *debug_port, int flags) > > > +{ > > > > I'm not really sure about making this a separate API call. We could just > > make it PQtrace() and increment the libpq s

Re: libpq debug log

On 2021-Feb-09, Kyotaro Horiguchi wrote: > This looks like a fusion of PQtrace and PQtraceEX. By the way, the > timestamp flag is needed at log emittion. So we can change the state > anytime. > > PQtrace(conn, of); > PQtraceSetFlags(conn, PQTRACE_SUPPRESS_TIMESTAMPS); I like this idea better tha

Re: pg_replication_origin_drop API potential race condition

On Tue, Feb 9, 2021 at 4:16 PM Alvaro Herrera wrote: > > On 2021-Feb-09, Amit Kapila wrote: > > > Now, we can do this optimization if we want but I am not sure if > > origin_drop would be a frequent enough operation that we add such an > > optimization. For now, I have added a note in the comments

Re: Single transaction in the tablesync worker?

On Tue, Feb 9, 2021 at 12:02 PM Peter Smith wrote: > > Here are my feedback comments for the V29 patch. > > > > FILE: logical-replication.sgml > > +slots have generated names: > pg_%u_sync_%u_%llu > +(parameters: Subscription oid, > +Table relid, system > identifiersysid) > + >

Re: Clean up code

On 09/02/2021 00:09, CK Tan wrote: Hi, can someone point me to the code that cleans up temp files should a query crashed unexpectedly? Thanks! Autovacuum does that. Search for "orphan" in do_autovacuum() function. - Heikki

Re: Improvements and additions to COPY progress reporting

út 9. 2. 2021 v 12:51 odesílatel 0010203112132233 napsal: > > On Tue, 9 Feb 2021 at 09:32, Josef Šimánek wrote: > > > > po 8. 2. 2021 v 19:35 odesílatel Matthias van de Meent > > napsal: > > > Lastly, 0005 adds 'io_target' to the reported information, that is, > > > FILE, PROGRAM, STDIO or CALLB

Re: Improvements and additions to COPY progress reporting

On Tue, 9 Feb 2021 at 09:32, Josef Šimánek wrote: > > po 8. 2. 2021 v 19:35 odesílatel Matthias van de Meent > napsal: > > Lastly, 0005 adds 'io_target' to the reported information, that is, > > FILE, PROGRAM, STDIO or CALLBACK. Although this can relatively easily > > be determined based on the c

Re: repeated decoding of prepared transactions

On Tue, Feb 9, 2021 at 11:29 AM Ashutosh Bapat wrote: > > On Tue, Feb 9, 2021 at 8:32 AM Amit Kapila wrote: > > > > On Mon, Feb 8, 2021 at 8:36 PM Markus Wanner > > wrote: > > > > > > Hello Amit, > > > > > > thanks for your very quick response. > > > > > > On 08.02.21 11:13, Amit Kapila wrote: >

Re: TRUNCATE on foreign table

IIUC, "truncatable" would be set to "false" for relations which do not have physical storage e.g. views but will be true for regular tables. When we are importing schema we need to set "truncatable" appropriately. Is that something we will support with this patch? Why would one want to truncate a

RE: Parallel INSERT (INTO ... SELECT ...)

> > > postgres=# explain verbose insert into testscan select a from x > > > where > > > a<8 or (a%2=0 and a>19990); > > > QUERY PLAN > > > > > > > -- > > > --

Re: TRUNCATE on foreign table

On Tue, Feb 9, 2021 at 5:31 PM Ashutosh Bapat wrote: > Why would one want to truncate a foreign table instead of truncating > actual table wherever it is? I think when the deletion on foreign tables (which actually deletes rows from the remote table?) is allowed, it does make sense to have a way

Re: pg_replication_origin_drop API potential race condition

On 2021-Feb-09, Amit Kapila wrote: > On Tue, Feb 9, 2021 at 4:16 PM Alvaro Herrera wrote: > > By all means let's get the bug fixed. > > I am planning to push this in HEAD only as there is no user reported > problem and this is actually more about giving correct information to > the user rather

Re: TRUNCATE on foreign table

On Tue, Feb 9, 2021 at 5:49 PM Bharath Rupireddy wrote: > > On Tue, Feb 9, 2021 at 5:31 PM Ashutosh Bapat > wrote: > > Why would one want to truncate a foreign table instead of truncating > > actual table wherever it is? > > I think when the deletion on foreign tables (which actually deletes > ro

Re: Improvements and additions to COPY progress reporting

On Tue, 9 Feb 2021 at 08:12, Bharath Rupireddy wrote: > > On Tue, Feb 9, 2021 at 12:06 AM Matthias van de Meent > wrote: > > With [0] we got COPY progress reporting. Before the column names of > > this newly added view are effectively set in stone with the release of > > pg14, I propose the follo

Re: [HACKERS] Custom compression methods

On Tue, Feb 9, 2021 at 2:08 PM Dilip Kumar wrote: > > On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar wrote: > > > > On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar wrote: > > > > > > On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: > > > > > > > > Even more review comments, still looking mostly at 0001

Re: Parallel INSERT (INTO ... SELECT ...)

On Tue, Feb 9, 2021 at 5:49 PM Hou, Zhijie wrote: > > > > > postgres=# explain verbose insert into testscan select a from x > > > > where > > > > a<8 or (a%2=0 and a>19990); > > > > QUERY PLAN > > > > > > -

Re: Perform COPY FROM encoding conversions in larger chunks

On Sun, Feb 7, 2021 at 2:13 PM Heikki Linnakangas wrote: > > On 02/02/2021 23:42, John Naylor wrote: > > > > In copyfromparse.c, this is now out of date: > > > > * Read the next input line and stash it in line_buf, with conversion to > > * server encoding. This comment for CopyReadLine() is s

Re: adding wait_start column to pg_locks

On 2021/02/09 19:11, Fujii Masao wrote: On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetches are not atomic on some platforms.

Routine usage information schema tables

Several information schema views track dependencies between functions/procedures and objects used by them. These had not been implemented so far because PostgreSQL doesn't track objects used in a function body. However, formally, these also show dependencies used in parameter default expression

Re: TRUNCATE on foreign table

> IIUC, "truncatable" would be set to "false" for relations which do not > have physical storage e.g. views but will be true for regular tables. "truncatable" option is just for the foreign table and it's not related with whether it's on a physical storage or not. "postgres_fdw" already has "updat

ERROR: invalid spinlock number: 0

Hi, Commit 3b733fcd04 caused the buildfarm member "rorqual" to report the following error and fail the regression test. The cause of this issue is a bug in that commit. ERROR: invalid spinlock number: 0 But while investigating the issue, I found that this error could happen even in the cur

Re: Routine usage information schema tables

> On 02/09/2021 3:06 PM Peter Eisentraut > wrote: > > > Several information schema views track dependencies between > functions/procedures and objects used by them. These had not been > [0001-Routine-usage-information-schema-tables.patch] Spotted one typo: included here ony if included he

Re: adding wait_start column to pg_locks

On 2021-02-09 22:54, Fujii Masao wrote: On 2021/02/09 19:11, Fujii Masao wrote: On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-02-03 11:23, Fujii Masao wrote: 64-bit fetc

Re: WIP: BRIN multi-range indexes

On Wed, Feb 3, 2021 at 7:54 PM Tomas Vondra wrote: > > [v-20210203] Hi Tomas, I have some random comments from reading the patch, but haven't gone into detail in the newer aspects. I'll do so in the near future. The cfbot seems to crash on this patch during make check, but it doesn't crash for

RE: Support tab completion for upper character inputs in psql

At Sun, 07 Feb 2021 13:55:00 -0500, Tom Lane wrote in > > This looks like you're trying to force case-insensitive behavior > whether that is appropriate or not. Does not sound like a good idea. I'm still confused about the APPROPRIATE behavior of tab completion. It seems ALTER table/tablespace

[patch] bit XOR aggregate functions

Hi, I personally use it as a checksum for a large unordered set, where performance and simplicity is prioritized over collision resilience. Maybe there are other ways to use them. Best, Alex diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 1ab31a9056..f33358f8db 100644 --- a/

Re: About to add WAL write/fsync statistics to pg_stat_wal view

On Thu, Feb 4, 2021 at 4:45 PM Masahiro Ikeda wrote: > I pgindented the patches. > > ... XLogWrite, which is invoked during an XLogFlush request (see ...). This is also incremented by the WAL receiver during replication. ("which normally called" should be "which is normally called" or "which no

Re: Perform COPY FROM encoding conversions in larger chunks

On 09/02/2021 15:40, John Naylor wrote: On Sun, Feb 7, 2021 at 2:13 PM Heikki Linnakangas > wrote: > > On 02/02/2021 23:42, John Naylor wrote: > > > > In copyfromparse.c, this is now out of date: > > > >   * Read the next input line and stash it in line_buf, with co

Re: Add MAIN_RELATION_CLEANUP and SECONDARY_RELATION_CLEANUP options to VACUUM

On 2/8/21, 9:19 PM, "Michael Paquier" wrote: > On Mon, Feb 08, 2021 at 06:59:45PM +, Bossart, Nathan wrote: >> These suggestions seem reasonable to me. I've applied them in v9. > > Sounds good to me, so applied. Thanks! Nathan

Re: repeated decoding of prepared transactions

On Tue, Feb 9, 2021 at 6:57 AM Amit Kapila wrote: > I think similar happens without any of the work done in PG-14 as well > if we restart the apply worker before the commit completes on the > subscriber. After the restart, we will send the start_decoding_at > point based on some previous commit wh

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

вт, 9 февр. 2021 г. в 01:46, Mark Dilger : > > > > On Feb 8, 2021, at 2:46 AM, Pavel Borisov > wrote: > > > > 0002 - is a temporary hack for testing. It will allow inserting > duplicates in a table even if an index with the exact name "idx" has a > unique constraint (generally it is prohibited to

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

To make tests stable I also removed lsn output under warning level. PFA v3 of a patch -- Best regards, Pavel Borisov Postgres Professional: http://postgrespro.com v3-0001-Make-amcheck-checking-UNIQUE-constraint-for-btree.patch Description: Binary data

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

Hi, Minor comment: + if (errflag == true) + ereport(ERROR, I think 'if (errflag)' should suffice. Cheers On Tue, Feb 9, 2021 at 10:44 AM Pavel Borisov wrote: > вт, 9 февр. 2021 г. в 01:46, Mark Dilger : > >> >> >> > On Feb 8, 2021, at 2:46 AM, Pavel Borisov >> wrote: >> > >> > 0002 -

Re: [POC] verifying UTF-8 using SIMD instructions

On Mon, Feb 8, 2021 at 6:17 AM Heikki Linnakangas wrote: > > I also tested the fallback implementation from the simdjson library > (included in the patch, if you uncomment it in simdjson-glue.c): > > mixed | ascii > ---+--- > 447 |46 > (1 row) > > I think we should at least try t

Re: [HACKERS] Custom compression methods

Please remember to trim unnecessary quoted material. On Sun, Feb 7, 2021 at 6:45 AM Dilip Kumar wrote: > [ a whole lot of quoted stuff ] > > I have tested the performance, pglz vs lz4 > > Test1: With a small simple string, pglz doesn't select compression but > lz4 select as no min limit > Table:

Re: [POC] verifying UTF-8 using SIMD instructions

On 09/02/2021 22:08, John Naylor wrote: Maybe there's a smarter way to check for zeros in C. Or maybe be more careful about cache -- running memchr() on the whole input first might not be the best thing to do. The usual trick is the haszero() macro here: https://graphics.stanford.edu/~seander

Re: Is txid_status() actually safe? / What is 011_crash_recovery.pl testing?

On Mon, Feb 8, 2021 at 4:52 PM Andres Freund wrote: > The 011_crash_recovery.pl test test starts a transaction, creates a > table, fetches the transaction's xid. Then shuts down the server in > immediate mode. It then asserts that after crash recovery the previously > assigned xid is shown as abor

Re: [HACKERS] Custom compression methods

I see the thread got broken somehow (or cfbot thought it did), so I added the new thread, and this is now passing all tests. (I think using the v22 patches). http://cfbot.cputube.org/dilip-kumar.html On Fri, Feb 05, 2021 at 11:07:53AM -0500, Robert Haas wrote: > > Also, I think we may want to ma

Re: [HACKERS] Custom compression methods

On Tue, Feb 9, 2021 at 3:37 PM Justin Pryzby wrote: > I think you misunderstood: I mean that the WIP patch should default to > --enable-lz4, to exercise on a few CI. It's hardly useful to run CI with the > feature disabled. I assume that the patch would be committed with default > --disable-lz4.

Re: [POC] verifying UTF-8 using SIMD instructions

I wrote: > > On Mon, Feb 8, 2021 at 6:17 AM Heikki Linnakangas wrote: > One of his earlier demos [1] (in simdutf8check.h) had a version that used mostly SSE2 with just three intrinsics from SSSE3. That's widely available by now. He measured that at 0.7 cycles per byte, which is still good compared

64-bit XIDs in deleted nbtree pages

There is a long standing problem with the way that nbtree page deletion places deleted pages in the FSM for recycling: The use of a 32-bit XID within the deleted page (in the special area's/BTPageOpaqueData struct's btpo.xact field) is not robust against XID wraparound, which can lead to permanentl

Tightening up allowed custom GUC names

Over in [1] it was noted that the system behaves rather oddly if you try to do ALTER USER/DATABASE SET with a custom GUC name containing "=" or "-". I think we should just disallow such cases. Relaxing the restriction is harder than it might seem: * The convention for entries in pg_db_role_settin

Re: Tightening up allowed custom GUC names

On Tue, Feb 09, 2021 at 05:34:37PM -0500, Tom Lane wrote: > Now granting that the best answer is just to forbid these cases, > there are still a couple of decisions about how extensive the > prohibition ought to be: > > * We could forbid these characters only when you try to actually > put such a

Re: WIP: BRIN multi-range indexes

On 2/9/21 3:46 PM, John Naylor wrote: On Wed, Feb 3, 2021 at 7:54 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > > [v-20210203] Hi Tomas, I have some random comments from reading the patch, but haven't gone into detail in the newer aspects. I'll do so in the near future

Re: Single transaction in the tablesync worker?

On Tue, Feb 9, 2021 at 8:32 PM Amit Kapila wrote: > > On Tue, Feb 9, 2021 at 12:02 PM Peter Smith wrote: > > > > Here are my feedback comments for the V29 patch. > > > > Thanks. > > > > > 3. > > Previously the tablesync origin name format was encapsulated in a > > common function. IMO it was clea

pg_cryptohash_final possible out-of-bounds access (per Coverity)

Hi Hackers, Per Coverity. Coverity complaints about pg_cryptohash_final function. And I agree with Coverity, it's a bad design. Its allows this: #define MY_RESULT_LENGTH 32 function pgtest(char * buffer, char * text) { pg_cryptohash_ctx *ctx; uint8 digest[MY_RESULT_LENGTH]; ctx = pg_cryptohash

RE: Parallel INSERT (INTO ... SELECT ...)

From: Hou, Zhijie/侯 志杰 > Till now, what I found is that: > With tang's conf, when doing parallel insert, the walrecord is more than > serial > insert (IMO, this is the main reason why it has performance degradation) See > the attatchment for the plan info. > > I have tried alter the target table

Re: Clean up code

On Mon, Feb 8, 2021, at 7:09 PM, CK Tan wrote: > Hi, can someone point me to the code that cleans up temp files should a query > crashed unexpectedly? Thanks! See this patch [1]. [1] https://www.postgresql.org/message-id/CAH503wDKdYzyq7U-QJqGn%3DGm6XmoK%2B6_6xTJ-Yn5WSvoHLY1Ww%40mail.gmail.com

Re: adding wait_start column to pg_locks

On 2021/02/09 23:31, torikoshia wrote: On 2021-02-09 22:54, Fujii Masao wrote: On 2021/02/09 19:11, Fujii Masao wrote: On 2021/02/09 18:13, Fujii Masao wrote: On 2021/02/09 17:48, torikoshia wrote: On 2021-02-05 18:49, Fujii Masao wrote: On 2021/02/05 0:03, torikoshia wrote: On 2021-0

Re: 64-bit XIDs in deleted nbtree pages

On Tue, Feb 9, 2021 at 2:14 PM Peter Geoghegan wrote: > The first patch teaches nbtree to use 64-bit transaction IDs here, and > so makes it impossible to leak deleted nbtree pages. This patch is the > nbtree equivalent of commit 6655a729, which made GiST use 64-bit XIDs > due to exactly the same

Re: Single transaction in the tablesync worker?

On Tue, Feb 9, 2021 at 10:38 AM Peter Smith wrote: > > On Mon, Feb 8, 2021 at 11:42 AM Peter Smith wrote: > > > > On Sun, Feb 7, 2021 at 2:38 PM Peter Smith wrote: > > > > > > On Sat, Feb 6, 2021 at 2:10 AM Petr Jelinek > > > wrote: > > > > > > > > Hi, > > > > > > > > Some minor comments about

Re: New IndexAM API controlling index vacuum strategies

On Sat, Feb 6, 2021 at 5:02 AM Peter Geoghegan wrote: > > On Wed, Feb 3, 2021 at 8:18 PM Masahiko Sawada wrote: > > > I'm starting to think that the right short term goal should not > > > directly involve bottom-up index deletion. We should instead return to > > > the idea of "unifying" the vacuu

Re: repeated decoding of prepared transactions

On Wed, Feb 10, 2021 at 12:08 AM Robert Haas wrote: > > On Tue, Feb 9, 2021 at 6:57 AM Amit Kapila wrote: > > I think similar happens without any of the work done in PG-14 as well > > if we restart the apply worker before the commit completes on the > > subscriber. After the restart, we will send

RE: Parallel INSERT (INTO ... SELECT ...)

> > Till now, what I found is that: > > With tang's conf, when doing parallel insert, the walrecord is more > > than serial insert (IMO, this is the main reason why it has > > performance degradation) See the attatchment for the plan info. > > > > I have tried alter the target table to unlogged and

Re: pg_replication_origin_drop API potential race condition

On Tue, Feb 9, 2021 at 5:53 PM Alvaro Herrera wrote: > > On 2021-Feb-09, Amit Kapila wrote: > > > On Tue, Feb 9, 2021 at 4:16 PM Alvaro Herrera > > wrote: > > > > By all means let's get the bug fixed. > > > > I am planning to push this in HEAD only as there is no user reported > > problem and th

Re: Is Recovery actually paused?

At Tue, 9 Feb 2021 12:27:21 +0530, Bharath Rupireddy wrote in > What I meant was that if we were to add waiting logic inside > pg_wal_replay_pause, we should also have a timeout with some default > value, to avoid pg_wal_replay_pause waiting forever in the waiting > loop. Within that timeout, if

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

At Tue, 9 Feb 2021 22:01:45 -0300, Ranier Vilela wrote in > Hi Hackers, > > Per Coverity. > > Coverity complaints about pg_cryptohash_final function. > And I agree with Coverity, it's a bad design. > Its allows this: > > #define MY_RESULT_LENGTH 32 > > function pgtest(char * buffer, char * te

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

At Wed, 10 Feb 2021 12:13:44 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 9 Feb 2021 22:01:45 -0300, Ranier Vilela wrote > in > > Hi Hackers, > > > > Per Coverity. > > > > Coverity complaints about pg_cryptohash_final function. > > And I agree with Coverity, it's a bad design. > > Its

Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

Hi: This patch is the first patch in UniqueKey patch series[1], since I need to revise that series many times but the first one would be not that often, so I'd like to submit this one for review first so that I don't need to maintain it again and again. v1-0001-Introduce-notnullattrs-field-in-Rel

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

On Wed, Feb 10, 2021 at 11:18 AM Andy Fan wrote: > Hi: > > This patch is the first patch in UniqueKey patch series[1], since I need > to revise > that series many times but the first one would be not that often, so I'd > like to > submit this one for review first so that I don't need to maintain

Re: Parallel INSERT (INTO ... SELECT ...)

On Tue, Feb 9, 2021 at 10:30 AM Greg Nancarrow wrote: > On Tue, Feb 9, 2021 at 1:04 AM Amit Langote wrote: > > That brings me to to this part of the hunk: > > > > + /* > > +* If there is no underlying SELECT, a parallel table-modification > > +* operation is not possible (nor desirable)

Re: Parallel INSERT (INTO ... SELECT ...)

On Wed, Jan 6, 2021 at 7:39 PM Antonin Houska wrote: > > > @@ -252,6 +252,7 @@ set_plan_references(PlannerInfo *root, Plan *plan) > PlannerGlobal *glob = root->glob; > int rtoffset = list_length(glob->finalrtable); > ListCell *lc; > + Plan

Re: [POC] verifying UTF-8 using SIMD instructions

On Tue, Feb 9, 2021 at 4:22 PM Heikki Linnakangas wrote: > > On 09/02/2021 22:08, John Naylor wrote: > > Maybe there's a smarter way to check for zeros in C. Or maybe be more > > careful about cache -- running memchr() on the whole input first might > > not be the best thing to do. > > The usual t

Re: Single transaction in the tablesync worker?

On Tue, Feb 9, 2021 at 10:38 AM Peter Smith wrote: > PSA an alternative patch. This one adds a new member to > WalRcvExecResult and so is able to detect the "slot does not exist" > error. This patch also applies on top of V28, if you want it. Did some testing with this patch on top of v29. I cou

Re: Is Recovery actually paused?

On Wed, Feb 10, 2021 at 8:19 AM Kyotaro Horiguchi wrote: > > At Tue, 9 Feb 2021 12:27:21 +0530, Bharath Rupireddy > wrote in > > What I meant was that if we were to add waiting logic inside > > pg_wal_replay_pause, we should also have a timeout with some default > > value, to avoid pg_wal_replay

Re: Preserve attstattarget on REINDEX CONCURRENTLY

On Sun, Feb 07, 2021 at 09:39:36AM +0900, Michael Paquier wrote: > I'll see about applying this stuff after the next version is tagged > then. The new versions have been tagged, so done as of bd12080 and back-patched. I have added a note in the commit log about the approach to use index_create()

Re: Parallel INSERT (INTO ... SELECT ...)

On Wed, Feb 10, 2021 at 2:39 PM Amit Langote wrote: > > > The code check that you have identified above ensures that the INSERT > > has an underlying SELECT, because the planner won't (and shouldn't > > anyway) generate a parallel plan for INSERT...VALUES, so there is no > > point doing any parall

Re: repeated decoding of prepared transactions

On Wed, Feb 10, 2021 at 8:02 AM Amit Kapila wrote: > On Wed, Feb 10, 2021 at 12:08 AM Robert Haas > wrote: > > > > On Tue, Feb 9, 2021 at 6:57 AM Amit Kapila > wrote: > > > I think similar happens without any of the work done in PG-14 as well > > > if we restart the apply worker before the comm

Re: pg_cryptohash_final possible out-of-bounds access (per Coverity)

At Tue, 9 Feb 2021 22:01:45 -0300, Ranier Vilela wrote in > Hi Hackers, > > Per Coverity. > > Coverity complaints about pg_cryptohash_final function. > And I agree with Coverity, it's a bad design. > Its allows this: > > #define MY_RESULT_LENGTH 32 > > function pgtest(char * buffer, char * te

Re: [HACKERS] Custom compression methods

On Wed, Feb 10, 2021 at 1:42 AM Robert Haas wrote: > > Please remember to trim unnecessary quoted material. Okay, I will. > On Sun, Feb 7, 2021 at 6:45 AM Dilip Kumar wrote: > > [ a whole lot of quoted stuff ] > > > > Conclusion: > > 1. In most cases lz4 is faster and doing better compression a

Re: TRUNCATE on foreign table

On Tue, Feb 9, 2021 at 7:45 PM Kazutaka Onishi wrote: > > > IIUC, "truncatable" would be set to "false" for relations which do not > > have physical storage e.g. views but will be true for regular tables. > > "truncatable" option is just for the foreign table and it's not related with > whether i

RE: Parallel INSERT (INTO ... SELECT ...)

From: Hou, Zhijie/侯 志杰 > table and index's size after parallel insert > -- > postgres=# select pg_size_pretty(pg_indexes_size('testscan_index')); > pg_size_pretty > > 4048 kB > (1 row) > > postgres=# select pg_size_pretty(pg_relation_size('testscan_index'));

RE: Parallel INSERT (INTO ... SELECT ...)

> > -- > > postgres=# select pg_size_pretty(pg_indexes_size('testscan_index')); > > pg_size_pretty > > > > 4048 kB > > (1 row) > > > > postgres=# select pg_size_pretty(pg_relation_size('testscan_index')); > > pg_size_pretty > > > > 4768 kB >

Re: Single transaction in the tablesync worker?

On Wed, Feb 10, 2021 at 7:41 AM Peter Smith wrote: > > On Tue, Feb 9, 2021 at 10:38 AM Peter Smith wrote: > > > > PSA v2 of this WalRcvExceResult patch (it is same as v1 but includes > some PG doc updates). > This applies OK on top of v30 of the main patch. > Thanks, I have integrated these chan

Re: [patch] bit XOR aggregate functions

At Tue, 9 Feb 2021 15:25:19 +, Alexey Bashtanov wrote in > I personally use it as a checksum for a large unordered set, where > performance and simplicity is prioritized over collision resilience. > Maybe there are other ways to use them. FWIW the BIT_XOR can be created using CREATE AGGREGA

RE: Parallel INSERT (INTO ... SELECT ...)

From: Hou, Zhijie/侯 志杰 > Sorry, Maybe the tablename is a little confused, > but 'testscan_index' is actually a table's name. ... > Did I miss something ? No, I don't think so. I just wanted to know the facts correctly. Your EXPLAIN output shows that the target table is testscan as follows. Ho

Re: Online checksums patch - once again

On Tue, Feb 09, 2021 at 10:54:50AM +0200, Heikki Linnakangas wrote: > (I may have said this before, but) My overall high-level impression of this > patch is that it's really cmmplex for a feature that you use maybe once in > the lifetime of a cluster. I'm happy to review but I'm not planning to > c

Re: Libpq support to connect to standby server as priority

On Tue, Feb 9, 2021 at 5:47 AM Greg Nancarrow wrote: > > On Mon, Feb 8, 2021 at 8:17 PM vignesh C wrote: > > > > > > > > I think what we want to do is mark default_transaction_read_only as > > > GUC_REPORT, instead. That will give a reliable report of what the > > > state of its GUC is, and you

Re: Libpq support to connect to standby server as priority

Thanks for the comments Greg, please find my comments inline below. On Tue, Feb 9, 2021 at 2:27 PM Greg Nancarrow wrote: > > On Mon, Feb 8, 2021 at 8:17 PM vignesh C wrote: > > > > > > > > I think what we want to do is mark default_transaction_read_only as > > > GUC_REPORT, instead. That will g

Re: repeated decoding of prepared transactions

On Wed, Feb 10, 2021 at 3:43 PM Ashutosh Bapat wrote: > I think we need to treat a prepared transaction slightly different from an > uncommitted transaction when sending downstream. We need to send a whole > uncommitted transaction downstream again because previously applied changes > must ha

  1   2   >