Migration Oracle multitenant database to PostgreSQL ?

2020-11-24 Thread ROS Didier
Hi I would like to know if it is possible to migrate Oracle multitenant database (with multiple PDB) to PostgreSQL ? Thanks in advance Best Regards D ROS EDF Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les

Re: Migration Oracle multitenant database to PostgreSQL ?

2020-11-24 Thread Thomas Kellerer
ROS Didier schrieb am 24.11.2020 um 09:09: > I would like to know if it is possible to migrate Oracle multitenant > database (with multiple PDB) to PostgreSQL ? Postgres' databases are very similar to Oracle's PDBs. Probably the biggest difference is, that you can't shutdown a single database as

回复: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2020-11-24 Thread Junfeng Yang
Hi hackers, Can anyone help to verify this?

RE: POC: postgres_fdw insert batching

2020-11-24 Thread tsunakawa.ta...@fujitsu.com
From: Tomas Vondra > 1) We're calling it "batch_size" but the API function is named > postgresGetMaxBulkInsertTuples(). Perhaps we should rename the function > to postgresGetModifyBatchSize()? That has the advantage it'd work if we > ever add support for batching to UPDATE/DELETE. Actually, I was

Re: Deduplicate aggregates and transition functions in planner

2020-11-24 Thread Heikki Linnakangas
On 19/11/2020 12:38, Heikki Linnakangas wrote: So barring objections, I'm going to push the attached updated patch that includes the removal of AggrefExprState, and leave CookedAggrefs or other further refactorings for the future. Done. Thanks! - Heikki

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Yugo NAGATA
On Wed, 11 Nov 2020 19:10:35 +0300 Konstantin Knizhnik wrote: Thank you for reviewing this patch! > > The patch is not applied to the current master because makeFuncCall > prototype is changed, > I fixed it by adding COAERCE_CALL_EXPLICIT. The rebased patch was submitted. > Ooops! Now TPS ar

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Yugo NAGATA
On Thu, 12 Nov 2020 15:37:42 +0300 Konstantin Knizhnik wrote: > Well, creation of proper indexes for table is certainly responsibility > of DBA. > But users may not consider materialized view as normal table. So the > idea that index should > be explicitly created for materialized view seems to

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Konstantin Knizhnik
On 24.11.2020 12:21, Yugo NAGATA wrote: I replaced it with RowExlusiveLock and ... got 1437 TPS with 10 connections. It is still about 7 times slower than performance without incremental view. But now the gap is not so dramatic. And it seems to be clear that this exclusive lock on matview is

Re: [HACKERS] logical decoding of two-phase transactions

2020-11-24 Thread Ajin Cherian
On Mon, Nov 23, 2020 at 10:35 PM Amit Kapila wrote: > For the first two, as the xact is still not visible to others so we > don't need to make it behave like a committed txn. To make the (DDL) > changes visible to the current txn, the message > REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID copies the

Re: psql: add \si, \sm, \st and \sr functions to show CREATE commands for indexes, matviews, triggers and tables

2020-11-24 Thread Anastasia Lubennikova
On 18.08.2020 17:25, Tom Lane wrote: a.pervush...@postgrespro.ru writes: [ si_st_sm_sr_v2.patch ] I hadn't particularly noticed this thread before, but I happened to look through this patch, and I've got to say that this proposed feature seems like an absolute disaster from a maintenance standp

Re: Implementing Incremental View Maintenance

2020-11-24 Thread Yugo NAGATA
On Tue, 24 Nov 2020 12:46:57 +0300 Konstantin Knizhnik wrote: > > > On 24.11.2020 12:21, Yugo NAGATA wrote: > > > >> I replaced it with RowExlusiveLock and ... got 1437 TPS with 10 > >> connections. > >> It is still about 7 times slower than performance without incremental view. > >> But now t

Re: LogwrtResult contended spinlock

2020-11-24 Thread Anastasia Lubennikova
On 04.09.2020 20:13, Andres Freund wrote: Hi, On 2020-09-04 10:05:45 -0700, Andres Freund wrote: On 2020-09-03 14:34:52 -0400, Alvaro Herrera wrote: Looking at patterns like this if (XLogCtl->LogwrtRqst.Write < EndPos) XLogCtl->LogwrtRqst.Write = EndPos; It seems poss

Re: scram-sha-256 broken with FIPS and OpenSSL 1.0.2

2020-11-24 Thread Michael Paquier
On Sat, Nov 21, 2020 at 10:19:42AM +0900, Michael Paquier wrote: > What you meant and what I meant was slightly different here. I meant > publishing a header in src/include/common/ that would get installed, > and I'd rather avoid that. And you mean to have the header for local > consumption in sr

RE: Parallel Inserts in CREATE TABLE AS

2020-11-24 Thread Hou, Zhijie
Hi, I'm very interested in this feature, and I'm looking at the patch, here are some comments. 1. + if (!TupIsNull(outerTupleSlot)) + { + (void) node->ps.dest->receiveSlot(outerTupleSlot, node->ps.dest); +

Remove cache_plan argument comments to ri_PlanCheck

2020-11-24 Thread Li Japin
Hi, hackers I found that the cache_plan argument to ri_PlanCheck already been remove since 5b7ba75f7ff854003231e8099e3038c7e2eba875. I think we can remove the comments tor cache_plan to ri_PlanCheck. diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c index 7

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Dilip Kumar
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: Most of the comments looks fine to me but I have a slightly different opinion for one of the comment so replying only for that. > I'm worried about how expensive this might be, and I think we could > make it cheaper. The reason why I think this

Re: pgbench and timestamps (bounced)

2020-11-24 Thread Anastasia Lubennikova
On 11.09.2020 16:59, Fabien COELHO wrote: Hello Tom, It requires a mutex around the commands, I tried to do some windows implementation which may or may not work. Ugh, I'd really rather not do that.  Even disregarding the effects of a mutex, though, my initial idea for fixing this has a big

Re: Libpq support to connect to standby server as priority

2020-11-24 Thread Anastasia Lubennikova
On 30.09.2020 10:57, Greg Nancarrow wrote: Thanks for your thoughts, patches and all the pointers. I'll be looking at all of them. (And yes, the comma instead of bitwise OR is of course an error, somehow made and gone unnoticed; the next field in the struct is an enum, so accepts any int value).

Re: libpq compression

2020-11-24 Thread Konstantin Knizhnik
Based on Andres review I have implemented the following changes in libpq_compression: 1. Make it possible to specify list of compression algorithms in connection string. 2. Make it possible to specify compression level. 3. Use "_pq_.compression" instead of "compression"  in startup package. 4.

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-11-24 Thread Magnus Hagander
On Fri, Nov 20, 2020 at 4:46 PM Peter Eisentraut wrote: > > On 2020-11-09 13:05, Magnus Hagander wrote: > > PFA a rebased version of this patch on top of what has happened since, > > and changing the pg_upgrade parameter to be --no-scripts. > > It seems were are still finding out more nuances abou

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Robert Haas
On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar wrote: > About (4), one option is that we directly call the correct handler > function for the built-in type directly from > toast_(de)compress(_slice) functions but in that case, we are > duplicating the code, another option is that we call the > GetCom

About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
Hi: For example, we added a new field in a node in primnodes.h struct FuncExpr { + int newf; }; then we modified the copy/read/out functions for this node. In _readFuncExpr, we probably add something like static FuncExpr _readFuncExpr(..) { .. + READ_INT_FILED(newf); }; Then we will get a

Re: Remove cache_plan argument comments to ri_PlanCheck

2020-11-24 Thread Amit Kapila
On Tue, Nov 24, 2020 at 4:46 PM Li Japin wrote: > > Hi, hackers > > I found that the cache_plan argument to ri_PlanCheck already been remove since > 5b7ba75f7ff854003231e8099e3038c7e2eba875. I think we can remove the comments > tor cache_plan to ri_PlanCheck. > > diff --git a/src/backend/utils/a

Re: vac_update_datfrozenxid will raise "wrong tuple length" if pg_database tuple contains toast attribute.

2020-11-24 Thread Amit Kapila
On Tue, Nov 24, 2020 at 2:07 PM Junfeng Yang wrote: > > Hi hackers, > > Can anyone help to verify this? > I think one way to get feedback is to register this patch for the next commit fest (https://commitfest.postgresql.org/31/) -- With Regards, Amit Kapila.

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-24 Thread Fujii Masao
On 2020/11/21 2:32, Matthias van de Meent wrote: Hi, The pg_stat_progress_cluster view can report incorrect heap_blks_scanned values when synchronize_seqscans is enabled, because it allows the sequential heap scan to not start at block 0. This can result in wraparounds in the heap_blks_scanne

Re: Migration Oracle multitenant database to PostgreSQL ?

2020-11-24 Thread Bruce Momjian
On Tue, Nov 24, 2020 at 09:22:26AM +0100, Thomas Kellerer wrote: > > ROS Didier schrieb am 24.11.2020 um 09:09: > > I would like to know if it is possible to migrate Oracle multitenant > > database (with multiple PDB) to PostgreSQL ? > Postgres' databases are very similar to Oracle's PDBs. > > Pr

Re: [PATCH] LWLock self-deadlock detection

2020-11-24 Thread Ashutosh Bapat
This looks useful. LWLockCheckSelfDeadlock() could use LWLockHeldByMe variant instead of copying that code with possibly a change in that function to return the required information. I am also seeing a pattern Assert(LWLockHeldByMe*()) LWLockAcquire() at some places. Should we change LWLockAcquir

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-11-24 Thread Bruce Momjian
On Tue, Nov 24, 2020 at 01:32:45PM +0100, Magnus Hagander wrote: > I think it boils down to that today the output from initdb is entirely > geared towards people running initdb directly and starting their > server manually, and very few people outside the actual PostgreSQL > developers ever do that

Re: [doc] plan invalidation when statistics are update

2020-11-24 Thread Fujii Masao
On 2020/11/19 14:33, torikoshia wrote: On 2020-11-18 11:35, Fujii Masao wrote: Thanks for your comment! On 2020/11/18 11:04, torikoshia wrote: Hi, AFAIU, when the planner statistics are updated, generic plans are invalidated and PostgreSQL recreates. However, the manual doesn't seem to e

Re: LogwrtResult contended spinlock

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-24, Anastasia Lubennikova wrote: > On 04.09.2020 20:13, Andres Freund wrote: > > Re general routine: On second thought, it might actually be worth having > > it. Even just for LSNs - there's plenty places where it's useful to > > ensure a variable is at least a certain size. I think

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-11-24 Thread Magnus Hagander
On Tue, Nov 24, 2020 at 3:12 PM Bruce Momjian wrote: > > On Tue, Nov 24, 2020 at 01:32:45PM +0100, Magnus Hagander wrote: > > I think it boils down to that today the output from initdb is entirely > > geared towards people running initdb directly and starting their > > server manually, and very fe

Re: walsender bug: stuck during shutdown

2020-11-24 Thread Alvaro Herrera
Hello, On 2020-Nov-24, Fujii Masao wrote: > Thanks for working on this! > Could you tell me the discussion thread where Chloe Dives reported the issue > to? > Sorry I could not find that.. It was not public -- sorry I didn't make that clear. > I'd like to see the procedure to reproduce the iss

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-24, Andy Fan wrote: > then we modified the copy/read/out functions for this node. In > _readFuncExpr, > we probably add something like > [ ... ] > Then we will get a compatible issue if we create a view with the node in > the older version and access the view with the new binary. W

Re: bug in pageinspect's "tuple data" feature

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-24, Michael Paquier wrote: > On Mon, Nov 23, 2020 at 09:11:26AM +0200, Heikki Linnakangas wrote: > > On 21/11/2020 21:32, Alvaro Herrera wrote: > >> This is pretty unhelpful; it would be better not to try to print the > >> data instead of dying. With that, at least you can know where

Re: Terminate the idle sessions

2020-11-24 Thread David G. Johnston
On Mon, Nov 23, 2020 at 11:22 PM Li Japin wrote: > > How about use “foreign-data wrapper” replace “postgres_fdw”? > I don't see much value in avoiding mentioning that specific term - my proposal turned it into an example instead of being exclusive. > - This parameter should be set to z

Re: [patch] CLUSTER blocks scanned progress reporting

2020-11-24 Thread Matthias van de Meent
On Tue, 24 Nov 2020 at 15:05, Fujii Masao wrote: > > On 2020/11/21 2:32, Matthias van de Meent wrote: > > Hi, > > > > The pg_stat_progress_cluster view can report incorrect > > heap_blks_scanned values when synchronize_seqscans is enabled, because > > it allows the sequential heap scan to not star

Re: abstract Unix-domain sockets

2020-11-24 Thread David G. Johnston
On Mon, Nov 23, 2020 at 9:00 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > Or is it the case that we always attempt to bind the TCP/IP port, > regardless of the presence of a socket file, in which case the failure for > port binding does cover the socket situation as well? > This ca

Re: Prevent printing "next step instructions" in initdb and pg_upgrade

2020-11-24 Thread Bruce Momjian
On Tue, Nov 24, 2020 at 04:05:26PM +0100, Magnus Hagander wrote: > pg_upgrade is a somewhat different but also interesting case. I think > the actual progress output is more interesting in pg_upgrade as it's > more likely to take measurable amounts of time. Whereas in initdb, > it's actually the "d

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-24 Thread Justin Pryzby
On Sat, Oct 31, 2020 at 01:36:11PM -0500, Justin Pryzby wrote: > > > From the grammar perspective ANY option is available for any command > > > that uses parenthesized option list. All the checks and validations > > > are performed at the corresponding command code. > > > This analyze_keyword is ac

Re: abstract Unix-domain sockets

2020-11-24 Thread Peter Eisentraut
On 2020-11-23 17:00, David G. Johnston wrote: So presently there is no functioning code to prevent two PostgreSQL instances from using the same socket so long as they do not also use the same data directory?  We only handle the case of an unclean crash - where the pid and socket are both left b

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Dilip Kumar
On Tue, Nov 24, 2020 at 7:14 PM Robert Haas wrote: > > On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar wrote: > > About (4), one option is that we directly call the correct handler > > function for the built-in type directly from > > toast_(de)compress(_slice) functions but in that case, we are > > d

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Peter Eisentraut
On 2020-11-24 01:52, Dagfinn Ilmari Mannsåker wrote: The Clang documentation¹ suggest an even neater solution, which would eliminate the repetitive empty pg_attribute_foo #defines in the trailing #else/#endif block in commit 1fa22a43a56e1fe44c7bb3a3d5ef31be5bcac41d: #ifndef __has_attribute #defi

Re: abstract Unix-domain sockets

2020-11-24 Thread David G. Johnston
On Tue, Nov 24, 2020 at 8:45 AM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > We're subject to whatever the kernel behavior is. If the kernel doesn't > report address conflicts for Unix-domain sockets, then we can't do > anything about that. Having an error message ready in case

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Tom Lane
David Rowley writes: > Pushed. walleye's been failing since this patchset went in: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=walleye&dt=2020-11-24%2000%3A25%3A31 ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmiss

Re: libpq compression

2020-11-24 Thread Robert Haas
On Tue, Nov 24, 2020 at 7:33 AM Konstantin Knizhnik wrote: > New version of the patch is attached. I read over the comments from Andres (and Peter) suggesting that this ought to be on-the-fly configurable. Here are some thoughts on making that work with the wire protocol: If the client potential

Re: PoC/WIP: Extended statistics on expressions

2020-11-24 Thread Justin Pryzby
On Mon, Nov 23, 2020 at 04:30:26AM +0100, Tomas Vondra wrote: > 0004 - Seems fine. IMHO not really "silly errors" but OK. This is one of the same issues you pointed out - shadowing a variable. Could be backpatched. On Mon, Nov 23, 2020 at 04:30:26AM +0100, Tomas Vondra wrote: > > +

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-24 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I took a quick look through this. This is just MHO, of course: > >> > >> * I don't think it's okay to change the existing signatures of > >> pg_ls_logdir() et al. > > > I d

Re: POC: postgres_fdw insert batching

2020-11-24 Thread Tomas Vondra
On 11/24/20 9:45 AM, tsunakawa.ta...@fujitsu.com wrote: > From: Tomas Vondra >> 1) We're calling it "batch_size" but the API function is named >> postgresGetMaxBulkInsertTuples(). Perhaps we should rename the function >> to postgresGetModifyBatchSize()? That has the advantage it'd work if we >>

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Robert Haas
On Tue, Nov 24, 2020 at 10:47 AM Dilip Kumar wrote: > For the compression routine name, I did not include "Am" because > currently, we are storing the compression method in the new catalog > "pg_compression" not in the pg_am. So are you suggesting that we > should store the compression methods a

Re: PoC/WIP: Extended statistics on expressions

2020-11-24 Thread Tomas Vondra
On 11/24/20 5:23 PM, Justin Pryzby wrote: > On Mon, Nov 23, 2020 at 04:30:26AM +0100, Tomas Vondra wrote: >> 0004 - Seems fine. IMHO not really "silly errors" but OK. > > This is one of the same issues you pointed out - shadowing a variable. > Could be backpatched. > > On Mon, Nov 23, 2020 at

Re: Strange behavior with polygon and NaN

2020-11-24 Thread Tom Lane
Kyotaro Horiguchi writes: > At Fri, 20 Nov 2020 15:57:46 -0500, Tom Lane wrote in >> I don't much like anything about float8_coef_mul(). > I have the same feeling on the function, but I concluded that > coefficients and coordinates should be regarded as different things in > the practical stand

Re: libpq compression

2020-11-24 Thread Daniil Zakhlystov
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, passed Spec compliant: tested, failed Documentation:tested, failed Submission review -- Is the patch in a patch format which has

Re: Online verification of checksums

2020-11-24 Thread David Steele
Hi Michael, On 11/23/20 8:10 PM, Michael Paquier wrote: On Mon, Nov 23, 2020 at 10:35:54AM -0500, Stephen Frost wrote: Also- what is the point of reading the page from shared buffers anyway..? All we need to do is prove that the page will be rewritten during WAL replay. If we can prove that,

Re: mark/restore failures on unsorted merge joins

2020-11-24 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> The problem is that the planner calls ExecSupportsMarkRestore to >> find out whether a Materialize node is needed, and that function >> looks no further than the Path type of T_Index[Only]Path in order to >> return true, even though in this case it's a GiST

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 24, 2020 at 10:47 AM Dilip Kumar wrote: >> For the compression routine name, I did not include "Am" because >> currently, we are storing the compression method in the new catalog >> "pg_compression" not in the pg_am. So are you suggesting that we >> should stor

Re: [PoC] Non-volatile WAL buffer

2020-11-24 Thread Tomas Vondra
On 11/24/20 7:34 AM, tsunakawa.ta...@fujitsu.com wrote: > From: Tomas Vondra >> So I wonder if using PMEM for the WAL buffer is the right way forward. >> AFAIK the WAL buffer is quite concurrent (multiple clients writing >> data), which seems to contradict the PMEM vs. DRAM trade-offs. >> >> Th

Re: mark/restore failures on unsorted merge joins

2020-11-24 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> Uh, why would you not just look to see if the ammarkpos/amrestrpos > Tom> fields are non-null? > We don't (in the back branches) seem to have a pointer to the > IndexAmRoutine handy, only the oid? Oh, sorry, I misread your comment to be

Re: libpq compression

2020-11-24 Thread Robert Haas
On Tue, Nov 24, 2020 at 12:35 PM Daniil Zakhlystov wrote: > To sum up, I think that the current implementation already introduces good > benefits. As I proposed in the Usability review, we may introduce the new > approaches later as separate compression 'algorithms'. I don't think the current p

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-24 Thread David Zhang
I verified the patch "v2-0001-Free-disk-space-for-dropped-relations-on-commit.patch" on master branch "0cc9932740f2bf572303b68438e4caf62de9". It works for me. Below is my test procedure and results. === Before the patch === #1 from psql console 1, create table and index then insert enough d

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Robert Haas
On Tue, Nov 24, 2020 at 1:21 PM Tom Lane wrote: > FWIW, I kind of agree with Robert's take on this. Heap and index AMs > are pretty fundamentally different animals, yet we don't have a problem > sticking them in the same catalog. I think anything that is related to > storage access could reasona

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-24 Thread Alexey Kondratov
On 2020-11-24 06:52, Bharath Rupireddy wrote: Thanks for the review comments. On Mon, Nov 23, 2020 at 9:57 PM Alexey Kondratov wrote: > v1-0001-postgres_fdw-function-to-discard-cached-connections.patch This patch looks pretty straightforward for me, but there are some things to be addressed

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-24 Thread Pavel Borisov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested Given we got two other reviews from Neil and David, I think I can fin

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-24, Tom Lane wrote: > Robert Haas writes: > > Oh, I thought it had been suggested in previous discussions that these > > should be treated as access methods rather than inventing a whole new > > concept just for this, and it seemed like a good idea to me. I guess I > > missed the fac

Re: enable_incremental_sort changes query behavior

2020-11-24 Thread Tom Lane
James Coleman writes: > On Mon, Nov 23, 2020 at 2:24 PM Tom Lane wrote: >> 1. James was wondering, far upthread, why we would do projections >> pre-sort or post-sort. I think the answers can be found by studying >> planner.c's make_sort_input_target(), which separates out what we want >> to do p

Re: mark/restore failures on unsorted merge joins

2020-11-24 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> Oh, sorry, I misread your comment to be that you wanted to add a Tom> field to IndexAmRoutine. You're right, the real issue here is that Tom> ExecSupportsMarkRestore lacks any convenient access to the needed Tom> info, and we need to add a bool to IndexOpt

Re: libpq compression

2020-11-24 Thread Daniil Zakhlystov
I completely agree that backward-compatibility is important here. I think that it is a good idea to clarify how the compression establishment works in the current version of the patch: 1. Frontend send the startup packet which may look like this: _pq_.compression = 'zlib,zstd' (I omitted the va

Re: mark/restore failures on unsorted merge joins

2020-11-24 Thread Tom Lane
Andrew Gierth writes: > I guess that's close enough; this should suffice then. Looks about right. Not sure if we need to bother with a regression test case; once that's in, it'd be hard to break it. regards, tom lane

Ready For Committer patches (CF 2020-11)

2020-11-24 Thread Anastasia Lubennikova
Hi, with this message, I want to draw attention to the RFC patches on the current commitfest. It would be good if committers could take a look at them. While doing a sweep through the CF, I have kicked a couple of entries back to Waiting on author, so now the list is correct. Now we have 17

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread David Rowley
On Wed, 25 Nov 2020 at 04:48, Peter Eisentraut wrote: > > On 2020-11-24 01:52, Dagfinn Ilmari Mannsåker wrote: > > The Clang documentation¹ suggest an even neater solution, which would > > eliminate the repetitive empty pg_attribute_foo #defines in the trailing > > #else/#endif block in commit 1fa

Re: bug in pageinspect's "tuple data" feature

2020-11-24 Thread Robert Haas
On Sat, Nov 21, 2020 at 2:32 PM Alvaro Herrera wrote: > If you have a sufficiently broken data page, pageinspect throws an > error when trying to examine the page: > > ERROR: invalid memory alloc request size 18446744073709551451 > > This is pretty unhelpful; it would be better not to try to prin

Re: mark/restore failures on unsorted merge joins

2020-11-24 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> I guess that's close enough; this should suffice then. Tom> Looks about right. Not sure if we need to bother with a regression Tom> test case; once that's in, it'd be hard to break it. We could check the EXPLAIN output (since the Materialize node would show

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread David Rowley
On Wed, 25 Nov 2020 at 04:55, Tom Lane wrote: > > walleye's been failing since this patchset went in: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=walleye&dt=2020-11-24%2000%3A25%3A31 > > ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith > -Wdeclaration-after-statement -Werror=

Re: mark/restore failures on unsorted merge joins

2020-11-24 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> Looks about right. Not sure if we need to bother with a regression > Tom> test case; once that's in, it'd be hard to break it. > We could check the EXPLAIN output (since the Materialize node would show > up), but it's not easy to get sta

Re: libpq compression

2020-11-24 Thread Konstantin Knizhnik
On 24.11.2020 21:35, Robert Haas wrote: On Tue, Nov 24, 2020 at 12:35 PM Daniil Zakhlystov wrote: To sum up, I think that the current implementation already introduces good benefits. As I proposed in the Usability review, we may introduce the new approaches later as separate compression 'al

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Tom Lane
David Rowley writes: > On Wed, 25 Nov 2020 at 04:55, Tom Lane wrote: >> walleye's been failing since this patchset went in: >> I have no idea what to make of that, but it looks more like a compiler bug >> than anything else. > I wondered if #if !defined(__MINGW32__) && !defined(__MINGW64__) woul

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-24, Tom Lane wrote: > David Rowley writes: > > On Wed, 25 Nov 2020 at 04:55, Tom Lane wrote: > >> walleye's been failing since this patchset went in: > >> I have no idea what to make of that, but it looks more like a compiler bug > >> than anything else. > > > I wondered if #if !def

Re: libpq compression

2020-11-24 Thread Konstantin Knizhnik
On 24.11.2020 20:34, Daniil Zakhlystov wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, passed Spec compliant: tested, failed Documentation:tested, failed Submission

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Nov-24, Tom Lane wrote: >> I'd make any such fix as narrow as possible (ie MINGW64 only, based on >> present evidence). It'd be nice to have a compiler version upper bound >> too, in the hopes that they'd fix it in future. Maybe something like >> "#if defined(__M

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-23, Andres Freund wrote: > On 2020-11-23 12:30:05 -0300, Alvaro Herrera wrote: > > PROC_IN_LOGICAL_DECODING: > > Oddly enough, I think the reset of PROC_IN_LOGICAL_DECODING in > > ReplicationSlotRelease might be the most problematic one of the lot. > > That's because a proc's xmin tha

Re: Libpq support to connect to standby server as priority

2020-11-24 Thread Tom Lane
Anastasia Lubennikova writes: > Hi, this entry is "Waiting on Author" and the thread was inactive for a > while. As far as I see, the patch needs some further work. > Are you going to continue working on it, or should I mark it as > "returned with feedback" until a better time? I'm inclined to

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-23, Tom Lane wrote: > Alvaro Herrera writes: > > GetCurrentVirtualXIDs, ComputeXidHorizons, GetSnapshotData: > > > In these cases, what we need is that the code computes some xmin (or > > equivalent computation) based on a set of transactions that exclude > > those marked with the f

Re: Libpq support to connect to standby server as priority

2020-11-24 Thread Alvaro Herrera
On 2020-Nov-24, Tom Lane wrote: > I'm inclined to go ahead and commit the 0001 patch I posted at [1] > (ie, change the implementation of GUC_REPORT to avoid intra-query > reports), since that addresses a performance problem that's > independent of the goal here. The rest of this seems to still >

Re: Libpq support to connect to standby server as priority

2020-11-24 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Nov-24, Tom Lane wrote: >>> As it stands, 0001 reduces the ParameterStatus message traffic to >>> at most one per GUC per query, but it doesn't attempt to eliminate >>> duplicate ParameterStatus messages altogether. We could do that >>> as a pretty simple adjustme

Add table access method as an option to pgbench

2020-11-24 Thread David Zhang
Hi Hackers, I noticed that there is a table access method API added starting from PG12. In other words, Postgresql open the door for developers to add their own access methods, for example, zheap, zedstore etc. However, the current pgbench doesn't have an option to allow user to specify which

RE: A few new options for CHECKPOINT

2020-11-24 Thread tsunakawa.ta...@fujitsu.com
From: Bossart, Nathan > The main purpose of this patch is to give users more control over their > manually > requested checkpoints or restartpoints. I suspect the most useful option is > IMMEDIATE, which can help avoid checkpoint- related IO spikes. However, I > didn't see any strong reason to

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
On Tue, Nov 24, 2020 at 11:11 PM Alvaro Herrera wrote: > On 2020-Nov-24, Andy Fan wrote: > > > then we modified the copy/read/out functions for this node. In > > _readFuncExpr, > > we probably add something like > > > [ ... ] > > > Then we will get a compatible issue if we create a view with the

RE: [PoC] Non-volatile WAL buffer

2020-11-24 Thread tsunakawa.ta...@fujitsu.com
From: Tomas Vondra > It's interesting that they only place the tail of the log on PMEM, i.e. > the PMEM buffer has limited size, and the rest of the log is not on > PMEM. It's a bit as if we inserted a PMEM buffer between our wal buffers > and the WAL segments, and kept the WAL segments on regular

Re: A few new options for CHECKPOINT

2020-11-24 Thread Bossart, Nathan
On 11/24/20, 4:03 PM, "tsunakawa.ta...@fujitsu.com" wrote: > From: Bossart, Nathan >> The main purpose of this patch is to give users more control over their >> manually >> requested checkpoints or restartpoints. I suspect the most useful option is >> IMMEDIATE, which can help avoid checkpoint

Re: Libpq support to connect to standby server as priority

2020-11-24 Thread Tom Lane
I wrote: > Alvaro Herrera writes: >> Agreed. If this is just a few hundred bytes of server-side local memory >> per backend, it seems definitely worth it. > Yeah, given the current set of GUC_REPORT variables, it's hard to see > the storage for their last-reported values amounting to much. The

RE: A few new options for CHECKPOINT

2020-11-24 Thread tsunakawa.ta...@fujitsu.com
From: Bossart, Nathan > It may be useful for backups taken with the "consistent snapshot" > approach. As noted in the documentation [0], running CHECKPOINT > before taking the snapshot can reduce recovery time. However, users > might wish to avoid the IO spike caused by an immediate checkpoint.

Re: [PoC] Non-volatile WAL buffer

2020-11-24 Thread Ashwin Agrawal
On Sun, Nov 22, 2020 at 5:23 PM Tomas Vondra wrote: > I'm not entirely sure whether the "pmemdax" (i.e. unpatched instance > with WAL on PMEM DAX device) is actually safe, but I included it anyway > to see what difference is. I am curious to learn more on this aspect. Kernels have provided supp

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Andy Fan
On Wed, Nov 25, 2020 at 8:10 AM Andy Fan wrote: > > > On Tue, Nov 24, 2020 at 11:11 PM Alvaro Herrera > wrote: > >> On 2020-Nov-24, Andy Fan wrote: >> >> > then we modified the copy/read/out functions for this node. In >> > _readFuncExpr, >> > we probably add something like >> >> > [ ... ] >> >

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Tom Lane
Alvaro Herrera writes: >> On Wed, 25 Nov 2020 at 04:55, Tom Lane wrote: >>> walleye's been failing since this patchset went in: >>> I have no idea what to make of that, but it looks more like a compiler bug >>> than anything else. > Apparently the bug was fixed days after it was reported, > http

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread David Rowley
On Wed, 25 Nov 2020 at 14:28, Tom Lane wrote: > So maybe, rather than hacking up the attribute stuff for > a bug that might bite us again anyway in future, we ought > to press walleye's owner to install a more recent compiler. I think that seems like a better idea. I had thoughts about installin

Re: About adding a new filed to a struct in primnodes.h

2020-11-24 Thread Tom Lane
Andy Fan writes: > What I mean here is something like below. What exactly would be the value of that? There is work afoot, or at least on people's to-do lists, to mechanize creation of the outfuncs/readfuncs/etc code directly from the Node struct declarations. So special cases for particular fi

Re: [PoC] Non-volatile WAL buffer

2020-11-24 Thread Tomas Vondra
On 11/25/20 1:27 AM, tsunakawa.ta...@fujitsu.com wrote: > From: Tomas Vondra >> It's interesting that they only place the tail of the log on PMEM, >> i.e. the PMEM buffer has limited size, and the rest of the log is >> not on PMEM. It's a bit as if we inserted a PMEM buffer between our >> wal buff

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread David Rowley
On Wed, 25 Nov 2020 at 14:35, David Rowley wrote: > > On Wed, 25 Nov 2020 at 14:28, Tom Lane wrote: > > So maybe, rather than hacking up the attribute stuff for > > a bug that might bite us again anyway in future, we ought > > to press walleye's owner to install a more recent compiler. > > I thin

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Tom Lane
David Rowley writes: > On Wed, 25 Nov 2020 at 14:28, Tom Lane wrote: >> So maybe, rather than hacking up the attribute stuff for >> a bug that might bite us again anyway in future, we ought >> to press walleye's owner to install a more recent compiler. > I think that seems like a better idea. I

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2020-11-24 Thread Craig Ringer
On Wed, Nov 25, 2020 at 2:43 AM Alexey Kondratov wrote: > On 2020-11-24 06:52, Bharath Rupireddy wrote: > > Thanks for the review comments. > > > > On Mon, Nov 23, 2020 at 9:57 PM Alexey Kondratov > > wrote: > >> > >> > v1-0001-postgres_fdw-function-to-discard-cached-connections.patch > >> > >>

Re: Keep elog(ERROR) and ereport(ERROR) calls in the cold path

2020-11-24 Thread Tom Lane
David Rowley writes: > So, how about the attached today and I'll email Joseph about walleye > and see if he can upgrade to a newer minGW version. WFM. (Note I already cc'd Joseph on this thread.) regards, tom lane

  1   2   >