Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2021-12-18 Thread Damir Belyalov
Hello. Wrote a patch implementing COPY with ignoring errors in rows using block subtransactions. Syntax: COPY [table] FROM [file/stdin] WITH IGNORE_ERROS; Examples: CREATE TABLE check_ign_err (n int, m int, k int); COPY check_ign_err FROM STDIN WITH IGNORE_ERRORS; 1 1 1 2 2 2 2 3 3 3 \. WARNING:

Re: Addition of --no-sync to pg_upgrade for test speedup

2021-12-18 Thread Michael Paquier
On Fri, Dec 17, 2021 at 09:47:05AM -0500, Bruce Momjian wrote: > On Fri, Dec 17, 2021 at 10:21:04AM +0100, Peter Eisentraut wrote: >> I think that is reasonable. Thanks. I have applied that, as that really helped here. >> Maybe we could have some global option, like some environment variable, th

Refactoring of compression options in pg_basebackup

2021-12-18 Thread Michael Paquier
Hi all, (Added Georgios in CC.) When working on the support of LZ4 for pg_receivewal, walmethods.c has gained an extra parameter for the compression method. It gets used on the DirectoryMethod instead of the compression level to decide which type of compression is used. One thing that I left out

Re: Adding CI to our tree

2021-12-18 Thread Andrew Dunstan
On 12/17/21 14:34, Andres Freund wrote: > Hi, > > On 2021-12-17 09:36:05 -0500, Tom Lane wrote: >> Andrew Dunstan writes: >>> Maye I have missed it, but why are we using ccache here? That seems a >>> bit pointless in an ephemeral instance. >> I believe Munro's cfbot tooling is able to save and r

Re: Column Filtering in Logical Replication

2021-12-18 Thread Alvaro Herrera
On 2021-Dec-18, Tomas Vondra wrote: > On 12/18/21 02:34, Alvaro Herrera wrote: > > On 2021-Dec-17, Tomas Vondra wrote: > > > > If the server has a *separate* security mechanism to hide the > > > > columns (per-column privs), it is that feature that will protect > > > > the data, not the logical-r

Re: sequences vs. synchronous replication

2021-12-18 Thread Tomas Vondra
On 12/18/21 07:00, Tomas Vondra wrote: On 12/18/21 05:52, Tom Lane wrote: Tomas Vondra writes: The problem is exactly the same as in [1] - the aborted transaction generated WAL, but RecordTransactionAbort() ignores that and does not update LogwrtResult.Write, with the reasoning that aborte

Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-18 Thread Chapman Flack
Hi, I sometimes do some testing as nobody, on a distro where getpwent(nobody)->pw_dir is a directory that nobody can't write. So I end up setting $HOME to a directory that, um, is writable. When I start psql, strace shows $HOME being honored when looking for .terminfo and .inputrc, and getpwent()

Re: Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-18 Thread Chapman Flack
On 12/18/21 15:57, Chapman Flack wrote: > I see that I can set > a HISTFILE variable (or set PSQL_HISTORY in the environment), > and can set PSQLRC in the environment (but not as a variable), > and nothing can set the .pgpass location well, not in the psql docs, but in the environment variable sec

Re: Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-18 Thread David G. Johnston
On Sat, Dec 18, 2021 at 2:07 PM Chapman Flack wrote: > On 12/18/21 15:57, Chapman Flack wrote: > > I see that I can set > > a HISTFILE variable (or set PSQL_HISTORY in the environment), > > and can set PSQLRC in the environment (but not as a variable), > > and nothing can set the .pgpass location

Re: Is my home $HOME or is it getpwent()->pw_dir ?

2021-12-18 Thread Chapman Flack
On 12/18/21 16:16, David G. Johnston wrote: > psql docs saith: > > "This utility, like most other PostgreSQL utilities, also uses the > environment variables supported by libpq (see Section 34.15)." I'm sure that's adequate as far as that goes. I just happened to miss it when composing the longer

Re: sequences vs. synchronous replication

2021-12-18 Thread Tom Lane
Tomas Vondra writes: > Here's a PoC demonstrating this idea. I'm not convinced it's the right > way to deal with this - it surely seems more like a duct tape fix than a > clean solution. But it does the trick. I was imagining something a whole lot simpler, like "don't try to cache unused sequen

Re: sequences vs. synchronous replication

2021-12-18 Thread Tomas Vondra
On 12/18/21 22:27, Tom Lane wrote: Tomas Vondra writes: Here's a PoC demonstrating this idea. I'm not convinced it's the right way to deal with this - it surely seems more like a duct tape fix than a clean solution. But it does the trick. I was imagining something a whole lot simpler, like

Re: sequences vs. synchronous replication

2021-12-18 Thread Tom Lane
Tomas Vondra writes: > What do you mean by "not caching unused sequence numbers"? Reducing > SEQ_LOG_VALS to 1, i.e. WAL-logging every sequence increment? Right. > That'd work, but I wonder how significant the impact will be. As I said, we've accepted worse in order to have stable replication

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

2021-12-18 Thread Justin Pryzby
Remove bogus ACL check for AMs. Rebased on cf0cab868. Use ForkNumber rather than int. Update comments and commit message. Also move the Size column of \l and \dt >From a1d33780e4c61a7bdb64ee6ab5c248e1175a3efd Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 13 Jul 2021 21:25:48 -0500 Subjec

Getting rid of regression test input/ and output/ files

2021-12-18 Thread Tom Lane
I did some desultory investigation of the idea proposed at [1] that we should refactor the regression test scripts to try to reduce their interdependencies. I soon realized that one of the stumbling blocks to this is that we've tended to concentrate data-loading COPY commands, as well as C functio

Re: should we document an example to set multiple libraries in shared_preload_libraries?

2021-12-18 Thread Maciek Sakrejda
On Thu, Dec 9, 2021 at 7:42 AM Bharath Rupireddy wrote: > How about ALTER SYSTEM SET shared_preload_libraries command itself > checking for availability of the specified libraries (after fetching > library names from the parsed string value) with stat() and then > report an error if any of the lib

Re: [EXTERNAL] Re: should we document an example to set multiple libraries in shared_preload_libraries?

2021-12-18 Thread Maciek Sakrejda
On Fri, Dec 10, 2021 at 10:10 AM Godfrin, Philippe E wrote: > I may have missed something in this stream, but is this a system controlled > by Patroni? In my case, no: it's a pretty vanilla PGDG install on Ubuntu 20.04. Thanks for the context, though. Thanks, Maciek

Re: sequences vs. synchronous replication

2021-12-18 Thread Amit Kapila
On Sat, Dec 18, 2021 at 7:24 AM Tomas Vondra wrote: > > while working on logical decoding of sequences, I ran into an issue with > nextval() in a transaction that rolls back, described in [1]. But after > thinking about it a bit more (and chatting with Petr Jelinek), I think > this issue affects p

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2021-12-18 Thread Pavel Stehule
Hi so 18. 12. 2021 v 9:55 odesílatel Damir Belyalov napsal: > Hello. > > Wrote a patch implementing COPY with ignoring errors in rows using block > subtransactions. > It is great so you are working on this patch. Unfortunately, I am afraid this simple design is not optimal. Using subtransaction

Re: Schema variables - new implementation for Postgres 15

2021-12-18 Thread Erik Rijkers
Op 19-12-2021 om 07:23 schreef Pavel Stehule: I am sending new versions of patches. I hope I solved all Tomas's objections. 1. The schema variables were renamed to session variables 2. I fixed issues related to creating, dropping variables under subtransactions + regress tests 3. I fixed is

Re: Schema variables - new implementation for Postgres 15

2021-12-18 Thread Pavel Stehule
ne 19. 12. 2021 v 8:09 odesílatel Erik Rijkers napsal: > Op 19-12-2021 om 07:23 schreef Pavel Stehule: > > > > > I am sending new versions of patches. > > > > I hope I solved all Tomas's objections. > > > > 1. The schema variables were renamed to session variables > > 2. I fixed issues related to

Re: Schema variables - new implementation for Postgres 15

2021-12-18 Thread Pavel Stehule
ne 19. 12. 2021 v 8:13 odesílatel Pavel Stehule napsal: > > > ne 19. 12. 2021 v 8:09 odesílatel Erik Rijkers napsal: > >> Op 19-12-2021 om 07:23 schreef Pavel Stehule: >> >> > >> > I am sending new versions of patches. >> > >> > I hope I solved all Tomas's objections. >> > >> > 1. The schema var

Re: Schema variables - new implementation for Postgres 15

2021-12-18 Thread Erikjan Rijkers
Op 19-12-2021 om 08:13 schreef Pavel Stehule: ne 19. 12. 2021 v 8:09 odesílatel Erik Rijkers >  > [0001-schema-variables-20211219.patch]  > [0002-schema-variables-20211219.patch] Hi Pavel, I get an error during test 'session_variables'. (on the upside, my own lit