Re: refactoring - share str2*int64 functions

On Wed, Sep 04, 2019 at 12:49:17PM +0200, Fabien COELHO wrote: > From a compiler perspective, the (un)likely tip is potentially useful on any > test. We know when parsing a that it is very unlikely that the string > conversion would fail, so we can tell that, so that the compiler knows which > bran

enhance SPI to support EXECUTE commands

Dear hackers, I found that such a statement would get 0 in PL/pgSQL. PREPARE smt_del(int) AS DELETE FROM t1; EXECUTE 'EXECUTE smt_del(100)'; GET DIAGNOSTICS j = ROW_COUNT; In fact, this is a problem with SPI, it does not support getting result of the EXECUTE command. I made a little enhancemen

Re: using explicit_bzero

On 2019-09-05 04:12, Michael Paquier wrote: > On Wed, Sep 04, 2019 at 04:38:21PM -0400, Alvaro Herrera wrote: >> Marked RfC. Can we get on with this? > > FWIW, I have been able to test this one on Windows with MSVC and > things are handled correctly. committed -- Peter Eisentraut

Re: [HACKERS] WIP: Aggregation push-down

Alvaro Herrera wrote: > This stuff seems very useful. How come it sits unreviewed for so long? I think the review is hard for people who are not interested in the planner very much. And as for further development, there are a few design decisions that can hardly be resolved without Tom Lane's c

Re: Locking a row with KEY SHARE NOWAIT blocks

On Tue, Sep 3, 2019 at 6:58 PM Heikki Linnakangas wrote: > > When you lock a row with FOR KEY SHARE, and the row's non-key columns > have been updated, heap_lock_tuple() walks the update chain to mark all > the in-progress tuple versions also as locked. But it doesn't pay > attention to the NOWAIT

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

On Wed, Sep 4, 2019 at 2:40 AM Alvaro Herrera wrote: > Fujita-san, amul, > > CFbot complains that Fujita-san submitted a patch that doesn't apply, > which makes sense since the necessary previous patch was only referred > to without being resubmitted. I suggest to always post all patches > toget

Re: ERROR: multixact X from before cutoff Y found to be still running

On Thu, Sep 5, 2019 at 1:01 PM Jeremy Schneider wrote: > On 9/4/19 17:37, Nathan Bossart wrote: > Currently, if you hold a multixact open long enough to generate an > "oldest multixact is far in the past" message during VACUUM, you may > see the following ERROR: > > WARNING: oldest multix

Re: Add "password_protocol" connection parameter to libpq

On Wed, 2019-08-21 at 16:12 +0900, Michael Paquier wrote: > This counts for other auth requests as well like krb5, no? I think > that we should also add the "disable" flavor for symmetry, and > also... Added "disable" option, and I refactored so that it would look explicitly for an expected auth

Re: block-level incremental backup

On Wed, Sep 4, 2019 at 10:08 PM Michael Paquier wrote: > > For generating a > > file, you can always emit the newest and "best" tar format, but for > > reading a file, you probably want to be prepared for older or cruftier > > variants. Maybe not -- I'm not super-familiar with the tar on-disk > >

Re: refactoring - share str2*int64 functions

On Wed, Sep 04, 2019 at 02:08:39AM -0700, Andres Freund wrote: >> +static bool >> +str2int64(const char *str, int64 *val) >> +{ >> +pg_strtoint_status stat = pg_strtoint64(str, val); >> + > > I find it weird to have a wrapper that's named 'str2...' that then calls > 'strto' to imp

Re: pg_promote() can cause busy loop

On Thu, Sep 5, 2019 at 10:26 AM Michael Paquier wrote: > > On Thu, Sep 05, 2019 at 09:46:26AM +0900, Fujii Masao wrote: > > I found small issue in pg_promote(). If postmaster dies > > while pg_promote() is waiting for the standby promotion to finish, > > pg_promote() can cause busy loop. This happ

Re: using explicit_bzero

On Wed, Sep 04, 2019 at 04:38:21PM -0400, Alvaro Herrera wrote: > Marked RfC. Can we get on with this? FWIW, I have been able to test this one on Windows with MSVC and things are handled correctly. -- Michael signature.asc Description: PGP signature

Re: pg_promote() can cause busy loop

On Thu, Sep 05, 2019 at 10:53:19AM +0900, Fujii Masao wrote: > It's ok to use PG_RETURN_BOOL(false) instead of breaking out of the loop > in that case. Which would make the code simpler. Okay. I would have done so FWIW. > But I don't think it's worth warning about postmaster death here > because

Re: block-level incremental backup

On Tue, Sep 03, 2019 at 08:59:53AM -0400, Robert Haas wrote: > I think pg_basebackup is using homebrew code to generate tar files, > but I'm reluctant to do that for reading tar files. Yes. This code has not actually changed since its introduction. Please note that we also have code which reads d

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

On Wed, Sep 04, 2019 at 04:50:45PM -0400, Alvaro Herrera wrote: > According to CFbot, the Windows build fails with this patch. Please > fix. To save a couple of clicks: "C:\projects\postgresql\pageinspect.vcxproj" (default target) (56) -> (Link target) -> heapfuncs.obj : error LNK2001: unresolv

Re: pg_promote() can cause busy loop

On Thu, Sep 05, 2019 at 09:46:26AM +0900, Fujii Masao wrote: > I found small issue in pg_promote(). If postmaster dies > while pg_promote() is waiting for the standby promotion to finish, > pg_promote() can cause busy loop. This happens because > pg_promote() does nothing when WaitLatch() detects >

Re: Tid scan improvements

On Wed, 4 Sep 2019 at 10:34, Alvaro Herrera wrote: > > On 2019-Aug-01, Edmund Horner wrote: > > > Hi everyone, > > > > Sadly it is the end of the CF and I have not had much time to work on > > this. I'll probably get some time in the next month to look at the > > heapam changes. > > > > Should we

Re: [HACKERS] CLUSTER command progress monitor

On Wed, Sep 04, 2019 at 09:18:39AM -0400, Robert Haas wrote: > I think this is all going in the wrong direction. It's the > responsibility of the people who are calling the pgstat_progress_* > functions to only do so when it's appropriate. Having the > pgstat_progress_* functions try to untangle

pg_promote() can cause busy loop

Hi, I found small issue in pg_promote(). If postmaster dies while pg_promote() is waiting for the standby promotion to finish, pg_promote() can cause busy loop. This happens because pg_promote() does nothing when WaitLatch() detects the postmaster death event. I think that pg_promote() should bail

Re: Unexpected "shared memory block is still in use"

I wrote: > This still isn't committable as-is, since the test will just curl up > and die on machines lacking IPC::SharedMem. After a bit of research, here's a version that takes a stab at fixing that. There may be cleaner ways to do it, but this successfully skips the test if it can't import the

Re: Plug-in common/logging.h with vacuumlo and oid2name

On Wed, Sep 04, 2019 at 02:37:12PM +0200, Peter Eisentraut wrote: >>> Do we need set_pglocale_pgservice() calls here if we're not doing NLS? >>> Does the logging stuff require it? I'm not sure. >> >> The logging part does not require it, but this can be used for >> PGSYSCONFDIR, no? > > How does

Re: Client Certificate Authentication Using Custom Fields (i.e. other than CN)

Hi David, Glad you are open to the idea! My proposal would be an additional authentication setting for certauth (alongside the current map option) which lets you specify which subject field to match on. I'll take a look at what the patch would look like, but this is incredibly tangential to what

Re: unexpected rowlock mode when trigger is on the table

On 2019-Sep-03, Tomáš Záluský wrote: > postgres=# begin; > BEGIN > postgres=# update master set detail_id=null, name='y' where id=1000; > UPDATE 1 > > In another psql console, I run: > > postgres=# select * from pgrowlocks('master'); > locked_row | locker | multi | xids | modes | pids > ---

Re: Unexpected "shared memory block is still in use"

I wrote: > Attached is a draft patch to change both shmem and sema key selection > to be based on data directory inode rather than port. > ... > I'm not quite sure what's going on in src/test/recovery/t/017_shm.pl. > As expected, the test for port number non-collision no longer sees > a failure. A

Re: BUG #15858: could not stat file - over 4GB

Thanks for looking into this. On Fri, Aug 23, 2019 at 11:49 PM Tom Lane wrote: > > Directly editing the configure script is Not Done ... or at least, > such changes wouldn't survive the next correctly-done configure > update. You have to edit configure.in (or one of the sub-files in > config/) a

Re: Index Skip Scan

Surely it isn't right to add members prefixed with "ioss_" to struct IndexScanState. I'm surprised about this "FirstTupleEmitted" business. Wouldn't it make more sense to implement index_skip() to return the first tuple if the scan is just starting? (I know little about executor, apologies if th

Re: progress report for ANALYZE

There were some minor problems in v5 -- bogus Docbook as well as outdated rules.out, small "git diff --check" complaint about whitespace. This v6 (on today's master) fixes those, no other changes. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

On 2019-Sep-04, Alvaro Herrera wrote: > Attached v3 again, for CFbot's benefit. No changes from last time. According to CFbot, the Windows build fails with this patch. Please fix. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, T

Re: Client Certificate Authentication Using Custom Fields (i.e. other than CN)

On Wed, Sep 04, 2019 at 05:24:15PM +0100, George Hafiz wrote: > Hello, > > It is currently only possible to authenticate clients using certificates > with the CN. > > I would like to propose that the field used to identify the client is > configurable, e.g. being able to specify DN as the appropr

Re: using explicit_bzero

On 2019-Aug-24, Peter Eisentraut wrote: > On 2019-08-14 05:00, Michael Paquier wrote: > > On Tue, Aug 13, 2019 at 10:30:39AM +0200, Peter Eisentraut wrote: > >> Another patch, to attempt to fix the Windows build. > > > > I have not been able to test the compilation, but the changes look > > good

Re: [HACKERS] WIP: Aggregation push-down

This stuff seems very useful. How come it sits unreviewed for so long? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: pg_basebackup ignores the existing data directory permissions

On 2019-Apr-03, Robert Haas wrote: > I am not sure what solution is best here, but it is hard to imagine > that the status quo is the right thing. This patch has been dormant for months. There's been at lot of discussion but it doesn't seem conclusive; it doesn't look like we know what we actual

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

On Wed, Sep 04, 2019 at 11:37:48AM +0200, Rafia Sabih wrote: On Tue, 30 Jul 2019 at 02:17, Tomas Vondra wrote: On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote: > > ... > >I wonder if we're approaching this wrong. Maybe we should not reverse >engineer queries for the various places

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

Attached v3 again, for CFbot's benefit. No changes from last time. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From 018077d786f874cb314b5f61b5ef85f42c62bbe5 Mon Sep 17 00:00:00 2001 From: Masahiko Sawada Da

Re: [Patch] Invalid permission check in pg_stats for functional indexes

On 2019-Sep-03, Pierre Ducroquet wrote: > > IIUC, the patch introduces an additional privilege check for the > > underlying objects involved in the expression/functional index. If the > > user has 'select' privileges on all of the columns/objects included in > > the expression/functional index, th

Re: [Patch] Invalid permission check in pg_stats for functional indexes

BTW you labelled this in the CF app as targetting "stable", but I don't think this is backpatchable. I think we should fix it in master and call it a day. Changing system view definitions in stable versions is tough. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Devel

Client Certificate Authentication Using Custom Fields (i.e. other than CN)

Hello, It is currently only possible to authenticate clients using certificates with the CN. I would like to propose that the field used to identify the client is configurable, e.g. being able to specify DN as the appropriate field. The reason being is that in some organisations, where you might

Re: Planning counters in pg_stat_statements (using pgss_store)

Hello I think the most important question for this topic is performance penalty. It was a long story, first test on my desktop was too volatile. I setup separate PC with DB only and test few cases. PC spec: 2-core Intel Core 2 Duo E6550, 4GB ram, mechanical HDD All tests on top 7dedfd22b79822b7f

Re: [PATCH] Connection time for \conninfo

On 2019-Sep-04, Michael Paquier wrote: > On Tue, Sep 03, 2019 at 10:13:57PM -0400, Rodrigo Ramírez Norambuena wrote: > > I've work in the a little patch to add into the \conninfo of psql > > command the connection time against a server backend > > > > Maybe could add after, the precheck to if th

having issues with PG12 debian packaging repository

Hello hackers I am getting sporadic errors when I tried to use PG12 bionic debian repository. Here is the error message that is result of apt-get update. --- Failed to fetch http://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/main/binary-i386/Packages.gz File has unexpected size (26

Re: Default JIT setting in V12

Hi, On 2019-09-04 07:51:16 -0700, Andres Freund wrote: > On 2019-09-04 09:56:28 -0400, Jeff Janes wrote: > > I think it is intuitive, and with empirical evidence, that we do not want > > to JIT compile at all unless we are going to optimize the compiled code. > > There's pretty clear counter-evid

auxiliary processes in pg_stat_ssl

I just noticed that we list auxiliary processes in pg_stat_ssl: 55432 13devel 28627=# select * from pg_stat_ssl ; pid │ ssl │ version │ cipher │ bits │ compression │ client_dn │ client_serial │ issuer_dn ───┼─┼─┼┼──┼─┼───

Re: Unexpected "shared memory block is still in use"

Peter Eisentraut writes: > I agree with this patch and the reasons for it. OK, thanks for reviewing. > A related point, perhaps we should change the key printed into > postmaster.pid to be in hexadecimal format ("0x08x") so that it matches > what ipcs prints. Hmm, that depends on whose ipcs you

Re: Default JIT setting in V12

Hi, On 2019-09-04 09:56:28 -0400, Jeff Janes wrote: > I think it is intuitive, and with empirical evidence, that we do not want > to JIT compile at all unless we are going to optimize the compiled code. There's pretty clear counter-evidence however as well :( I think it's probably more sensible

Re: BUG #15977: Inconsistent behavior in chained transactions

I made another patch for suggested behavior (COMMIT/ROLLBACK AND CHAIN now gives us an error when used in an implicit block). 2019年9月4日(水) 16:53 Andres Freund : > Hi, > > On 2019-09-03 11:54:57 +0200, Peter Eisentraut wrote: > > On 2019-08-29 16:58, Fabien COELHO wrote: > > > > > >> Thanks, I got

Re: block-level incremental backup

On Tue, Sep 3, 2019 at 12:46 PM Ibrar Ahmed wrote: > I did that and have experience working on the TAR format. I was curious > about what > "best/newest" you are talking. Well, why not go look it up? On my MacBook, tar is documented to understand three different tar formats: gnutar, ustar, and

Default JIT setting in V12

Since JIT is on by default in v12, I wanted to revisit the issue raised in https://www.postgresql.org/message-id/CAMkU=1zVhQ5k5d=YyHNyrigLUNTkOj4=YB17s9--3ts8H-SO=q...@mail.gmail.com When the total estimated cost is between jit_above_cost and jit_optimize_above_cost, I get a substantial regression

Re: Replication & recovery_min_apply_delay

On 04.09.2019 1:22, Alvaro Herrera wrote: On 2019-Aug-02, Michael Paquier wrote: On Wed, Jul 31, 2019 at 04:43:26PM -0400, Alvaro Herrera wrote: As for the test module, the one I submitted takes a lot of time to run (well, 60s) and I don't think it's a good idea to include it as something t

Re: [HACKERS] CLUSTER command progress monitor

On Tue, Sep 3, 2019 at 1:02 AM Masahiko Sawada wrote: > After more thought, even if we don't start a new command progress when > there is another one already started the progress update functions > could be called and these functions don't specify the command type. > Therefore, the progress inform

Re: row filtering for logical replication

Em ter, 3 de set de 2019 às 00:32, Euler Taveira escreveu: > > Ops... exact. That was an oversight while poking with different types of > slots. > Here is a rebased version including this small fix. -- Euler Taveira Timbira - http://www.timbira.com.br/ P

Re: pglz performance

> 4 сент. 2019 г., в 17:40, Peter Eisentraut > написал(а): > > On 2019-09-04 11:22, Andrey Borodin wrote: >>> What about the two patches? Which one is better? >> On our observations pglz_decompress_hacked.patch is best for most of tested >> platforms. >> Difference is that pglz_decompress_h

Re: pglz performance

On 2019-09-04 11:22, Andrey Borodin wrote: >> What about the two patches? Which one is better? > On our observations pglz_decompress_hacked.patch is best for most of tested > platforms. > Difference is that pglz_decompress_hacked8.patch will not appply optimization > if decompressed match is not

Re: Plug-in common/logging.h with vacuumlo and oid2name

On 2019-09-04 14:17, Michael Paquier wrote: >> progname can probably be made into a local variable now. > > Can it? vacuumlo() still uses the progname from _param for the > connection string. Yeah, probably best to leave it as is for now. >> Do we need set_pglocale_pgservice() calls here if we'

Re: Plug-in common/logging.h with vacuumlo and oid2name

On Wed, Sep 04, 2019 at 10:17:57AM +0200, Peter Eisentraut wrote: > On 2019-08-20 03:28, Michael Paquier wrote: > > + pg_log_error("\nfailed to remove lo %u: > > %s", lo, > > + > > PQerrorMessage(conn));

Re: [HACKERS] CLUSTER command progress monitor

On Wed, Sep 4, 2019 at 3:48 PM Michael Paquier wrote: > > On Tue, Sep 03, 2019 at 02:52:28PM +0900, Michael Paquier wrote: > > Indeed, good catch. This is wrong since b6fb647 which has introduced > > the progress reports. I'll fix that one and back-patch if there are > > no objections. > > OK, a

Re: REINDEX filtering in the backend

On 2019-08-30 02:10, Michael Paquier wrote: > On Thu, Aug 29, 2019 at 10:52:55AM +0200, Julien Rouhaud wrote: >> That was already suggested by Thomas and seconded by Peter E., see >> https://www.postgresql.org/message-id/2b1504ac-3d6c-11ec-e1ce-3daf132b3d37%402ndquadrant.com. >> >> I personally thi

Re: block-level incremental backup

On Tue, Sep 3, 2019 at 12:11 PM Dilip Kumar wrote: > > On Fri, Aug 16, 2019 at 3:54 PM Jeevan Chalke > wrote: > > > 0003: > +/* > + * When to send the whole file, % blocks modified (90%) > + */ > +#define WHOLE_FILE_THRESHOLD 0.9 > > How this threshold is selected. Is it by some test? > > > - ma

Re: Unexpected "shared memory block is still in use"

I agree with this patch and the reasons for it. A related point, perhaps we should change the key printed into postmaster.pid to be in hexadecimal format ("0x08x") so that it matches what ipcs prints. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Suppo

Re: fix for BUG #3720: wrong results at using ltree

On Tue, Jul 16, 2019 at 8:52 PM Nikita Glukhov wrote: > > On 09.07.2019 17:57, Oleg Bartunov wrote: > > On Mon, Jul 8, 2019 at 7:22 AM Thomas Munro > wrote: > > On Sun, Apr 7, 2019 at 3:46 AM Tom Lane > wrote: > > =?UTF-8?Q?Filip_Rembia=C5=82kowski?= > writes: > > Here is my attempt to f

Re: fix for BUG #3720: wrong results at using ltree

Please create separate commitfest entry.

Re: refactoring - share str2*int64 functions

Bonjour Michaël, Attached a rebased version which implements the int64/uint64 stuff. It is basically the previous patch without the overflow inlined functions. - if (!strtoint64(yytext, true, &yylval->ival)) + if (unlikely(pg_strtoint64(yytext, &yylval->ival) != PG_STRTOINT_OK)) It

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

On Tue, 30 Jul 2019 at 02:17, Tomas Vondra wrote: > On Sun, Jul 21, 2019 at 01:34:22PM +0200, Tomas Vondra wrote: > > > > ... > > > >I wonder if we're approaching this wrong. Maybe we should not reverse > >engineer queries for the various places, but just start with a set of > >queries that we wa

Re: Proposal: roll pg_stat_statements into core

Hi, On 2019-09-02 12:07:17 -0400, Tom Lane wrote: > Euler Taveira writes: > > At least if pg_stat_statements > > was in core you could load it by default and have a GUC to turn it > > on/off without restarting the server (that was Magnus proposal and > > Andres agreed). > > That assertion is 100

Re: pg_get_databasebyid(oid)

Hello Thank you for attention! I marked CF entry as returned with feedback. regards, Sergei

Re: WIP: Data at rest encryption

On Wed, 04 Sep 2019 00:56:15 +0800 Alvaro Herrera wrote On 2019-Aug-02, Shawn Wang wrote: > Hi Antonin, > It is very glad to see the new patch. I used the public patches a long time > ago. > I did some tests like the stream replication, much

Re: pglz performance

Hi, Peter! Thanks for looking into this. > 4 сент. 2019 г., в 14:09, Peter Eisentraut > написал(а): > > On 2019-06-24 10:44, Andrey Borodin wrote: >>> 18 мая 2019 г., в 11:44, Andrey Borodin написал(а): >>> >> Hi! >> Here's rebased version of patches. >> >> Best regards, Andrey Borodin. >

Re: pglz performance

On 2019-06-24 10:44, Andrey Borodin wrote: >> 18 мая 2019 г., в 11:44, Andrey Borodin написал(а): >> > Hi! > Here's rebased version of patches. > > Best regards, Andrey Borodin. I think this is the most recent patch for the CF entry . What about the

Re: refactoring - share str2*int64 functions

Hi, On 2019-09-03 20:10:37 +0200, Fabien COELHO wrote: > @@ -113,7 +113,7 @@ parse_output_parameters(List *options, uint32 > *protocol_version, >errmsg("conflicting or > redundant options"))); > protocol_version_given = true;

Re: d25ea01275 and partitionwise join

Hi Amit, On Wed, Sep 4, 2019 at 3:30 PM Richard Guo wrote: > Hi Amit, > > On Wed, Sep 4, 2019 at 10:01 AM Amit Langote > wrote: > >> Fujita-san, >> >> To avoid losing track of this, I've added this to November CF. >> >> https://commitfest.postgresql.org/25/2278/ >> >> I know there is one more p

Re: Plug-in common/logging.h with vacuumlo and oid2name

On 2019-08-20 03:28, Michael Paquier wrote: > + pg_log_error("\nfailed to remove lo %u: > %s", lo, > + > PQerrorMessage(conn)); This newline should be removed. progname can probably be made into a l

Re: refactoring - share str2*int64 functions

On Tue, Sep 03, 2019 at 08:10:37PM +0200, Fabien COELHO wrote: > Attached a rebased version which implements the int64/uint64 stuff. It is > basically the previous patch without the overflow inlined functions. - if (!strtoint64(yytext, true, &yylval->ival)) + if (unlikely(pg_strtoint64(yyt

Re: BUG #15977: Inconsistent behavior in chained transactions

Hi, On 2019-09-03 11:54:57 +0200, Peter Eisentraut wrote: > On 2019-08-29 16:58, Fabien COELHO wrote: > > > >> Thanks, I got it. I have never made a patch before so I'll keep it in my > >> mind. Self-contained patch is now attached. > > > > v3 applies, compiles, "make check" ok. > > > > I turn

Re: d25ea01275 and partitionwise join

Hi Amit, On Wed, Sep 4, 2019 at 10:01 AM Amit Langote wrote: > Fujita-san, > > To avoid losing track of this, I've added this to November CF. > > https://commitfest.postgresql.org/25/2278/ > > I know there is one more patch beside the partitionwise join fix, but > I've set the title to suggest t