Re: pgbench rate limiting changes transaction latency computation

2019-06-12 Thread Heikki Linnakangas
On 12/06/2019 09:23, Fabien COELHO wrote: But that was just because I was showing a simplistic example. E.g. here's a log of a vacuum finishing, and then another starting a few seconds later (both vacuums lasting a fair while): progress: 139.0 s, 2438.4 tps, txtime 13.033 ms stddev 3.830, lat 17

Re: Adaptive query optimization

2019-06-12 Thread legrand legrand
>>I tried to create much simpler version of AQO based on auto_explain >>extension. >>This extension provide all necessary infrastructure to analyze >>statements with long execution time. >>I have added two new modes to auto_explain: >>1. Auto generation of multicolumn statistics for variables usi

Re: [PATCH] Speedup truncates of relation forks

2019-06-12 Thread Masahiko Sawada
On Wed, Jun 12, 2019 at 12:25 PM Tsunakawa, Takayuki wrote: > > From: Tomas Vondra [mailto:tomas.von...@2ndquadrant.com] > > Years ago I've implemented an optimization for many DROP TABLE commands > > in a single transaction - instead of scanning buffers for each relation, > > the code now accumul

Re: BEFORE UPDATE trigger on postgres_fdw table not work

2019-06-12 Thread Etsuro Fujita
Amit-san, On Wed, Jun 12, 2019 at 3:33 PM Amit Langote wrote: > On Wed, Jun 12, 2019 at 3:14 PM Etsuro Fujita wrote: > > * Reworded the comments a bit in postgresPlanFoereignModify the > > original patch modified > > + * statement, and for UPDATE if BEFORE ROW UPDATE triggers since those > +

Re: fix psql \conninfo & \connect when using hostaddr

2019-06-12 Thread Noah Misch
On Tue, Jun 11, 2019 at 01:59:20PM +0200, Dmitry Dolgov wrote: > > On Tue, Jun 11, 2019 at 6:46 AM Fabien COELHO wrote: > > > Although I guess it can be avoided by `-reuse-previous=off`, probably it > > > makese sense to update the docs. > > > > Yep, that is one option. The other is to revert or

Re: fix psql \conninfo & \connect when using hostaddr

2019-06-12 Thread Dmitry Dolgov
> On Wed, Jun 12, 2019 at 9:45 AM Noah Misch wrote: > > On Tue, Jun 11, 2019 at 01:59:20PM +0200, Dmitry Dolgov wrote: > > > On Tue, Jun 11, 2019 at 6:46 AM Fabien COELHO wrote: > > > > > Although I guess it can be avoided by `-reuse-previous=off`, probably it > > > > makese sense to update the d

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Siarhei Siniak
I've added debug prints to cube extension.g_custom_cube_a_f8 g_custom_cube_picksplit are the only called methods after that it prints import psycopg2 import logging import numpy import unittest import python.utils.logging import python.custom_db.backends import python.custom_db.backends.postgres

Re: openssl valgrind failures on skink are due to openssl issue

2019-06-12 Thread Michael Paquier
On Tue, Jun 11, 2019 at 02:07:29PM -0700, Andres Freund wrote: > On 2019-06-11 16:55:28 -0400, Tom Lane wrote: >> Andres Freund writes: >>> I can't think of a better way to fix skink for now than just disabling >>> openssl for skink, until 1.1.1d is released. Thanks for digging into the details o

set parameter for all existing session

2019-06-12 Thread alex lock
I check the “alter database, alter role " and "set " command, but none of them can set the parameters to all the existing sessions. do we have a way to do that? looks the "assign_hook" can be used to customize this, is it a right way to do that?

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Siarhei Siniak
I've added debug prints to cube extension. g_custom_cube_a_f8 and g_custom_cube_picksplit are the only called methods. After that it prints: ERROR:  failed to add item to index page in "lc_idx" Cube extension modifications:     #define MAX_DIM (512) Python test source code has been attached

Re: pg_basebackup failure after setting default_table_access_method option

2019-06-12 Thread Michael Paquier
On Mon, Jun 10, 2019 at 11:49:03PM -0700, Andres Freund wrote: > Well, all four. Given it's just copied code I don't see much code in > splitting the commit anymore. Thanks for pushing the fix, the result looks fine. > I noticed some other uglyness: check_timezone calls interval_in(), > without a

Re: fix psql \conninfo & \connect when using hostaddr

2019-06-12 Thread Fabien COELHO
Hello Noah, Although I guess it can be avoided by `-reuse-previous=off`, probably it makese sense to update the docs. Yep, that is one option. The other is to revert or alter the subtle change, but ISTM that it made sense in some use case, so I wanted some time to think about it and test.

Re: set parameter for all existing session

2019-06-12 Thread Pavel Stehule
Hi st 12. 6. 2019 v 9:58 odesílatel alex lock napsal: > I check the “alter database, alter role " and "set " command, but none of > them can set the parameters to all the existing sessions. do we have a > way to do that? looks the "assign_hook" can be used to customize this, is > it a right

RE: [PATCH] Speedup truncates of relation forks

2019-06-12 Thread Jamison, Kirk
On Tuesday, June 11, 2019 7:23 PM (GMT+9), Adrien Nayrat wrote: > > Attached is a patch to speed up the performance of truncates of relations. > > Thanks for working on this! Thank you also for taking a look at my thread. > > If you want to test with large number of relations, > > you may use

Re: set parameter for all existing session

2019-06-12 Thread alex lock
On Wed, Jun 12, 2019 at 4:25 PM Pavel Stehule wrote: > Hi > > st 12. 6. 2019 v 9:58 odesílatel alex lock napsal: > >> I check the “alter database, alter role " and "set " command, but none of >> them can set the parameters to all the existing sessions. do we have a >> way to do that? looks th

[PATCH] vacuumlo: print the number of large objects going to be removed

2019-06-12 Thread Timur Birsh
Hello, If tables has a lot of rows with large objects (>1_000_000) that removed throughout the day, it would be useful to know how many LOs going to be removed. First patch - print the number of large objects going to be removed, second patch - print how many LOs removed in percent. Can anyone p

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Andrey Borodin
Hi! > 9 июня 2019 г., в 23:05, Siarhei Siniak написал(а): > > I've been using cube extension recompiled with > #define MAX_DIM 256. > > But with a version 11.3 I'm getting the following error: > failed to add item to index page in So, you have changed source code (removing dim constraint) and

RE: BEFORE UPDATE trigger on postgres_fdw table not work

2019-06-12 Thread shohei.mochizuki
Fujita-san, > On Mon, Jun 10, 2019 at 9:04 PM Etsuro Fujita > wrote: > > I'll look into the patch more closely tomorrow. > > I did that, but couldn't find any issue about the patch. Here is an updated > version of the patch. Changes are: > > * Reworded the comments a bit in postgresPlanFoereig

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Siarhei Siniak
A uniform set of points with a dimension 128 and type cube. That has a size of 50 ** 3. Can be queried for a nearest neighbor at a speed of 10 queries per second with limit varying from 1 to 25. It works better than when no index used at all. So gist gives here a speed up. The documentation of po

Re: check_recovery_target_lsn() does a PG_CATCH without a throw

2019-06-12 Thread Peter Eisentraut
On 2019-06-11 08:11, Andres Freund wrote: > While working on fixing [1] I noticed that 2dedf4d9a899 "Integrate > recovery.conf into postgresql.conf" added two non-rethrowing PG_CATCH > uses. That's not OK. Right. Here is a patch that addresses this by copying the relevant code from pg_lsn_in() an

Re: Adaptive query optimization

2019-06-12 Thread Konstantin Knizhnik
On 12.06.2019 0:43, Tomas Vondra wrote: I don't think "learning phase" is an issue, in fact I think that's something we need to do - it ensures we have enough data to make good decisions. What is wrong with learning phase is that it requires some DBA assistance: somebody should determine wh

catalog files simplification

2019-06-12 Thread Peter Eisentraut
The current catalog files all do this: CATALOG(pg_aggregate,2600,AggregateRelationId) { ... } FormData_pg_aggregate; typedef FormData_pg_aggregate *Form_pg_aggregate; The bottom part of this seems redundant. With the attached patch, we can generate that automatically, so

Re: Minimal logical decoding on standbys

2019-06-12 Thread Amit Khandekar
On Tue, 11 Jun 2019 at 12:24, Amit Khandekar wrote: > > On Mon, 10 Jun 2019 at 10:37, Amit Khandekar wrote: > > > > On Tue, 4 Jun 2019 at 21:28, Andres Freund wrote: > > > > > > Hi, > > > > > > On 2019-06-04 15:51:01 +0530, Amit Khandekar wrote: > > > > After giving more thought on this, I think

Re: fix psql \conninfo & \connect when using hostaddr

2019-06-12 Thread Fabien COELHO
Hello, I got that. I'm working on it, and on the other issues you raised. The issue I see is what do we want when a name resolves to multiple addresses. The answer is not fully obvious to me right now. I'll try to send a patch over the week-end. At Alvaro's request, here is a quick WIP pat

Re: Adaptive query optimization

2019-06-12 Thread Kuntal Ghosh
Hello, On Wed, Jun 12, 2019 at 5:06 PM Konstantin Knizhnik wrote: > On 12.06.2019 0:43, Tomas Vondra wrote: > I don't think "learning phase" is an issue, in fact I think that's > something we need to do - it ensures we have enough data to make good > decisions. > > What is wrong with learning pha

Re: [pg_rewind] cp: cannot stat ‘pg_wal/RECOVERYHISTORY’: No such file or directory

2019-06-12 Thread tushar
On 06/10/2019 04:37 PM, Kuntal Ghosh wrote: When we define a restore command, we tell the server to copy a file a WAL file from the archive. So, it should be restore_command='cp tmp/archive_dir1/%f %p' This is the reason you're getting this following error. Ohh. Mea Culpa. Thanks for pointing

Re: tableam: abstracting relation sizing code

2019-06-12 Thread Robert Haas
On Tue, Jun 11, 2019 at 7:17 PM Andres Freund wrote: > Just to understand: What version are you targeting? It seems pretty > clearly v13 material to me? My current plan is to commit this to v13 as soon as the tree opens. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Po

Re: catalog files simplification

2019-06-12 Thread Robert Haas
On Wed, Jun 12, 2019 at 7:52 AM Peter Eisentraut wrote: > The current catalog files all do this: > > CATALOG(pg_aggregate,2600,AggregateRelationId) > { > ... > } FormData_pg_aggregate; > > typedef FormData_pg_aggregate *Form_pg_aggregate; > > The bottom part of this seems r

Re: catalog files simplification

2019-06-12 Thread Tom Lane
Robert Haas writes: > On Wed, Jun 12, 2019 at 7:52 AM Peter Eisentraut > wrote: >> The current catalog files all do this: >> >> CATALOG(pg_aggregate,2600,AggregateRelationId) >> { >> ... >> } FormData_pg_aggregate; >> >> typedef FormData_pg_aggregate *Form_pg_aggregate; >> >> The bottom part o

Are there still non-ELF BSD systems?

2019-06-12 Thread Peter Eisentraut
The shared library code has some support for non-ELF BSD systems. I suspect that this is long obsolete. Could we remove it? See attached. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From 3f21b2e13226dc1e0f5e

Re: catalog files simplification

2019-06-12 Thread Tom Lane
I wrote: > Robert Haas writes: >> Maybe the macro definition could be split across several lines instead >> of having one really long line? > I think that would complicate Catalog.pm; not clear if it's worth it. Oh, cancel that --- in an uncaffeinated moment, I thought you were asking about spli

Fix inconsistencies for v12 (pass 2)

2019-06-12 Thread Alexander Lakhin
Hello Amit, Can you also review the following fixes?: 2.1. bt_binsrch_insert -> _bt_binsrch_insert (an internal inconsistency) 2.2. EWOULDBOCK -> EWOULDBLOCK (a typo) 2.3. FORGET_RELATION_FSYNC & FORGET_DATABASE_FSYNC -> SYNC_FORGET_REQUEST (orphaned after 3eb77eba) 2.4. GetNewObjectIdWithIndex ->

Re: Binary support for pgoutput plugin

2019-06-12 Thread Dave Cramer
On Mon, 10 Jun 2019 at 07:49, Petr Jelinek wrote: > Hi, > > On 10/06/2019 13:27, Dave Cramer wrote: > > So back to binary output. > > > > From what I can tell the place to specify binary options would be in the > > create publication and or in replication slots? > > > > The challenge as I see it

Re: Are there still non-ELF BSD systems?

2019-06-12 Thread Tom Lane
Peter Eisentraut writes: > The shared library code has some support for non-ELF BSD systems. I > suspect that this is long obsolete. Could we remove it? See attached. I checked around a bit ... all of the *BSD systems in the buildfarm report ELF_SYS='true', so it's safe to say that the code yo

Re: upgrades in row-level locks can deadlock

2019-06-12 Thread Oleksii Kliukin
Oleksii Kliukin wrote: > Hi, > > Alvaro Herrera wrote: > >> On 2019-May-22, Oleksii Kliukin wrote: >> >>> X1: select id from job where name = 'a' for key share; >>> Y: select id from job where name = 'a' for update; -- starts waiting for X1 >>> X2: select id from job where name = 'a' for key

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Andrey Borodin
> 12 июня 2019 г., в 15:11, Siarhei Siniak написал(а): > > A uniform set of points with a dimension 128 and type cube. That has a size > of 50 ** 3. Can be queried for a nearest neighbor at a speed of 10 queries > per second with limit varying from 1 to 25. > It works better than when no ind

RE: proposal: pg_restore --convert-to-text

2019-06-12 Thread Daniel Verite
José Arthur Benetasso Villanova wrote: > On Thu, 28 Feb 2019, Imai, Yoshikazu wrote: > > > Is there no need to rewrite the Description in the Doc to state we should > > specify either -d or -f option? > > (and also it might be better to write if -l option is given, neither -d nor > > -f

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Siarhei Siniak
>ANN is beyond concepts of SQL standard: database with index must return same >results as without index. >I can add ANN to github.com/x4m/ags which is a fork of GiST.How to recompile >that extension and not to get a name conflict with a standard one? I've renamed everything for cube extension.

Re: upgrades in row-level locks can deadlock

2019-06-12 Thread Alvaro Herrera
On 2019-Jun-12, Oleksii Kliukin wrote: > Thank you! I can make it even simpler; s1 just acquires for share lock, s3 > gets for update one and s2 takes for share lock first, and then tries to > acquire for update one; once s1 finishes, s3 deadlocks. Cool. I think it would be worthwhile to includ

Re: proposal: pg_restore --convert-to-text

2019-06-12 Thread Andrew Gierth
> "Daniel" == Daniel Verite writes: Daniel> While testing pg_restore on v12, I'm stumbling on this too. Daniel> pg_restore without argument fails like that: Daniel> $ pg_restore Daniel> pg_restore: error: one of -d/--dbname and -f/--file must be specified Yeah, that's not good. How abo

Re: proposal: pg_restore --convert-to-text

2019-06-12 Thread Alvaro Herrera
On 2019-Jun-12, Daniel Verite wrote: > While testing pg_restore on v12, I'm stumbling on this too. Thanks for testing. > pg_restore without argument fails like that: > > $ pg_restore > pg_restore: error: one of -d/--dbname and -f/--file must be specified > > But that's not right since "pg_rest

Re: Race conditions with checkpointer and shutdown

2019-06-12 Thread Alvaro Herrera
On 2019-Apr-29, Tom Lane wrote: > Ashwin Agrawal writes: > > On Mon, Apr 29, 2019 at 10:36 AM Tom Lane wrote: > >> Can you try applying a1a789eb5ac894b4ca4b7742f2dc2d9602116e46 > >> to see if it fixes the problem for you? > > > Yes, will give it a try on greenplum and report back the result. >

Re: Quitting the thes

2019-06-12 Thread Alvaro Herrera
On 2019-May-14, Michael Paquier wrote: > Thanks, committed. I have noticed an extra one in reorderbuffer.c. Some grepping found a bit more; patch attached. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-06-12 Thread Bruce Momjian
On Wed, Jun 5, 2019 at 11:54:04AM +0900, Masahiko Sawada wrote: > On Fri, May 10, 2019 at 2:42 AM Bruce Momjian wrote: > > I think we need to step back and see what we want to do. There are six > > levels of possible encryption: > > > > 1. client-side column encryption > > 2. server-side colum

Re: GiST limits on contrib/cube with dimension > 100?

2019-06-12 Thread Tom Lane
Andrey Borodin writes: >> 9 июня 2019 г., в 23:05, Siarhei Siniak написал(а): >> I've been using cube extension recompiled with >> #define MAX_DIM 256. >> But with a version 11.3 I'm getting the following error: >> failed to add item to index page in > So, you have changed source code (removing

Re: Race conditions with checkpointer and shutdown

2019-06-12 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Apr-29, Tom Lane wrote: >> Ashwin Agrawal writes: >>> Have we decided if this will be applied to back branches? >> My feeling about it is "maybe eventually, but most definitely not >> the week before a set of minor releases". Some positive experience >> with Gre

Re: hyrax vs. RelationBuildPartitionDesc

2019-06-12 Thread Robert Haas
On Tue, Jun 11, 2019 at 1:57 PM Tom Lane wrote: > I think the existing code is horribly ugly and this is even worse. > It adds cycles to RelationDecrementReferenceCount which is a hotspot > that has no business dealing with this; the invariants are unclear; > and there's no strong reason to think

Re: Race conditions with checkpointer and shutdown

2019-06-12 Thread Alvaro Herrera
On 2019-Jun-12, Tom Lane wrote: > Alvaro Herrera writes: > > Maybe now's a good time to get it back-patched? > > Should we do that now, or wait till after next week's releases? IMO this has been hammered enough in master, and we still have a few days in the back-branches for buildfarm, that it

Re: hyrax vs. RelationBuildPartitionDesc

2019-06-12 Thread Tom Lane
Robert Haas writes: > On Tue, Jun 11, 2019 at 1:57 PM Tom Lane wrote: > I think the change is responsive to your previous complaint that the > timing of stuff getting freed is not very well pinned down. With this > change, it's much more tightly pinned down: it happens when the > refcount goes

Re: Quitting the thes

2019-06-12 Thread Guillaume Lelarge
Le mer. 12 juin 2019 à 20:45, Alvaro Herrera a écrit : > On 2019-May-14, Michael Paquier wrote: > > > Thanks, committed. I have noticed an extra one in reorderbuffer.c. > > Some grepping found a bit more; patch attached. > > Thanks a lot, this is very good. I've got some fixes to do :) -- Gui

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Tue, May 21, 2019 at 02:22:53PM -0700, Peter Geoghegan wrote: > On Tue, May 21, 2019 at 1:51 PM Bruce Momjian wrote: > > > My concern here (which I believe Alexander shares) is that it doesn't > > > make sense to group these two items together. They're two totally > > > unrelated pieces of work

Re: pgbench rate limiting changes transaction latency computation

2019-06-12 Thread Alvaro Herrera
On 2019-Jun-12, Heikki Linnakangas wrote: > That was my thought too, when looking at this example. When there is already > a long queue of transactions, the txtime of individual transactions doesn't > matter much. The most important thing under that condition is how fast the > system can dissolve

Re: Race conditions with checkpointer and shutdown

2019-06-12 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Jun-12, Tom Lane wrote: >> Should we do that now, or wait till after next week's releases? > IMO this has been hammered enough in master, and we still have a few > days in the back-branches for buildfarm, that it's okay to do it now. Poking at that, I find that a

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Tue, May 21, 2019 at 12:57:56PM -0700, Andres Freund wrote: > Hi, > > On 2019-05-21 15:47:34 -0400, Bruce Momjian wrote: > > On Mon, May 20, 2019 at 03:17:19PM -0700, Andres Freund wrote: > > > Hi, > > > > > > Note that I've added a few questions to individuals involved with > > > specific poi

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Wed, May 22, 2019 at 04:50:14PM +0900, Ian Barwick wrote: > On 5/22/19 4:26 PM, Michael Paquier wrote: > > On Wed, May 22, 2019 at 09:19:53AM +0900, Ian Barwick wrote: > > > the last two items are performance improvements not related to > > > authentication; > > > presumably the VACUUM item wou

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Tue, May 28, 2019 at 08:58:23AM -0700, Andres Freund wrote: > > > > Add function > linkend="functions-info-partition">pg_partition_root() > to return top-most parent of a partition tree (Michaël Paquier) > > > > > > > >

Re: POC: Cleaning up orphaned files using undo logs

2019-06-12 Thread Robert Haas
On Mon, May 27, 2019 at 5:59 AM Amit Kapila wrote: > Apart from fixing the above comments, the patch is rebased on latest > undo patchset. As of now, I have split the binaryheap.c changes into > a separate patch. We are stilll enhancing the patch to compute > oldestXidHavingUnappliedUndo which t

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Fri, Jun 7, 2019 at 12:04:33PM +0200, Adrien Nayrat wrote: > On 5/11/19 10:33 PM, Bruce Momjian wrote: > > I have posted a draft copy of the PG 12 release notes here: > > > > http://momjian.us/pgsql_docs/release-12.html > > > > They are committed to git. It includes links to the main doc

Re: PG 12 draft release notes

2019-06-12 Thread Peter Geoghegan
On Wed, Jun 12, 2019 at 1:16 PM Bruce Momjian wrote: > First, my apologies in getting to this so late. Peter Geoghegan > supplied me with slides and a video to study, and I now understand how > complex the btree improvements are. Here is a video of Peter's > presentation at PGCon: > > ht

Re: Question about some changes in 11.3

2019-06-12 Thread Tom Lane
Mat Arye writes: > On Mon, Jun 3, 2019 at 4:07 PM Tom Lane wrote: >> Hm. I'd say this was already broken by the invention of >> apply_scanjoin_target_to_paths; perhaps 11-before-11.3 managed to >> still work for you, but it's not hard to envision applications of >> set_rel_pathlist_hook for whic

Re: Should we warn against using too many partitions?

2019-06-12 Thread David Rowley
On Wed, 12 Jun 2019 at 17:49, Amit Langote wrote: > I noticed a typo: > > "...able to handle partition hierarchies up a few thousand partitions" > > s/up/up to/g > > I'm inclined to add one more word though, as: > > "...able to handle partition hierarchies with up to a few thousand partitions" > >

Re: [PATCH] ruleutils: Fix subqueries with shadowed aliases

2019-06-12 Thread Tom Lane
=?iso-8859-1?Q?Philip_Dub=E9?= writes: > Discovered while looking into issue here: > https://github.com/citusdata/citus/pull/2733 > For completeness I'll quote the example code to demonstrate the issue: > ... > Where the 2nd join_alias should be renamed to join_alias_1 Good catch! The proposed

Re: Adaptive query optimization

2019-06-12 Thread Tomas Vondra
On Wed, Jun 12, 2019 at 06:14:41PM +0530, Kuntal Ghosh wrote: Hello, On Wed, Jun 12, 2019 at 5:06 PM Konstantin Knizhnik wrote: On 12.06.2019 0:43, Tomas Vondra wrote: I don't think "learning phase" is an issue, in fact I think that's something we need to do - it ensures we have enough data to

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Wed, Jun 12, 2019 at 03:06:34PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 1:16 PM Bruce Momjian wrote: > > First, my apologies in getting to this so late. Peter Geoghegan > > supplied me with slides and a video to study, and I now understand how > > complex the btree improvements

Re: proposal: new polymorphic types - commontype and commontypearray

2019-06-12 Thread Tom Lane
Greg Stark writes: > The proposals I see above are "commontype", "supertype", "anycommontype", > and various abbreviations of those. I would humbly add "compatibletype". > Fwiw I kind of like commontype. > Alternately an argument could be made that length and typing convenience > isn't really a fa

Re: pg_dump: fail to restore partition table with serial type

2019-06-12 Thread Tom Lane
Alvaro Herrera writes: > There was indeed one more problem, that only the pg10 pg_upgrade test > detected. Namely, binary-upgrade dump didn't restore for locally > defined constraints: they were dumped twice, first in the table > definition and later by the ALTER TABLE ADD CONSTRAINT bit for bina

Re: PG 12 draft release notes

2019-06-12 Thread Peter Geoghegan
On Wed, Jun 12, 2019 at 5:22 PM Bruce Momjian wrote: > I had become so confused by this item that I needed a few weeks to > settle on what was actually going on. I put a lot of time into my pgCon talk, especially on the diagrams. Seems like that paid off. Even Heikki was confused by my explanatio

Re: Custom table AMs need to include heapam.h because of BulkInsertState

2019-06-12 Thread David Rowley
On Mon, 10 Jun 2019 at 11:45, David Rowley wrote: > > On Sat, 8 Jun 2019 at 04:51, Andres Freund wrote: > > David, any opinions on how to best fix this? It's not extremely obvious > > how to do so best in the current setup of the partition actually being > > hidden somewhere a few calls away (i.e

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Wed, Jun 12, 2019 at 06:34:27PM -0700, Peter Geoghegan wrote: > > I was wrong. I was thinking of this commit: > > > > commit d2086b08b0 > > Author: Alexander Korotkov > > Date: Sat Jul 28 00:31:40 2018 +0300 > > > > Reduce path length for locking leaf B-tree pages during inse

Re: PG 12 draft release notes

2019-06-12 Thread Peter Geoghegan
On Wed, Jun 12, 2019 at 7:29 PM Bruce Momjian wrote: > OK, I mentioned something about increased locality now. Patch attached. Looks good -- locality is a good catch-all term. Thanks! -- Peter Geoghegan

Re: PG 12 draft release notes

2019-06-12 Thread Bruce Momjian
On Wed, Jun 12, 2019 at 07:42:31PM -0700, Peter Geoghegan wrote: > On Wed, Jun 12, 2019 at 7:29 PM Bruce Momjian wrote: > > OK, I mentioned something about increased locality now. Patch attached. > > Looks good -- locality is a good catch-all term. Great, patch applied. -- Bruce Momjian

Re: pg_dump: fail to restore partition table with serial type

2019-06-12 Thread Alvaro Herrera
On 2019-Jun-12, Tom Lane wrote: > Alvaro Herrera writes: > > There was indeed one more problem, that only the pg10 pg_upgrade test > > detected. Namely, binary-upgrade dump didn't restore for locally > > defined constraints: they were dumped twice, first in the table > > definition and later by

Re: Adaptive query optimization

2019-06-12 Thread Kuntal Ghosh
On Thu, Jun 13, 2019 at 5:49 AM Tomas Vondra wrote: > > For example, we might require 1000 samples for a given node (say, scan > with some quals), before we start using it to tweak the estimates. Once > we get the number of estimates, we can continue collecting more data, > and once in a while upd

Re: Parallel grouping sets

2019-06-12 Thread David Rowley
On Wed, 12 Jun 2019 at 14:59, Richard Guo wrote: > Implementation 1 > Parallel aggregation has already been supported in PostgreSQL and it is > implemented by aggregating in two stages. First, each worker performs an > aggregation step, producing a partial result for each group of which > that pr

Re: [PATCH] vacuumlo: print the number of large objects going to be removed

2019-06-12 Thread Timur Birsh
12.06.2019, 14:31, "Timur Birsh" : > Please cc, I am not subscribed to the list. I have subscribed to the mailing list, there is no need to cc me. Thank you.

Re: fix psql \conninfo & \connect when using hostaddr

2019-06-12 Thread Noah Misch
On Wed, Jun 12, 2019 at 02:44:49PM +0200, Fabien COELHO wrote: > there is no simple way to know whether hostaddr was set at > the libPQ level > A solution could be to have a PQdoestheconnectionuseshostaddr(conn) > function, but I cannot say I'd be thrilled. PQconninfo() is the official way to ret

Re: Race conditions with checkpointer and shutdown

2019-06-12 Thread Michael Paquier
On Wed, Jun 12, 2019 at 04:26:23PM -0400, Tom Lane wrote: > Poking at that, I find that a1a789eb5 back-patches reasonably painlessly > into v11 and v10, but trying to bring it back to 9.6 encounters a pile of > merge failures. Also, looking at the git logs shows that we did a hell > of a lot of su

Re: Cleaning up and speeding up string functions

2019-06-12 Thread David Rowley
On Thu, 6 Jun 2019 at 08:54, Alvaro Herrera wrote: > > On 2019-May-26, David Rowley wrote: > > > On Sun, 26 May 2019 at 04:50, Tom Lane wrote: > > > > Here the cost is code space rather than programmer-visible complexity, > > > but I still doubt that it's worth it. > > > > I see on today's master

RE: libpq debug log

2019-06-12 Thread Iwata, Aya
Hi, I rebased my patch from head. Please find my attached patch. Regard, Aya Iwata v5-libpq-PQtrace-output-one-line.patch Description: v5-libpq-PQtrace-output-one-line.patch

RE: [PATCH] Speedup truncates of relation forks

2019-06-12 Thread Tsunakawa, Takayuki
From: Masahiko Sawada [mailto:sawada.m...@gmail.com] > We do RelationTruncate() also when we truncate heaps that are created > in the current transactions or has a new relfilenodes in the current > transaction. So I think there is a room for optimization Thomas > suggested, although I'm not sure it

Re: Race conditions with checkpointer and shutdown

2019-06-12 Thread Michael Paquier
On Wed, Jun 12, 2019 at 01:42:01PM -0400, Alvaro Herrera wrote: > I looked at the buildfarm failures for the recoveryCheck stage. It > looks like there is only one failure for branch master after this > commit, which was chipmunk saying: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=

Re: [PATCH] vacuumlo: print the number of large objects going to be removed

2019-06-12 Thread Michael Paquier
Hi, On Thu, Jun 13, 2019 at 10:49:46AM +0600, Timur Birsh wrote: > 12.06.2019, 14:31, "Timur Birsh" : >> Please cc, I am not subscribed to the list. > > I have subscribed to the mailing list, there is no need to cc me. Welcome. Nice to see that you have subscribed to the lists. Please note tha

Re: PG 12 draft release notes

2019-06-12 Thread Michael Paquier
On Wed, Jun 12, 2019 at 05:25:37PM -0400, Bruce Momjian wrote: > Since we did not backpatch this fix, I am hesitant to spell out exactly > how to exploit this DOS attack. Yes, people can read it in the email > archives, and commit messages, but I don't see the value in spelling it > out the releas

Re: Quitting the thes

2019-06-12 Thread Michael Paquier
On Wed, Jun 12, 2019 at 02:45:27PM -0400, Alvaro Herrera wrote: > Some grepping found a bit more; patch attached. Indeed. There were much more. I just got to look with stuff like that: find . -name "*.c" | xargs egrep "(\b[a-zA-Z]+) \1\b" But I did not find any more spots. Indentation is incor

Re: check_recovery_target_lsn() does a PG_CATCH without a throw

2019-06-12 Thread Michael Paquier
On Wed, Jun 12, 2019 at 01:16:54PM +0200, Peter Eisentraut wrote: > Right. Here is a patch that addresses this by copying the relevant code > from pg_lsn_in() and timestamptz_in() directly into the check hooks. > It's obviously a bit unfortunate not to be able to share that code, > but it's not ac