Re: Fix for pg_stat_activity putting client hostaddr into appname field

2018-03-29 Thread Michael Paquier
On Tue, Mar 27, 2018 at 03:47:07PM +1300, Edmund Horner wrote: > But the stats array includes auxiliary processes, which means it has > NumBackendStatSlots items. The pointers for the aux process strings > are out of range. In the case of my query, the pointers for > st_appname in the aux process

Re: Commit fest 2017-11

2018-03-29 Thread Alexander Korotkov
Hi, Fabien! On Fri, Dec 1, 2017 at 9:10 AM, Fabien COELHO wrote: > And the last 21 patches have been classified as well. Here is the >> final score for this time: >> Committed: 55. >> Moved to next CF: 103. >> Rejected: 1. >> Returned with Feedback: 47. >> Total: 206. >> >> Thanks to all the con

Re: Commit fest 2017-11

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 10:06 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > Hi, Fabien! > > On Fri, Dec 1, 2017 at 9:10 AM, Fabien COELHO wrote: > >> And the last 21 patches have been classified as well. Here is the >>> final score for this time: >>> Committed: 55. >>> Moved to nex

Re: Commit fest 2017-11

2018-03-29 Thread Simon Riggs
On 29 March 2018 at 09:19, Magnus Hagander wrote: > > > On Thu, Mar 29, 2018 at 10:06 AM, Alexander Korotkov > wrote: >> >> Hi, Fabien! >> >> On Fri, Dec 1, 2017 at 9:10 AM, Fabien COELHO wrote: And the last 21 patches have been classified as well. Here is the final score for this

Re: Commit fest 2017-11

2018-03-29 Thread Alexander Korotkov
On Thu, Mar 29, 2018 at 11:19 AM, Magnus Hagander wrote: > On Thu, Mar 29, 2018 at 10:06 AM, Alexander Korotkov < > a.korot...@postgrespro.ru> wrote: > >> Hi, Fabien! >> >> On Fri, Dec 1, 2017 at 9:10 AM, Fabien COELHO >> wrote: >> >>> And the last 21 patches have been classified as well. Here i

Re: Commit fest 2017-11

2018-03-29 Thread Michael Paquier
On Thu, Mar 29, 2018 at 09:38:28AM +0100, Simon Riggs wrote: > I suggest someone does another analysis that shows how many patch > reviews have been conducted by patch authors, so we can highlight > people who are causing the problem yet not helping solve the problem. This data is already partiall

Re: Commit fest 2017-11

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 10:37 AM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Thu, Mar 29, 2018 at 11:19 AM, Magnus Hagander > wrote: > >> On Thu, Mar 29, 2018 at 10:06 AM, Alexander Korotkov < >> a.korot...@postgrespro.ru> wrote: >> >>> Hi, Fabien! >>> >>> On Fri, Dec 1, 2017 at

Typo in shared_record_table_compare() commentary

2018-03-29 Thread Arthur Zakirov
Hello hackers, During studying dshash I've found a little typo. There is no SharedRecordTableKey struct in the code, I think the commentary refers to SharedRecordTableKey struct. The patch is attached. -- Arthur Zakirov Postgres Professional: http://www.postgrespro.com Russian Postgres Company

Re: Commit fest 2017-11

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 10:38 AM, Simon Riggs wrote: > On 29 March 2018 at 09:19, Magnus Hagander wrote: > > > > > > On Thu, Mar 29, 2018 at 10:06 AM, Alexander Korotkov > > wrote: > >> > >> Hi, Fabien! > >> > >> On Fri, Dec 1, 2017 at 9:10 AM, Fabien COELHO > wrote: > > And the last

Re: [HACKERS] A design for amcheck heapam verification

2018-03-29 Thread Simon Riggs
On 29 March 2018 at 01:49, Peter Geoghegan wrote: >>> IndexBuildHeapRangeScan() doesn't mention anything about CIC's heap >>> ShareUpdateExclusiveLock (it just says SharedLock), because that lock >>> strength doesn't have anything to do with IndexBuildHeapRangeScan() >>> when it operates with an

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-29 Thread Thomas Munro
On Wed, Mar 28, 2018 at 6:18 AM, Robert Haas wrote: > On Tue, Mar 27, 2018 at 11:17 AM, Tom Lane wrote: >> regression=# create sequence s1; >> CREATE SEQUENCE >> regression=# begin; >> BEGIN >> regression=# set force_parallel_mode to 1; >> SET >> regression=# declare c cursor for select nextval('

Re: Typo in shared_record_table_compare() commentary

2018-03-29 Thread Thomas Munro
On Thu, Mar 29, 2018 at 10:16 PM, Arthur Zakirov wrote: > During studying dshash I've found a little typo. There is no > SharedRecordTableKey struct in the code, I think the commentary refers > to SharedRecordTableKey struct. Right, thanks! > The patch is attached. +1 -- Thomas Munro http://w

Re: Typo in shared_record_table_compare() commentary

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 11:37 AM, Thomas Munro < thomas.mu...@enterprisedb.com> wrote: > On Thu, Mar 29, 2018 at 10:16 PM, Arthur Zakirov > wrote: > > During studying dshash I've found a little typo. There is no > > SharedRecordTableKey struct in the code, I think the commentary refers > > to Sha

Re: Commit fest 2017-11

2018-03-29 Thread Sergei Kornilov
Hello I can not find "Reports" in bottom any page of CF app... Such stats covers only reviews marked in CF app? Through "Comment"->"Review" buttons? I'm afraid this statistics will be inaccurate for new users (like me). Wiki page https://wiki.postgresql.org/wiki/Reviewing_a_Patch say > Send revi

Re: Commit fest 2017-11

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 11:47 AM, Sergei Kornilov wrote: > Hello > I can not find "Reports" in bottom any page of CF app... > > Such stats covers only reviews marked in CF app? Through > "Comment"->"Review" buttons? I'm afraid this statistics will be inaccurate > for new users (like me). Wiki pag

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-29 Thread Simon Riggs
On 28 March 2018 at 12:00, Pavan Deolasee wrote: > v27 attached, though review changes are in > the add-on 0005 patch. This all looks good now, thanks for making all of those changes. I propose [v27 patch1+patch3+patch5] as the initial commit candidate for MERGE, with other patches following la

Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-03-29 Thread Rushabh Lathia
Hi, Consider the below test: CREATE TABLE foo (a INT, b INT, c VARCHAR) PARTITION BY LIST(a); CREATE TABLE foo_p1 PARTITION OF foo FOR VALUES IN (1,2); CREATE TABLE foo_p2 PARTITION OF foo FOR VALUES IN (3,4); INSERT INTO foo select i,i,i from generate_series(1,4)i; CREATE TABLE foo_d (like foo)

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-29 Thread Teodor Sigaev
The next question what I see: why do not we lock entry leaf pages in some cases? I've modified patch to predicate lock each leaf (entry or posting) page. Now patch reaches commitable state from my point view. -- Teodor Sigaev E-mail: teo...@sigaev.ru

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-29 Thread Ashutosh Bapat
On Wed, Mar 28, 2018 at 7:21 PM, Ashutosh Bapat wrote: > > Ah sorry, I was wrong about remote_conds. remote_conds and local_conds > are basically the conditions on the relation being pushed down. > havingQuals are conditions on a grouped relation so treating them like > baserestrictinfo or join co

Re: Proposal: http2 wire format

2018-03-29 Thread Hannu Krosing
> > > * room for other resultset formats later. Like Damir, I really want to > add > > protobuf or json serializations of result sets at some point, mainly so > we > > can return "entity graphs" in graph representation rather than left-join > > projection. > > -1. I don't think this belongs in post

Re: [HACKERS] path toward faster partition pruning

2018-03-29 Thread David Rowley
On 29 March 2018 at 21:35, Amit Langote wrote: > Beside fixing that, I have decided to get rid of the > PartititionPruneStepOpNe (a special kind of base pruning step that was > being used to prune list partitions using a set of <> operator clauses) > and related functions. Instead pruning for <>

hot_standby_feedback vs excludeVacuum and snapshots

2018-03-29 Thread Greg Stark
I'm poking around to see debug a vacuuming problem and wondering if I've found something more serious. As far as I can tell the snapshots on HOT standby are built using a list of running xids that the primary builds and puts in the WAL and that seems to include all xids from transactions running i

Fix src/test/subscription/t/003_constraints.pl header comment

2018-03-29 Thread Alexander Korotkov
Hi! src/test/subscription/t/003_constraints.pl starts from the line # Basic logical replication test It seems that this line was copy-pasted from src/test/subscription/t/ 001_rep_changes.pl. Fix is attached. -- Alexander Korotkov Postgres Professional: http://www.postgrespro.com The Russian

Re: Prefix operator for text and spgist support

2018-03-29 Thread Ildus Kurbangaliev
On Fri, 23 Mar 2018 11:45:33 +0300 Alexander Korotkov wrote: > On Thu, Mar 22, 2018 at 7:09 PM, Teodor Sigaev > wrote: > > > Patch looks resonable, but I see some place to improvement: > > spg_text_leaf_consistent() only needs to check with > > text_startswith() if reconstucted value came to le

Re: committing inside cursor loop

2018-03-29 Thread Andrew Gierth
> "Peter" == Peter Eisentraut writes: Peter> Committed, thanks! So... what is a pl/* that does _not_ use pinned cursors for cursor loops supposed to do in this case? -- Andrew (irc:RhodiumToad)

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-03-29 Thread Teodor Sigaev
Conception of max-retry option seems strange for me. if number of retries reaches max-retry option, then we just increment counter of failed transaction and try again (possibly, with different random numbers). At the end we should distinguish number of error transaction and failed transaction, t

Re: csv format for psql

2018-03-29 Thread Daniel Verite
David G. Johnston wrote: > Could someone post how captions, rows-only, and footer pset settings factor > into this? Specifically are they fixed to on/off or will they hide/show if > users request them explicitly? This is described in the doc with: + csv format writes columns se

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-29 Thread Alvaro Herrera
I don't quite understand the new call in gininsert -- I mean I see that it wants to check for conflicts even when fastupdate is set, but why? Maybe, just maybe, it would be better to add a new flag to the GinCheckForSerializableConflictIn function, that's passed differently for this one callsite, a

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-03-29 Thread Thomas Munro
On Thu, Mar 29, 2018 at 6:58 PM, Craig Ringer wrote: > On 28 March 2018 at 11:53, Tom Lane wrote: >> >> Craig Ringer writes: >> > TL;DR: Pg should PANIC on fsync() EIO return. >> >> Surely you jest. > > No. I'm quite serious. Worse, we quite possibly have to do it for ENOSPC as > well to avoid s

Re: csv format for psql

2018-03-29 Thread Daniel Verite
David G. Johnston wrote: > Or, really, just make --csv take an optional argument which, if present, is > the delimiter. I don't think psql can support optional arguments because psql --option foo would be ambiguous, since foo could be the option's value or the name of a database to connec

Re: csv format for psql

2018-03-29 Thread Pavel Stehule
2018-03-29 14:17 GMT+02:00 Daniel Verite : > David G. Johnston wrote: > > > Or, really, just make --csv take an optional argument which, if present, > is > > the delimiter. > > I don't think psql can support optional arguments because > psql --option foo > would be ambiguous, since foo cou

Re: JIT compiling with LLVM v12

2018-03-29 Thread Jesper Pedersen
Hi Andres, On 03/28/2018 05:27 PM, Andres Freund wrote: On 2018-03-27 10:34:26 -0700, Andres Freund wrote: On 2018-03-27 10:05:47 -0400, Peter Eisentraut wrote: On 3/13/18 19:40, Andres Freund wrote: I've pushed a revised and rebased version of my JIT patchset. What is the status of this it

Re: csv format for psql

2018-03-29 Thread Daniel Verite
Isaac Morland wrote: > The actual reason I'm posting this is because some of the discussion has > made me a bit confused: there is already a CSV format defined for the COPY > command and used by the psql \copy. I just want to check that what is being > discussed here would use the exact sa

Re: JIT compiling with LLVM v12.2

2018-03-29 Thread John Naylor
Hi Andres, I spent some time over pouring over the JIT README, and I've attached a patch with some additional corrections as well as some stylistic suggestions. The latter may be debatable, but I'm sure you can take and pick as you see fit. If there are cases where I misunderstood your intent, mayb

Re: Parallel Aggregates for string_agg and array_agg

2018-03-29 Thread Tomas Vondra
On 03/29/2018 05:49 AM, David Rowley wrote: > On 29 March 2018 at 03:05, Tomas Vondra wrote: >> On 03/28/2018 03:54 PM, Tom Lane wrote: >>> I had in mind to look at exprType() of the argument. >> >> Right, I'm fine with that. > > Attached is v9, which is based on Tom's v8 but includes the new t

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Alvaro Herrera
Moving thread to pgsql-hackers. Simon Riggs wrote: > On 28 March 2018 at 22:23, Andres Freund wrote: > > > Add documentation for the JIT feature. > > Very nice feature and most welcome but we should call it something > other than just "JIT" > > JIT means Just In Time, which could be applied to

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-29 Thread Jeevan Chalke
On Wed, Mar 28, 2018 at 7:21 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Mar 27, 2018 at 2:43 PM, Jeevan Chalke > wrote: > > > I am not sure on what we should Assetrt here. Note that we end-up here > only > > when doing grouping, and thus I don't think we need any Asser

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-29 Thread Jeevan Chalke
On Thu, Mar 29, 2018 at 4:13 PM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Wed, Mar 28, 2018 at 7:21 PM, Ashutosh Bapat > wrote: > > > > Ah sorry, I was wrong about remote_conds. remote_conds and local_conds > > are basically the conditions on the relation being pushed down. >

Re: [HACKERS] [PATCH] Incremental sort

2018-03-29 Thread Alexander Korotkov
On Wed, Mar 28, 2018 at 7:17 PM, Andres Freund wrote: > On 2018-03-28 16:28:01 +0300, Teodor Sigaev wrote: > > > BTW, patch had conflicts with master. Please, find rebased version > attached. > > > > Despite by patch conflist patch looks commitable, has anybody objections > to > > commit it? > >

Re: [HACKERS] Pluggable storage

2018-03-29 Thread David Steele
On 3/28/18 8:39 PM, Michael Paquier wrote: > On Wed, Mar 28, 2018 at 12:23:56PM -0400, David Steele wrote: >> I think this entry should be moved the the next CF. I'll do that >> tomorrow unless there are objections. > > Instead of moving things to the next CF by default, perhaps it would > make m

Re: Parallel Aggregates for string_agg and array_agg

2018-03-29 Thread David Rowley
On 30 March 2018 at 02:00, Tomas Vondra wrote: > On 03/29/2018 05:49 AM, David Rowley wrote: >> Attached is v9, which is based on Tom's v8 but includes the new tests >> and I think the required fix to disable use of the serial/deserial >> function for array_agg(). >> > > I have only looked at the

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-29 Thread Alexander Korotkov
On Thu, Mar 29, 2018 at 1:38 PM, Teodor Sigaev wrote: > The next question what I see: why do not we lock entry leaf pages in some >> cases? >> > > I've modified patch to predicate lock each leaf (entry or posting) page. > Now patch reaches commitable state from my point view. I made some enhanc

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-03-29 Thread Craig Ringer
On 29 March 2018 at 20:07, Thomas Munro wrote: > On Thu, Mar 29, 2018 at 6:58 PM, Craig Ringer > wrote: > > On 28 March 2018 at 11:53, Tom Lane wrote: > >> > >> Craig Ringer writes: > >> > TL;DR: Pg should PANIC on fsync() EIO return. > >> > >> Surely you jest. > > > > No. I'm quite serious. W

Re: Parallel safety of binary_upgrade_create_empty_extension

2018-03-29 Thread Tom Lane
Thomas Munro writes: > Presumably also cursor_to_xmlschema. I also found some more > suspicious brin and gin modifying functions. So I think the list of > functions that needs to be marked 'u' so far is: > * binary_upgrade_create_empty_extension > * pg_import_system_collations > * brin_summariz

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread John Naylor
> I agree. In some email threads Andres has been using "JIT" as a verb, > too, such as "JITing expressions" and such; that's a bit shocking, in a > way. Honestly I don't care in a pgsql-hackers thread, I mean we all > understand what it means, but in user-facing docs and things we should > use co

Re: Cast jsonb to numeric, int, float, bool

2018-03-29 Thread Teodor Sigaev
Thanks for everyone, pushed with some editorization Teodor Sigaev wrote: I think, it should support from/to numeric/bool/text only. If we want to have casts to from numeric to other numeric types then it should be full set (int2, int4, int8, float4, float8). I was too optimistic about casting

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-29 Thread Tomas Vondra
On 03/29/2018 02:27 AM, Dean Rasheed wrote: > On 28 March 2018 at 15:50, Tomas Vondra wrote: >> After thinking about this a bit more, I'm not sure if updating the info >> based on recursive calls makes sense. The fullmatch flag was supposed to >> answer a simple question - can there be just a sing

Re: Function to track shmem reinit time

2018-03-29 Thread Tomas Vondra
On 03/28/2018 08:55 PM, David Steele wrote: > On 3/4/18 11:17 AM, Tomas Vondra wrote: >> >> Furthermore, the patch is yet another victim of fd1a421fe - fixing the >> pg_proc entries is trivial, but a new version is needed. >> >> I'd also like to see an example/explanation how this improves this >

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Alvaro Herrera
John Naylor wrote: > > I agree. In some email threads Andres has been using "JIT" as a verb, > > too, such as "JITing expressions" and such; that's a bit shocking, in a > > way. Honestly I don't care in a pgsql-hackers thread, I mean we all > > understand what it means, but in user-facing docs an

Typo in pg_backup_custom.c

2018-03-29 Thread Daniel Gustafsson
Attached fixes a typo, which according to Git is close to being allowed to drive. cheers ./daniel typo_pgbackupcustom.patch Description: Binary data

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Tom Lane
Alvaro Herrera writes: > Simon Riggs wrote: >> JIT means Just In Time, which could be applied to many concepts and >> has been in use for many years in a range of concepts. particularly in >> manufacturing/logistics and project management. > I agree. In some email threads Andres has been using "

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Stephen Frost
Greetings Tom, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Alvaro Herrera writes: > > Simon Riggs wrote: > >> JIT means Just In Time, which could be applied to many concepts and > >> has been in use for many years in a range of concepts. particularly in > >> manufacturing/logistics and project

Re: new buildfarm with gcc & clang "trunk" -Werror?

2018-03-29 Thread Fabien COELHO
Hello Peter, - fix these warnings (other than putting -Wno-format-truncation or similar workarounds...). I've been tracking gcc-8, and I have a patch ready, but these things change a bit with every compiler snapshot, so I'm waiting until things settle down. Ok, so I will not submit an

Re: Parallel Aggregates for string_agg and array_agg

2018-03-29 Thread Tomas Vondra
On 03/29/2018 03:09 PM, David Rowley wrote: > On 30 March 2018 at 02:00, Tomas Vondra wrote: >> On 03/29/2018 05:49 AM, David Rowley wrote: >>> Attached is v9, which is based on Tom's v8 but includes the new tests >>> and I think the required fix to disable use of the serial/deserial >>> functio

Re: committing inside cursor loop

2018-03-29 Thread Peter Eisentraut
On 3/29/18 07:37, Andrew Gierth wrote: >> "Peter" == Peter Eisentraut writes: > > Peter> Committed, thanks! > > So... what is a pl/* that does _not_ use pinned cursors for cursor loops > supposed to do in this case? Other than maybe using pinned cursors, one would have to look at the whole

Re: new buildfarm with gcc & clang "trunk" -Werror?

2018-03-29 Thread Tom Lane
Fabien COELHO writes: > So I understand that a -Werror beast is somehow not desirable. Not at this point; I for one would simply ignore it. *After* we have decided what project policy for this class of warnings is going to be, it might be valuable to have such a critter. But that's not decided

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-29 Thread Bruce Momjian
On Fri, Mar 2, 2018 at 12:13:26PM -0800, Peter Geoghegan wrote: > FWIW, I think that pgbench would become a lot more usable if someone > maintained a toolset for managing pgbench. Something similar to Greg > Smith's pgbench-tools project, but with additional features for > instrumenting the server

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-29 Thread Pavel Stehule
2018-03-29 16:07 GMT+02:00 Bruce Momjian : > On Fri, Mar 2, 2018 at 12:13:26PM -0800, Peter Geoghegan wrote: > > FWIW, I think that pgbench would become a lot more usable if someone > > maintained a toolset for managing pgbench. Something similar to Greg > > Smith's pgbench-tools project, but wit

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-03-29 Thread Alvaro Herrera
Rushabh Lathia wrote: > CREATE TABLE foo (a INT, b INT, c VARCHAR) PARTITION BY LIST(a); > CREATE TABLE foo_p1 PARTITION OF foo FOR VALUES IN (1,2); > CREATE TABLE foo_p2 PARTITION OF foo FOR VALUES IN (3,4); > INSERT INTO foo select i,i,i from generate_series(1,4)i; > > CREATE TABLE foo_d (like

Re: csv format for psql

2018-03-29 Thread Daniel Verite
David G. Johnston wrote: > Unaligned format could grow its own fieldsep if it wanted to but it can > also just use the default provided fieldsep var whose default value is > pipe. If it did grow one it would need to understand "not set" in order to > preserve existing behavior. We don't

Re: committing inside cursor loop

2018-03-29 Thread Andrew Gierth
> "Peter" == Peter Eisentraut writes: >> So... what is a pl/* that does _not_ use pinned cursors for cursor >> loops supposed to do in this case? Peter> Other than maybe using pinned cursors, one would have to look at Peter> the whole picture and see what makes sense. Never mind, I was

Re: Allow workers to override datallowconn

2018-03-29 Thread Tomas Vondra
On 03/02/2018 12:16 PM, Magnus Hagander wrote: > On Fri, Feb 23, 2018 at 7:55 PM, Magnus Hagander > wrote: > > On Fri, Feb 23, 2018 at 7:52 PM, Tom Lane > wrote: > > Magnus Hagander > w

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-03-29 Thread Daniel Gustafsson
> On 27 Mar 2018, at 13:50, Eren Başak wrote: > > pg_cancel_backend() is defined proisstrict, while pg_cancel_backend_msg() is > > not. I think we must be able to perform the cancellation if the message is > > NULL, so made it two functions. > > I agree that we should preserve the old usage as

Re: PATCH: Configurable file mode mask

2018-03-29 Thread David Steele
On 3/28/18 1:59 AM, Michael Paquier wrote: > On Tue, Mar 27, 2018 at 04:21:09PM -0400, David Steele wrote: >> These updates address Michael's latest review and implement group access >> for pg_basebackup, pg_receivewal, and pg_recvlogical. A new internal >> GUC, data_directory_group_access, allows

Re: JIT compiling with LLVM v12

2018-03-29 Thread Pierre Ducroquet
On Thursday, March 29, 2018 2:39:17 PM CEST Jesper Pedersen wrote: > Hi Andres, > > On 03/28/2018 05:27 PM, Andres Freund wrote: > > On 2018-03-27 10:34:26 -0700, Andres Freund wrote: > >> On 2018-03-27 10:05:47 -0400, Peter Eisentraut wrote: > >>> On 3/13/18 19:40, Andres Freund wrote: > I'v

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread David Steele
On 3/29/18 9:51 AM, Stephen Frost wrote: > Greetings Tom, all, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Alvaro Herrera writes: >>> Simon Riggs wrote: JIT means Just In Time, which could be applied to many concepts and has been in use for many years in a range of concepts. particula

Re: csv format for psql

2018-03-29 Thread David G. Johnston
On Thu, Mar 29, 2018 at 7:30 AM, Daniel Verite wrote: > David G. Johnston wrote: > > > Unaligned format could grow its own fieldsep if it wanted to but it can > > also just use the default provided fieldsep var whose default value is > > pipe. If it did grow one it would need to understa

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-03-29 Thread Rushabh Lathia
On Thu, Mar 29, 2018 at 7:46 PM, Alvaro Herrera wrote: > Rushabh Lathia wrote: > > > CREATE TABLE foo (a INT, b INT, c VARCHAR) PARTITION BY LIST(a); > > CREATE TABLE foo_p1 PARTITION OF foo FOR VALUES IN (1,2); > > CREATE TABLE foo_p2 PARTITION OF foo FOR VALUES IN (3,4); > > INSERT INTO foo sel

Re: JIT compiling with LLVM v12

2018-03-29 Thread Jesper Pedersen
Hi, On 03/29/2018 11:03 AM, Pierre Ducroquet wrote: Clang is needed to emit the LLVM bitcode required for inlining. The "-emit- llvm" flag is used for that. A dual compilation is required for inlining to work, one compilation with gcc/clang/msvc/… to build the postgresql binary, one with clang t

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

2018-03-29 Thread Alvaro Herrera
Rushabh Lathia wrote: > On Thu, Mar 29, 2018 at 7:46 PM, Alvaro Herrera > wrote: > > Hmm, offhand I don't quite see why this error fails to be thrown. > > ATTACH PARTITION should throw an error, because partition table "foo" > already have two partition with key values (1, 2,3 4). And table "foo_

Re: [HACKERS] [POC] Faster processing at Gather node

2018-03-29 Thread Bruce Momjian
On Fri, Mar 2, 2018 at 05:21:28PM -0500, Tels wrote: > Hello Robert, > > On Fri, March 2, 2018 12:22 pm, Robert Haas wrote: > > On Wed, Feb 28, 2018 at 10:06 AM, Robert Haas > > wrote: > >> [ latest patches ] > > > > Committed. Thanks for the review. > > Cool :) > > There is a typo, tho: > >

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

2018-03-29 Thread Catalin Iacob
On Thu, Mar 29, 2018 at 2:07 PM, Thomas Munro wrote: > I found your discussion with kernel hacker Jeff Layton at > https://lwn.net/Articles/718734/ in which he said: "The stackoverflow > writeup seems to want a scheme where pages stay dirty after a > writeback failure so that we can try to fsync t

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Simon Riggs
On 29 March 2018 at 01:50, Robert Haas wrote: > On Tue, Mar 27, 2018 at 11:53 PM, Pavan Deolasee > wrote: >> Yeah, we should not do that. The patch surely does not intend to replay any >> more WAL than what we do today. The idea is to just use a different >> mechanism to find the prior checkpoint

Re: SET TRANSACTION in PL/pgSQL

2018-03-29 Thread Peter Eisentraut
On 3/15/18 17:49, Alexander Korotkov wrote: > I didn't dig deeply into this subject.  But should we rather teach SPI > to execute > utility statements without taking snapshot when not necessary.  That seems > like what executor do for client provided queries.  And that seems a bit > unlogical > tha

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-29 Thread Konstantin Knizhnik
On 11.01.2018 22:41, Peter Eisentraut wrote: On 12/22/17 23:57, Tomas Vondra wrote: PART 1: adding logical_work_mem memory limit (0001) --- Currently, limiting the amount of memory consumed by logical decoding is tricky (or you might say impossi

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Tom Lane
Simon Riggs writes: > I know the approach is new and surprising but I thought about it a lot > before proposing it and I couldn't see any holes; still can't. Please > give this some thought so we can get comfortable with this idea and > increase performance as a result. Thanks. The long and the s

Re: [HACKERS] path toward faster partition pruning

2018-03-29 Thread Tomas Vondra
Hi, I think there's a bug in generate_pruning_steps_from_opexprs, which does this for PARTITION_STRATEGY_HASH: for_each_cell(lc1, lc) { pc = lfirst(lc1); /* * Note that we pass nullkeys for step_nullkeys, * because we need to tell hash partition bound s

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-29 Thread Teodor Sigaev
Alvaro Herrera wrote: I don't quite understand the new call in gininsert -- I mean I see that it wants to check for conflicts even when fastupdate is set, but why? If fastupdate is set then we check conflict with whole index, not a particular pages in it. Predicate lock on penging list pages w

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

2018-03-29 Thread Tom Lane
Claudio Freire writes: > On Wed, Mar 28, 2018 at 6:54 PM, Tom Lane wrote: >> After 0001, >> there's no reason to assume that vacuum is particularly likely to get >> cancelled between having made cleanups and having updated the upper FSM >> levels. (Maybe the odds are a bit more for the no-indexe

Re: Typo in pg_backup_custom.c

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 3:43 PM, Daniel Gustafsson wrote: > Attached fixes a typo, which according to Git is close to being allowed to > drive. > Applied, thanks. -- Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/

Re: Question about WalSndWriteData

2018-03-29 Thread Konstantin Knizhnik
On 21.03.2018 10:08, Konstantin Knizhnik wrote: On 21.03.2018 04:50, Peter Eisentraut wrote: On 3/16/18 12:08, Konstantin Knizhnik wrote: pq_putmessage_noblock copies data from ctx->out buffer to libpq buffers. After it we write timestamp to ctx->out buffer. And comments says that we shoul

Re: Fix src/test/subscription/t/003_constraints.pl header comment

2018-03-29 Thread Magnus Hagander
On Thu, Mar 29, 2018 at 1:32 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > Hi! > > src/test/subscription/t/003_constraints.pl starts from the line > > # Basic logical replication test > > It seems that this line was copy-pasted from src/test/subscription/t/001_ > rep_changes.pl. > F

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Tomas Vondra
On 03/29/2018 06:42 PM, Tom Lane wrote: > Simon Riggs writes: >> I know the approach is new and surprising but I thought about it a lot >> before proposing it and I couldn't see any holes; still can't. Please >> give this some thought so we can get comfortable with this idea and >> increase perfor

Feature Request - DDL deployment with logical replication

2018-03-29 Thread Jeremy Finzel
Hello! I have not seen much discussion about what the plans are for being able to manage schema changes when using logical replication. In our own infrastructure, mechanisms that have been provided to manage DDL statements at the same transactional point as they happen on the master have been imm

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-29 Thread Alvaro Herrera
Teodor Sigaev wrote: > > Alvaro Herrera wrote: > > I don't quite understand the new call in gininsert -- I mean I see that > > it wants to check for conflicts even when fastupdate is set, but why? > If fastupdate is set then we check conflict with whole index, not a > particular pages in it. Predi

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Robert Haas
On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane wrote: > I'd go a little further and drop "JIT" from user-facing documentation > altogether. Instead refer to the feature as "compilation of expressions" > or some such. JIT is just jargon. Plus, the timing of the compilation is > actually the least imp

RE: pgstat_report_activity() and parallel CREATE INDEX (was: Parallel index creation & pg_stat_activity)

2018-03-29 Thread Phil Florent
Thanks, ran the original test and it works great after the patch. Phil De : Robert Haas Envoyé : jeudi 22 mars 2018 18:17 À : Peter Geoghegan Cc : Andres Freund; Phil Florent; PostgreSQL Hackers Objet : Re: pgstat_report_activity() and parallel CREATE INDEX (was

Re: [HACKERS] A design for amcheck heapam verification

2018-03-29 Thread Peter Geoghegan
On Thu, Mar 29, 2018 at 2:28 AM, Simon Riggs wrote: > I understand we are adding a check to verify heap against index and > this will take longer than before. When it runs does it report > progress of the run via pg_stat_activity, so we can monitor how long > it will take? My point to Pavan was t

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Andres Freund
On 2018-03-29 13:26:31 -0400, Robert Haas wrote: > On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane wrote: > > I'd go a little further and drop "JIT" from user-facing documentation > > altogether. Instead refer to the feature as "compilation of expressions" > > or some such. JIT is just jargon. Plus,

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-29 Thread Robert Haas
On Thu, Mar 29, 2018 at 1:13 PM, Tomas Vondra wrote: > On 03/29/2018 06:42 PM, Tom Lane wrote: >> The long and the short of it is that this is a very dangerous-looking >> proposal, we are at the tail end of a development cycle, and there are >> ~100 other patches remaining in the commitfest that a

Re: Proposal: http2 wire format

2018-03-29 Thread Andres Freund
On 2018-03-28 20:34:13 -0400, Peter Eisentraut wrote: > On 3/28/18 12:09, Andres Freund wrote: > > Yea, not the most descriptive... Returning multiple different resultsets > > from a function / procedure. Inability to do so is a serious limitation > > of postgres in comparison to some other languag

Re: ALTER TABLE ADD COLUMN fast default

2018-03-29 Thread Andres Freund
Hi, On 2018-03-29 10:16:23 +1030, Andrew Dunstan wrote: > On Wed, Mar 28, 2018 at 5:30 PM, Andres Freund wrote: > > Hi, > > > > On 2018-03-26 09:02:10 +1030, Andrew Dunstan wrote: > >> Thanks for this, all looks good. Here is the consolidate patch > >> rebased. If there are no further comments I

Re: Typo in src/backend/access/hash/README

2018-03-29 Thread Bruce Momjian
On Mon, Mar 5, 2018 at 12:23:31PM +0530, Amit Kapila wrote: > On Mon, Mar 5, 2018 at 9:10 AM, Justin Pryzby wrote: > > On Mon, Mar 05, 2018 at 04:30:28PM +1300, Thomas Munro wrote: > >> Hi, > >> > >> Not sure what word was missed here, but I guess "count": > >> > >> our the number of buckets sto

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2018-03-29 13:26:31 -0400, Robert Haas wrote: > > On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane wrote: > > > I'd go a little further and drop "JIT" from user-facing documentation > > > altogether. Instead refer to the feature as "compilatio

Re: Incorrect use of "an" and "a" in code comments and docs

2018-03-29 Thread Bruce Momjian
On Mon, Mar 5, 2018 at 07:23:22PM +1300, David Rowley wrote: > On 5 March 2018 at 18:42, Michael Paquier wrote: > > On Mon, Mar 05, 2018 at 01:58:54PM +0900, Michael Paquier wrote: > >> Attached is a proposal of patch to fix all those things. > > > > And of course I forgot to attach the actual pa

Re: [HACKERS] [PATCH] Incremental sort

2018-03-29 Thread Alexander Kuzmenkov
Hi Alexander, I took a quick look at the patch. Some things I fixed myself in the attached patch v.21. Here is the summary: Typo in compare_fractional_path_costs() should be fixed as a separate patch. Remove unused function estimate_pathkeys_groups. Extra MemoryContextReset before tuplesort_en

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Robert Haas
On Thu, Mar 29, 2018 at 3:00 PM, Stephen Frost wrote: > What we've done elsewhere when there's been similar jargon is to say > something along the lines of: > > "compiling of routines (also known as Just-In-Time or JIT compilation)" That strike me as roughly analogous to saying: "hiding rows use

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread Andres Freund
On 2018-03-29 15:00:36 -0400, Stephen Frost wrote: > Greetings, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2018-03-29 13:26:31 -0400, Robert Haas wrote: > > > On Thu, Mar 29, 2018 at 9:44 AM, Tom Lane wrote: > > > > I'd go a little further and drop "JIT" from user-facing documentation

Re: Incorrect use of "an" and "a" in code comments and docs

2018-03-29 Thread Bruce Momjian
On Mon, Mar 5, 2018 at 01:58:54PM +0900, Michael Paquier wrote: > Hi all, > > While looking at something else, I have one one occurence of $subject. > Looking more deeply at the code I have found 10 more of them, like: > - sinval is a signal invalidation, so it seems to me that "a" is > correct,

Re: Incorrect use of "an" and "a" in code comments and docs

2018-03-29 Thread Bruce Momjian
On Mon, Mar 5, 2018 at 07:42:40PM +1300, Thomas Munro wrote: > On Mon, Mar 5, 2018 at 6:51 PM, Abhijit Menon-Sen > wrote: > > At 2018-03-05 14:42:14 +0900, mich...@paquier.xyz wrote: > >> > - sinval is a signal invalidation, so it seems to me that "a" is > >> > correct, not "an". > > > > I guess

  1   2   >