Re: [HACKERS] WIP: [[Parallel] Shared] Hash

Hi, Here is a new patch series responding to feedback from Peter and Andres: 1. Support pgstat_report_tempfile and log_temp_files, which I had overlooked as Peter pointed out. 2. Use a patch format that is acceptable to git am, per complaint off-list from Andres. (Not actually made with git f

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

Hi All, I have tried to test 'group_update_clog_v11.1.patch' shared upthread by Amit on a high end machine. I have tested the patch with various savepoints in my test script. The machine details along with test scripts and the test results are shown below, Machine details: 24 sockets

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >> slots on standbys in, so we can maintain slots on stand

Re: [HACKERS] Multiple false-positive warnings from Valgrind

On Tue, Mar 21, 2017 at 10:57 PM, Aleksander Alekseev wrote: > Recently I've decided to run PostgreSQL under Valgrind according to wiki > description [1]. Lots of warnings are generated [2] but it is my > understanding that all of them are false-positive. For instance I've > found these two report

Re: [HACKERS] Measuring replay lag

On 23 March 2017 at 06:42, Simon Riggs wrote: > On 23 March 2017 at 01:02, Thomas Munro wrote: > >> Thanks! Please find attached v7, which includes a note we can point >> at when someone asks why it doesn't show 00:00:00, as requested. > > Thanks. > > Now I look harder the handling for logical l

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

On Wed, Mar 22, 2017 at 9:00 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/22/17 14:09, Robert Haas wrote: > >> The opposite means primary. I can flip the GUC name to "is_primary", if > >> that's clearer. > > Hmm, I don't find that clearer. "hot standby" has a very speci

Re: [HACKERS] bug/oversight in TestLib.pm and PostgresNode.pm

On 2017-03-23 03:28, Michael Paquier wrote: On Thu, Mar 23, 2017 at 12:51 AM, Erik Rijkers wrote: While trying to test pgbench's stderr (looking for 'creating tables' in output of the initialisation step) I ran into these two bugs (or perhaps better 'oversights'). + if (defined $expected

Re: [HACKERS] Fix for grammatical error in code-comment

On Wed, Mar 22, 2017 at 2:47 PM, Emil Iggland wrote: > I found a grammatical error in one of the code comments. > > The comment is made regarding the run-time of the algorithm, and > references big-O performance. > However the comment text uses "effective", instead of the probably more > correct

Re: [HACKERS] createlang/droplang deprecated

On Mon, Mar 20, 2017 at 1:37 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/18/17 09:00, Peter Eisentraut wrote: > > I just noticed that createlang and droplang have been listed as > > deprecated since PG 9.1. > > > > Do we dare remove them? > > Patch > > +1 -- Magnus H

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But I think it is important > to get independent validation from your side too, just to ensure I am not > making an

Re: [HACKERS] ICU integration

I am fine with this version of the patch. The issues I have with it, which I mentioned earlier in this thread, seem to be issues with ICU rather than with this patch. For example there seems to be no way for ICU to validate the syntax of the BCP 47 locales (or ICU's old format). But I think we

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 16:07, Craig Ringer wrote: > If preferred I can instead add > > proc.h: > > #define PROC_RESERVED 0x20 > > procarray.h: > > #define PROCARRAY_REPLICATION_SLOTS 0x20 > > and then test for (flags & PROCARRAY_REPLICATION_SLOTS) Attached done that way. -- Craig Ringer

Re: [HACKERS] Measuring replay lag

> Second thoughts... I'll just make LagTrackerWrite externally > available, so a plugin can send anything it wants to the tracker. > Which means I'm explicitly removing the "logical replication support" > from this patch. Done. Here's the patch I'm looking to commit, with some docs and minor code

Re: [HACKERS] identity columns

On 3/22/17, Peter Eisentraut wrote: > On 3/22/17 03:59, Vitaly Burovoy wrote: >> Column's IDENTITY behavior is very similar to a DEFAULT one. We write >> "SET DEFAULT" and don't care whether it was set before or not, because >> we can't have many of them for a single column. Why should we do that

Re: [HACKERS] Measuring replay lag

On Thu, Mar 23, 2017 at 10:50 PM, Simon Riggs wrote: >> Second thoughts... I'll just make LagTrackerWrite externally >> available, so a plugin can send anything it wants to the tracker. >> Which means I'm explicitly removing the "logical replication support" >> from this patch. > > Done. > > Here'

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Thanks Amit. v19 addresses some of the comments below. On Thu, Mar 23, 2017 at 10:28 AM, Amit Kapila wrote: > On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila > wrote: > > On Tue, Mar 21, 2017 at 6:47 PM, Pavan Deolasee > > wrote: > >> > >>> > >> > >> Please find attached rebased patches. > >> > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:02 PM, Amit Kapila wrote: > > > That sounds like you are dodging the actual problem. I mean you can > put that same PageIsFull() check in master code as well and then you > will most probably again see the same regression. Well I don't see it that way. There was a spe

Re: [HACKERS] createlang/droplang deprecated

> On 20 Mar 2017, at 01:37, Peter Eisentraut > wrote: > > On 3/18/17 09:00, Peter Eisentraut wrote: >> I just noticed that createlang and droplang have been listed as >> deprecated since PG 9.1. >> >> Do we dare remove them? > > Patch LGTM, +1 cheers ./daniel -- Sent via pgsql-hackers mail

Re: [HACKERS] Monitoring roles patch

On Wed, Mar 22, 2017 at 3:46 PM, Dave Page wrote: > On Wed, Mar 22, 2017 at 1:15 PM, Stephen Frost wrote: >> >> I did specifically ask for explicit roles to be made to enable such >> capability and that the pg_monitor role be GRANT'd those roles instead >> of hacking the pg_monitor OID into those

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

On 03/21/2017 08:02 AM, Haribabu Kommi wrote: Solution -1) Just ignore dumping these CREATE DATABASE commands and provide the user information in the documentation to create "postgres" and "template1" database in the target in case if they don't exist. If this kind of cases are very rare. Soluti

Re: [HACKERS] segfault in hot standby for hash indexes

On Thu, Mar 23, 2017 at 9:17 AM, Amit Kapila wrote: > > On Thu, Mar 23, 2017 at 8:43 AM, Amit Kapila wrote: > > On Wed, Mar 22, 2017 at 3:39 PM, Ashutosh Sharma > > wrote: > >> Hi, > >> > >> On Wed, Mar 22, 2017 at 8:41 AM, Amit Kapila > >> wrote: > >> > >> To fix this, I think we should pass

Re: [HACKERS] Parallel Append implementation

On 23 March 2017 at 05:55, Robert Haas wrote: > On Wed, Mar 22, 2017 at 4:49 AM, Amit Khandekar > wrote: >> Attached is the updated patch that handles the changes for all the >> comments except the cost changes part. Details about the specific >> changes are after the cost-related points discuss

Re: [HACKERS] pgbench - allow to store select results into variables

On Thu, Mar 16, 2017 at 12:45 AM, Fabien COELHO wrote: > > Hello Rafia, > >> I was reviewing v7 of this patch, to start with I found following white >> space errors when applying with git apply, >> /home/edb/Desktop/patches/others/pgbench-into-7.patch:66: trailing >> whitespace. > > > Yep. > > I d

Re: [HACKERS] Partition-wise aggregation/grouping

On Tue, Mar 21, 2017 at 1:47 PM, Antonin Houska wrote: > Jeevan Chalke wrote: > > > Declarative partitioning is supported in PostgreSQL 10 and work is > already in > > progress to support partition-wise joins. Here is a proposal for > partition-wise > > aggregation/grouping. Our initial performa

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

On Wed, Mar 22, 2017 at 9:54 PM, Robert Haas wrote: > On Wed, Mar 22, 2017 at 12:20 PM, Robert Haas wrote: >> On Wed, Mar 22, 2017 at 1:31 AM, Michael Paquier >> wrote: >>> Okay, switched as ready for committer. One note for the committer >>> though: keeping the calls of pgstat_bestart() out of

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

On Thu, Mar 23, 2017 at 8:19 PM, Kuntal Ghosh wrote: > Hence, to be consistent with others, bgworker processes can be > initialized from BackgroundWorkerInitializeConnectionBy[Oid]. Yeah, I am fine with that. Thanks for the updated versions. -- Michael -- Sent via pgsql-hackers mailing list (

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 4:08 PM, Pavan Deolasee wrote: > > > On Thu, Mar 23, 2017 at 3:02 PM, Amit Kapila > wrote: > >> >> >> That sounds like you are dodging the actual problem. I mean you can >> put that same PageIsFull() check in master code as well and then you >> will most probably again s

[HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

Hi pgsql-hackers, I'm planning to apply to GSOC'17 and my proposal consists currently of two parts: (1) Add errors handling to COPY as a minimum program Motivation: Using PG on the daily basis for years I found that there are some cases when you need to load (e.g. for a further analytics) a bu

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

On Tue, Mar 21, 2017 at 8:34 PM, Robert Haas wrote: > On Sun, Mar 19, 2017 at 12:01 PM, Magnus Hagander > wrote: > > I think maybe we should output a message when the slot is created, at > least > > in verbose mode, to make sure people realize that happened. Does that > seem > > reasonable? > >

Re: [HACKERS] Monitoring roles patch

Dave, * Dave Page (dp...@pgadmin.org) wrote: > On Wed, Mar 22, 2017 at 3:46 PM, Dave Page wrote: > > On Wed, Mar 22, 2017 at 1:15 PM, Stephen Frost wrote: > >> I did specifically ask for explicit roles to be made to enable such > >> capability and that the pg_monitor role be GRANT'd those roles

Re: [HACKERS] [GSoC] Push-based query executor discussion

Oleg Bartunov writes: I don't know exactly if the results of GSoC project should be committed, Technically, they are not required: https://developers.google.com/open-source/gsoc/faq Are mentoring organizations required to use the code produced by students? No. While we hope that all the

Re: [HACKERS] postgres_fdw bug in 9.6

On 2017/03/21 18:40, Etsuro Fujita wrote: Ok, I'll update the patch. One thing I'd like to revise in addition to that is (1) add to JoinPathExtraData a flag member to indicate whether to give the FDW a chance to consider a remote join, which will be set to true if the joinrel's fdwroutine is not

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

Hi 2017-03-23 12:33 GMT+01:00 Alexey Kondratov : > Hi pgsql-hackers, > > I'm planning to apply to GSOC'17 and my proposal consists currently of two > parts: > > (1) Add errors handling to COPY as a minimum program > > Motivation: Using PG on the daily basis for years I found that there are > some

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

On Thursday, March 23, 2017 4:50:20 AM EDT Magnus Hagander wrote: > We should probably consider if there is some way we can implement > these two things the same way. If we're inventing a new variable that > gets pushed on each connection, perhaps we can use that one and avoid > the SHOW command? I

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

>> 1) Is there anyone out of PG comunity who will be interested in such >> project and can be a menthor? >> 2) These two points have a general idea – to simplify work with a large >> amount of data from a different sources, but mybe it would be better to >> focus on the single task? >> > > I spent

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

On 23 March 2017 at 19:33, Alexey Kondratov wrote: > (1) Add errors handling to COPY as a minimum program Huge +1 if you can do it in an efficient way. I think the main barrier to doing so is that the naïve approach creates a subtransaction for every row, which is pretty dire in performance ter

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

On 2017/03/21 18:38, Etsuro Fujita wrote: On 2017/03/16 22:23, Arthur Zakirov wrote: Can you rebase the patch? It is not applied now. Ok, will do. Thanks for the report! Done. Also, I added regression tests and revised code and comments a bit. (As for create_foreignscan_path(), I haven'

Re: [HACKERS] Logical replication existing data copy

On 3/21/17 21:38, Peter Eisentraut wrote: > This patch is looking pretty good to me, modulo the failing pg_dump tests. > > Attached is a fixup patch. I have mainly updated some comments and > variable naming for (my) clarity. No functional changes. Committed all that. -- Peter Eisentraut

Re: [HACKERS] pageinspect and hash indexes

Hi Amit, >>> + >>> + /* Check if it is an unused hash page. */ >>> + if (pageopaque->hasho_flag == LH_UNUSED_PAGE) >>> + return page; >>> >>> >>> I don't see we need to have a separate check for unused page, why >>> can't it be checked with other page types in below check. >>> >>> if (pagetype !=

Re: [HACKERS] Monitoring roles patch

Hi On Thu, Mar 23, 2017 at 12:23 PM, Stephen Frost wrote: > >> diff --git a/contrib/pg_stat_statements/pg_stat_statements.c >> b/contrib/pg_stat_statements/pg_stat_statements.c >> index 221ac98d4a..cec94d5896 100644 >> --- a/contrib/pg_stat_statements/pg_stat_statements.c >> +++ b/contrib/pg_sta

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:44 PM, Pavan Deolasee wrote: > On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila > wrote: >> > >> 3. >> + /* >> + * HASH indexes compute a hash value of the key and store that in the >> + * index. So >> we must first obtain the hash of the value obtained from the >> + * heap

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

> On 23 Mar 2017, at 15:53, Craig Ringer wrote: > > On 23 March 2017 at 19:33, Alexey Kondratov > wrote: > >> (1) Add errors handling to COPY as a minimum program > > Huge +1 if you can do it in an efficient way. > > I think the main barrier to doing so is that the naïve approach > creates

Re: [HACKERS] Partitioned tables and relfilenode

Hi! I have noticed that there is scheduled unlinking of nonexistent physical storage under partitioned table when we execute DROP TABLE statement on this partitioned table. Though this action doesn't generate any error under typical behavior of postgres because the error of storage's lack is

[HACKERS] Schedule and Release Management Team for PG10

The Postgres release team has decided that last year's appointment of a Release Management Team worked pretty well, so we're going to run the version-10 release cycle the same way. The members of the RMT for this year will be Peter Eisentraut, Robert Haas, and Noah Misch. As previously agreed at

Re: [HACKERS] Partitioned tables and relfilenode

Hi, On Thu, Mar 23, 2017 at 11:27 PM, Maksim Milyutin wrote: > Hi! > > I have noticed that there is scheduled unlinking of nonexistent physical > storage under partitioned table when we execute DROP TABLE statement on this > partitioned table. Though this action doesn't generate any error under >

Re: [HACKERS] Page Scan Mode in Hash Index

Hi, On 03/22/2017 09:32 AM, Ashutosh Sharma wrote: Done. Please refer to the attached v2 version of patch. Thanks. 1) 0001-Rewrite-hash-index-scans-to-work-a-page-at-a-time.patch: this patch rewrites the hash index scan module to work in page-at-a-time mode. It basically introduces two new

[HACKERS] minor spelling error fix (btis -> bits)

Attached please find a minor spelling error fix, changing "btis" to "bits". -- Jon From f590a6dce6677bc5b8a409d40fd651ecb69b27bb Mon Sep 17 00:00:00 2001 From: Jon Nelson Date: Sun, 23 Mar 2014 08:23:48 -0500 Subject: [PATCH] - fix minor spelling error --- src/backend/utils/adt/network.c |

Re: [HACKERS] Guidelines for GSoC student proposals / Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

On Wed, Mar 22, 2017 at 2:24 AM, Mengxing Liu wrote: > I've finished a draft proposal for "Eliminate O(N^2) scaling from > rw-conflict tracking in serializable transactions". You can find > it from GSOC website or by the link below. > https://docs.google.com/document/d/17TAs3EJIokwPU7UTUmnlVY3El

Re: [HACKERS] minor spelling error fix (btis -> bits)

On 23 March 2017 at 15:05, Jon Nelson wrote: > Attached please find a minor spelling error fix, changing "btis" to "bits". Applied, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hacker

Re: [HACKERS] [COMMITTERS] pgsql: Add missing support for new node fields

Another idea here: Instead of making COPY_PARSE_PLAN_TREES a compile-time option, make it a run-time option, and make that somehow selectable while running the test suite. It would be much easier to run the test suite with an option on the command line, instead of having to manually edit a header

Re: [HACKERS] perlcritic

> On 21 Mar 2017, at 19:20, David Steele wrote: > > On 3/6/17 12:02 PM, Dagfinn Ilmari Mannsåker wrote: >> ilm...@ilmari.org (Dagfinn Ilmari Mannsåker) writes: >> >>> Hi Peter, >>> >>> Peter Eisentraut writes: >>> I posted this about 18 months ago but then ran out of steam. [ ] Here

[HACKERS] Fwd: GSOC 2017 project ideas

Given the low activity in pgsql-students, forward here to get more feedbacks. -- Forwarded message -- From: Charles Cui Date: 2017-03-23 8:58 GMT-07:00 Subject: GSOC 2017 project ideas To: pgsql-stude...@postgresql.org Hi guys, This is Charles Cui from Tsinghua University, C

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hi This is my first review (Magnus said in his presentation

[HACKERS] Order-preserving function transforms and EquivalenceClass

Hi, I am on a team developing an open-source extension for time-series data storage in PostgreSQL (https://github.com/timescaledb/timescaledb). We are trying to extend/hook into the planner so that it understands that date_trunc('minute', time) has the same ordering as time (or rather that a sort

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

Thank you, pushed Andrew Borodin wrote: 2017-03-22 22:48 GMT+05:00 Teodor Sigaev : hasEmptyChild? and hasNonEmptyChild (BTW, isAnyNonempy has missed 't') Yes, I think this naming is good. It's clear what's in common in these flags and what's different. And if the whole posting tree is empty

Re: [HACKERS] Backend crash on non-exclusive backup cancel

Hi! I believe patch looks good and it's ready to commit. As I understand, it fixes bug introduced by commit 7117685461af50f50c03f43e6a622284c8d54694 Date: Tue Apr 5 20:03:49 2016 +0200 Implement backup API functions for non-exclusive backups And, suppose, it should be backpatched to 9.6

[HACKERS] parallel "return query" is no good

Commit 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b allowed a parallel plan to be generated when for a RETURN QUERY or RETURN QUERY EXECUTE statement in a PL/pgsql block. As it turns out, the analysis that led to this decision was totally wrong-headed, because the plan will always be executed using SP

Re: [HACKERS] parallel "return query" is no good

On Thu, Mar 23, 2017 at 12:50 PM, Robert Haas wrote: > Commit 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b allowed a parallel > plan to be generated when for a RETURN QUERY or RETURN QUERY EXECUTE > statement in a PL/pgsql block. As it turns out, the analysis that led > to this decision was totally w

Re: [HACKERS] parallel "return query" is no good

* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Mar 23, 2017 at 12:50 PM, Robert Haas wrote: > > Commit 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b allowed a parallel > > plan to be generated when for a RETURN QUERY or RETURN QUERY EXECUTE > > statement in a PL/pgsql block. As it turns out, t

Re: [HACKERS] parallel "return query" is no good

On 2017-03-23 13:03:19 -0400, Robert Haas wrote: > On Thu, Mar 23, 2017 at 12:50 PM, Robert Haas wrote: > > Commit 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b allowed a parallel > > plan to be generated when for a RETURN QUERY or RETURN QUERY EXECUTE > > statement in a PL/pgsql block. As it turns ou

Re: [HACKERS] WIP: Faster Expression Processing v4

Looking at some of the coding choices you made here, I see that you're making a hard assumption that called functions never scribble on their fcinfo->arg/argnull arrays. I do not believe that we've ever had such an assumption before. Are we comfortable with that? If so, we'd better document it.

Re: [HACKERS] parallel "return query" is no good

On 03/23/2017 10:03 AM, Robert Haas wrote: On Thu, Mar 23, 2017 at 12:50 PM, Robert Haas wrote: Commit 7aea8e4f2daa4b39ca9d1309a0c4aadb0f7ed81b allowed a parallel plan to be generated when for a RETURN QUERY or RETURN QUERY EXECUTE statement in a PL/pgsql block. As it turns out, the analysis t

Re: [HACKERS] WIP: Faster Expression Processing v4

Hi, On 2017-03-23 13:14:59 -0400, Tom Lane wrote: > Looking at some of the coding choices you made here, I see that you're > making a hard assumption that called functions never scribble on their > fcinfo->arg/argnull arrays. I do not believe that we've ever had such > an assumption before. I th

Re: [HACKERS] parallel "return query" is no good

Robert Haas wrote: > I guess the downside of back-patching this is that it could cause a > plan change for somebody which ends up being worse. On the whole, > serial execution of queries intended to be run in parallel isn't > likely to work out well, but it's always possible somebody has a cases

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

Hi, On Tue, Feb 7, 2017 at 9:23 AM, Robert Haas wrote: > On Mon, Feb 6, 2017 at 10:40 PM, Amit Kapila wrote: >>> Maybe we should call them "unused pages". >> >> +1. If we consider some more names for that column then probably one >> alternative could be "empty pages". > > Yeah, but I think "unu

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

On Thu, Mar 23, 2017 at 12:41 AM, Rafia Sabih wrote: > Agree, done. OK, committed execute-once-v3.patch after some further study. See also https://www.postgresql.org/message-id/CA%2BTgmoZ_ZuH%2BauEeeWnmtorPsgc_SmP%2BXWbDsJ%2BcWvWBSjNwDQ%40mail.gmail.com which resulted from that study. Regardin

Re: [HACKERS] Page Scan Mode in Hash Index

On Thu, Mar 23, 2017 at 8:29 PM, Jesper Pedersen wrote: > Hi, > > On 03/22/2017 09:32 AM, Ashutosh Sharma wrote: >> >> Done. Please refer to the attached v2 version of patch. >> > > Thanks. > 1) 0001-Rewrite-hash-index-scans-to-work-a-page-at-a-time.patch: this patch rewrites the hash in

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Hi Pavan, On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But I think it is important > to get independent validation from your side too, just to ensure I am not > m

Re: [HACKERS] Page Scan Mode in Hash Index

Hi, On 03/23/2017 02:11 PM, Ashutosh Sharma wrote: On Thu, Mar 23, 2017 at 8:29 PM, Jesper Pedersen wrote: 0001v2: In hashgettuple() you can remove the 'currItem' and 'offnum' from the 'else' part, and do the assignment inside if (so->numKilled < MaxIndexTuplesPerPage) instead. Done. P

Re: [HACKERS] Hash support for grouping sets

> "Mark" == Mark Dilger writes: Mark> You define DiscreteKnapsack to take integer weights and double Mark> values, and perform the usual Dynamic Programming algorithm to Mark> solve. But the only place you call this, you pass in NULL for Mark> the values, indicating that all the values a

Re: [HACKERS] createlang/droplang deprecated

On 3/23/17 06:41, Daniel Gustafsson wrote: >> On 20 Mar 2017, at 01:37, Peter Eisentraut >> wrote: >> >> On 3/18/17 09:00, Peter Eisentraut wrote: >>> I just noticed that createlang and droplang have been listed as >>> deprecated since PG 9.1. >>> >>> Do we dare remove them? >> >> Patch > > LGTM

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer wrote: > Changes made per discussion. Committed 0001. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Page Scan Mode in Hash Index

> Hi, > > On 03/23/2017 02:11 PM, Ashutosh Sharma wrote: >> >> On Thu, Mar 23, 2017 at 8:29 PM, Jesper Pedersen >> wrote: >>> >>> 0001v2: >>> >>> In hashgettuple() you can remove the 'currItem' and 'offnum' from the >>> 'else' >>> part, and do the assignment inside >>> >>> if (so->numKilled < Ma

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 11:44 PM, Mithun Cy wrote: > Hi Pavan, > On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee > wrote: > > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > > RAM, attached SSD) and results are shown below. But I think it is > important > > to get ind

Re: [HACKERS] Hash support for grouping sets

On 2017-03-23 03:43:57 +, Andrew Gierth wrote: > > "Andres" == Andres Freund writes: > > Andres> Changes to advance_aggregates() are, in my experience, quite > Andres> likely to have performance effects. This needs some > Andres> performance tests. > [...] > Andres> Looks like it co

Re: [HACKERS] Hash support for grouping sets

> On Mar 23, 2017, at 11:22 AM, Andrew Gierth > wrote: > >> "Mark" == Mark Dilger writes: > > Mark> You define DiscreteKnapsack to take integer weights and double > Mark> values, and perform the usual Dynamic Programming algorithm to > Mark> solve. But the only place you call this, you p

Re: [HACKERS] Hash support for grouping sets

On 2017-03-23 05:09:46 +, Andrew Gierth wrote: > > "Andres" == Andres Freund writes: > > >> - Assert(newphase == 0 || newphase == aggstate->current_phase + 1); > >> + Assert(newphase <= 1 || newphase == aggstate->current_phase + 1); > > Andres> I think this somewhere in the file heade

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 7:53 PM, Amit Kapila wrote: > On Thu, Mar 23, 2017 at 3:44 PM, Pavan Deolasee > > > > > Yes, this is a very fair point. The way I proposed to address this > upthread > > is by introducing a set of threshold/scale GUCs specific to WARM. So > users > > can control when to in

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

Copying Fabrízio Mello here, who spent some time trying to work on reloptions too. He may have something to say about the new functionality that this patch provides. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Service

Re: [HACKERS] Parallel Append implementation

On 23 March 2017 at 16:26, Amit Khandekar wrote: > On 23 March 2017 at 05:55, Robert Haas wrote: >> On Wed, Mar 22, 2017 at 4:49 AM, Amit Khandekar wrote: >>> Attached is the updated patch that handles the changes for all the >>> comments except the cost changes part. Details about the specific

Re: [HACKERS] [PATCH] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

On Thu, Mar 23, 2017 at 3:58 PM, Alvaro Herrera wrote: > > Copying Fabrízio Mello here, who spent some time trying to work on > reloptions too. He may have something to say about the new > functionality that this patch provides. > Thanks Álvaro, I'll look the patch and try to help in some way.

[HACKERS] Privilege checks on array coercions

There is a test in privileges.sql (currently lines 589-625 in privileges.out) that seems to be dependent on the fact that the ArrayCoerceExpr logic doesn't check for EXECUTE privilege on the per-element type coercion function if it's dealing with a NULL input array. While fooling with Andres' fast

Re: [HACKERS] ICU integration

On 3/23/17 05:34, Andreas Karlsson wrote: > I am fine with this version of the patch. The issues I have with it, > which I mentioned earlier in this thread, seem to be issues with ICU > rather than with this patch. For example there seems to be no way for > ICU to validate the syntax of the BCP

Re: [HACKERS] Privilege checks on array coercions

On 2017-03-23 15:26:51 -0400, Tom Lane wrote: > There is a test in privileges.sql (currently lines 589-625 in > privileges.out) that seems to be dependent on the fact that the > ArrayCoerceExpr logic doesn't check for EXECUTE privilege on the > per-element type coercion function if it's dealing wit

Re: [HACKERS] [POC] A better way to expand hash indexes.

Hi Amit please find the new patch On Tue, Mar 21, 2017 at 8:16 AM, Amit Kapila wrote: > It is not only about above calculation, but also what the patch is > doing in function _hash_get_tbuckets(). By the way function name also > seems unclear (mainly *tbuckets* in the name). Fixed I have introd

Re: [HACKERS] Hash support for grouping sets

> "Mark" == Mark Dilger writes: Mark> Is there a performance test case where this patch should shine Mark> brightest? I'd like to load a schema with lots of data, and run Mark> a grouping sets query, both before and after applying the patch, Mark> to see what the performance advantage is

Re: [HACKERS] ICU integration

On Thu, Mar 23, 2017 at 12:34 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/23/17 05:34, Andreas Karlsson wrote: > > I am fine with this version of the patch. The issues I have with it, > > which I mentioned earlier in this thread, seem to be issues with ICU > > rather tha

Re: [HACKERS] Potential data loss of 2PC files

Hmm, it doesn't work (but appplies) on current HEAD: % uname -a FreeBSD *** 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #0 r315651: Tue Mar 21 02:44:23 MSK 2017 teodor@***:/usr/obj/usr/src/sys/XOR amd64 % pg_config --configure '--enable-depend' '--enable-cassert' '--enable-debug' '--enable-tap

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-03-23 17:26 GMT+01:00 Pierre Ducroquet : > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: tested, passed > Spec compliant: tested, passed > Documentation:tested, passed > > H

Re: [HACKERS] Re: [BUGS] Problem in using pgbench's --connect(-C) and --rate=rate(-R rate) options together.

Hi, the patch looks good except why do you remove initialization of is_throttled? Suppose, just a typo? --- a/src/bin/pgbench/pgbench.c +++ b/src/bin/pgbench/pgbench.c @@ -1967,7 +1967,6 @@ top: st->listen = false; st->sleeping = false; st->throttling = false; - st

Re: [HACKERS] increasing the default WAL segment size

On 3/22/17 17:33, David Steele wrote: > I think if we don't change the default size it's very unlikely I would > use alternate WAL segment sizes or recommend that anyone else does, at > least in v10. > > I simply don't think it would get the level of testing required to be > production worthy

Re: [HACKERS] ICU integration

On Fri, Mar 24, 2017 at 8:34 AM, Peter Eisentraut wrote: > Committed. On a macOS system using MacPorts, I have "icu" installed. MacPorts is a package manager that installs stuff under /opt/local. I have /opt/local/bin in $PATH so that pkg-config can be found. I run ./configure with --with-icu

Re: [HACKERS] ICU integration

On 3/23/17 16:45, Thomas Munro wrote: > On Fri, Mar 24, 2017 at 8:34 AM, Peter Eisentraut > wrote: >> Committed. > > On a macOS system using MacPorts, I have "icu" installed. MacPorts is > a package manager that installs stuff under /opt/local. I have > /opt/local/bin in $PATH so that pkg-confi

Re: [HACKERS] increasing the default WAL segment size

Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 3/22/17 17:33, David Steele wrote: > > and I doubt that most tool writers would be quick to > > add support for a feature that very few people (if any) use. > > I'm not one of those tool writers, although I have written my

Re: [HACKERS] Measuring replay lag

On Thu, Mar 23, 2017 at 10:50 PM, Simon Riggs wrote: >> Second thoughts... I'll just make LagTrackerWrite externally >> available, so a plugin can send anything it wants to the tracker. >> Which means I'm explicitly removing the "logical replication support" >> from this patch. > > Done. > > Here'

Re: [HACKERS] WIP: Faster Expression Processing v4

Stylistic thought ... I am wondering if it wouldn't be a good idea to replace EEOP_CASE_WHEN_STEP, EEOP_CASE_THEN_STEP, EEOP_COALESCE, and EEOP_ARRAYREF_CHECKINPUT with instructions defined in a less usage-dependent way as EEOP_JUMP unconditional jump EEOP_JUMP_IF_NUL

Re: [HACKERS] Potential data loss of 2PC files

On Fri, Mar 24, 2017 at 5:08 AM, Teodor Sigaev wrote: > Hmm, it doesn't work (but appplies) on current HEAD: > [...] > Data page checksums are disabled. > > fixing permissions on existing directory /spool/pg_data ... ok > creating subdirectories ... ok > selecting default max_connections ... 100 >

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On Wed, Mar 22, 2017 at 7:00 AM, Michael Paquier wrote: > On Wed, Mar 22, 2017 at 6:24 AM, Andrew Dunstan > wrote: >> This is really a pretty small patch all things considered, and pretty >> low-risk (although I haven;t been threough the code in fine detail yet). >> In the end I'm persuaded by An

Re: [HACKERS] [PATCH] few fts functions for jsonb

On 03/21/2017 06:28 PM, Dmitry Dolgov wrote: > > On 21 March 2017 at 03:03, Andrew Dunstan > > wrote: > > > > However, I think it should probably be broken up into a couple of > pieces - > > one for the generic json/jsonb transforms infrastructure (which pr

Re: [HACKERS] Logical replication existing data copy

Hi, On Thu, Mar 23, 2017 at 09:00:16AM -0400, Peter Eisentraut wrote: > On 3/21/17 21:38, Peter Eisentraut wrote: > > This patch is looking pretty good to me, modulo the failing pg_dump tests. > > > > Attached is a fixup patch. I have mainly updated some comments and > > variable naming for (my)

  1   2   >