Re: [HACKERS] allow referring to functions without arguments when unique

2017-03-14 Thread Michael Paquier
On Fri, Mar 10, 2017 at 2:03 PM, Peter Eisentraut wrote: > On 3/7/17 00:32, Michael Paquier wrote: >>> @@ -7198,6 +7198,33 @@ function_with_argtypes: >>> n->objargs = extractArgTypes($2); >>> $$ = n; >>> } >>> This may not have arguments list

Re: [HACKERS] background sessions

2017-03-14 Thread Pavel Stehule
2017-03-13 21:22 GMT+01:00 Robert Haas : > On Sat, Mar 11, 2017 at 10:11 AM, Pavel Stehule > wrote: > > 2017-03-09 14:52 GMT+01:00 Peter Eisentraut > > : > >> > >> On 3/8/17 14:22, Pavel Stehule wrote: > >> > 1. will be background session process closed automatically when parent > >> > process is

Re: [HACKERS] increasing the default WAL segment size

2017-03-14 Thread tushar
On 03/14/2017 11:14 AM, Beena Emerson wrote: Hello, Attached is the updated patch. It fixes the issues and also updates few code comments. Can you please check with the new patch? Thanks, both issues has been fixed now. -- regards,tushar EnterpriseDB https://www.enterprisedb.com/ The Ente

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-14 Thread Amit Langote
On 2017/03/13 19:30, Amit Langote wrote: >> Here is the updated patch. >> >> Since this patch proposes to avoid creating scan nodes for non-leaf tables >> in a partition tree, they won't be referenced anywhere in the resulting >> plan tree. So the executor will not lock those tables in the >> sele

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-14 Thread Michael Paquier
On Fri, Mar 10, 2017 at 2:36 PM, Kuntal Ghosh wrote: > On Fri, Mar 10, 2017 at 3:11 AM, Andres Freund wrote: >> On 2017-03-09 16:37:29 -0500, Tom Lane wrote: >>> Robert Haas writes: >>> > On Thu, Mar 9, 2017 at 2:30 PM, Peter Eisentraut >>> > wrote: >>> >> In practice, I think it's common to do

Re: [HACKERS] PATCH: Configurable file mode mask

2017-03-14 Thread Tsunakawa, Takayuki
From: David Steele [mailto:da...@pgmasters.net] > >> 3.The default location of the SSL key file is $PGDATA, so the permission > of the key file is likely to become 0640. But the current postgres requires > it to be 0600. See src/backend/libpq/be-secure-openssl.c. > > > > Yes, that needs to be add

[HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-14 Thread Mengxing Liu
I send this email to Tony, too. Because he promised to help me with testing and benchmarking. > > >> The worst problems have been > >> seen with 32 or more cores on 4 or more sockets with a large number > >> of active connections. I don't know whether you have access to a > >> machine capable

Re: [HACKERS] Gather Merge

2017-03-14 Thread Rushabh Lathia
On Mon, Mar 13, 2017 at 10:56 PM, Robert Haas wrote: > On Fri, Mar 10, 2017 at 7:59 AM, Rushabh Lathia > wrote: > > Error coming from create_gather_merge_plan() from below condition: > > > > if (memcmp(sortColIdx, gm_plan->sortColIdx, > >numsortkeys * sizeof(AttrNumber)) != 0

Re: [HACKERS] parallelize queries containing initplans

2017-03-14 Thread Amit Kapila
On Fri, Feb 10, 2017 at 4:34 PM, Amit Kapila wrote: > > I could see two possibilities to determine whether the plan (for which > we are going to generate an initplan) contains a reference to a > correlated var param node. One is to write a plan or path walker to > determine any such reference and

Re: [HACKERS] scram and \password

2017-03-14 Thread Heikki Linnakangas
On 03/14/2017 04:47 AM, Tom Lane wrote: Robert Haas writes: I'm not talking about changing the default, just having it be possible to use \password with the new system as it was with the old, whatever exactly we think that means. Seems to me the intended behavior of \password is to use the be

Re: [HACKERS] [PATCH] SortSupport for macaddr type

2017-03-14 Thread Heikki Linnakangas
Looks straightforward at a quick read-through. I have just a couple of questions. How much of the performance gain comes from avoiding the FunctionCallInvoke overhead, by simply having SortSupport with a comparison function, and how much comes from the "abbreviation"? Also, is it worth the cyc

Re: [HACKERS] multivariate statistics (v25)

2017-03-14 Thread David Rowley
On 13 March 2017 at 23:00, David Rowley wrote: > > 0003: > > No more time today. Will try and get to those soon. > 0003: I've now read this patch. My main aim here was to learn what it does and how it works. I need to spend much longer understanding how your calculating the functional dependenci

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-14 Thread DEV_OPS
Hi Mengxing Please read my comments : On 3/14/17 17:34, Mengxing Liu wrote: > I send this email to Tony, too. Because he promised to help me with testing > and benchmarking. > The worst problems have been seen with 32 or more cores on 4 or more sockets with a large number of acti

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-14 Thread Kuntal Ghosh
On Tue, Mar 14, 2017 at 1:50 PM, Michael Paquier wrote: > Here is a first pass on this patch. Thanks Michael for the review. > > void > -pgstat_bestart(void) > +pgstat_procstart(void) > I would not have thought that this patch justifies potentially > breaking extensions. Since I'm using this met

Re: [HACKERS] [PATCH] Use $ parameters as replacement characters for pg_stat_statements

2017-03-14 Thread Robert Haas
On Mon, Mar 13, 2017 at 6:05 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Mar 4, 2017 at 1:52 PM, Peter Geoghegan wrote: >>> I'd be in favor of a change >>> that makes it easier to copy and paste a query, to run EXPLAIN and so >>> on. Lukas probably realizes that there are no guarantees

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Alvaro Herrera
Andres Freund wrote: > EEO_SWITCH(op->opcode) > { > EEO_CASE(EEO_DONE): > goto out; Oh my. > which is a bit annoying. (the EEO_CASE is either a jump label or a case > statement, depending on computed goto availability). > > It seems we could either: > 1) live with the damage >

Re: [HACKERS] [COMMITTERS] pgsql: Improve postmaster's logging of listen socket creation.

2017-03-14 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > I don't mind the new output, but I kinda wonder whether it's a good idea > > to include the '.s.PGSQL.5432' bit in the host and/or whether we > > shouldn't include the port in the TCP cases as well > > Yeah, I've been thinking tha

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Alvaro Herrera
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c index 3d6a3801c0..d205101b89 100644 --- a/src/backend/executor/execUtils.c +++ b/src/backend/executor/execUtils.c @@ -47,7 +47,14 @@ #include "utils/rel.h" -static bool get_last_attnums(Node *node, ProjectionInfo

Re: [HACKERS] Gather Merge

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 5:47 AM, Rushabh Lathia wrote: > Thanks Robert for the patch and the explanation. > > I studied the patch and that look right to me. I performed manual testing, > run the scripts which I created during the gather merge patch also run > the tpch queries to make sure that it

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-14 Thread Robert Haas
On Mon, Mar 13, 2017 at 10:22 PM, Craig Ringer wrote: > I'll introduce a new LWLock, ClogTruncationLock, which will be held > from when we advance the new clogOldestXid field through to when clog > truncation completes. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-14 Thread Ashutosh Sharma
Hi, Attached is the v6 patch for microvacuum in hash index rebased on top of 'v10 patch for WAL in hash index - [1]' and 'v1 patch for WAL consistency check for hash index - [2]'. [1] - https://www.postgresql.org/message-id/CAA4eK1%2Bk5wR4-kAjPqLoKemuHayQd6RkQQT9gheTfpn%2B72o1UA%40mail.gmail.com

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-14 Thread Ashutosh Bapat
Thanks for the review. > > Some very high-level thoughts based on a look through these patches: > > In 0001, you've removed a comment about how GEQO needs special > handling, but it doesn't look as if you've made any compensating > change elsewhere. That seems unlikely to be correct. If GEQO nee

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-14 Thread Ashutosh Bapat
On Tue, Mar 14, 2017 at 6:28 AM, Amit Langote wrote: > On 2017/03/14 9:17, Robert Haas wrote: >> On Mon, Mar 13, 2017 at 3:24 PM, Robert Haas wrote: >>> Haven't looked at 0007 yet. >> >> Overall this one looks pretty good and straightforward. > > In the following code of find_partition_scheme():

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-14 Thread Alvaro Herrera
Ashutosh Sharma wrote: > Yes. But, as i said earlier I am getting negative checksum value for > page_header as well. Isn't that wrong. For eg. When I debug the > following query, i could pd_checksum value as '40074' in gdb where > page_header shows it as '-25462'. Yes; the point is that this is a

Re: [HACKERS] Page Scan Mode in Hash Index

2017-03-14 Thread Ashutosh Sharma
Hi, >> I've assigned to review this patch. >> At first, I'd like to notice that I like idea and general design. >> Secondly, patch set don't apply cleanly to master. Please, rebase it. > > > Thanks for showing your interest towards this patch. I would like to > inform that this patch has got depe

[HACKERS] Authentication tests, and plain 'password' authentication with a SCRAM verifier

2017-03-14 Thread Heikki Linnakangas
Hi, I didn't include the authentication TAP tests that Michael wrote in the main SCRAM commit last week. The main issue was that the new test was tacked on the src/test/recovery test suite, for lack of a better place. I propose that we add a whole new src/test/authentication directory for it.

Re: [HACKERS] IF NOT EXISTS option for CREATE SERVER and CREATE USER MAPPING statements

2017-03-14 Thread Ashutosh Bapat
> > Thanks for catching that. > It was caused by a conflict on applying of the patch. > Updated versions of both patches are attached. > We do not need extra line + + other usages of this do not have an extra line. Removed the extra line in the attached patch. I noticed that the earlier err

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-03-14 Thread Ashutosh Bapat
Added this to 2017/07 commitfest. On Fri, Mar 10, 2017 at 10:03 AM, Ashutosh Bapat wrote: >>> >>> The new name merge_fdw_options() is shorter than the one I chose, but >>> we are not exactly merging options for an upper relation since there >>> isn't the other fpinfo to merge from. But I think we

Re: [HACKERS] dropping partitioned tables without CASCADE

2017-03-14 Thread Ashutosh Bapat
Added this to 2017/7 commitfest to keep a track of it. On Wed, Mar 8, 2017 at 3:39 PM, Amit Langote wrote: > On 2017/03/08 18:27, Ashutosh Bapat wrote: >>> >>> About the other statement you changed, I just realized that we should >>> perhaps do one more thing. Show the Number of partitions, even

[HACKERS] Re: Authentication tests, and plain 'password' authentication with a SCRAM verifier

2017-03-14 Thread Michael Paquier
On Tue, Mar 14, 2017 at 9:36 PM, Heikki Linnakangas wrote: > While looking at the test, I noticed that the SCRAM patch didn't include > support for logging in with plain 'password' authentication, when the user > has a SCRAM verifier stored in pg_authid. That was an oversight. If the > client give

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-14 Thread David Steele
On 3/2/17 11:34 AM, Petr Jelinek wrote: > On 02/03/17 13:23, Craig Ringer wrote: >> >> I particularly dislike calling a commit callback for an abort. So I'd >> like to look further into the interface side of things. I'm inclined >> to suggest adding new callbacks for 2pc prepare, commit and rollbac

Re: [HACKERS] Gather Merge

2017-03-14 Thread Tom Lane
Robert Haas writes: > Cool, thanks for the review. I'm not quite confident that we've found > all of the bugs here yet, but I think we're moving in the right > direction. I guess the real question here is why isn't Gather Merge more like Append and MergeAppend? That is, why did you complicate m

Re: [HACKERS] [POC] hash partitioning

2017-03-14 Thread David Steele
On 3/3/17 8:33 AM, amul sul wrote: > On Fri, Mar 3, 2017 at 5:00 PM, Greg Stark > It also has the advantage that it's easier to see how to add more > partitions. You just split all the ranges and (and migrate the > data...). There's even the possibility of having uneven partitions if

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-03-14 Thread David Steele
On 3/4/17 12:43 AM, Robert Haas wrote: > On Wed, Mar 1, 2017 at 11:28 PM, Alvaro Herrera > wrote: >> Alvaro Herrera wrote: >>> Jinyu Zhang wrote: >>> Update the patch_brin_optimze_mem according to your comment. >>> >>> I have added this patch to the commitfest, which I've been intending to >>

[HACKERS] Re: [patch] reorder tablespaces in basebackup tar stream for backup_label

2017-03-14 Thread David Steele
On 3/4/17 2:20 AM, Michael Paquier wrote: > On Sat, Mar 4, 2017 at 1:13 AM, Bernd Helmle wrote: >> Ah right, i assumed there must be something, otherwise the comment >> won't be there ;) >> >> We could special case that part to distinguish fetch/stream mode, but i >> fear that leads to more confus

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-03-14 Thread Alvaro Herrera
David Steele wrote: > On 3/4/17 12:43 AM, Robert Haas wrote: > > I think we should regard this resubmission as untimely, unless there > > is an argument that it amounts to a bug fix. > > I agree and I'm also confused about which author this is waiting on. Is > it Jinyu or Álvaro? I don't think

Re: [HACKERS] wait events for disk I/O

2017-03-14 Thread Rahila Syed
Hello, I applied and tested this patch on latest sources and it works fine. Following are some comments, >+ /* Wait event for SNRU */ >+ WAIT_EVENT_READ_SLRU_PAGE, Typo in the comment. >FileWriteback(v->mdfd_vfd, seekpos, (off_t) BLCKSZ * nflush, WAIT_EVENT_FLUSH_DATA_BLOCK); This call is i

Re: [HACKERS] PATCH: pageinspect / add page_checksum and bt_page_items(bytea)

2017-03-14 Thread Ashutosh Sharma
On Mar 14, 2017 5:37 PM, "Alvaro Herrera" wrote: Ashutosh Sharma wrote: > Yes. But, as i said earlier I am getting negative checksum value for > page_header as well. Isn't that wrong. For eg. When I debug the > following query, i could pd_checksum value as '40074' in gdb where > page_header show

Re: [HACKERS] PATCH: Configurable file mode mask

2017-03-14 Thread David Steele
On 3/14/17 4:23 AM, Tsunakawa, Takayuki wrote: > From: David Steele [mailto:da...@pgmasters.net] 3.The default location of the SSL key file is $PGDATA, so the permission >> of the key file is likely to become 0640. But the current postgres requires >> it to be 0600. See src/backend/libpq/be-

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Heikki Linnakangas
On 03/14/2017 08:53 AM, Andres Freund wrote: Besides that, this version has: - pgindented most of the affected pieces (i.e. all significant new code has been reindent, not all touched one) I think you'll need to add all the inner structs ExprEvalStep typedefs.list to indent them right. My

[HACKERS] Bug: timestamp without time zone column allows timestamptz default

2017-03-14 Thread David Fetter
Folks, What happens now: CREATE TABLE bar(id int DEFAULT 'a'); ERROR: invalid input syntax for integer: "a" Good so far. Now with a different data type. CREATE TABLE bar(t TIMESTAMP WITHOUT TIME ZONE DEFAULT CURRENT_TIMESTAMP); CREATE TABLE This is a foot gun we need not have

Re: [HACKERS] Bug: timestamp without time zone column allows timestamptz default

2017-03-14 Thread Tom Lane
David Fetter writes: > CREATE TABLE bar(t TIMESTAMP WITHOUT TIME ZONE DEFAULT CURRENT_TIMESTAMP); > This is a foot gun we need not have. Yes, we do need to have it. (1) if we remove the implicit cast from timestamptz to timestamp, the villagers will be on our doorsteps with the usual equipm

Re: [HACKERS] Backend crash on non-exclusive backup cancel

2017-03-14 Thread Anastasia Lubennikova
As far as I understand, in this thread were discussed two bugs of pg_stop_backup(). Thanks to the clear descriptions above, I easily reproduced both of them. BUG#1: Server crashes on assertion on call of pg_stop_backup(false) after interrupted call of pg_stop_backup(false). TRAP: FailedAssertion

Re: [HACKERS] scram and \password

2017-03-14 Thread Joe Conway
On 03/14/2017 03:15 AM, Heikki Linnakangas wrote: > On 03/14/2017 04:47 AM, Tom Lane wrote: >> Robert Haas writes: >>> I'm not talking about changing the default, just having it be possible >>> to use \password with the new system as it was with the old, whatever >>> exactly we think that means. >

Re: [HACKERS] New CORRESPONDING clause design

2017-03-14 Thread Surafel Temesgen
hi Some errors are related to just CORRESPONDING without any columns. So using > expr doesn't help here. So parse node CORRESPONDING can solve both issues. > > In current implementation pointing to a node means pointing to a node’s first element so I don’t think we can be able to point to CORRESP

Re: [HACKERS] scram and \password

2017-03-14 Thread Tom Lane
Joe Conway writes: > On 03/14/2017 03:15 AM, Heikki Linnakangas wrote: >> If the server isn't set up to do SCRAM authentication, i.e. there are no >> "scram" entries in pg_hba.conf, and you set yourself a SCRAM verifier, >> you have just locked yourself out of the system. I think that's a >> non-s

Re: [HACKERS] PATCH: Configurable file mode mask

2017-03-14 Thread David Steele
On 3/13/17 3:03 PM, Tom Lane wrote: > David Steele writes: >> On 3/13/17 2:16 PM, Tom Lane wrote: >>> I also don't especially want to have to analyze cases like "what >>> happens if user initdb'd with mask X but then changes the GUC and >>> restarts the postmaster?". Maybe the right thing is to n

Re: [HACKERS] scram and \password

2017-03-14 Thread Joe Conway
On 03/14/2017 08:40 AM, Tom Lane wrote: > Joe Conway writes: >> On 03/14/2017 03:15 AM, Heikki Linnakangas wrote: >>> It would be a lot more sensible, if there was a way to specify in >>> pg_hba.conf, "scram-or-md5". We punted on that for PostgreSQL 10, but >>> perhaps we should try to cram that i

Re: [HACKERS] Should buffer of initialization fork have a BM_PERMANENT flag

2017-03-14 Thread Robert Haas
On Mon, Mar 13, 2017 at 8:51 PM, Michael Paquier wrote: >> I agree with that, but I propose the attached version instead. It >> seems cleaner to have the entire test for setting BM_PERMANENT in one >> place rather than splitting it up as you did. > > Fine for me. You may want to update the commen

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add test case for two phase commit. Also by Masahiko Sawada.

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 2:18 AM, Michael Paquier wrote: > On Tue, Mar 14, 2017 at 5:04 AM, Michael Meskes wrote: >> Add test case for two phase commit. Also by Masahiko Sawada. >> >> Branch >> -- >> master >> >> Details >> --- >> http://git.postgresql.org/pg/commitdiff/42fcad68a9c0e0ebecf

Re: [HACKERS] [COMMITTERS] pgsql: Fix cardinality estimates for parallel joins.

2017-03-14 Thread Robert Haas
On Tue, Jan 17, 2017 at 11:49 AM, Robert Haas wrote: > On Mon, Jan 16, 2017 at 7:23 AM, Amit Kapila wrote: >> On Sat, Jan 14, 2017 at 12:07 AM, Robert Haas wrote: >>> Fix cardinality estimates for parallel joins. >>> >> >> + /* >> +* In the case of a parallel plan, the row count ne

Re: [HACKERS] Gather Merge

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 9:56 AM, Tom Lane wrote: > Robert Haas writes: >> Cool, thanks for the review. I'm not quite confident that we've found >> all of the bugs here yet, but I think we're moving in the right >> direction. > > I guess the real question here is why isn't Gather Merge more like

Re: [HACKERS] Upgrading postmaster's log messages about bind/listen errors

2017-03-14 Thread Tom Lane
I wrote: >> I think that what would actually be of some use nowadays is a LOG-level >> message emitted if the wraparound *isn't* activated immediately at start. >> But otherwise, we should follow the rule that silence is golden. > Concretely, how about the attached? It preserves the original > "p

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Alvaro Herrera
After looking at how index_fetch_heap and heap_hot_search_buffer interact, I can't say I'm in love with the idea. I started thinking that we should not have index_fetch_heap release the buffer lock only to re-acquire it five lines later, so it should keep the buffer lock, do the recheck and only r

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Robert Haas
On Mon, Mar 13, 2017 at 11:48 PM, Amit Kapila wrote: > We didn't found any issue with the above testing. Great! I've committed the latest version of the patch, with some cosmetic changes. It would be astonishing if there weren't a bug or two left, but I think overall this is very solid work, an

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Andres Freund
Hi, On 2017-03-14 16:58:54 +0200, Heikki Linnakangas wrote: > On 03/14/2017 08:53 AM, Andres Freund wrote: > > Besides that, this version has: > > - pgindented most of the affected pieces (i.e. all significant new code > > has been reindent, not all touched one) > > I think you'll need to add

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Andres Freund
On 2017-03-14 08:28:02 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > > EEO_SWITCH(op->opcode) > > { > > EEO_CASE(EEO_DONE): > > goto out; > > Oh my. > > > which is a bit annoying. (the EEO_CASE is either a jump label or a case > > statement, depending on computed goto

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Tom Lane
Robert Haas writes: > Great! I've committed the latest version of the patch, with some > cosmetic changes. Woo hoo! That's been a bee in the bonnet for, um, decades. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Andres Freund
On 2017-03-14 08:44:24 -0300, Alvaro Herrera wrote: > Patch 0003 is huge. I suspect you mean 0004? If so - yes :(. I unfortunately don't think there's a useful way to split it in smaller chunks - I originally moved ops over one-by-one, but that required a lot of duplicated structs and such...

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 1:40 PM, Tom Lane wrote: > Robert Haas writes: >> Great! I've committed the latest version of the patch, with some >> cosmetic changes. > > Woo hoo! That's been a bee in the bonnet for, um, decades. Yeah. I'm pretty happy to see that go in. It's become pretty clear to

Re: [HACKERS] background sessions

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 3:31 AM, Pavel Stehule wrote: > Very often strategy can be recheck of parent process in some waiting > cycles. It should not to impact performance. I think that's going to be hard to arrange, and I think it isn't necessary. If the leader wants to arrange for the worker t

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Tom Lane
Robert Haas writes: > It's become pretty clear to me that there are a bunch of other things > about hash indexes which are not exactly great, the worst of which is > the way they grow by DOUBLING IN SIZE. Uh, what? Growth should happen one bucket-split at a time. > Other things that are not so

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Tom Lane
Andres Freund writes: > On 2017-03-14 08:44:24 -0300, Alvaro Herrera wrote: >> It would be good to have someone at least read it before pushing, but >> I don't think anyone other than you has done so. > I'd love for somebody else > to look through it, I tried asking multiple times... Seems like

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Heikki Linnakangas
On 03/14/2017 07:31 PM, Andres Freund wrote: On 2017-03-14 16:58:54 +0200, Heikki Linnakangas wrote: * How tight are we on space in the ExprEvalStep union? Currently, the jump-threading preparation replaces the opcodes with the goto labels, but it would be really nice to keep the original opcode

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 2:14 PM, Tom Lane wrote: > Robert Haas writes: >> It's become pretty clear to me that there are a bunch of other things >> about hash indexes which are not exactly great, the worst of which is >> the way they grow by DOUBLING IN SIZE. > > Uh, what? Growth should happen on

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 14, 2017 at 2:14 PM, Tom Lane wrote: >> Robert Haas writes: >>> It's become pretty clear to me that there are a bunch of other things >>> about hash indexes which are not exactly great, the worst of which is >>> the way they grow by DOUBLING IN SIZE. >> Uh, wha

Re: [HACKERS] Parallel seq. plan is not coming against inheritance or partition table

2017-03-14 Thread Robert Haas
On Mon, Mar 13, 2017 at 10:18 AM, Amit Kapila wrote: > Agreed, so I have rebased your patch and passed heap_pages as -1 for > index_only scans as discussed. Also, Rafia has tested with attached > patch that parallel index and parallel index only scans are picked for > TPC-H queries. I have also

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Petr Jelinek
On 10/03/17 20:02, Peter Eisentraut wrote: > On 2/27/17 22:10, Stephen Frost wrote: >> Peter, >> >> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: >>> On 2/18/17 18:06, Stephen Frost wrote: I'm not convinced that it really makes sense to have PUBLICATION of a table be indepe

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Pavan Deolasee
On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera wrote: > > @@ -234,6 +236,21 @@ index_beginscan(Relation heapRelation, > > scan->heapRelation = heapRelation; > > scan->xs_snapshot = snapshot; > > > > + /* > > + * If the index supports recheck, make sure that index tuple is >

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 2:41 PM, Petr Jelinek wrote: > My understanding of what Shephen is proposing is, you have "ownerA" of > tableA and "ownerB" of tableB, then you want role "publishe"r to be able > to publish those, so you simply grant it the "ownerA" and "ownerB" > roles. Obviously that migh

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Petr Jelinek
On 14/03/17 19:47, Robert Haas wrote: > On Tue, Mar 14, 2017 at 2:41 PM, Petr Jelinek > wrote: >> My understanding of what Shephen is proposing is, you have "ownerA" of >> tableA and "ownerB" of tableB, then you want role "publishe"r to be able >> to publish those, so you simply grant it the "owne

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-14 Thread Andres Freund
Hi, On 2017-03-14 20:28:51 +0200, Heikki Linnakangas wrote: > On 03/14/2017 07:31 PM, Andres Freund wrote: > > On 2017-03-14 16:58:54 +0200, Heikki Linnakangas wrote: > > > * How tight are we on space in the ExprEvalStep union? Currently, the > > > jump-threading preparation replaces the opcodes w

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Petr Jelinek
On 14/03/17 19:49, Petr Jelinek wrote: > On 14/03/17 19:47, Robert Haas wrote: >> On Tue, Mar 14, 2017 at 2:41 PM, Petr Jelinek >> wrote: >>> My understanding of what Shephen is proposing is, you have "ownerA" of >>> tableA and "ownerB" of tableB, then you want role "publishe"r to be able >>> to p

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > On Tue, Mar 14, 2017 at 2:14 PM, Tom Lane wrote: > >> Robert Haas writes: > >>> It's become pretty clear to me that there are a bunch of other things > >>> about hash indexes which are not exactly great, the worst of which is > >>>

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-14 Thread Robert Haas
On Mon, Mar 13, 2017 at 10:36 AM, Ashutosh Sharma wrote: > Couple of review comments,, > > You may also need to update the documentation as now we are also going > to support wal consistency check for hash index. The current > documentation does not include hash index. > > +only records or

Re: [HACKERS] Authentication tests, and plain 'password' authentication with a SCRAM verifier

2017-03-14 Thread Jeff Janes
On Tue, Mar 14, 2017 at 5:36 AM, Heikki Linnakangas wrote: > Hi, > > I didn't include the authentication TAP tests that Michael wrote in the > main SCRAM commit last week. The main issue was that the new test was > tacked on the src/test/recovery test suite, for lack of a better place. I > propos

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Stephen Frost
Greetings, * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: > On 14/03/17 19:47, Robert Haas wrote: > > On Tue, Mar 14, 2017 at 2:41 PM, Petr Jelinek > > wrote: > >> My understanding of what Shephen is proposing is, you have "ownerA" of > >> tableA and "ownerB" of tableB, then you want role "

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 2:56 PM, Petr Jelinek wrote: > Note that I am not necessarily saying it's better though, just trying to > explain. It definitely has drawbacks, as in order to grant publish on > one table you might be granting lots of privileges on various objects by > granting the role. So

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> It's true that as soon as we need another overflow page, that's going to >> get dropped beyond the 2^{N+1}-1 point, and the *apparent* size of the >> index will grow quite a lot. But any modern filesystem should handle >> that with

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > However, what I'm not clear about is whether this is a situation > that's likely to come up much in practice. I would have thought that > publications and subscriptions would typically be configured by roles > with quite high levels of pri

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera > wrote: > > I have already commented about the executor involvement in btrecheck(); > > that doesn't seem good. I previously suggested to pass the EState down > > from caller, but that's not a great idea either since you st

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Alvaro Herrera
Pavan Deolasee wrote: > BTW I wanted to share some more numbers from a recent performance test. I > thought it's important because the latest patch has fully functional chain > conversion code as well as all WAL-logging related pieces are in place > too. I ran these tests on a box borrowed from To

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> It's true that as soon as we need another overflow page, that's going to > >> get dropped beyond the 2^{N+1}-1 point, and the *apparent* size of the > >> index will grow quite a lot. But

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Pavan Deolasee
On Tue, Mar 14, 2017 at 7:19 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > > BTW I wanted to share some more numbers from a recent performance test. I > > thought it's important because the latest patch has fully functional > chain > > conversion code as well as all WAL-logging related pi

Re: [HACKERS] Authentication tests, and plain 'password' authentication with a SCRAM verifier

2017-03-14 Thread Heikki Linnakangas
On 03/14/2017 09:02 PM, Jeff Janes wrote: It is somewhat disconcerting that the client will send a plain-text password to a mis-configured (or mal-configured) server, but I don't think there is anything this patch series can reasonably do about that. Yeah. That's one pretty glaring hole with li

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Petr Jelinek
On 14/03/17 20:09, Robert Haas wrote: > On Tue, Mar 14, 2017 at 2:56 PM, Petr Jelinek > wrote: >> Note that I am not necessarily saying it's better though, just trying to >> explain. It definitely has drawbacks, as in order to grant publish on >> one table you might be granting lots of privileges

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-14 Thread Mark Kirkwood
On 15/03/17 06:29, Robert Haas wrote: Great! I've committed the latest version of the patch, with some cosmetic changes. It would be astonishing if there weren't a bug or two left, but I think overall this is very solid work, and I think it's time to put this out there and see how things go.

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 8:02 AM, Ashutosh Sharma wrote: > Attached is the v6 patch for microvacuum in hash index rebased on top > of 'v10 patch for WAL in hash index - [1]' and 'v1 patch for WAL > consistency check for hash index - [2]'. > > [1] - > https://www.postgresql.org/message-id/CAA4eK1%2

Re: [HACKERS] logical replication access control patches

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 3:37 PM, Petr Jelinek wrote: > On 14/03/17 20:09, Robert Haas wrote: >> On Tue, Mar 14, 2017 at 2:56 PM, Petr Jelinek >> wrote: >>> Note that I am not necessarily saying it's better though, just trying to >>> explain. It definitely has drawbacks, as in order to grant publi

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-14 Thread Robert Haas
On Tue, Mar 14, 2017 at 7:22 AM, Kuntal Ghosh wrote: > I do have extended localBackendStatusTable with slots for non-backend > processes. But, I've renamed it as localProcStatusTable since it > includes all processes. I'll keep the variable name as > localBackendStatusTable in the updated patch to

Re: [HACKERS] [COMMITTERS] pgsql: Add amcheck extension to contrib.

2017-03-14 Thread Andres Freund
On 2017-03-13 14:09:39 -0700, Andres Freund wrote: > Hi, > > On 2017-03-13 15:45:01 -0400, Tom Lane wrote: > > I could be wrong, but the most obvious explanation for this failure is > > that autovacuum had a lock on the table or index when we looked. > > Even if that isn't why axolotl failed in th

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-14 Thread Peter Geoghegan
On Tue, Mar 14, 2017 at 12:19 PM, Alvaro Herrera wrote: > Impressive results. Agreed. It seems like an important invariant for WARM is that any duplicate index values ought to have different TIDs (actually, it's a bit stricter than that, since btrecheck() cares about simple binary equality). IST

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-14 Thread Andres Freund
On 2017-03-13 00:35:06 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-03-11 22:14:07 -0500, Tom Lane wrote: > >> This looks generally sane to me, although I'm not very happy about folding > >> the "$(MKDIR_P) output_iso" call into pg_isolation_regress_check --- that > >> seems weird an

Re: [HACKERS] scram and \password

2017-03-14 Thread Jeff Janes
On Tue, Mar 14, 2017 at 3:15 AM, Heikki Linnakangas wrote: > On 03/14/2017 04:47 AM, Tom Lane wrote: > >> Robert Haas writes: >> >>> I'm not talking about changing the default, just having it be possible >>> to use \password with the new system as it was with the old, whatever >>> exactly we thi

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-14 Thread Kevin Grittner
On Tue, Mar 14, 2017 at 6:00 AM, DEV_OPS wrote: > On 3/14/17 17:34, Mengxing Liu wrote: > The worst problems have been > seen with 32 or more cores on 4 or more sockets with a large number > of active connections. I don't know whether you have access to a > machine capable of put

Re: [HACKERS] scram and \password

2017-03-14 Thread Jeff Janes
On Tue, Mar 14, 2017 at 8:40 AM, Tom Lane wrote: > Joe Conway writes: > > > I was also thinking about that. Basically a primary method and a > > fallback. If that were the case, a gradual transition could happen, and > > if we want \password to enforce best practice it would be ok. > > Why exact

Re: [HACKERS] Index usage for elem-contained-by-const-range clauses

2017-03-14 Thread Tom Lane
Pritam Baral writes: > The topic has been previously discussed[0] on the -performance mailing list, > about four years ago. > In that thread, Tom suggested[0] the planner could be made to "expand > "intcol <@ > 'x,y'::int4range" into "intcol between x and y", using something similar > to the > ind

Re: [HACKERS] background sessions

2017-03-14 Thread Pavel Stehule
2017-03-14 19:08 GMT+01:00 Robert Haas : > On Tue, Mar 14, 2017 at 3:31 AM, Pavel Stehule > wrote: > > Very often strategy can be recheck of parent process in some waiting > > cycles. It should not to impact performance. > > I think that's going to be hard to arrange, and I think it isn't > nece

[HACKERS] improve comments of snapbuild.c

2017-03-14 Thread Erik Rijkers
Improvements (grammar/typos) in the comments in snapbuild.c To be applied to master. thanks, Erik Rijkers --- src/backend/replication/logical/snapbuild.c.orig 2017-03-14 21:53:42.590196415 +0100 +++ src/backend/replication/logical/snapbuild.c 2017-03-14 21:57:57.906539208 +0100 @@ -34,7 +34,7

Re: [HACKERS] scram and \password

2017-03-14 Thread Tom Lane
Jeff Janes writes: > On Tue, Mar 14, 2017 at 8:40 AM, Tom Lane wrote: >> Why exactly would anyone want "md5 only"? I should think that "scram >> only" is a sensible pg_hba setting, if the DBA feels that md5 is too >> insecure, but I do not see the point of "md5 only" in 2017. I think >> we shou

  1   2   >