Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Andres Freund
Hi, On 2017-02-15 08:48:44 -0500, Robert Haas wrote: > We got rid of the major existing use of page locks in > 6d46f4783efe457f74816a75173eb23ed8930020, which extirpated them from > hash indexes, and I was kind of hoping they could go away altogether, > but we can't do that as long as GIN is using

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Stephen Frost
All, * Andres Freund (and...@anarazel.de) wrote: > On 2017-02-15 08:48:44 -0500, Robert Haas wrote: > > The other way of fixing this problem is to have each worker generate a > > subset of the tuples and funnel them all back to the leader through a > > Gather node; the leader then does all the ins

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Andres Freund
On February 15, 2017 5:20:20 PM PST, Stephen Frost wrote: >In many cases, I expect this would work just as well, if not better, >than trying to actually do writes in parallel. Why? IPCing tuples around is quite expensive. Or do you mean because it'll be more suitable because of the possible p

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > On February 15, 2017 5:20:20 PM PST, Stephen Frost wrote: > >In many cases, I expect this would work just as well, if not better, > >than trying to actually do writes in parallel. > > Why? IPCing tuples around is quite expensive. Or do you m

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Andres Freund
On 2017-02-15 20:28:43 -0500, Stephen Frost wrote: > Andres, > > * Andres Freund (and...@anarazel.de) wrote: > > On February 15, 2017 5:20:20 PM PST, Stephen Frost > > wrote: > > >In many cases, I expect this would work just as well, if not better, > > >than trying to actually do writes in paral

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2017-02-15 20:28:43 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > On February 15, 2017 5:20:20 PM PST, Stephen Frost > > > wrote: > > > >In many cases, I expect this would work just as well, if not bette

Re: [HACKERS] CREATE TABLE with parallel workers, 10.0?

2017-02-15 Thread Andres Freund
On 2017-02-15 20:35:16 -0500, Stephen Frost wrote: > Perhaps, but until we've got a system worked out for having the workers > do the writes, we aren't getting anything. Being able to have the > leader do the writes based on the tuples fed back from the workers is > clearly better than nothing. N

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-02-15 Thread Robert Haas
On Mon, Feb 13, 2017 at 10:22 AM, Amit Kapila wrote: > As discussed, attached are refactoring patches and a patch to enable > WAL for the hash index on top of them. Thanks. I think that the refactoring patches shouldn't add START_CRIT_SECTION() and END_CRIT_SECTION() calls; let's leave that for

Re: [HACKERS] increasing the default WAL segment size

2017-02-15 Thread Alvaro Herrera
Now that we've renamed "xlog" to "wal" in user-facing elements, I think we should strive to use the name "wal" internally too in new code, not "xlog" anymore. This patch introduces several variables, macros, functions that ought to change names now -- XLogSegmentOffset should be WALSegmentOffset f

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Amit Langote
On 2017/02/15 13:31, Robert Haas wrote: > On Mon, Feb 13, 2017 at 5:57 AM, Amit Langote > wrote: >> On 2017/02/13 14:21, Amit Langote wrote: >>> On 2017/02/10 19:19, Simon Riggs wrote: * The OID inheritance needs work - you shouldn't need to specify a partition needs OIDS if the parent h

Re: [HACKERS] increasing the default WAL segment size

2017-02-15 Thread Robert Haas
On Wed, Feb 15, 2017 at 8:46 PM, Alvaro Herrera wrote: > Now that we've renamed "xlog" to "wal" in user-facing elements, I think > we should strive to use the name "wal" internally too in new code, not > "xlog" anymore. This patch introduces several variables, macros, > functions that ought to ch

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Haribabu Kommi
On Wed, Feb 15, 2017 at 11:35 PM, Amit Kapila wrote: > On Wed, Feb 15, 2017 at 12:03 PM, Seki, Eiji > wrote: > > Amit Kapila wrote: > >> How will you decide just based on oldest xmin whether the tuple is > visible or not? How will you take decisions about tuples which have xmax > set? > > > > I

Re: [HACKERS] [Bug fix] PQsendQuery occurs error when target_session_attrs is set to read-write

2017-02-15 Thread Michael Paquier
On Thu, Feb 16, 2017 at 1:08 AM, Robert Haas wrote: > Great, thanks! This looks good to me, so committed. Thanks. > Is there any > chance you can use your amazing TAP-test-creation powers to do some > automated testing of this feature? For example, maybe we could at > least set up a master and

Re: [HACKERS] increasing the default WAL segment size

2017-02-15 Thread Andres Freund
On 2017-02-15 22:46:38 -0300, Alvaro Herrera wrote: > Now that we've renamed "xlog" to "wal" in user-facing elements, I think > we should strive to use the name "wal" internally too in new code, not > "xlog" anymore. This patch introduces several variables, macros, > functions that ought to change

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-02-15 Thread Thomas Munro
On Sat, Feb 4, 2017 at 2:45 PM, Peter Geoghegan wrote: > It might just have been that the table was too small to be an > effective target for parallel sequential scan with so many workers, > and so a presorted best case CREATE INDEX, which isn't that different, > also fails to see much benefit (co

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Michael Paquier
On Thu, Feb 16, 2017 at 10:48 AM, Haribabu Kommi wrote: > Because of the above reason, we need a new API or some change in API > to provide the Oldest xmin by ignoring the ANALYZE transactions, so that > it will reduce the size of WOS and improves the VCI query performance. A new API in core is n

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-02-15 Thread Andres Freund
Hi, On 2017-02-13 23:57:00 +1300, Thomas Munro wrote: > Here's a new version to fix the problems reported by Rafia above. The > patch descriptions are as before but it starts from 0002 because 0001 > was committed as 7c5d8c16 (thanks, Andres). FWIW, I'd appreciate if you'd added a short commit m

Re: [HACKERS] DROP SUBSCRIPTION and ROLLBACK

2017-02-15 Thread Masahiko Sawada
On Thu, Feb 16, 2017 at 7:52 AM, Petr Jelinek wrote: > On 15/02/17 06:43, Masahiko Sawada wrote: >> On Tue, Feb 14, 2017 at 1:13 AM, Fujii Masao wrote: >>> On Mon, Feb 13, 2017 at 4:05 PM, Masahiko Sawada >>> wrote: On Sat, Feb 11, 2017 at 8:18 PM, Petr Jelinek wrote: > On 10/02/

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-02-15 Thread Andres Freund
Hi, Just to summarize what you could read between the lines in the previous mail: From a higher level POV the design here makes sense to me, I do however think there's a good chunk of code-level improvements needed. Regards, Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Parallel Index Scans

2017-02-15 Thread Amit Kapila
On Thu, Feb 16, 2017 at 12:27 AM, Robert Haas wrote: > On Wed, Feb 15, 2017 at 1:39 PM, Robert Haas wrote: >> On Wed, Feb 15, 2017 at 7:11 AM, Amit Kapila >> wrote:> >>> support related patch. In anycase, to avoid confusion I am attaching >>> all the three patches with this e-mail. >> >> Commi

Re: [HACKERS] SCRAM authentication, take three

2017-02-15 Thread Michael Paquier
On Wed, Feb 15, 2017 at 8:28 PM, Heikki Linnakangas wrote: > Ah, found it. It was because of this change: > > Having two error codes with the same SQLSTATE is not cool, and tripped the > assertion in PL/python. I removed the new error code, it was only used in > one place, and ERRCODE_PROTOCOL_VIO

Re: [HACKERS] Parallel Append implementation

2017-02-15 Thread Ashutosh Bapat
On Wed, Feb 15, 2017 at 6:40 PM, Robert Haas wrote: > On Wed, Feb 15, 2017 at 4:43 AM, Amit Khandekar > wrote: >>> On 14 February 2017 at 22:35, Robert Haas wrote: For example, suppose that I have a scan of two children, one of which has parallel_workers of 4, and the other of which h

[HACKERS] case_preservation_and_insensitivity = on

2017-02-15 Thread Joel Jacobson
Hi hackers, "A system that is not case-preserving is necessarily case-insensitive, but it is possible and common for a system to be case-insensitive, yet case-preserving" [1] Imagine if you could turn on a GUC that would turn PostgreSQL into such a system, where the case would be preserved by def

Re: [HACKERS] ICU integration

2017-02-15 Thread Peter Eisentraut
On 1/31/17 16:59, Thomas Munro wrote: > I assume (but haven't checked) that ucol_nextSortKeyPart accesses only > the start of the string via the UCharIterator passed in, unless you > have the rare reverse-accent-sort feature enabled (maybe used only in > fr_CA, it looks like it is required to scan

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-02-15 Thread Seki, Eiji
Simon Riggs wrote: > Please persuade us with measurements that allowing this impact on > ANALYZE would really improve performance at least in your case, and > also examine the effect of this on the accuracy and usefulness of the > gathered statistics. I explain results of the test that Haribabu m

Re: [HACKERS] increasing the default WAL segment size

2017-02-15 Thread Tom Lane
Andres Freund writes: > On 2017-02-15 22:46:38 -0300, Alvaro Herrera wrote: >> Now that we've renamed "xlog" to "wal" in user-facing elements, I think >> we should strive to use the name "wal" internally too in new code, not >> "xlog" anymore. This patch introduces several variables, macros, >> f

Re: [HACKERS] case_preservation_and_insensitivity = on

2017-02-15 Thread Tom Lane
Joel Jacobson writes: > Case Preservation + Case Insensitivity = A good combination > Thoughts? Have you read any of our innumerable previous discussions about this? The last one was barely a month ago, cf https://www.postgresql.org/message-id/flat/ACF85C502E55A143AB9F4ECFE960660A17282D%40mailser

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-15 Thread Karl O. Pinc
On Wed, 15 Feb 2017 15:23:00 -0500 Robert Haas wrote: > +ereport(WARNING, > +(errcode(ERRCODE_INTERNAL_ERROR), > + errmsg("corrupted data found in \"%s\"", > +LOG_METAINFO_DATAFILE))); > > elog seems fine here. Ther

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-02-15 Thread Tom Lane
"Karl O. Pinc" writes: > On a different topic, I pulled from master and now > I see that git finds the following untracked: > src/bin/pg_basebackup/pg_receivexlog > src/bin/pg_resetxlog/ > src/bin/pg_xlogdump/ Those got renamed, cf https://git.postgresql.org/gitweb/?p=postgresql

Re: [HACKERS] ICU integration

2017-02-15 Thread Peter Geoghegan
On Wed, Feb 15, 2017 at 9:17 PM, Peter Eisentraut wrote: > I have changed it to use ucol_nextSortKeyPart() when appropriate. I think it would be fine if the second last argument was "sizeof(Datum)", not "Min(sizeof(Datum), sss->buflen2)" here: > + if (GetDatabaseEncoding() == PG_UT

Re: [HACKERS] Parallel Append implementation

2017-02-15 Thread Amit Khandekar
On 15 February 2017 at 18:40, Robert Haas wrote: > On Wed, Feb 15, 2017 at 4:43 AM, Amit Khandekar > wrote: >>> On 14 February 2017 at 22:35, Robert Haas wrote: For example, suppose that I have a scan of two children, one of which has parallel_workers of 4, and the other of which has

Re: [HACKERS] ICU integration

2017-02-15 Thread Peter Geoghegan
On Wed, Feb 15, 2017 at 9:17 PM, Peter Eisentraut wrote: >> Clearly when you upgrade your system from (say) Debian 8 to Debian 9 >> and the ICU major version changes we expect to have to REINDEX, but >> does anyone know if such data changes are ever pulled into the minor >> version package upgrade

Re: [HACKERS] UPDATE of partition key

2017-02-15 Thread Amit Khandekar
On 15 February 2017 at 20:26, David Fetter wrote: > When an UPDATE can't happen, there are often ways to hint at > what went wrong and how to correct it. Violating a uniqueness > constraint would be one example. > > When an UPDATE can't happen and the depth of the subtree is a > plausible candida

Re: [HACKERS] Documentation improvements for partitioning

2017-02-15 Thread Amit Langote
On 2017/02/16 2:08, Robert Haas wrote: > On Wed, Feb 15, 2017 at 11:34 AM, Alvaro Herrera > wrote: >> I think new-style partitioning is supposed to consider each partition as >> an implementation detail of the table; the fact that you can manipulate >> partitions separately does not really mean th

Re: [HACKERS] UPDATE of partition key

2017-02-15 Thread Amit Langote
On 2017/02/16 15:50, Amit Khandekar wrote: > On 15 February 2017 at 20:26, David Fetter wrote: >> When an UPDATE can't happen, there are often ways to hint at >> what went wrong and how to correct it. Violating a uniqueness >> constraint would be one example. >> >> When an UPDATE can't happen and

Re: [HACKERS] Parallel Index-only scan

2017-02-15 Thread Rahila Syed
I reviewed the patch. Overall it looks fine to me. One comment, >- if (index->amcanparallel && >- !index_only_scan && >+ if ((index->amcanparallel || >+ index_only_scan) && Why do we need to check for index_only_scan in the above condition. IIUC, index->amcanpara

<    1   2