Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 4:31 PM, Vladimir Borodin wrote: > 31 янв. 2017 г., в 9:50, Michael Paquier > написал(а): > >> I am marking this patch as returned with feedback. > > Michael, sorry, but why? Because I have been through many patches today. > If I understood everything right, the main que

Re: [HACKERS] Push down more full joins in postgres_fdw

2017-01-31 Thread Etsuro Fujita
On 2017/01/30 21:05, Ashutosh Bapat wrote: On Mon, Jan 30, 2017 at 5:00 PM, Etsuro Fujita wrote: On 2017/01/27 21:25, Etsuro Fujita wrote: Sorry, I started thinking we went in the wrong direction. I added to deparseSelectStmtForRel build_subquery_tlists, which creates a tlist for each subquer

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Craig Ringer
On 31 Jan. 2017 19:29, "Michael Paquier" wrote: On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer wrote: > Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when > wal_level >= logical I don't think that's the end of the world. But > since we already have almost everything we need in

[HACKERS] An issue in remote query optimization

2017-01-31 Thread Abbas Butt
Hi, Postgres_fdw optimizes remote queries by pushing down the where clause. This feature does not work consistently when the query is executed from within a pl/pgsql function. The optimization works when the function executes the query for the first 5 times, and fails afterwards. Example: Step 1:

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Konstantin Knizhnik
On 31.01.2017 09:29, Michael Paquier wrote: On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer wrote: Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when wal_level >= logical I don't think that's the end of the world. But since we already have almost everything we need in memory

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-01-31 Thread Kyotaro HORIGUCHI
Hello, thank you for the revised patch. The only comment from me is about comments on the new over*2D funnctions. At Mon, 30 Jan 2017 21:12:31 +0300, Nikita Glukhov wrote in <4450e7a6-01e7-0fb2-a01e-98fb5405d...@postgrespro.ru> > On 30.01.2017 12:04, Kyotaro HORIGUCHI wrote: > > Hello, > > > >

Re: [HACKERS] Radix tree for character conversion

2017-01-31 Thread Kyotaro HORIGUCHI
At Tue, 31 Jan 2017 12:25:46 +0900, Michael Paquier wrote in > On Mon, Jan 30, 2017 at 3:37 PM, Kyotaro HORIGUCHI > wrote: > > Hello, this is the revised version of character conversion using radix tree. > > Thanks for the new version, I'll look at it once I am done with the > cleanup of the c

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2017-01-31 Thread Kyotaro HORIGUCHI
At Tue, 31 Jan 2017 15:07:55 +0900, Michael Paquier wrote in > On Tue, Jan 31, 2017 at 2:58 PM, Pavel Stehule > wrote: > > I found a error - I sent mail only to author 2016-12-31 :( - It is my > > mistake. I am sorry > > Ah... Thanks for the update. No problem. Ouch. Sorry for missing you co

Re: [HACKERS] patch: function xmltable

2017-01-31 Thread Pavel Stehule
2017-01-24 21:38 GMT+01:00 Alvaro Herrera : > Pavel Stehule wrote: > > > * SELECT (xmltable(..)).* + regress tests > > * compilation and regress tests without --with-libxml > > Thanks. I just realized that this is doing more work than necessary -- > I think it would be simpler to have tableexpr f

Re: [HACKERS] IF (NOT) EXISTS in psql-completion

2017-01-31 Thread Pavel Stehule
2017-01-31 11:10 GMT+01:00 Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp>: > At Tue, 31 Jan 2017 15:07:55 +0900, Michael Paquier < > michael.paqu...@gmail.com> wrote in 8v0X6A4gKQb2Uc=mc+...@mail.gmail.com> > > On Tue, Jan 31, 2017 at 2:58 PM, Pavel Stehule > wrote: > > > I found a error -

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Etsuro Fujita
On 2017/01/31 18:24, Abbas Butt wrote: Postgres_fdw optimizes remote queries by pushing down the where clause. This feature does not work consistently when the query is executed from within a pl/pgsql function. The optimization works when the function executes the query for the first 5 times, and

Re: [HACKERS] parallelize queries containing initplans

2017-01-31 Thread Amit Kapila
On Wed, Dec 28, 2016 at 5:20 PM, Amit Kapila wrote: > To start > with let us see the plan of TPC-H query (Q-22) and understand how it > can be improved. > > Limit >InitPlan 1 (returns $0) > -> Finalize Aggregate >-> Gather > Workers Planned: 2 >

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Abbas Butt
On Tue, Jan 31, 2017 at 2:25 AM, Etsuro Fujita wrote: > On 2017/01/31 18:24, Abbas Butt wrote: > >> Postgres_fdw optimizes remote queries by pushing down the where clause. >> This feature does not work consistently when the query is executed from >> within a pl/pgsql function. The optimization wo

Re: [HACKERS] parallelize queries containing subplans

2017-01-31 Thread Amit Kapila
On Tue, Jan 24, 2017 at 10:59 AM, Amit Kapila wrote: > On Thu, Jan 19, 2017 at 4:51 PM, Dilip Kumar wrote: >> On Thu, Jan 19, 2017 at 3:05 PM, Dilip Kumar wrote: >>> During debugging I found that subplan created for below part of the >>> query is parallel_unsafe, Is it a problem or there is some

Re: [HACKERS] Parallel bitmap heap scan

2017-01-31 Thread Dilip Kumar
On Tue, Jan 31, 2017 at 7:47 AM, Haribabu Kommi wrote: > Thanks for the update. I have some comments > Thanks for the review. > > 0002-hash-support-alloc-free-v14.patch: > > > + if (tb->alloc) > + { > > The memory for tb->alloc is allocated always, is the if check still > required? In parallel c

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Etsuro Fujita
On 2017/01/31 19:53, Abbas Butt wrote: On Tue, Jan 31, 2017 at 2:25 AM, Etsuro Fujita mailto:fujita.ets...@lab.ntt.co.jp>> wrote: On 2017/01/31 18:24, Abbas Butt wrote: Postgres_fdw optimizes remote queries by pushing down the where clause. This feature does not wor

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-01-31 Thread Nikita Glukhov
On 31.01.2017 13:04, Kyotaro HORIGUCHI wrote: The following comment, /* Can any range from range_box to be overlower than this argument? */ This might be better to be using the same wording to its users, for example the comment for overLeft4D is the following. | /* Can any rectangle from rec

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Abbas Butt
On Tue, Jan 31, 2017 at 3:15 AM, Etsuro Fujita wrote: > On 2017/01/31 19:53, Abbas Butt wrote: > >> On Tue, Jan 31, 2017 at 2:25 AM, Etsuro Fujita >> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >> On 2017/01/31 18:24, Abbas Butt wrote: >> > > Postgres_fdw optimizes remote queries by p

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Daniel Verite
Tom Lane wrote: > Moreover, the committed patch is inconsistent in that it forbids > only one of the above. Why is it okay to treat unset as "off", > but not okay to treat the default empty-string value as "on"? Treating unset (NULL in the value) as "off" comes from the fact that except

[HACKERS] [Bug fix] PQsendQuery occurs error when target_session_attrs is set to read-write

2017-01-31 Thread Higuchi, Daisuke
Hello, This this is my first posting to the mailing list. I am interested in multiple hosts of libpq [1], then I found the bug in this feature. When I set "target_session_attrs" to "any" and call PQsendQuery, my application is succeeded. However, when I set "target_session_attrs" to "read-writ

Re: [HACKERS] One-shot expanded output in psql using \G

2017-01-31 Thread Daniel Verite
Stephen Frost wrote: > That's not how '\dx' works, as I pointed out, so I don't see having the > second character being 'x' to imply "\x mode" makes sense. \gx means "like \g but output with expanded display" It turns out that it's semantically close to "\g with \x" so I refered to it lik

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Rahila Syed
Hello, >Agreed, that it makes sense to consider only the number of pages to >scan for computation of parallel workers. I think for index scan we >should consider both index and heap pages that need to be scanned >(costing of index scan consider both index and heap pages). I thin >where consider

Re: [HACKERS] macaddr 64 bit (EUI-64) datatype support

2017-01-31 Thread Haribabu Kommi
On Tue, Jan 31, 2017 at 4:13 PM, Vitaly Burovoy wrote: > On 1/27/17, Haribabu Kommi wrote: > > Updated patches are attached. > > > Hello, > > I'm almost ready to mark it as Ready for committer. > The final round. Thanks for the review. > 1. > >+DATA(insert OID = 774 ( macaddr8 ... > >+#defin

[HACKERS] Logical Replication and Character encoding

2017-01-31 Thread Shinoda, Noriyoshi
Hi hackers, I tried a committed Logical Replication environment. I found that replication between databases of different encodings did not convert encodings in character type columns. Is this behavior correct? I expected that the character 0xe6bca2 (UTF-8) would be converted to the same chara

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Ashutosh Bapat
On Tue, Jan 31, 2017 at 5:23 PM, Abbas Butt wrote: > > > On Tue, Jan 31, 2017 at 3:15 AM, Etsuro Fujita > wrote: >> >> On 2017/01/31 19:53, Abbas Butt wrote: >>> >>> On Tue, Jan 31, 2017 at 2:25 AM, Etsuro Fujita >>> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >>> On 2017/01/31 18:24, Abbas B

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-01-31 Thread Ashutosh Bapat
> > 0003-hj-refactor-memory-accounting-v4.patch: > > Modify the existing hash join code to work in terms of chunks when > estimating and later tracking memory usage. This is probably more > accurate than the current tuple-based approach, because it tries to > take into account the space used by ch

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-01-31 Thread Konstantin Knizhnik
On 30.01.2017 19:21, Heikki Linnakangas wrote: On 01/24/2017 04:47 PM, Konstantin Knizhnik wrote: Interesting.. What should happen here is that for the backend's own insertion slot, the "insertingat" value should be greater than the requested flush point ('upto' variable). That's because befor

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Abbas Butt
Sorry for the confusion. ANALYZE works for the foreign table 'foreign_numbers'. test=# analyze foreign_numbers; ANALYZE test=# On Tue, Jan 31, 2017 at 5:04 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Jan 31, 2017 at 5:23 PM, Abbas Butt > wrote: > > > > > > On Tue, Ja

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Thu, Jan 26, 2017 at 2:38 AM, Alvaro Herrera > wrote: > > The simple_heap_update + CatalogUpdateIndexes pattern is getting > > obnoxious. How about creating something like catalog_heap_update which > > does both things at once, and stop bothering each callsite with the

Re: [HACKERS] patch: function xmltable

2017-01-31 Thread Alvaro Herrera
Pavel Stehule wrote: > 2017-01-24 21:38 GMT+01:00 Alvaro Herrera : > > I think it would be good to have a more complex test case in regress -- > > let's say there is a table with some simple XML values, then we use > > XMLFOREST (or maybe one of the table_to_xml functions) to generate a > > large

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Pavan Deolasee
On Tue, Jan 31, 2017 at 7:21 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > On Thu, Jan 26, 2017 at 2:38 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > The simple_heap_update + CatalogUpdateIndexes pattern is getting > > > obnoxious. How about creating something like

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Tue, Jan 31, 2017 at 7:21 PM, Alvaro Herrera > wrote: > > CatalogUpdateIndexes was just added as a convenience function on top of > > a very common pattern. If we now have a reason to create a second one > > because there are now two very common patterns, it seems reas

Re: [HACKERS] Refactoring of replication commands using printsimple

2017-01-31 Thread Robert Haas
On Tue, Jan 31, 2017 at 12:19 AM, Michael Paquier wrote: > This is a follow up of the refactoring that has been discussed in the > thread to increase the default size of WAL segments: > https://www.postgresql.org/message-id/cab7npqq4hynrlq+w1jrryvsysoxuqa40pyb2uw5uqkkag4h...@mail.gmail.com > > The

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-01-31 Thread Robert Haas
On Tue, Jan 31, 2017 at 12:15 AM, Peter Geoghegan wrote: >> Should this 64KB minimum be mentioned in the documentation? > > You mean user-visible documentation, and not just tuplesort.h? I don't > think that that's necessary. That's a ludicrously low amount of memory > for a worker to be limited t

Re: [HACKERS] Deadlock in XLogInsert at AIX

2017-01-31 Thread Konstantin Knizhnik
One more assertion failure: ExceptionalCondition(conditionName = "!(OldPageRqstPtr <= XLogCtl->InitializedUpTo)", errorType = "FailedAssertion", fileName = "xlog.c", lineNumber = 1887), line 54 in "assert.c" (dbx) p OldPageRqstPtr 153551667200 (dbx) p XLogCtl->InitializedUpTo 153551667200 (d

Re: [HACKERS] An issue in remote query optimization

2017-01-31 Thread Tom Lane
Abbas Butt writes: > Postgres_fdw optimizes remote queries by pushing down the where clause. > This feature does not work consistently when the query is executed from > within a pl/pgsql function. The optimization works when the function > executes the query for the first 5 times, and fails afterw

Re: [HACKERS] patch: function xmltable

2017-01-31 Thread Pavel Stehule
2017-01-31 14:57 GMT+01:00 Alvaro Herrera : > Pavel Stehule wrote: > > 2017-01-24 21:38 GMT+01:00 Alvaro Herrera : > > > > I think it would be good to have a more complex test case in regress -- > > > let's say there is a table with some simple XML values, then we use > > > XMLFOREST (or maybe one

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Robert Haas
On Thu, Jan 5, 2017 at 12:54 AM, Kuntal Ghosh wrote: > I've attached the patch with the modified changes. PFA. Can this patch check contrib/bloom? +/* + * Mask some line pointer bits, particularly those marked as + * used on a master and unused on a standby. + */

Re: [HACKERS] multivariate statistics (v19)

2017-01-31 Thread Tomas Vondra
On 01/31/2017 07:52 AM, Amit Langote wrote: On 2017/01/31 6:57, Tomas Vondra wrote: On 01/30/2017 09:37 PM, Alvaro Herrera wrote: Looks good to me. I don't think we need to keep the names very short -- I would propose "standistinct", "stahistogram", "stadependencies". Yeah, I got annoyed by

Re: [HACKERS] sequence data type

2017-01-31 Thread Peter Eisentraut
And here is a rebased patch for the original feature. I think this addresses all raised concerns and suggestions now. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From ce2680ef072a9a4dc2cb879a70610d71ad24

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-31 Thread Robert Haas
On Tue, Jan 31, 2017 at 1:48 AM, Michael Paquier wrote: > I partially agree with this paragraph, at least there are advantages > to do so for cases where the data fits in shared buffers. Even for > data sets fitting in RAM that can be an advantage as the buffers would > get evicted from Postgres'

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2017-01-31 Thread Robert Haas
On Thu, Jan 26, 2017 at 5:53 AM, Emre Hasegeli wrote: > Though, I know the community is against behaviour changing GUCs. I > will not spend more time on this, before I get positive feedback from > others. As if on cue, let me say that a behavior-changing GUC sounds like a terrible idea to me. I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Pavan Deolasee
On Tue, Jan 31, 2017 at 7:37 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > > > > Sounds good. Should I submit that as a separate patch on current master? > > Yes, please. > > Attached. Two new APIs added. - CatalogInsertHeapAndIndex which does a simple_heap_insert followed by catalog up

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-31 Thread Mithun Cy
On Tue, Jan 31, 2017 at 9:47 PM, Robert Haas wrote: > Now, I assume that this patch sorts the I/O (although I haven't > checked that) and therefore I expect that the prewarm completes really > fast. If that's not the case, then that's bad. But if it is the > case, then it's not really hurting yo

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
"Daniel Verite" writes: > Tom Lane wrote: >> One possible compromise that would address your concern about display >> is to modify the hook API some more so that variable hooks could actually >> substitute new values. Then for example the bool-variable hooks could >> effectively replace "\s

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Pavan Deolasee wrote: > Two new APIs added. > > - CatalogInsertHeapAndIndex which does a simple_heap_insert followed by > catalog updates > - CatalogUpdateHeapAndIndex which does a simple_heap_update followed by > catalog updates > > There are only a handful callers remain for simple_heap_insert

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Daniel Verite
I wrote: > This would allow the hook to distinguish between initialization and > unsetting, which in turn will allow it to deny the \unset in the > cases when it doesn't make any sense conceptually (like AUTOCOMMIT). I notice that in the commited patch, you added the ability for DeleteVar

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-31 Thread Corey Huinker
On Tue, Jan 31, 2017 at 1:04 AM, Fabien COELHO wrote: > > This is code lifted from variable.c's ParseVariableBool(). When the other >>> patch for "psql hooks" is committed (the one that detects when the string >>> wasn't a valid boolean), this code will go away and we'll just use >>> ParseVariab

Re: [HACKERS] COPY as a set returning function

2017-01-31 Thread Corey Huinker
> > > Here is a 4: Refactoring BeginCopyFrom so as instead of a Relation are > used a TupleDesc, a default expression list, and a relation name. You > could as well make NextCopyFrom() smarter so as it does nothing if no > expression contexts are given by the caller, which is the case of your > fun

Re: [HACKERS] Speedup twophase transactions

2017-01-31 Thread Robert Haas
On Mon, Jan 23, 2017 at 7:00 AM, Nikhil Sontakke wrote: > 4) Minor nit-pick on existing code. > > (errmsg_plural("%u two-phase state file was written " > "for > long-running prepared transactions", >

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2017-01-31 Thread Emre Hasegeli
> Backing up a bit here, have we lost track of the problem that we're > trying to solve? Tom gave his opinion here: > > https://www.postgresql.org/message-id/3895.1464791...@sss.pgh.pa.us > > But I don't see that the latest patch I can find does anything to fix > that. This is what he wrote: > A

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
"Daniel Verite" writes: > I notice that in the commited patch, you added the ability > for DeleteVariable() to reject the deletion if the hook > disagrees. Right. > But this can't happen in practice because as mentioned just upthread > the hook called with NULL doesn't know if the variable is g

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-31 Thread Tom Lane
Corey Huinker writes: > On Tue, Jan 31, 2017 at 1:04 AM, Fabien COELHO wrote: >> The ParseVariableBool function has been updated, and the new version is >> much cleaner, including all fixes that I suggested in your copy, so you can >> use it in your patch. > I see there's still a lot of activity

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2017-01-31 Thread Robert Haas
On Tue, Jan 31, 2017 at 1:06 PM, Emre Hasegeli wrote: > This is what he wrote: > >> As I understand it, the key problem is that tests like "is point on line" >> would basically never succeed except in the most trivial cases, because of >> roundoff error. That's not very nice, and it might cascade

[HACKERS][REVIEW] macaddr 64 bit (EUI-64) datatype support

2017-01-31 Thread Vitaly Burovoy
Hello, I've reviewed the patch[1]. Result of testing: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed The patch introduce a new type macaddr8 for EUI-64 addresses[2] (assuming OU

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Craig Ringer
On 31 Jan. 2017 22:43, "Konstantin Knizhnik" wrote: On 31.01.2017 09:29, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer > wrote: > >> Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when >> wal_level >= logical I don't think that's the end of the worl

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
I wrote: > Attached is a draft patch for that. I chose to make a second hook rather > than complicate the assign hook API, mainly because it allows more code > sharing --- all the bool vars can share the same substitute hook, and > so can the three-way vars as long as "on" and "off" are the approp

Re: [HACKERS] patch: function xmltable

2017-01-31 Thread Pavel Stehule
Hi 2017-01-31 14:57 GMT+01:00 Alvaro Herrera : > Pavel Stehule wrote: > > 2017-01-24 21:38 GMT+01:00 Alvaro Herrera : > > > > I think it would be good to have a more complex test case in regress -- > > > let's say there is a table with some simple XML values, then we use > > > XMLFOREST (or maybe

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Alvaro Herrera wrote: > Unless there are objections I will push this later this afternoon. Done. Let's get on with the show -- please post a rebased WARM. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- S

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Andres Freund
On 2017-01-31 14:10:01 -0300, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > > Two new APIs added. > > > > - CatalogInsertHeapAndIndex which does a simple_heap_insert followed by > > catalog updates > > - CatalogUpdateHeapAndIndex which does a simple_heap_update followed by > > catalog updates

Re: [HACKERS] ICU integration

2017-01-31 Thread Thomas Munro
On Tue, Jan 10, 2017 at 9:45 AM, Peter Geoghegan wrote: > * I think it's worth looking into ucol_nextSortKeyPart(), and using > that as an alternative to ucol_getSortKey(). It doesn't seem any > harder, and when I tested it it was clearly faster. (I think that > ucol_nextSortKeyPart() is more or l

[HACKERS] Should `pg_upgrade --check` check relation filenodes are present?

2017-01-31 Thread Craig de Stigter
Hi list We attempted to pg_upgrade a database on a replication slave, and got the error: error while creating link for relation "." >> ("/var/lib/postgresql-ext/PG_9.2_201204301/19171/141610397" to >> "/var/lib/postgresql-ext/PG_9.5_201510051/16401/9911696"): No such file or >> directory > > > Th

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-01-31 Thread Jim Nasby
On 11/29/16 9:58 AM, Jeff Janes wrote: Considering a single SSD can do 70% of that limit, I would say yes. Next question becomes... should there even be an upper limit? Where the contortions needed to prevent calculation overflow become annoying? I'm not a big fan of nannyism in

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Andres Freund wrote: > On 2017-01-31 14:10:01 -0300, Alvaro Herrera wrote: > > Hmm, I was thinking we would get rid of CatalogUpdateIndexes altogether. > > Two of the callers are in the new routines (which I propose to rename to > > CatalogTupleInsert and CatalogTupleUpdate); the only remaining on

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Andres Freund
On 2017-01-31 19:10:05 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2017-01-31 14:10:01 -0300, Alvaro Herrera wrote: > > > > Hmm, I was thinking we would get rid of CatalogUpdateIndexes altogether. > > > Two of the callers are in the new routines (which I propose to rename to > > > C

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
BTW ... while I've been fooling with this issue, I've gotten a bit annoyed at the fact that "\set" prints the variables in, essentially, creation order. That makes the list ugly and hard to find things in, and it exposes some psql implementation details to users. I propose the attached simple pat

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Tom Lane
Andres Freund writes: > Hm, sorry for missing this earlier. I think CatalogUpdateIndexes() is > fairly widely used in extensions - it seems like a pretty harsh change > to not leave some backward compatibility layer in place. If an extension is doing that, it is probably constructing tuples to p

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Robert Haas
On Tue, Jan 24, 2017 at 4:51 PM, Robert Haas wrote: > On Mon, Jan 23, 2017 at 1:03 AM, Amit Kapila wrote: >> In spite of being careful, I missed reorganizing the functions in >> genam.h which I have done in attached patch. > > Cool. Committed parallel-generic-index-scan.2.patch. Thanks. Review

Re: [HACKERS] Should `pg_upgrade --check` check relation filenodes are present?

2017-01-31 Thread Tom Lane
Craig de Stigter writes: > We attempted to pg_upgrade a database on a replication slave, and got the > error: > error while creating link for relation "." > ("/var/lib/postgresql-ext/PG_9.2_201204301/19171/141610397" to > "/var/lib/postgresql-ext/PG_9.5_201510051/16401/9911696"): No such file or

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > Hm, sorry for missing this earlier. I think CatalogUpdateIndexes() is > > fairly widely used in extensions - it seems like a pretty harsh change > > to not leave some backward compatibility layer in place. > > If an extension is

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> (I'm a little more concerned by Alvaro's apparent position that WARM >> is a done deal; I didn't think so. This particular change seems like >> good cleanup anyhow, however.) > Agreed. BTW, the reason I think it's good cleanup is

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Andres Freund
On 2017-01-31 17:21:28 -0500, Tom Lane wrote: > Andres Freund writes: > > Hm, sorry for missing this earlier. I think CatalogUpdateIndexes() is > > fairly widely used in extensions - it seems like a pretty harsh change > > to not leave some backward compatibility layer in place. > > If an extens

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-01-31 Thread Thomas Munro
On Wed, Feb 1, 2017 at 2:10 AM, Ashutosh Bapat wrote: >> >> 0003-hj-refactor-memory-accounting-v4.patch: >> [...] >> > I looked at this patch. I agree that it accounts the memory usage more > accurately. Here are few comments. Thanks for the review! > spaceUsed is defined with comment > Size

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Tom Lane wrote: > BTW, the reason I think it's good cleanup is that it's something that my > colleagues at Salesforce also had to do as part of putting PG on top of a > different storage engine that had different ideas about index handling. > Essentially it's providing a bit of abstraction as to w

Re: [HACKERS] sequence data type

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:11 AM, Peter Eisentraut wrote: > And here is a rebased patch for the original feature. I think this > addresses all raised concerns and suggestions now. Thanks for the new version. That looks good to me after an extra lookup. -- Michael -- Sent via pgsql-hackers mail

Re: [HACKERS] Refactoring of replication commands using printsimple

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 11:59 PM, Robert Haas wrote: > Sorry, I have a little more nitpicking. Thanks for the input. > How about having > printsimple() use pq_sendcountedtext() instead of pq_sendint() > followed by pq_sendbytes(), as it does for TEXTOID? > > Other than that, this looks fine to m

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 6:22 PM, Craig Ringer wrote: > That's where you've misunderstood - it isn't committed yet. The point or > this change is to allow us to do logical decoding at the PREPARE TRANSACTION > point. The xact is not yet committed or rolled back. Yes, I got that. I was looking for

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:17 AM, Robert Haas wrote: > Well, the question even for that case is whether it really costs > anything. My bet is that it is nearly free when it doesn't help, but > that could be wrong. My experience running pgbench tests is that > prewarming all of pgbench_accounts on

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:06 AM, Robert Haas wrote: > On Thu, Jan 5, 2017 at 12:54 AM, Kuntal Ghosh > wrote: >> I've attached the patch with the modified changes. PFA. > > Can this patch check contrib/bloom? Only full pages are applied at redo by the generic WAL facility. So you would finish by c

Re: [HACKERS] Logical Replication and Character encoding

2017-01-31 Thread Kyotaro HORIGUCHI
Hello, At Tue, 31 Jan 2017 12:46:18 +, "Shinoda, Noriyoshi" wrote in > I tried a committed Logical Replication environment. I found > that replication between databases of different encodings did > not convert encodings in character type columns. Is this > behavior correct? The output

Re: [HACKERS] Logical Replication and Character encoding

2017-01-31 Thread Kyotaro HORIGUCHI
At Wed, 01 Feb 2017 12:05:40 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170201.120540.183393194.horiguchi.kyot...@lab.ntt.co.jp> > Hello, > > At Tue, 31 Jan 2017 12:46:18 +, "Shinoda, Noriyoshi" > wrote in > > > I tried a committed Logical Replication environment. I fou

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-01-31 Thread Michael Paquier
On Thu, Jan 26, 2017 at 6:49 PM, Masahiko Sawada wrote: > Sorry, I attached wrong version patch of pg_fdw_xact_resovler. Please > use attached patch. This patch has been moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] pageinspect: Hash index support

2017-01-31 Thread Michael Paquier
On Sun, Jan 29, 2017 at 11:09 AM, Ashutosh Sharma wrote: > okay. Thanks. I have done changes on top of this patch. Moved to CF 2017-03 as there is a new patch, no reviews. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: ht

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-31 Thread Michael Paquier
On Tue, Jan 24, 2017 at 4:58 PM, Kyotaro HORIGUCHI wrote: > Six new syscaches in 665d1fa was conflicted and 3-way merge > worked correctly. The new syscaches don't seem to be targets of > this patch. To be honest, I am not completely sure what to think about this patch. Moved to next CF as there

Re: [HACKERS] Supporting huge pages on Windows

2017-01-31 Thread Michael Paquier
On Mon, Jan 30, 2017 at 10:46 AM, Tsunakawa, Takayuki wrote: > From: Amit Kapila [mailto:amit.kapil...@gmail.com] >> I think it is better to document in some way if we decide to go-ahead with >> the patch. > > Sure, I added these sentences. Patch has been moved to CF 2017-03. There is a recent ne

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2017-01-31 Thread Michael Paquier
On Fri, Jan 27, 2017 at 10:26 AM, Haribabu Kommi wrote: > Thanks for the review. > Let's wait for the committer's opinion. I have moved this patch to CF 2017-03 to wait for this to happen. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] PATCH: two slab-like memory allocators

2017-01-31 Thread Michael Paquier
On Tue, Dec 13, 2016 at 10:32 AM, Petr Jelinek wrote: > Okay, this version looks good to me, marked as RfC. The patches still apply, moved to CF 2017-03 with same status: RfC. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Measuring replay lag

2017-01-31 Thread Michael Paquier
On Sat, Jan 21, 2017 at 10:49 AM, Thomas Munro wrote: > Ok. I see that there is a new compelling reason to move the ring > buffer to the sender side: then I think lag tracking will work > automatically for the new logical replication that just landed on > master. I will try it that way. Thanks

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2017-01-31 Thread Michael Paquier
On Tue, Nov 29, 2016 at 1:33 PM, Michael Paquier wrote: > On Fri, Nov 25, 2016 at 4:41 PM, Michael Paquier > wrote: >> On Fri, Nov 25, 2016 at 4:02 PM, Ideriha, Takeshi >> wrote: >>> I applied your fixed patch and new one, and confirmed the applied source >>> passed the tests successfully. And

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:31 PM, Michael Paquier wrote: > On Tue, Nov 29, 2016 at 1:33 PM, Michael Paquier > wrote: >> Patch moved to CF 2017-01. > > And nothing has happened since, the patch rotting a bit because of a > conflict in pg_dump's TAP tests. Attached is a rebased version. Moved to CF

Re: [HACKERS] pg_xlogdump follow into the future

2017-01-31 Thread Michael Paquier
On Fri, Dec 2, 2016 at 1:36 PM, Haribabu Kommi wrote: > Patch received feedback at the end of commitfest. > Closed in 2016-11 commitfest with "moved to next CF". > Please feel free to update the status once you submit the updated patch. And the thread has died as well weeks ago. I am marking that

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-01-31 Thread Michael Paquier
On Tue, Jan 17, 2017 at 9:18 PM, Amit Kapila wrote: > On Tue, Jan 17, 2017 at 11:39 AM, Dilip Kumar wrote: >> On Wed, Jan 11, 2017 at 10:55 AM, Dilip Kumar wrote: >>> I have reviewed the latest patch and I don't have any more comments. >>> So if there is no objection from other reviewers I can m

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 2:15 PM, Peter Geoghegan wrote: > On Mon, Jan 30, 2017 at 8:46 PM, Thomas Munro > wrote: >> On Wed, Jan 4, 2017 at 12:53 PM, Peter Geoghegan wrote: >>> Attached is V7 of the patch. >> >> I am doing some testing. First, some superficial things from first pass: >> >> [Vari

Re: [HACKERS] Proposal : Parallel Merge Join

2017-01-31 Thread Michael Paquier
On Thu, Jan 5, 2017 at 12:57 AM, Dilip Kumar wrote: > On Wed, Jan 4, 2017 at 12:02 PM, Amit Kapila wrote: >> Review comments: >> 1. >> + bool is_partial); >> + >> >> Seems additional new line is not required. > Fixed This patch has a patch, no new reviews. Moved to CF 2017-03. -- Michael --

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

2017-01-31 Thread Michael Paquier
On Wed, Jan 25, 2017 at 2:53 PM, Michael Paquier wrote: > The latest patch available still applies, one person has added his > name (John G) in October though there have been no reviews. There have > been a couple of arguments against this patch, and the thread has had > no activity for the last m

Re: [HACKERS] Reporting planning time with EXPLAIN

2017-01-31 Thread Michael Paquier
On Wed, Jan 4, 2017 at 7:59 PM, Ashutosh Bapat wrote: > Here are patches for following Those patches have received no code-level reviews, so moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.p

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-01-31 Thread Ashutosh Bapat
> >> However, ExecHashIncreaseNumBatches() may change the >> number of buckets; the patch does not seem to account for spaceUsed changes >> because of that. > > That's what this hunk is intended to do: > > @@ -795,6 +808,12 @@ ExecHashIncreaseNumBuckets(HashJoinTable hashtable) > TRACE_POST

Re: [HACKERS] postgres_fdw bug in 9.6

2017-01-31 Thread Michael Paquier
On Mon, Jan 23, 2017 at 6:56 PM, Etsuro Fujita wrote: > Other changes: > > * Also modified CreateLocalJoinPath so that we pass outer/inner paths, not > outer/inner rels, because it would be more flexible for the FDW to build the > local-join path from paths it chose. > * Fixed a bug that I missed

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 9:36 AM, Alvaro Herrera wrote: >> I propose that we should finish the job by inventing CatalogTupleDelete(), >> which for the moment would be a trivial wrapper around >> simple_heap_delete(), maybe just a macro for it. >> >> If there's no objections I'll go make that happen

Re: [HACKERS] Push down more full joins in postgres_fdw

2017-01-31 Thread Michael Paquier
On Mon, Jan 30, 2017 at 8:30 PM, Etsuro Fujita wrote: > Other changes: > * I went back to make_outerrel_subquery and make_innerrel_subquery, which > are flags to indicate whether to deparse the input relations as subqueries. > is_subquery_rel would work well for handling the cases of full joins wi

  1   2   >