Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-06 Thread Kouhei Kaigai
Hanada-san, Thanks for your dedicated efforts for remote-join feature. Below are the comments from my side. * Bug - mixture of ctid system column and whole row-reference In case when "ctid" system column is required, deparseSelectSql() adds ctid reference on the base relation scan level. On the

[HACKERS] Ignoring some binaries generated in src/test

2015-04-06 Thread Michael Paquier
Hi all, A couple of binaries in src/test, that are not part of the main make flow, can be built but they are actually not ignored in the tree: examples/testlibpq examples/testlibpq2 examples/testlibpq3 examples/testlibpq4 examples/testlo examples/testlo64 locale/test-ct

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Michael Paquier
On Mon, Apr 6, 2015 at 9:10 PM, Fujii Masao wrote: > I'm not familiar with native language support (sorry), but don't we need to > add the shortcut of gettext into every calls of pg_log and pg_fatal, e.g., > change pg_fatal("xxx") to pg_fatal(_("xxx"))? I know that fprintf() in > pg_Log_v() has su

[HACKERS] Typo in a comment in set_rel_size()

2015-04-06 Thread Amit Langote
Hi, Attached fixes what I suppose is a typo: * so set up a single dummy path for it. Here we only check this for * regular baserels; if it's an otherrel, CE was already checked in -* set_append_rel_pathlist(). +* set_append_rel_size(). * Thanks, Amit

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Michael Paquier
On Mon, Apr 6, 2015 at 10:01 PM, Alvaro Herrera wrote: > I'm not sure about translation of generic strings such as "%s: %s". My > first impression is that they shouldn't be translated, but maybe it is > important that they are for languages I don't know nothing about such as > Japanese. I misunde

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Sawada Masahiko
On Tue, Apr 7, 2015 at 7:53 AM, Jim Nasby wrote: > On 4/6/15 5:18 PM, Greg Stark wrote: >> >> Only I would suggest thinking of it in terms of two orthogonal boolean >> flags rather than three states. It's easier to reason about whether a >> table has a specific property than trying to control a st

[HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-04-06 Thread Tomas Vondra
Hi, attached is a first version of a patch that aims to improve cardinality estimates of joins by matching foreign keys between the tables (which was ignored by the planner until now). This significantly improves estimates when joining two tables using multi-column conditions, matching a for

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Simon Riggs
On 6 April 2015 at 20:38, David Steele wrote: >> The earlier version of pg_audit generated different output. >> Specifically, it allowed you to generate output for each object >> tracked; one line per object. That discussion covers recursive SQL. That is important too, but not what I am saying.

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread David Steele
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 4/6/15 4:47 PM, Simon Riggs wrote: > On 6 April 2015 at 16:34, Peter Eisentraut > wrote: >> "Audit" is a "big word". It might imply regulatory or standards >> compliance on some level. We already have ways to log >> everything. If customers w

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-04-06 Thread Fabrízio de Royes Mello
On Mon, Apr 6, 2015 at 10:21 AM, Sawada Masahiko wrote: > > On Fri, Mar 13, 2015 at 5:10 PM, Kyotaro HORIGUCHI > wrote: > > Hello, I have some trivial comments about the latest patch. > > > > At Thu, 12 Mar 2015 21:15:14 +0900, Sawada Masahiko < sawada.m...@gmail.com> wrote in > > sawada.mshk> O

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread David Steele
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 4/6/15 4:34 PM, Peter Eisentraut wrote: > On 2/14/15 9:34 PM, David Steele wrote: >> The patch I've attached satisfies the requirements that I've had >> from customers in the past. > > What I'm missing is a more precise description/documentation

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 5:18 PM, Greg Stark wrote: Only I would suggest thinking of it in terms of two orthogonal boolean flags rather than three states. It's easier to reason about whether a table has a specific property than trying to control a state machine in a predefined pathway. So I would say the two f

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Greg Stark
On 6 Apr 2015 09:17, "Jim Nasby" wrote: > > > No. You would be free to set a table as ReadOnly any time you wanted, without scanning anything. All that setting does is disable any DML on the table. > > The Frozen state would only be set by the vacuum code, IFF: > - The table state is ReadOnly *at

Re: [HACKERS] BRIN range operator class

2015-04-06 Thread Alvaro Herrera
Thanks for the updated patch; I will at it as soon as time allows. (Not really all that soon, regrettably.) Judging from a quick look, I think patches 1 and 5 can be committed quickly; they imply no changes to other parts of BRIN. (Not sure why 1 and 5 are separate. Any reason for this?) Also

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Alvaro Herrera
Simon Riggs wrote: > The present version can trigger an audit trail event for a statement, > without tracking the object that was being audited. This prevents you > from searching for "all SQL that touches table X", i.e. we know the > statements were generated, but not which ones they were. IMHO t

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Simon Riggs
On 6 April 2015 at 16:34, Peter Eisentraut wrote: > On 2/14/15 9:34 PM, David Steele wrote: >> The patch I've attached satisfies the requirements that I've had from >> customers in the past. > > What I'm missing is a more precise description/documentation of what > those requirements might be. > >

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Peter Eisentraut
On 2/14/15 9:34 PM, David Steele wrote: > The patch I've attached satisfies the requirements that I've had from > customers in the past. What I'm missing is a more precise description/documentation of what those requirements might be. "Audit" is a "big word". It might imply regulatory or standar

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 1:28 PM, Josh Berkus wrote: On 04/06/2015 10:07 AM, Jim Nasby wrote: Actually, I would start simply with ReadOnly and ReadWrite. As I understand it, the goal here is to prevent huge amounts of periodic freeze work due to XID wraparound. I don't think we need the Freeze state to accomp

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Josh Berkus
On 04/06/2015 11:35 AM, Alvaro Herrera wrote: > Josh Berkus wrote: > >> I agree with Jim that if we have a trustworthy Frozen Map, having a >> ReadOnly flag is of marginal value, unless such a ReadOnly flag allowed >> us to skip updating the individual row XIDs entirely. I can think of >> some wa

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Alvaro Herrera
Josh Berkus wrote: > I agree with Jim that if we have a trustworthy Frozen Map, having a > ReadOnly flag is of marginal value, unless such a ReadOnly flag allowed > us to skip updating the individual row XIDs entirely. I can think of > some ways to do that, but they have severe tradeoffs. If you

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Josh Berkus
On 04/06/2015 10:07 AM, Jim Nasby wrote: > Actually, I would start simply with ReadOnly and ReadWrite. > > As I understand it, the goal here is to prevent huge amounts of periodic > freeze work due to XID wraparound. I don't think we need the Freeze > state to accomplish that. > > With a single b

Re: [HACKERS] Exposing PG_VERSION_NUM in pg_config

2015-04-06 Thread Jim Nasby
On 4/1/15 1:25 AM, Andrew Gierth wrote: "Michael" == Michael Paquier writes: Michael> For an extension that has a single branch compatible with a Michael> set of multiple major versions of Postgres, the cases are Michael> custom values for REGRESS_OPTS and REGRESS depending on the Mich

Re: [HACKERS] pg_dump / copy bugs with "big lines" ?

2015-04-06 Thread Jim Nasby
On 3/31/15 3:46 AM, Ronan Dunklau wrote: >StringInfo uses int's to store length, so it could possibly be changed, >but then you'd just error out due to MaxAllocSize. > >Now perhaps those could both be relaxed, but certainly not to the extent >that you can shove an entire 1.6TB row into an output

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 12:29 PM, k...@rice.edu wrote: On Mon, Apr 06, 2015 at 12:07:47PM -0500, Jim Nasby wrote: ... As I understand it, the goal here is to prevent huge amounts of periodic freeze work due to XID wraparound. I don't think we need the Freeze state to accomplish that. With a single bit per pa

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread k...@rice.edu
On Mon, Apr 06, 2015 at 12:07:47PM -0500, Jim Nasby wrote: > ... > As I understand it, the goal here is to prevent huge amounts of > periodic freeze work due to XID wraparound. I don't think we need > the Freeze state to accomplish that. > > With a single bit per page in the Frozen Map, checking a

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 11:12 AM, Sawada Masahiko wrote: On Mon, Apr 6, 2015 at 10:17 PM, Jim Nasby wrote: On 4/6/15 1:46 AM, Sawada Masahiko wrote: On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes wrote: On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby wrote: On 4/3/15 12:59 AM, Sawada Masahiko wrote: +

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread David Steele
On 4/6/15 8:40 AM, Sawada Masahiko wrote: > On Fri, Apr 3, 2015 at 10:01 PM, David Steele wrote: >> On 4/3/15 3:59 AM, Sawada Masahiko wrote: >>> On Thu, Apr 2, 2015 at 2:46 AM, David Steele wrote: Let me know if you see any other issues. >>> >>> I pulled HEAD, and then tried to compile

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Sawada Masahiko
On Mon, Apr 6, 2015 at 10:17 PM, Jim Nasby wrote: > On 4/6/15 1:46 AM, Sawada Masahiko wrote: >> >> On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes wrote: >>> >>> On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby >>> wrote: On 4/3/15 12:59 AM, Sawada Masahiko wrote: > > > +

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-04-06 Thread Sawada Masahiko
On Fri, Mar 13, 2015 at 5:10 PM, Kyotaro HORIGUCHI wrote: > Hello, I have some trivial comments about the latest patch. > > At Thu, 12 Mar 2015 21:15:14 +0900, Sawada Masahiko > wrote in > sawada.mshk> On Thu, Mar 12, 2015 at 6:36 AM, Jim Nasby > wrote: >> >>> >Are the parenthesis necessary?

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Petr Jelinek
On 06/04/15 15:07, Amit Kapila wrote: On Mon, Apr 6, 2015 at 5:56 PM, Petr Jelinek mailto:p...@2ndquadrant.com>> wrote: > > On 06/04/15 12:33, Amit Kapila wrote: >> >> >> But I think the Update on target table with sample scan is >> supported via views which doesn't seem to be the right thi

Re: [HACKERS] Freeze avoidance of very large table.

2015-04-06 Thread Jim Nasby
On 4/6/15 1:46 AM, Sawada Masahiko wrote: On Sun, Apr 5, 2015 at 8:21 AM, Jeff Janes wrote: On Sat, Apr 4, 2015 at 3:10 PM, Jim Nasby wrote: On 4/3/15 12:59 AM, Sawada Masahiko wrote: + case HEAPTUPLE_LIVE: + case HEAPTUPLE_RECENT

Re: [HACKERS] initdb -S and tablespaces

2015-04-06 Thread Alvaro Herrera
Abhijit Menon-Sen wrote: Hi, > At 2015-04-03 13:32:32 -0300, alvhe...@2ndquadrant.com wrote: > > I also noticed that walkdir() thinks it is completely agnostic on > > what the passed action is; except that the comment at the bottom talks > > about how fsync on directories is important, which see

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Amit Kapila
On Mon, Apr 6, 2015 at 5:56 PM, Petr Jelinek wrote: > > On 06/04/15 12:33, Amit Kapila wrote: >> >> >> But I think the Update on target table with sample scan is >> supported via views which doesn't seem to be the right thing >> in case you just want to support it via FROM/USING, example >> >> pos

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Alvaro Herrera
Fujii Masao wrote: > On Mon, Apr 6, 2015 at 5:33 PM, Michael Paquier > wrote: > > On Mon, Apr 6, 2015 at 1:41 PM, Michael Paquier wrote: > >> I guess that you are working on a patch? If not, you are looking for one? > > > > Code-speaking, this gives the patch attached. > > Thanks! Here are the re

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Sawada Masahiko
On Fri, Apr 3, 2015 at 10:01 PM, David Steele wrote: > On 4/3/15 3:59 AM, Sawada Masahiko wrote: >> On Thu, Apr 2, 2015 at 2:46 AM, David Steele wrote: >>> Let me know if you see any other issues. >>> >> >> I pulled HEAD, and then tried to compile source code after applied >> following "deparsing

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Petr Jelinek
On 06/04/15 11:02, Simon Riggs wrote: On 2 April 2015 at 17:36, Petr Jelinek wrote: so here is version 11. Looks great. Comment on docs: The SELECT docs refer only to SYSTEM and BERNOULLI. It doesn't mention that if other methods are available they could be used also. The phrasing was "sam

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Petr Jelinek
On 06/04/15 12:33, Amit Kapila wrote: On Sat, Apr 4, 2015 at 8:25 PM, Petr Jelinek mailto:p...@2ndquadrant.com>> wrote: > > Yes I want extensibility here. And I think the tablesample method arguments are same thing as function arguments given that in the end they are arguments for the init func

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Fujii Masao
On Mon, Apr 6, 2015 at 5:33 PM, Michael Paquier wrote: > On Mon, Apr 6, 2015 at 1:41 PM, Michael Paquier wrote: >> I guess that you are working on a patch? If not, you are looking for one? > > Code-speaking, this gives the patch attached. Thanks! Here are the review comments: I'm not familiar wi

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Amit Kapila
On Sat, Apr 4, 2015 at 8:25 PM, Petr Jelinek wrote: > > On 04/04/15 14:57, Amit Kapila wrote: >> >> >> 1. >> +tablesample_clause: >> +TABLESAMPLE ColId '(' func_arg_list ')' opt_repeatable_clause >> >> It seems to me that you want to allow it to make it extendable >> to user defined Tablesample me

Re: [HACKERS] TABLESAMPLE patch

2015-04-06 Thread Simon Riggs
On 2 April 2015 at 17:36, Petr Jelinek wrote: > so here is version 11. Looks great. Comment on docs: The SELECT docs refer only to SYSTEM and BERNOULLI. It doesn't mention that if other methods are available they could be used also. The phrasing was "sampling method can be one of ." Are we re

Re: [HACKERS] pg_rewind and log messages

2015-04-06 Thread Michael Paquier
On Mon, Apr 6, 2015 at 1:41 PM, Michael Paquier wrote: > I guess that you are working on a patch? If not, you are looking for one? Code-speaking, this gives the patch attached. I eliminated a bunch of newlines in the log messages that seemed really unnecessary to me, simplifying a bit the whole. W

Re: [HACKERS] Auditing extension for PostgreSQL (Take 2)

2015-04-06 Thread Sawada Masahiko
On Fri, Apr 3, 2015 at 10:01 PM, David Steele wrote: > On 4/3/15 3:59 AM, Sawada Masahiko wrote: >> On Thu, Apr 2, 2015 at 2:46 AM, David Steele wrote: >>> Let me know if you see any other issues. >>> >> >> I pulled HEAD, and then tried to compile source code after applied >> following "deparsing

Re: [HACKERS] initdb -S and tablespaces

2015-04-06 Thread Abhijit Menon-Sen
Hi Álvaro. Thanks for taking a look at the patch. At 2015-04-03 13:32:32 -0300, alvhe...@2ndquadrant.com wrote: > > But then, since the name is already telling us that it's all about > PGDATA, maybe we can remove the "recursively" part of the name? Sure, that makes sense too. Since you and Andre