Re: [HACKERS] checkpointer continuous flushing

2015-08-24 Thread Fabien COELHO


Hello Amit,


Can the script be started on its own at all?


I have tried like below which results in same error, also I tried few
other variations but could not succeed.
./avg.py


Hmmm... Ensure that the script is readable and executable:

  sh> chmod a+rx ./avg.py

Also check the file:

  sh> file ./avg.py
  ./avg.py: Python script, UTF-8 Unicode text executable


Sure... This is *already* the case with the current checkpointer, the
schedule is performed with respect to the initial number of buffers it
think it will have to write, and if someone else writes these buffers then
the schedule is skewed a little bit, or more... I have not changed this


I don't know how good or bad it is to build  further on somewhat skewed
logic,


The logic is no more skewed that it is with the current version: your 
remark about the estimation which may be wrong in some cases is clearly 
valid, but it is orthogonal (independent, unrelated, different) to what is 
addressed by this patch.


I currently have no reason to believe that the issue you raise is a major 
performance issue, but if so it may be addressed by another patch by 
whoever want to do so.


What I have done is to demonstrate that generating a lot of random I/Os is 
a major performance issue (well, sure), and this patch addresses this 
point and provide major speedup (*3-5) and latency reductions (from +60% 
unavailability to nearly full availability) for high OLTP write load, by 
reordering and flushing checkpoint buffers in a sensible way.



but the point is that unless it is required why to use it.


This is really required to avoid predictable performance regressions, see 
below.



I do not think that Heikki version worked wrt to balancing writes over
tablespaces,


I also think that it doesn't balances over tablespaces, but the question 
is why do we need to balance over tablespaces, can we reliably predict 
in someway which indicates that performing balancing over tablespace can 
help the workload.


The reason for the tablespace balancing is that in the current postgres 
buffers are written more or less randomly, so it is (probably) implicitely 
and statistically balanced over tablespaces because of this randomness, 
and indeed, AFAIK, people with multi tablespace setup have not complained 
that postgres was using the disks sequentially.


However, once the buffers are sorted per file, the order becomes 
deterministic and there is no more implicit balancing, which means that if 
someone has a pg setup with several disks it will write sequentially on 
these instead of in parallel.


This regression was pointed out by Andres Freund, I agree that such a 
regression for high end systems must be avoided, hence the tablespace 
balancing.



I think here we are doing more engineering than required for this patch.


I do not think so, I think that Andres remark is justified to avoid a 
performance regression on high end systems which use tablespaces, which is 
really undesirable.


About the balancing code, it is not that difficult, even if it is not 
trivial: the point is to select the tablespace for which the progress 
ratio (written/to_write) is below the overall progress ratio, so that it 
catches up, and do so in a round robin maner, so that all tablespaces get 
to write things. I also have both written a proof and tested the logic (in 
a separate script).


--
Fabien.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Declarative partitioning

2015-08-24 Thread Simon Riggs
On 24 August 2015 at 00:53, Josh Berkus  wrote:

> On 08/21/2015 08:34 PM, Jim Nasby wrote:
> > On 8/18/15 12:31 PM, Josh Berkus wrote:
> >> Also this would be useful for range
> >> partitions:
> >>
> >> CREATE PARTITION ON parent_table USING ( start_value );
> >>
> >> ... where start_value is the start range of the new partition.  Again,
> >> easier for users to get correct.
> >
> > Instead of that, I think it would be more foolproof to do
> >
> > CREATE PARTITION ON parent_table FOR ( value1, ... );
> >
> > instead of trusting the user to get the exact start value correct.
> >
> > Though... I guess there could be value in allowing an exact start value
> > but throwing an error if it doesn't sit exactly on a boundary. Might
> > make it less likely to accidentally create the wrong partition.
>
> Well, I'm figuring that most people would use "CREATE NEXT PARTITION"
> instead.
>

ALTER TABLE foo ADD PARTITION NEXT;

when a sequence of partitions has been defined

-- 
Simon Riggshttp://www.2ndQuadrant.com/

PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Re: [HACKERS] psql - better support pipe line

2015-08-24 Thread Heikki Linnakangas

On 08/24/2015 08:06 AM, Pavel Stehule wrote:

Hi

I found so a set "psql" and "xargs" is pretty strong. But I miss a psql
option for simple returning current database and continuing in pipeline.

What I am doing:

psql postgres -At -c "select datname from pg_database" |
xargs -P 3 -I % psql % -At -c "select current_databe() from
pg_stat_all_tables
where relname = 'pg_attribute' and n_dead_tup > 10" |
xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %" |
xargs -P 3 -I % psql % -At -c "select curren_database() from ..."

it works perfectly - but the line

xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %"

is little bit ugly - with some psql option it can be cleaned to

xargs -P3 -I % psql % -q --echo-db -c "analyze pg_attribute" | ...

--echo-db requires -q option

What are you thinking about this idea?


Seems like a one-tricky-pony to me. You're just as likely to need to 
print a relation name or something else, as the current database.


I don't actually understandu what you'd put in the "..." above. One 
you've analyze'd the table, what more do you want to do?


Overall, once your pipeline gets that complicated, I'd rather write a 
little bash or perl script with for-loops and variables.


- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] psql - better support pipe line

2015-08-24 Thread Pavel Stehule
2015-08-24 12:49 GMT+02:00 Heikki Linnakangas :

> On 08/24/2015 08:06 AM, Pavel Stehule wrote:
>
>> Hi
>>
>> I found so a set "psql" and "xargs" is pretty strong. But I miss a psql
>> option for simple returning current database and continuing in pipeline.
>>
>> What I am doing:
>>
>> psql postgres -At -c "select datname from pg_database" |
>> xargs -P 3 -I % psql % -At -c "select current_databe() from
>> pg_stat_all_tables
>> where relname = 'pg_attribute' and n_dead_tup > 10" |
>> xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %" |
>> xargs -P 3 -I % psql % -At -c "select curren_database() from ..."
>>
>> it works perfectly - but the line
>>
>> xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %"
>>
>> is little bit ugly - with some psql option it can be cleaned to
>>
>> xargs -P3 -I % psql % -q --echo-db -c "analyze pg_attribute" | ...
>>
>> --echo-db requires -q option
>>
>> What are you thinking about this idea?
>>
>
> Seems like a one-tricky-pony to me. You're just as likely to need to print
> a relation name or something else, as the current database.
>

yes, you have true


> I don't actually understandu what you'd put in the "..." above. One you've
> analyze'd the table, what more do you want to do?
>

do other filtering and for example run VACUUM FULL when it can have a
significant effect

>
> Overall, once your pipeline gets that complicated, I'd rather write a
> little bash or perl script with for-loops and variables.


It is pipeline of trivial statements - so still it is pretty simple - and
with multiprocess suppport

Regards

Pavel


>
>
> - Heikki
>
>


Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-24 Thread Andrew Dunstan



On 08/23/2015 08:58 PM, Michael Paquier wrote:

2) No docs or related regression test yet. I will do that if there is
enough interest in this getting committed. So far no one except Andrew
and I have chimed in.

I think that's a good thing to have, now I have concerns about making
this data readable for non-superusers. Cloud deployments of Postgres
are logically going to block the access of this view.



I don't think it exposes any information of great security value.


+ Datum pg_config(PG_FUNCTION_ARGS);
+
+ PG_FUNCTION_INFO_V1(pg_config);

The declaration of the function is not needed, PG_FUNCTION_INFO_V1
takes care of it.




Umm, we shouldn't be using PG_FUNCTION_INFO_V1 in backend code at all, IIRC.

cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] psql - better support pipe line

2015-08-24 Thread Andrew Dunstan



On 08/24/2015 06:49 AM, Heikki Linnakangas wrote:

On 08/24/2015 08:06 AM, Pavel Stehule wrote:

Hi

I found so a set "psql" and "xargs" is pretty strong. But I miss a psql
option for simple returning current database and continuing in pipeline.

What I am doing:

psql postgres -At -c "select datname from pg_database" |
xargs -P 3 -I % psql % -At -c "select current_databe() from
pg_stat_all_tables
where relname = 'pg_attribute' and n_dead_tup > 10" |
xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %" |
xargs -P 3 -I % psql % -At -c "select curren_database() from ..."

it works perfectly - but the line

xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %"

is little bit ugly - with some psql option it can be cleaned to

xargs -P3 -I % psql % -q --echo-db -c "analyze pg_attribute" | ...

--echo-db requires -q option

What are you thinking about this idea?


Seems like a one-tricky-pony to me. You're just as likely to need to 
print a relation name or something else, as the current database.


I don't actually understandu what you'd put in the "..." above. One 
you've analyze'd the table, what more do you want to do?


Overall, once your pipeline gets that complicated, I'd rather write a 
little bash or perl script with for-loops and variables.






Yes, the use case for this is way too narrow.

cheers

andrew







--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Declarative partitioning

2015-08-24 Thread Andrew Dunstan



On 08/24/2015 03:46 AM, Simon Riggs wrote:



ALTER TABLE foo ADD PARTITION NEXT;

when a sequence of partitions has been defined





Or perhaps

   ALTER TABLE foo ADD PARTITION NEXT nn;


So you can set up, say, a week's worth of daily partitions at once.

I could also imagine a variant that allows you to specify partitions up 
to the one including some value.


cheers

andrew


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-24 Thread Tom Lane
Andrew Dunstan  writes:
> On 08/23/2015 08:58 PM, Michael Paquier wrote:
>> I think that's a good thing to have, now I have concerns about making
>> this data readable for non-superusers. Cloud deployments of Postgres
>> are logically going to block the access of this view.

> I don't think it exposes any information of great security value.

We just had that kerfuffle about whether WAL compression posed a security
risk; doesn't that imply that at least the data relevant to WAL position
has to be unreadable by non-superusers?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] psql - better support pipe line

2015-08-24 Thread Tom Lane
Heikki Linnakangas  writes:
> On 08/24/2015 08:06 AM, Pavel Stehule wrote:
>> it works perfectly - but the line
>> xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %"
>> is little bit ugly - with some psql option it can be cleaned to
>> xargs -P3 -I % psql % -q --echo-db -c "analyze pg_attribute" | ...
>> --echo-db requires -q option
>> What are you thinking about this idea?

> Seems like a one-tricky-pony to me. You're just as likely to need to 
> print a relation name or something else, as the current database.

Not only that, but:

(1) there is no reason to believe that the db name and only the db name
is needed to do another connection; what about port, host, user, etc?

(2) this commandeers the pipe connection to transmit out-of-band data,
making it impossible to use the pipe for its natural function, viz
transmitting ordinary data from one processing step to the next.  Sure,
there are use-cases where there's no such data and you can repurpose the
pipe like that, but that's an enormous limitation.

> Overall, once your pipeline gets that complicated, I'd rather write a 
> little bash or perl script with for-loops and variables.

Yeah, on the whole this seems like a band-aid to let a bad scripting
approach limp a few steps further before it collapses completely.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WIP: Rework access method interface

2015-08-24 Thread Tom Lane
Alexander Korotkov  writes:
> On Mon, Aug 10, 2015 at 7:50 PM, Tom Lane  wrote:
>> Hm.  So one way or the other we're going to end up violating relational
>> theory somewhere.  OK, I yield: let's say that pg_am has amname, amkind,
>> amhandler, and nothing else.  Then we will need SQL functions to expose
>> whatever information we think needs to be available to SQL code.

> There is second revision of this patch. Changes are so:

>  * AmRoutine was renamed to IndexAmRoutine assuming there could be other
> access methods in the future.
>  * amhandlers now return index_am_handler pseudotype.
>  * CHECK_PROCEDUREs are now is the place of original GET_REL_PROCEDUREs.
>  * amstrategies, amsupport, amcanorderbyop, amstorage, amkeytype are in
> both pg_am and IndexAmRoutine. Consistence of amhandler answer and pg_am
> tuple is checking.

[ scratches head... ]  I thought we'd just agreed we weren't going to keep
any of those pg_am columns?  If we keep them, we'll have to define what
they mean for sequence AMs etc.  ("Let them be NULL" would likely break
enough stuff that we might as well not have them.)

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-24 Thread Thor Lancelot Simon
On Thu, Aug 20, 2015 at 04:32:19PM +0100, Greg Stark wrote:
> 
> That's the problem. initdb tests how many connections can start up
> when writing the default config. But we assume that each process can
> use up to the rlimit file descriptors without running into a
> system-wide limit.

That sounds like a fairly bogus assumption -- unless the system-wide
limit is to be meaningless.

The default NetBSD limits on the VAX are probably still too low, however.

Thor


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] WIP: Rework access method interface

2015-08-24 Thread Alexander Korotkov
On Mon, Aug 24, 2015 at 5:15 PM, Tom Lane  wrote:

> Alexander Korotkov  writes:
> > On Mon, Aug 10, 2015 at 7:50 PM, Tom Lane  wrote:
> >> Hm.  So one way or the other we're going to end up violating relational
> >> theory somewhere.  OK, I yield: let's say that pg_am has amname, amkind,
> >> amhandler, and nothing else.  Then we will need SQL functions to expose
> >> whatever information we think needs to be available to SQL code.
>
> > There is second revision of this patch. Changes are so:
>
> >  * AmRoutine was renamed to IndexAmRoutine assuming there could be other
> > access methods in the future.
> >  * amhandlers now return index_am_handler pseudotype.
> >  * CHECK_PROCEDUREs are now is the place of original GET_REL_PROCEDUREs.
> >  * amstrategies, amsupport, amcanorderbyop, amstorage, amkeytype are in
> > both pg_am and IndexAmRoutine. Consistence of amhandler answer and pg_am
> > tuple is checking.
>
> [ scratches head... ]  I thought we'd just agreed we weren't going to keep
> any of those pg_am columns?  If we keep them, we'll have to define what
> they mean for sequence AMs etc.  ("Let them be NULL" would likely break
> enough stuff that we might as well not have them.)
>

>From the previous discussion I see following options:
1) Non-index access methods don't reuse pg_class.relam nor pg_am. Fully
relational compliant but complex catalog structure.
2) Non-index access methods reuse pg_class.relam but don't reuse pg_am.
This violates relational theory because single column reference multiple
tables.
3) Non-index access methods reuse both pg_class.relam and pg_am. This
violates relational theory because we store different objects in the same
table.

I'd say we already have precedent of #2. It's pg_depend which reference
objects of arbitrary types.
In the #3 we really shouldn't keep any specific to index am in pg_am.

But what we assume to be an access method in general? For instance, we have
foreign data wrappers which aren't access methods (but looks quite similar
from some degree).

--
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-24 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 08/24/2015 06:50 AM, Tom Lane wrote:
> Andrew Dunstan  writes:
>> On 08/23/2015 08:58 PM, Michael Paquier wrote:
>>> I think that's a good thing to have, now I have concerns about
>>> making this data readable for non-superusers. Cloud deployments
>>> of Postgres are logically going to block the access of this
>>> view.
> 
>> I don't think it exposes any information of great security
>> value.
> 
> We just had that kerfuffle about whether WAL compression posed a
> security risk; doesn't that imply that at least the data relevant
> to WAL position has to be unreadable by non-superusers?

So pg_config might be fully unrestricted, but pg_controldata might
need certain rows filtered based on superuser status? Do you think
those rows should be present but redacted, or completely filtered out?

Joe
- -- 
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJV2z3SAAoJEDfy90M199hlgmwP/iPI4gJAM00b1mWiPHYSEMjQ
pdVgPkFgfGQKyTizo7rEv1nJTQI3J9aUD7hvqYvPGlSOum0xei17fiRUIKnfqGgZ
7aSuhc97gZ7U5LvDsClovEUDEon+RIibZAYHKnKv2qYDwO/ZvfdFFQNi9TV0eREi
QrEYafNo3/PWqJtrJoqhXaXyXsZ33FKtaaesQZJXvUUkTaE42eviq0cPiz2lHEsq
szlGBnPkBS3qthAusApetAobZH9OymL4yl1BWwmBl3d2nEvQ4OVFGWo195It4XyQ
98bMzXse0PvBuKkcKrlTjxPdtR9UE/2FHojh7VLaj+JQeCGjehXNuogGPr7XHNSu
cbCvIWsxW7Vz1liwFxY9I7Aui6/4X/oPehrct4CqaihqoztP1JrkQpVJDBYWwAhH
Q/sRe8gUY8AWQHQljt9nuZvXmEYBnFbSf8tWVZ3/yhU1fK9dcl9B5doIHwKQXXtW
+BHx4mOX5gcSRvGQFkJO0auE3Y9dvfUtpV4xDC57OHekgKA+rZw/HtElwKIhgrHI
QoCd9PpJdG3UngX7ffsRuhJIhTUCSOKA2AIdceRyH4UgtqtHLzSU1tom3XMcQD+f
mJvlKMwSvqh2Qmd/ZiNhgN4APkGk1AmH26hMMhI9HIrAIghkmPDfssLxYcBgJyDd
lt8dJLQDnaddFLuvdQww
=KZVU
-END PGP SIGNATURE-


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-24 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 08/24/2015 04:38 AM, Andrew Dunstan wrote:
> On 08/23/2015 08:58 PM, Michael Paquier wrote:
>> + Datum pg_config(PG_FUNCTION_ARGS); + +
>> PG_FUNCTION_INFO_V1(pg_config);
>> 
>> The declaration of the function is not needed,
>> PG_FUNCTION_INFO_V1 takes care of it.
> 
> Umm, we shouldn't be using PG_FUNCTION_INFO_V1 in backend code at
> all, IIRC.

Right -- those lines are a cut-n-pasteo from the original pg_config as
an extension code. Will fix.

Thanks,

Joe

- -- 
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJV20DqAAoJEDfy90M199hlA9wP/RtnahsLzmbsXfPssTGUfdHu
nuiF5Sgpqn5/tMNakNVr/ACBiSZQeFUf3FQNRzyoOK6zMXCox4HbSFKi4u0UxpYV
CZZgIKByf4xaHjaZGpnY5dTteBQXdRv3Dp85hhmVbDHbO80+7e7zf0BI5QrUm14E
Nhv6PJn6NMBm/GJrvm76+lDt7DJWYygi/3Jupn/aQNpPCZ5bHP+e4e/NC2FMtW3y
Knm+KN5YEA0IWZKnM0s9kIfYeI9PE2tsF3jpdw8U7BTzziLf/6yTVXlJ/5xj0CfU
1kubgcZTp5UhDOWD7RuRQ6WUzbye/Yd/+9C9SNYltidZY7tnlbRyb0J6QerXNakM
tM+eXAbroXnfAZulq8YJO8nFPviXh6Y4F1pEXtpVJIuLTu9NXEDhRLPAzsSXciCa
yQuq98L2UmzpSr9i5ETMmjb7mUPcS9/IR/FQldNgP1/ARY2CTyL6hbdCJH8QieVo
plEUaYPz4QbKTyF/OZsuamDSpqun412Zs/LRgF5kQhIcI1Q0z9SJ4GwQUZb/bwQm
c0ztQnW8AKtzBgGVCYoJSKd4bD5w/Qtv8WdoZJzXnu3GOvq/laS+kCaBcYl3N7Rf
dwDKpYmWitmyIT0THzhdCiJ38rMgq/JjmhCJQiMJJxvvsadl/mPKUO6k4ZUMNw6O
BKrHf/JETN/Wnqd1IPqq
=ohKC
-END PGP SIGNATURE-


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] statistics for array types

2015-08-24 Thread Jeff Janes
On Thu, Aug 20, 2015 at 6:00 PM, Tomas Vondra 
wrote:

> Hi,
>
> On 08/11/2015 04:38 PM, Jeff Janes wrote:
>
>> When reviewing some recent patches, I decided the statistics gathered
>>  for arrays had some pre-existing shortcomings.
>>
>> The main one is that when the arrays contain rare elements there is
>> no histogram to fall back upon when the MCE array is empty, the way
>> there is for scalar stats.  So it has to punt completely and resort
>> to saying that it is 0.5% selectivity without recourse to any data at
>> all.
>>
>> The rationale for applying the threshold before things are eligible
>> for inclusion in the MCE array seems to be that this puts some
>> theoretical bound on the amount of error we are likely to have in
>> that element.  But I think it is better to exceed that theoretical
>> bound than it is to have no data at all.
>>
>> The attached patch forces there to be at least one element in MCE,
>> keeping the one element with the highest predicted frequency if the
>> MCE would otherwise be empty.  Then any other element queried for is
>> assumed to be no more common than this most common element.
>>
>
> We only really need the frequency, right? So do we really need to keep
> the actual MCV element? I.e. most_common_elem_freqs does not have the
> same number of values as most_common_elems anyway:
>
>   A list of the frequencies of the most common element values, i.e., the
>   fraction of rows containing at least one instance of the given value.
>   Two or three additional values follow the per-element frequencies;
>   these are the minimum and maximum of the preceding per-element
>   frequencies, and optionally the frequency of null elements.
>   (Null when most_common_elems is.)
>
> So we might modify it so that it's always defined - either it tracks the
> same values as today (when most_common_elems is defined), or the
> frequency of the most common element (when most_common_elems is NULL).
>

I had also considered that.  It requires more changes to make it happen,
and it seems to create a more complex contract on what those columns mean,
but without giving a corresponding benefit.


>
> This way we can keep the current theoretical error-bound on the MCE
> frequencies, and if that's not possible we can have at least the new
> value without confusing existing code.


But if the frequency of the most common element was grossly wrongly, then
whatever value we stick in there is still going to be grossly wrong.
Removing the value associated with it isn't going to stop it from being
wrong.  When we do query with the (incorrectly thought) first most common
element, either it will find and use the wrong value from slot 1, or it
will find nothing and fall back on the same wrong value from slot 3.


>
> I'd also briefly considered just having the part of the code that
>> pulls the stats out of pg_stats interpret a MCE array as meaning
>> that nothing is more frequent than the threshold, but that would mean
>> that that part of the code needs to know about how the threshold is
>> chosen, which just seems wrong. And it would need to know the
>> difference between NULL MCE because no stats were gathered, versus
>> because stats were gathered but nothing met the threshold.
>>
>
> I'm not sure whether this is the same thing I just proposed ...
>


No, that was yet another option.  "The only way this slot can be null is if
all values were present less than this number of times".  Or if analyze had
never been run.

Cheers,

Jeff


[HACKERS] Planned release for PostgreSQL 9.5

2015-08-24 Thread Paragon Corporation
Just checking to see if you guys have settled on a date for 9.5.0 release.

The PostGIS Dev team would like to release PostGIS 2.2 about or a week or
more before, but not too far ahead of 9.5.0  release.


Thanks,
Regina Obe




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Planned release for PostgreSQL 9.5

2015-08-24 Thread Tom Lane
"Paragon Corporation"  writes:
> Just checking to see if you guys have settled on a date for 9.5.0 release.

No.  Considering we don't have a beta out yet, it's not imminent ...

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-24 Thread Greg Stark
Attached is the pg_regress diff. I believe they are all user-visible
effects of non-iee fp math though I would have expected the rounding
to work right and I'm not clear how gist ends up returning rows in a
different order.

There are still two local changes. The SIGILL handler which is set to
the FPE handler function and initdb is forced to allocate a smaller
shared memory and smaller number of connections to avoid the kernel
panic. I'm running the regression tests with MAX_CONNECTIONS=2.

They take 7h20m to run the regression tests (The git pull actually
only took 7m and rebuilding took three hours)

test tablespace   ... ok
parallel group (20 tests, in groups of 2):  char boolean name varchar
int2 text int4 int8 oid float4 float8 bit txid numeric uuid enum money
rangetypes regproc pg_lsn
 boolean  ... ok
 char ... ok
 name ... ok
 varchar  ... ok
 text ... ok
 int2 ... FAILED
 int4 ... FAILED
 int8 ... FAILED
 oid  ... ok
 float4   ... FAILED
 float8   ... FAILED
 bit  ... ok
 numeric  ... FAILED
 txid ... ok
 uuid ... ok
 enum ... ok
 money... ok
 rangetypes   ... ok
 pg_lsn   ... ok
 regproc  ... ok
test strings  ... ok
test numerology   ... FAILED
parallel group (20 tests, in groups of 2):  lseg point box line path
polygon circle date time timetz timestamp timestamptz abstime interval
reltime tinterval macaddr inet comments tstypes
 point... FAILED
 lseg ... ok
 line ... FAILED
 box  ... ok
 path ... ok
 polygon  ... FAILED
 circle   ... FAILED
 date ... ok
 time ... ok
 timetz   ... ok
 timestamp... ok
 timestamptz  ... ok
 interval ... FAILED
 abstime  ... ok
 reltime  ... ok
 tinterval... ok
 inet ... ok
 macaddr  ... ok
 tstypes  ... ok
 comments ... ok
parallel group (6 tests, in groups of 2):  geometry horology regex
oidjoins type_sanity opr_sanity
 geometry ... FAILED
 horology ... ok
 regex... ok
 oidjoins ... ok
 type_sanity  ... ok
 opr_sanity   ... ok
test insert   ... ok
test insert_conflict  ... ok
test create_function_1... ok
test create_type  ... ok
test create_table ... ok
test create_function_2... ok
parallel group (2 tests):  copyselect copy
 copy ... ok
 copyselect   ... ok
parallel group (2 tests):  create_operator create_misc
 create_misc  ... ok
 create_operator  ... ok
parallel group (2 tests):  create_view create_index
 create_index ... ok
 create_view  ... ok
parallel group (13 tests, in groups of 2):  create_aggregate
create_function_3 create_cast constraints triggers inherit typed_table
create_table_like drop_if_exists vacuum rolenames updatable_views
roleattributes
 create_aggregate ... ok
 create_function_3... ok
 create_cast  ... ok
 constraints  ... ok
 triggers ... ok
 inherit  ... ok
 create_table_like... ok
 typed_table  ... ok
 vacuum   ... ok
 drop_if_exists   ... ok
 updatable_views  ... FAILED
 rolenames... ok
 roleattributes   ... ok
test sanity_check ... ok
test errors   ... ok
test select   ... ok
parallel group (20 tests, in groups of 2):  select_distinct
select_into select_distinct_on select_implicit select_having subselect
case union aggregates join random transactions portals arrays
hash_index btree_index namespace update delete prepared_xacts
 select_into  ... ok
 select_distinct  ... ok
 select_distinct_on   ... ok
 select_implicit  ... ok
 select_having... ok
 subselect... ok
 union... FAILED
 case ... ok
 join ... ok
 aggregates   ... FAILED
 transactions ... 

Re: [HACKERS] psql - better support pipe line

2015-08-24 Thread Pavel Stehule
2015-08-24 16:02 GMT+02:00 Tom Lane :

> Heikki Linnakangas  writes:
> > On 08/24/2015 08:06 AM, Pavel Stehule wrote:
> >> it works perfectly - but the line
> >> xargs -P 3 -I % sh -c "psql % -q -c 'analyze pg_attribute'; echo %"
> >> is little bit ugly - with some psql option it can be cleaned to
> >> xargs -P3 -I % psql % -q --echo-db -c "analyze pg_attribute" | ...
> >> --echo-db requires -q option
> >> What are you thinking about this idea?
>
> > Seems like a one-tricky-pony to me. You're just as likely to need to
> > print a relation name or something else, as the current database.
>
> Not only that, but:
>
> (1) there is no reason to believe that the db name and only the db name
> is needed to do another connection; what about port, host, user, etc?
>

I have to agree - the possibilities is much more than database name - so
one option is not good idea.


>
> (2) this commandeers the pipe connection to transmit out-of-band data,
> making it impossible to use the pipe for its natural function, viz
> transmitting ordinary data from one processing step to the next.  Sure,
> there are use-cases where there's no such data and you can repurpose the
> pipe like that, but that's an enormous limitation.
>

I wrote some bash or perl scripts and I don't think so described style is
less readable than other.

But it has one pretty advantage - paralelism without any line more, without
higher complexity.

Regards

Pavel



>
> > Overall, once your pipeline gets that complicated, I'd rather write a
> > little bash or perl script with for-loops and variables.
>
> Yeah, on the whole this seems like a band-aid to let a bad scripting
> approach limp a few steps further before it collapses completely.
>




>
> regards, tom lane
>


Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-24 Thread Tom Lane
Greg Stark  writes:
> Attached is the pg_regress diff. I believe they are all user-visible
> effects of non-iee fp math though I would have expected the rounding
> to work right and I'm not clear how gist ends up returning rows in a
> different order.

I concur that these are generally unsurprising given what we know about
VAX arithmetic.  The tests that give different integer rounding results
are specifically checking whether the platform does round-to-nearest-even
as specified by IEEE.  It's not surprising that pre-IEEE platforms might
not have chosen that behavior.  The other stuff is due to different
range and precision of FP math, get_floatX_infinity() returning HUGE_VAL
rather than a true infinity, get_floatX_nan() throwing a SIGFPE, etc.

The gist tests in question appear to me to be underdetermined by design
--- for example, the first one is

select p from gist_tbl where p <@ box(point(0,0), point(0.5, 0.5))
order by p <-> point(0.2, 0.2);

and so there is nothing wrong with ordering (0.15,0.15) and (0.25,0.25)
differently, because they're exactly the same distance from (0.2,0.2).
I'm not sure why we've not seen more platform-specific failures on that
test.  Given that it's only existed since Nov 2014, maybe we shouldn't
assume that it's been through the wars yet.  I'm tempted to change the
reference point to (0.201,0.201) or so, so that the correct sort order
is unambiguous.  Heikki, did you make it like that intentionally?

We could eliminate the unexpected FPEs on use of "NaN" if we configured
get_floatX_nan() to throw a "platform does not support NaN" error rather
than intentionally executing an undefined operation.  However, I'm not
sure why we'd bother unless we're going to make VAX a supported platform,
and personally I don't want to change the other tests that are failing
here.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-24 Thread Kouhei Kaigai
> On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas  wrote:
> >
> > On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila  
> > wrote:
> > >> Here is one other issue I found. Existing code assumes a TOC segment has
> > >> only one contents per node type, so it uses pre-defined key (like
> > >> PARALLEL_KEY_SCAN) per node type, however, it is problematic if we put
> > >> multiple PlannedStmt or PartialSeqScan node on a TOC segment.
> > >
> > > We have few keys in parallel-seq-scan patch
> > > (PARALLEL_KEY_TUPLE_QUEUE and PARALLEL_KEY_INST_INFO) for
> > > which multiple structures are shared between master and worker backends.
> > >
> > > Check if something similar can work for your use case.
> >
> > I think you are possibly missing the point.
> 
> It could be possible, but let me summarize what I thought would be required
> for above use case.  For Parallel Append, we need to push multiple
> planned statements in contrast to one planned statement as is done for
> current patch and then one or more parallel workers needs to work on each
> planned statement. So if we know in advance how many planned statements
> are we passing down (which we should), then using ParallelWorkerNumber
> (ParallelWorkerNumber % num_planned_statements or some other similar
> way), workers can find the the planned statement on which they need to work
> and similarly information for PartialSeqScan (which currently is parallel heap
> scan descriptor information).
>
My problem is that we have no identifier to point a particular element on
the TOC segment even if PARALLEL_KEY_PLANNEDSTMT or PARALLEL_KEY_SCAN can
have multiple items.
Please assume a situation when ExecPartialSeqScan() has to lookup
a particular item on TOC but multiple PartialSeqScan nodes can exist.

Currently, it does:
pscan = shm_toc_lookup(node->ss.ps.toc, PARALLEL_KEY_SCAN);

However, ExecPartialSeqScan() cannot know which is the index of mine,
or it is not reasonable to pay attention on other node in this level.
Even if PARALLEL_KEY_SCAN has multiple items, PartialSeqScan node also
needs to have identifier.

> >  I think KaiGai's correct,
> > and I pointed out the same problem to you before.  The parallel key
> > for the Partial Seq Scan needs to be allocated on the fly and carried
> > in the node, or we'll never be able to push multiple things below the
> > funnel.
> 
> Okay, immediately I don't see what is the best way to achieve this but
> let us discuss this separately on Parallel Seq Scan thread and let me
> know if you have something specific in your mind.  I will also give this
> a more thought.
>
I want to have 'node_id' in the Plan node, then unique identifier is
assigned on the field prior to serialization. It is a property of the
Plan node, so we can reproduce this identifier on the background worker
side using stringToNode(), then ExecPartialSeqScan can pull out a proper
field from the TOC segment by this node_id.
Probably, we can co-exist this structure without big changes.

1. Define PARALLEL_KEY_DYNAMIC_LEAST as a least value that is larger
   than any static TOC key (like PARALLEL_KEY_TUPLE_QUEUE).
2. Run plan-tree node walker on InitializeParallelWorkers, just before
   nodeToString(), to assign node_id larger than the above label and
   with increasing for each node.
3. Use node_id instead of the static PARALLEL_KEY_SCAN on
   ExecPartialSeqScan

Even though we need some more trivial fixes are needed, it seems to
me the above approach shall work.
Also, please note that I don't assume only PartialSeqScan want to
have its field on TOC segment, but some CustomScan node also wants
to have its own shared field when co-working under Funnel node.


On the other hand, I think it is too aggressive to complete the
initial work of this patch by the starting day of the next commit
fest, so I think the target is middle of October.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-24 Thread Kouhei Kaigai
Fujita-san,

How about your opinion towards the solution?
CF:Sep will start next week, so I'd like to make a consensus of
the direction, at least.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei 


> -Original Message-
> From: pgsql-hackers-ow...@postgresql.org
> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Kouhei Kaigai
> Sent: Thursday, August 13, 2015 10:13 AM
> To: Etsuro Fujita; Robert Haas
> Cc: PostgreSQL-development; 花田茂
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
> 
> Fujita-san,
> 
> The attached patch enhanced the FDW interface according to the direction
> below (but not tested yet).
> 
> >> In the summary, the following three enhancements are a straightforward
> >> way to fix up the problem he reported.
> >> 1. Add a special path to call recheckMtd in ExecScanFetch if scanrelid==0
> >> 2. Add a callback of FDW in ForeignRecheck() - to construct a record
> >> according to the fdw_scan_tlist definition and to evaluate its
> >> visibility, or to evaluate qualifier pushed-down if base relation.
> >> 3. Add List *fdw_paths in ForeignPath like custom_paths of CustomPaths,
> >> to construct plan nodes for EPQ evaluation.
> 
> Likely, what you need to do are...
> 1. Save the alternative path on fdw_paths when foreign join push-down.
>GetForeignJoinPaths() may be called multiple times towards a particular
>joinrel according to the combination of innerrel/outerrel.
>RelOptInfo->fdw_private allows to avoid construction of same remote
>join path multiple times. On the second or later invocation, it may be
>a good tactics to reference cheapest_startup_path and replace the saved
>one if later invocation have cheaper one, prior to exit.
> 2. Save the alternative Plan nodes on fdw_plans or lefttree/righttree
>somewhere you like at the GetForeignPlan()
> 3. Makes BeginForeignScan() to call ExecInitNode() towards the plan node
>saved at (2), then save the PlanState on fdw_ps, lefttree/righttree,
>or somewhere private area if not displayed on EXPLAIN.
> 4. Implement ForeignRecheck() routine. If scanrelid==0, it kicks the
>planstate node saved at (3) to generate tuple slot. Then, call the
>ExecQual() to check qualifiers being pushed down.
> 5. Makes EndForeignScab() to call ExecEndNode() towards the PlanState
>saved at (3).
> 
> I never think above steps are "too" complicated for people who can write
> FDW drivers. It is what developer usually does.
> 
> Thanks,
> --
> NEC Business Creation Division / PG-Strom Project
> KaiGai Kohei 
> 
> 
> > -Original Message-
> > From: Kaigai Kouhei(海外 浩平)
> > Sent: Wednesday, August 12, 2015 11:17 PM
> > To: 'Etsuro Fujita'; Robert Haas
> > Cc: PostgreSQL-development; 花田茂
> > Subject: RE: [HACKERS] Foreign join pushdown vs EvalPlanQual
> >
> > > -Original Message-
> > > From: pgsql-hackers-ow...@postgresql.org
> > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita
> > > Sent: Wednesday, August 12, 2015 8:26 PM
> > > To: Robert Haas; Kaigai Kouhei(海外 浩平)
> > > Cc: PostgreSQL-development; 花田茂
> > > Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
> > >
> > > On 2015/08/12 7:21, Robert Haas wrote:
> > > > On Fri, Aug 7, 2015 at 3:37 AM, Kouhei Kaigai  
> > > > wrote:
> > > >>> I could have a discussion with Fujita-san about this topic.
> > > >>>
> > > >> Also, let me share with the discussion towards entire solution.
> > > >>
> > > >> The primitive reason of this problem is, Scan node with scanrelid==0
> > > >> represents a relation join that can involve multiple relations, thus,
> > > >> its TupleDesc of the records will not fit base relations, however,
> > > >> ExecScanFetch() was not updated when scanrelid==0 gets supported.
> > > >>
> > > >> FDW/CSP on behalf of the Scan node with scanrelid==0 are responsible
> > > >> to generate records according to the fdw_/custom_scan_tlist that
> > > >> reflects the definition of relation join, and only FDW/CSP know how
> > > >> to combine these base relations.
> > > >> In addition, host-side expressions (like Plan->qual) are initialized
> > > >> to reference the records generated by FDW/CSP, so the least invasive
> > > >> approach is to allow FDW/CSP to have own logic to recheck, I think.
> > > >>
> > > >> Below is the structure of ExecScanFetch().
> > > >>
> > > >>ExecScanFetch(ScanState *node,
> > > >>  ExecScanAccessMtd accessMtd,
> > > >>  ExecScanRecheckMtd recheckMtd)
> > > >>{
> > > >>EState *estate = node->ps.state;
> > > >>
> > > >>if (estate->es_epqTuple != NULL)
> > > >>{
> > > >>/*
> > > >> * We are inside an EvalPlanQual recheck.  Return the test 
> > > >> tuple
> > > if
> > > >> * one is available, after rechecking any
> access-method-specific
> > > >> * conditions.
> > > >> */
> > > >>Index   scanrelid = ((Sc

[HACKERS] pg_controldata output alignment regression

2015-08-24 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Do we care that as of 9.5 pg_controldata output is not 100% aligned
anymore? The culprit is:

  Current track_commit_timestamp setting: off

Its value is shifted 2 characters to the right with respect to all the
others. I think it ought to be fixed but thought I'd get opinions first.

Joe

- -- 
Crunchy Data - http://crunchydata.com
PostgreSQL Support for Secure Enterprises
Consulting, Training, & Open Source Development
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJV28z9AAoJEDfy90M199hlFUkP/3prO8QakKsHG4+kDu2td2Oj
ITkOTm0fE67BxalHr2UkkH5kMrcbFaXsluzPbuCHW/i3VMemPiRaQmexpgoD7NUG
qAmMv5BNSYaU02iRp9Ay32g9Ohoh/OrZfD8MNCWyvTmxVB730I1bhRxl308S+Y3J
gqiy+qufVuTa4O65N2+5xGSUXpP352kL+/m6+xprb4LY7gaVpAoxd3wFrSj6A6O5
MJvOTHSoM67A8UXuCs2PVzyhb9U+egJ5IaAI7ItMgx7L+83ZziHEumqe3VYI+AW8
+vL5FtCl09rpE56npgG+2LxGSN/yhdiYOrSN3FqCG/UuuXKwwXnEv70i+71/NpAO
Ychb/5c8pmo7dZFR6H6mbtWYDjdaGurtCwe2uEG9C41cXDpztYZHePMgFwCAKwdm
syHGeWN9YWfKq8US2NkOiGcU2pTzoc6aQeU1U0lJSjEwCzn2d7aTTUJUxIHkLgcg
54GQ+qVbi4N+mmJ1ME39gK1tJObp4bEOGz1ZEryACi+xnneyfncxdP8lRaYwCkWS
YkYLAi+Ojic/20Eha/d4DuWLMjsNUgfBY2InsT8R1bmMDRInfuEaffclrQrk662x
GggmRkrJ0AQzLn+T8zo9N8G9veWeLy7He0gz2LtNCQzDKEIJ68FeIC/ZltJ14PF+
9WivdTJa5W3pkAohatKF
=Ax9S
-END PGP SIGNATURE-


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_controldata output alignment regression

2015-08-24 Thread Tom Lane
Joe Conway  writes:
> Do we care that as of 9.5 pg_controldata output is not 100% aligned
> anymore? The culprit is:
>   Current track_commit_timestamp setting: off
> Its value is shifted 2 characters to the right with respect to all the
> others. I think it ought to be fixed but thought I'd get opinions first.

Seems to me we could s/Current //g, or s/ setting//g, or both,
and get rid of the problem without adding more whitespace.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-24 Thread Amit Kapila
On Tue, Aug 25, 2015 at 6:19 AM, Kouhei Kaigai  wrote:
>
> > On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas 
wrote:
> >
> > It could be possible, but let me summarize what I thought would be
required
> > for above use case.  For Parallel Append, we need to push multiple
> > planned statements in contrast to one planned statement as is done for
> > current patch and then one or more parallel workers needs to work on
each
> > planned statement. So if we know in advance how many planned statements
> > are we passing down (which we should), then using ParallelWorkerNumber
> > (ParallelWorkerNumber % num_planned_statements or some other similar
> > way), workers can find the the planned statement on which they need to
work
> > and similarly information for PartialSeqScan (which currently is
parallel heap
> > scan descriptor information).
> >
> My problem is that we have no identifier to point a particular element on
> the TOC segment even if PARALLEL_KEY_PLANNEDSTMT or PARALLEL_KEY_SCAN can
> have multiple items.
> Please assume a situation when ExecPartialSeqScan() has to lookup
> a particular item on TOC but multiple PartialSeqScan nodes can exist.
>
> Currently, it does:
> pscan = shm_toc_lookup(node->ss.ps.toc, PARALLEL_KEY_SCAN);
>
> However, ExecPartialSeqScan() cannot know which is the index of mine,
> or it is not reasonable to pay attention on other node in this level.
> Even if PARALLEL_KEY_SCAN has multiple items, PartialSeqScan node also
> needs to have identifier.
>

Yes that's right and I think we can find out the same.  Basically we need to
know the planned statement number on which current worker is working and
that anyway we have to do before the worker can start the work.  One way is
as I have explained above that use ParallelWorkerNumber
(ParallelWorkerNumber % num_planned_statements) to find or might need
some sophisticated way to find that out, but definitely we need to know that
before start of execution by worker and once we know that we can use it
find the PARALLEL_KEY_SCAN or whatever key for this worker (as the
the position of PARALLEL_KEY_SCAN will be same as of planned stmt
for a worker).


> > >  I think KaiGai's correct,
> > > and I pointed out the same problem to you before.  The parallel key
> > > for the Partial Seq Scan needs to be allocated on the fly and carried
> > > in the node, or we'll never be able to push multiple things below the
> > > funnel.
> >
> > Okay, immediately I don't see what is the best way to achieve this but
> > let us discuss this separately on Parallel Seq Scan thread and let me
> > know if you have something specific in your mind.  I will also give this
> > a more thought.
> >
> I want to have 'node_id' in the Plan node, then unique identifier is
> assigned on the field prior to serialization. It is a property of the
> Plan node, so we can reproduce this identifier on the background worker
> side using stringToNode(), then ExecPartialSeqScan can pull out a proper
> field from the TOC segment by this node_id.
>

Okay, this can also work, but why to introduce identifier in plan node, if
it
can work without it.


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


Re: [HACKERS] Performance improvement for joins where outer side is unique

2015-08-24 Thread David Rowley
On 24 August 2015 at 14:29, Tom Lane  wrote:

> David Rowley  writes:
> > I have to admit I don't much like it either, originally I had this as an
> > extra property that was only seen in EXPLAIN VERBOSE.
>
> Seems like a reasonable design from here.


The attached patch has the format in this way.


> (Note that for non-text output,
> I'd say the field should come out unconditionally.  We only care about

abbreviating in text mode.)
>
>
If that's the case then why do we not enable verbose for all of the
non-text outputs?
It seems strange to start making exceptions on a case-by-case basis.

Regards

David Rowley

--
 David Rowley   http://www.2ndQuadrant.com/

 PostgreSQL Development, 24x7 Support, Training & Services


unique_joins_2015-08-25_feb3068.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers