Re: [HACKERS] Indirect indexes

2016-12-28 Thread Alvaro Herrera
Here's v3. This one applies on top of the "interesting-attrs" patch I sent a few hours ago: https://www.postgresql.org/message-id/20161228232018.4hc66ndrzpz4g4wn@alvherre.pgsql and contains a number of bugfixes that I discovered on my own (and which therefore require no further explanation, since

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-12-28 Thread Dilip Kumar
On Fri, Dec 23, 2016 at 8:28 AM, Amit Kapila wrote: > The results look positive. Do you think we can conclude based on all > the tests you and Dilip have done, that we can move forward with this > patch (in particular group-update) or do you still want to do more > tests? I am aware that in one

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Ashutosh Bapat
On Wed, Dec 28, 2016 at 10:55 PM, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Stephen Frost writes: >> > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> >> I think it's an awful choice of name; it has nothing to do with either >> >> the functionality or the printed name of

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Ashutosh Bapat
On Wed, Dec 28, 2016 at 10:30 PM, Tom Lane wrote: > Stephen Frost writes: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> I think it's an awful choice of name; it has nothing to do with either >>> the functionality or the printed name of the field. > >> As an example, we might some day wish to incl

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Robert Haas
On Wed, Dec 28, 2016 at 1:06 PM, Tom Lane wrote: > Jim Nasby writes: >> On 12/28/16 11:25 AM, Tom Lane wrote: >>> The idea of just capturing the wait start for heavyweight locks, and >>> not other lock types, still seems superior to any of the alternatives >>> that have been suggested ... > >> Is

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Amit Kapila
On Wed, Dec 28, 2016 at 8:41 PM, Stephen Frost wrote: > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: >> >> One can use this option as >> >> postgres=# explain (summary on) select * from pg_class c, pg_type t >> >> where c.reltype = t.oid; >> >> QUERY PL

Re: [HACKERS] Duplicate node tag assignments

2016-12-28 Thread Alvaro Herrera
Amit Kapila wrote: > On Wed, Dec 28, 2016 at 10:03 PM, Tom Lane wrote: > > Or we could just abandon the manually-assigned breaks in the list > > altogether, and let NodeTags run from 1 to whatever. This would > > slightly complicate debugging, in that the numeric values of node > > tags would ch

Re: [HACKERS] Duplicate node tag assignments

2016-12-28 Thread Amit Kapila
On Wed, Dec 28, 2016 at 10:03 PM, Tom Lane wrote: > By chance I happened to notice that the recent partition patch pushed > us over the number of available node tags between > > T_A_Expr = 900, > > and > > T_TriggerData = 950,/* in commands/trigger.h */ > > Specifically

Re: [HACKERS] Proposal : Parallel Merge Join

2016-12-28 Thread Dilip Kumar
On Thu, Dec 29, 2016 at 3:15 AM, Tomas Vondra wrote: > FWIW, I've done quite a bit of testing on this patch, and also on the other > patches adding parallel index scans and bitmap heap scan. I've been running > TPC-H and TPC-DS on 16GB data sets with each patch, looking for regressions > or crashe

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Amit Kapila
On Wed, Dec 28, 2016 at 10:55 PM, Tom Lane wrote: > Jim Nasby writes: >> On 12/28/16 7:10 AM, Amit Kapila wrote: >>> Can we think of introducing new guc trace_system_waits or something >>> like that which will indicate that the sessions will report the value >>> of wait_start in pg_stat_activity?

Re: [HACKERS] Support for pg_receivexlog --format=plain|tar

2016-12-28 Thread Michael Paquier
On Wed, Dec 28, 2016 at 9:31 PM, Magnus Hagander wrote: > Conditional tests? It probably wouldn't hurt to have them, but that would be > something more generic (like we'd need something to actually validate it -- > but it would make sense to have a test that, with compression enabled, would > veri

[HACKERS] rewrite HeapSatisfiesHOTAndKey

2016-12-28 Thread Alvaro Herrera
Pursuant to my comments at https://www.postgresql.org/message-id/20161223192245.hx4rbrxbrwtgwj6i@alvherre.pgsql and because of a stupid bug I found in my indirect indexes patch, I rewrote (read: removed) HeapSatisfiesHOTAndKey. The replacement function HeapDetermineModifiedColumns returns a bitmap

Re: [HACKERS] Improving RLS planning

2016-12-28 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Here's an updated version of the RLS planning patch that gets rid of >> the incorrect interaction with Query.hasRowSecurity and adjusts >> terminology as agreed. > I've spent a fair bit of time going over this change to understand

Re: [HACKERS] Proposal : Parallel Merge Join

2016-12-28 Thread Tomas Vondra
Hi, On 12/21/2016 04:53 PM, Dilip Kumar wrote: On Wed, Dec 21, 2016 at 8:39 PM, Robert Haas wrote: Committed the refactoring patch with some mild cosmetic adjustments. Thanks.. As to the second patch: +if (jointype == JOIN_UNIQUE_INNER) +jointype = JOIN_INNER; Isn't t

Re: [HACKERS] Logical tape pause/resume

2016-12-28 Thread Peter Geoghegan
On Wed, Dec 21, 2016 at 4:25 AM, Heikki Linnakangas wrote: > In the meanwhile, Robert committed the cap on the number of tapes. Since > that's in, I'm not sure if the pause/resume part of this is worthwhile. > Maybe it is. I rebased my parallel tuplesort patch on top of what you committed a few d

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-28 Thread Claudio Freire
On Wed, Dec 28, 2016 at 3:41 PM, Claudio Freire wrote: >> Anyway, I found the problem that had caused segfault. >> >> for (segindex = 0; segindex <= vacrelstats->dead_tuples.last_seg; tupindex = >> 0, segindex++) >> { >> DeadTuplesSegment *seg = >> &(vacrelstats->dead_tuples.dead_tuples[segind

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Pavel Stehule
2016-12-28 20:25 GMT+01:00 Jim Nasby : > On 12/28/16 12:51 PM, Pavel Stehule wrote: > >> Now, the incompatibility can be hard issue - it is big question if we >> lock some users on old versions because some users can save to lines of >> code. Introduction of ROW_COUNT is lowly incompatibility - it

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Jim Nasby
On 12/28/16 12:51 PM, Pavel Stehule wrote: Now, the incompatibility can be hard issue - it is big question if we lock some users on old versions because some users can save to lines of code. Introduction of ROW_COUNT is lowly incompatibility - it can be simply detected - but for example change of

Re: [HACKERS] Improving RLS planning

2016-12-28 Thread Tom Lane
Stephen Frost writes: > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: >> Hmm. I've not read any of the new code yet, but the fact that this >> test now reduces to a one-time filter makes it effectively useless as >> a test of qual evaluation order because it has deduced that it doesn't >> need

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Pavel Stehule
2016-12-28 18:54 GMT+01:00 Jim Nasby : > On 12/28/16 7:16 AM, Pavel Stehule wrote: > >> >> >> 2016-12-28 5:09 GMT+01:00 Jim Nasby > >: >> >> On 12/27/16 4:56 PM, Merlin Moncure wrote: >> >> On Tue, Dec 27, 2016 at 1:54 AM, Pavel Stehule >> mailt

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Pavel Stehule
2016-12-28 19:23 GMT+01:00 Jim Nasby : > On 12/28/16 12:15 PM, Pavel Stehule wrote: > >> GUC are fragile - the source code and settings can be separated. >> > > *Can* be, but they don't *have* to be. That's a huge feature, not a bug. > > Our #option is more robust, because source code holds all fl

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-28 Thread Claudio Freire
On Wed, Dec 28, 2016 at 10:26 AM, Anastasia Lubennikova wrote: > 27.12.2016 20:14, Claudio Freire: > > On Tue, Dec 27, 2016 at 10:41 AM, Anastasia Lubennikova > wrote: > > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x006941e7 in lazy_vacuum_heap (onerel=0x1ec2360, >

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 19:17 GMT+01:00 Jim Nasby : > On 12/28/16 11:29 AM, Fabien COELHO wrote: > >> >> Hello Jim, >> >> 1) Variables would be completely non-transactional. [...] A solution >>> to this problem would be to provide a plpgsql equivalent to plperl or >>> plpython's session hashes. >>> >> >> That

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Jim Nasby
On 12/28/16 12:15 PM, Pavel Stehule wrote: GUC are fragile - the source code and settings can be separated. *Can* be, but they don't *have* to be. That's a huge feature, not a bug. Our #option is more robust, because source code holds all flags required for execution. So I would to see a mech

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Joe Conway
On 12/28/2016 10:17 AM, Jim Nasby wrote: > Then IMHO what needs to happen is to have a discussion on actual syntax > instead of calling into question the value of the feature. Following > this thread has been very painful because the communications have not > been very clear. Focus on grammar would

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Jim Nasby
On 12/28/16 11:29 AM, Fabien COELHO wrote: Hello Jim, 1) Variables would be completely non-transactional. [...] A solution to this problem would be to provide a plpgsql equivalent to plperl or plpython's session hashes. That is what I have in mind with "session variables" à la MS/MY SQL, but

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Pavel Stehule
2016-12-28 18:54 GMT+01:00 Jim Nasby : > On 12/28/16 7:16 AM, Pavel Stehule wrote: > >> ** The real problem is that we have no mechanism for allowing a PL's >> language/syntax/API to move forward without massive backwards >> compatibility problems. ** >> >> >> We have not, but there ar

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Tom Lane
Jim Nasby writes: > On 12/28/16 11:25 AM, Tom Lane wrote: >> The idea of just capturing the wait start for heavyweight locks, and >> not other lock types, still seems superior to any of the alternatives >> that have been suggested ... > Is some kind of alarm a viable option for the others? If set

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Jim Nasby
On 12/28/16 11:25 AM, Tom Lane wrote: The idea of just capturing the wait start for heavyweight locks, and not other lock types, still seems superior to any of the alternatives that have been suggested ... Is some kind of alarm a viable option for the others? If setting the alarm is cheap, you

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Jim Nasby
On 12/28/16 7:16 AM, Pavel Stehule wrote: 2016-12-28 5:09 GMT+01:00 Jim Nasby mailto:jim.na...@bluetreble.com>>: On 12/27/16 4:56 PM, Merlin Moncure wrote: On Tue, Dec 27, 2016 at 1:54 AM, Pavel Stehule mailto:pavel.steh...@gmail.com>> wrote: Which is why this is an ex

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Jim Nasby
On 12/28/16 7:16 AM, Pavel Stehule wrote: ** The real problem is that we have no mechanism for allowing a PL's language/syntax/API to move forward without massive backwards compatibility problems. ** We have not, but there are few possibilities: 1. enhance #option command 2. we can

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 17:53 GMT+01:00 Jim Nasby : > On 12/28/16 9:57 AM, Fabien COELHO wrote: > >> * Other later triggers, etc, also reference USER_IS_AUDITOR >>> >> >> The variable is not directly referenced, one would have to call >> isUserAuditor() to access the private session value, but then you can >>

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Fabien COELHO
Hello Jim, 1) Variables would be completely non-transactional. [...] A solution to this problem would be to provide a plpgsql equivalent to plperl or plpython's session hashes. That is what I have in mind with "session variables" à la MS/MY SQL, but at the SQL level, not PL/pgSQL level. 2

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I think it's an awful choice of name; it has nothing to do with either > >> the functionality or the printed name of the field. > > > As an example, we might some day wish to inclu

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Tom Lane
Jim Nasby writes: > On 12/28/16 7:10 AM, Amit Kapila wrote: >> Can we think of introducing new guc trace_system_waits or something >> like that which will indicate that the sessions will report the value >> of wait_start in pg_stat_activity? > In my experience the problem with those kind of setti

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Jim Nasby
On 12/28/16 7:10 AM, Amit Kapila wrote: Can we think of introducing new guc trace_system_waits or something like that which will indicate that the sessions will report the value of wait_start in pg_stat_activity? The default value of such a parameter can be false which means wait_start will be s

Re: [HACKERS] Hooks

2016-12-28 Thread Jim Nasby
On 12/28/16 10:43 AM, David Fetter wrote: Callbacks aren't easy to find either. Should callbacks be another chapter in the docs? That would also be nice, but I suspect that will be harder than finding all the hooks. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in A

Re: [HACKERS] Hooks

2016-12-28 Thread Jim Nasby
On 12/27/16 11:14 PM, David Fetter wrote: Sure, but that seems like an effort somewhat orthogonal to the one I proposed, which is to get some user-facing i.e. SGML docs up for the current hooks. My point was that a (presumably small) amount of effort towards earmarking hooks in code so that a

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I think it's an awful choice of name; it has nothing to do with either >> the functionality or the printed name of the field. > As an example, we might some day wish to include a summary of buffer > information at the bottom too wh

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Jim Nasby
On 12/28/16 9:57 AM, Fabien COELHO wrote: * Other later triggers, etc, also reference USER_IS_AUDITOR The variable is not directly referenced, one would have to call isUserAuditor() to access the private session value, but then you can GRANT/REVOKE whatever you want on the access function. Wh

Re: [HACKERS] Duplicate node tag assignments

2016-12-28 Thread Tom Lane
Andres Freund writes: > On 2016-12-28 11:33:31 -0500, Tom Lane wrote: >> By chance I happened to notice that the recent partition patch pushed >> us over the number of available node tags between >> So I'm leaning to the second, more drastic, solution. Thoughts? > Alternatively we could add a -W

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > >>> I am not sure whether using *summary* to print just planning time is a > >>> good idea. Another option could be SUMMARY_PLAN_TIME. > > > Using 'summary' seems

Re: [HACKERS] [PATCH] Fix for documentation of timestamp type

2016-12-28 Thread Cynthia Shang
The latest patch attachment has a couple typos in it ("storead" instead of "stored"). I interpreted the final suggestion in the thread to mean 1) default stores in microseconds 2) deprecated compile-time option stores as seconds. If these assumptions are correct then the suggestion in the threa

Re: [HACKERS] Hooks

2016-12-28 Thread David Fetter
On Wed, Dec 28, 2016 at 12:19:11PM +0800, Craig Ringer wrote: > On 28 December 2016 at 12:15, Jim Nasby wrote: > > > Can we reduce the scope of this to a manageable starting point? > > I'm guessing that all existing hooks share certain characteristics > > that it'd be pretty easy to detect. If yo

Re: [HACKERS] Duplicate node tag assignments

2016-12-28 Thread Andres Freund
Hi, On 2016-12-28 11:33:31 -0500, Tom Lane wrote: > By chance I happened to notice that the recent partition patch pushed > us over the number of available node tags between > > T_A_Expr = 900, > So I'm leaning to the second, more drastic, solution. Thoughts? Alternatively we could add a

[HACKERS] Duplicate node tag assignments

2016-12-28 Thread Tom Lane
By chance I happened to notice that the recent partition patch pushed us over the number of available node tags between T_A_Expr = 900, and T_TriggerData = 950,/* in commands/trigger.h */ Specifically we now have some of the replication grammar node type codes confli

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 16:57 GMT+01:00 Fabien COELHO : > > My 0.02€ to try to illustrate a possible private session variable based > implementation for this use case: > > * Session starts >> > > \c app > > * app does SELECT setup_user('user-auth-key-data', 'some-other-blob') >> > > SELECT setup_user('fjshdfjk

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Fabien COELHO
My 0.02€ to try to illustrate a possible private session variable based implementation for this use case: * Session starts \c app * app does SELECT setup_user('user-auth-key-data', 'some-other-blob') SELECT setup_user('fjshdfjkshfjks', 'jklfsjfklsjfk'); ** setup_user is SECURITY DEFI

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 16:42 GMT+01:00 Fabien COELHO : > > Hello, > > Why should they? If it is a session variable, being created when needed or >>> used with the right type could be enough? >>> >> >> You cannot to trust some fuzzy object - or you have to play hard game with >> securing content - hashing, cod

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Fabien COELHO
Hello, Why should they? If it is a session variable, being created when needed or used with the right type could be enough? You cannot to trust some fuzzy object - or you have to play hard game with securing content - hashing, coding, decoding - it is slow, cpu intensive I'm afraid that I d

Re: [HACKERS] make more use of RoleSpec struct

2016-12-28 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > > > The only functional issue might be the removal of the IsA() checks. If > > > we don't cast any Node before passing it to any of those functions, > > > there sho

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Alvaro Herrera
Tom Lane wrote: > How about just saying that the existing TIMING option turns this on, > if it's specified without ANALYZE? Right now that combination draws > an error: > > regression=# explain (timing on) select 1; > ERROR: EXPLAIN option TIMING requires ANALYZE > > so there's no

Re: [HACKERS] make more use of RoleSpec struct

2016-12-28 Thread Alvaro Herrera
Stephen Frost wrote: > * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > > The only functional issue might be the removal of the IsA() checks. If > > we don't cast any Node before passing it to any of those functions, > > there should be no problem because any misfeasance will be reported as a

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Tom Lane
Stephen Frost writes: > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: >>> I am not sure whether using *summary* to print just planning time is a >>> good idea. Another option could be SUMMARY_PLAN_TIME. > Using 'summary' seems entirely reasonable to me, I think it's an awful choice

Re: [HACKERS] Hooks

2016-12-28 Thread David Fetter
On Wed, Dec 28, 2016 at 03:07:52PM +0900, Michael Paquier wrote: > On Wed, Dec 28, 2016 at 2:14 PM, David Fetter wrote: > > Here's everything that matches ^\w+_hook$ that I've found so far in > > git master. There are very likely false positives in this list. > > > > [... long list of hooks ...]

Re: [HACKERS] make more use of RoleSpec struct

2016-12-28 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Peter Eisentraut wrote: > > Here is a small cleanup patch to make more use of the RoleSpec > > struct/node throughout the parser to avoid casts and make the code more > > self-documenting. > > This makes sense to me. I started to do this at som

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Stephen Frost
* Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > >> One can use this option as > >> postgres=# explain (summary on) select * from pg_class c, pg_type t > >> where c.reltype = t.oid; > >> QUERY PLAN > >> -

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Craig Ringer
On 28 December 2016 at 22:04, Pavel Stehule wrote: > For security the variable should be persistent. > > If you would to do statical analyse (what you usually would), then variable > should be persistent. > > Currently the big issue of plpgsql_check is work with temporary tables. > Local objects

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 15:38 GMT+01:00 Fabien COELHO : > > For security the variable should be persistent. >> > > Why should they? If it is a session variable, being created when needed or > used with the right type could be enough? > You cannot to trust some fuzzy object - or you have to play hard game with

Re: [HACKERS] make more use of RoleSpec struct

2016-12-28 Thread Alvaro Herrera
Peter Eisentraut wrote: > Here is a small cleanup patch to make more use of the RoleSpec > struct/node throughout the parser to avoid casts and make the code more > self-documenting. This makes sense to me. I started to do this at some point when I was writing RoleSpec; eventually got annoyed abo

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Fabien COELHO
For security the variable should be persistent. Why should they? If it is a session variable, being created when needed or used with the right type could be enough? If you would to do statical analyse (what you usually would), then variable should be persistent. I do not understand what

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Fabien COELHO
Also, I'm not yet convinced that simple privatizable transcient/session variables would not be enough to fit the use case, [...] So... maybe? The main question then becomes how you integrate access control. For what it's worth, permissions on persistent functions could be used to control a

[HACKERS] make more use of RoleSpec struct

2016-12-28 Thread Peter Eisentraut
Here is a small cleanup patch to make more use of the RoleSpec struct/node throughout the parser to avoid casts and make the code more self-documenting. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From 6f65b0

Re: [HACKERS] [PATCH] Fix minor race in commit_ts SLRU truncation vs lookups

2016-12-28 Thread Petr Jelinek
On 28/12/16 15:01, Craig Ringer wrote: > Hi all > > There's a minor race between commit_ts SLRU truncation and concurrent > commit_ts lookups, where a lookup can check the lower valid bound xid > without knowing it's already been truncated away. This would result in > a SLRU lookup error. > > It'

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 15:00 GMT+01:00 Craig Ringer : > On 28 December 2016 at 21:19, Fabien COELHO wrote: > > > Also, I'm not yet convinced that simple privatizable transcient/session > > variables would not be enough to fit the use case, so that for the same > > price there would be session variables for a

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Craig Ringer
On 28 December 2016 at 21:19, Fabien COELHO wrote: > Also, I'm not yet convinced that simple privatizable transcient/session > variables would not be enough to fit the use case, so that for the same > price there would be session variables for all, not only special ones with > permissions. Since

[HACKERS] [PATCH] Fix minor race in commit_ts SLRU truncation vs lookups

2016-12-28 Thread Craig Ringer
Hi all There's a minor race between commit_ts SLRU truncation and concurrent commit_ts lookups, where a lookup can check the lower valid bound xid without knowing it's already been truncated away. This would result in a SLRU lookup error. It's pretty low-harm since it's hard to trigger and the on

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 4:40 GMT+01:00 Craig Ringer : > Fabien, I don't really see the point of "persistent variables". What > benefit do they add over relations? > > You can add a simple function to fetch a tuple if you want it not to > look like a subquery. Do it with heap access in C if you like, save > the

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Ashutosh Bapat
>> >> One can use this option as >> postgres=# explain (summary on) select * from pg_class c, pg_type t >> where c.reltype = t.oid; >> QUERY PLAN >> -- >> Hash Join (cost=17.12..35.70 rows=319

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-28 Thread Anastasia Lubennikova
27.12.2016 20:14, Claudio Freire: On Tue, Dec 27, 2016 at 10:41 AM, Anastasia Lubennikova wrote: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x006941e7 in lazy_vacuum_heap (onerel=0x1ec2360, vacrelstats=0x1ef6e00) at vacuumlazy.c:1417 1417tblk = Item

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Pavel Stehule
2016-12-28 14:19 GMT+01:00 Fabien COELHO : > > Hello Craig, > > Fabien, I don't really see the point of "persistent variables". What >> benefit do they add over relations? >> > > A relation is a set of values, a variable is a scalar with one value. > > It is always possible to declare a set and us

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Michael Banck
On Wed, Dec 28, 2016 at 02:08:55PM +0100, Michael Banck wrote: > On Tue, Dec 27, 2016 at 09:26:21AM -0500, Stephen Frost wrote: > > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > > > We report planning and execution time when EXPLAIN ANALYZE is issued. > > > We do not have facility to

Re: [HACKERS] proposal: session server side variables

2016-12-28 Thread Fabien COELHO
Hello Craig, Fabien, I don't really see the point of "persistent variables". What benefit do they add over relations? A relation is a set of values, a variable is a scalar with one value. It is always possible to declare a set and use it as a singleton, but somehow it seems cleaner to ask f

Re: [HACKERS] merging some features from plpgsql2 project

2016-12-28 Thread Pavel Stehule
2016-12-28 5:09 GMT+01:00 Jim Nasby : > On 12/27/16 4:56 PM, Merlin Moncure wrote: > >> On Tue, Dec 27, 2016 at 1:54 AM, Pavel Stehule >> wrote: >> >>> First I describe my initial position. I am strongly against introduction >>> "new" language - plpgsql2 or new plpgsql, or any else. The trust of

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Michael Banck
On Tue, Dec 27, 2016 at 09:26:21AM -0500, Stephen Frost wrote: > * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > > We report planning and execution time when EXPLAIN ANALYZE is issued. > > We do not have facility to report planning time as part EXPLAIN > > output. In order to get the pl

Re: [HACKERS] pg_stat_activity.waiting_start

2016-12-28 Thread Amit Kapila
On Sat, Dec 24, 2016 at 7:46 AM, Tom Lane wrote: > Stephen Frost writes: >> * Tom Lane (t...@sss.pgh.pa.us) wrote: >>> The difficulty with that is it'd require a gettimeofday() call for >>> every wait start. Even on platforms where those are relatively cheap, >>> the overhead would be nasty ---

Re: [HACKERS] Reporting planning time with EXPLAIN

2016-12-28 Thread Amit Kapila
On Tue, Dec 27, 2016 at 1:27 PM, Ashutosh Bapat wrote: > Hi, > We report planning and execution time when EXPLAIN ANALYZE is issued. > We do not have facility to report planning time as part EXPLAIN > output. In order to get the planning time, one has to issue EXPLAIN > ANALYZE which involves exec

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-28 Thread Anastasia Lubennikova
27.12.2016 16:54, Alvaro Herrera: Anastasia Lubennikova wrote: I ran configure using following set of flags: ./configure --enable-tap-tests --enable-cassert --enable-debug --enable-depend CFLAGS="-O0 -g3 -fno-omit-frame-pointer" And then ran make check. Here is the stacktrace: Program termin

Re: [HACKERS] Support for pg_receivexlog --format=plain|tar

2016-12-28 Thread Magnus Hagander
On Wed, Dec 28, 2016 at 6:58 AM, Michael Paquier wrote: > On Wed, Dec 28, 2016 at 3:12 AM, Magnus Hagander > wrote: > > On Tue, Dec 27, 2016 at 1:16 PM, Michael Paquier < > michael.paqu...@gmail.com> > > wrote: > >> On Tue, Dec 27, 2016 at 6:34 PM, Magnus Hagander > >> wrote: > >> > On Tue, Dec

Re: [HACKERS] parallelize queries containing subplans

2016-12-28 Thread Rafia Sabih
On Wed, Dec 28, 2016 at 11:47 AM, Amit Kapila wrote: > > Currently, queries that have references to SubPlans or > AlternativeSubPlans are considered parallel-restricted. I think we > can lift this restriction in many cases especially when SubPlans are > parallel-safe. To make this work, we need

Re: [HACKERS] Measuring replay lag

2016-12-28 Thread Thomas Munro
On Thu, Dec 22, 2016 at 10:14 AM, Thomas Munro wrote: > On Thu, Dec 22, 2016 at 2:14 AM, Fujii Masao wrote: >> I agree that the capability to measure the remote_apply lag is very useful. >> Also I want to measure the remote_write and remote_flush lags, for example, >> in order to diagnose the cau

[HACKERS] parallelize queries containing initplans

2016-12-28 Thread Amit Kapila
By seeing the subject line, one might wonder why we need to consider parallelizing the queries containing initplans differently from queries containing subplans considering that I have posted a mail to achieve later a few hours back. The reason is that both are treated differently with respect to

Re: [HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-28 Thread Fabien COELHO
Hello Tom, [...] It's touching every single utility statement type, which is not only pretty invasive in itself, but will break any pending patches that add more utility statement types. Yep, but it is limited to headers and the break is trivial... And you've not followed through on the imp

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-12-28 Thread Craig Ringer
On 23 December 2016 at 18:00, Craig Ringer wrote: > I'll have to follow up with a patch soon, as it's Toddler o'Clock. Here we go. This patch advances oldestXid, under XidGenLock, _before_ truncating clog. txid_status() holds XidGenLock from when it tests oldestXid until it's done looking up c

Re: [HACKERS] Faster methods for getting SPI results

2016-12-28 Thread Craig Ringer
On 28 December 2016 at 12:32, Jim Nasby wrote: > On 12/27/16 9:10 PM, Craig Ringer wrote: >> >> On 28 December 2016 at 09:58, Jim Nasby wrote: >> >>> I've looked at this some more, and ITSM that the only way to do this >>> without >>> some major surgery is to create a new type of Destination spec

Re: [HACKERS] postgres_fdw bug in 9.6

2016-12-28 Thread Ashutosh Bapat
On Wed, Dec 28, 2016 at 1:29 PM, Etsuro Fujita wrote: > On 2016/12/28 15:54, Ashutosh Bapat wrote: >> >> On Wed, Dec 28, 2016 at 9:20 AM, Etsuro Fujita >> wrote: >>> >>> On 2016/12/27 22:03, Ashutosh Bapat wrote: If mergejoin_allowed is true and mergeclauselist is non-NIL but hashc

Re: [HACKERS] postgres_fdw bug in 9.6

2016-12-28 Thread Etsuro Fujita
On 2016/12/28 15:54, Ashutosh Bapat wrote: On Wed, Dec 28, 2016 at 9:20 AM, Etsuro Fujita wrote: On 2016/12/27 22:03, Ashutosh Bapat wrote: If mergejoin_allowed is true and mergeclauselist is non-NIL but hashclauselist is NIL (that's rare but there can be types has merge operators but not hash