Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Etsuro Fujita
On 2015/09/29 13:55, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita On 2015/09/29 9:13, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of R

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Torsten Zuehlsdorff
Hello, On 29.09.2015 00:34, Tom Lane wrote: Jim Nasby writes: On 9/28/15 11:43 AM, Andres Freund wrote: It has been stated pretty clearly in this thread by a number of senior community people that we're not going to use a closed source system. GitLab OTOH is released under a MIT license, s

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Torsten Zuehlsdorff
On 29.09.2015 05:54, Tom Lane wrote: Stephen Frost writes: * Jim Nasby (jim.na...@bluetreble.com) wrote: 2 years ago is when they first released the enterprise edition, which according to [1] had "The most important new feature is that you can now add members to groups of projects." It need

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Andres Freund
On 2015-09-28 21:48:00 -0500, Jim Nasby wrote: > On 9/28/15 8:49 PM, Robert Haas wrote: > >If at some point we back-patch this further, then it potentially > >becomes a live issue, but I would like to respectfully inquire what > >exactly you think making it a PANIC would accomplish? There are a lo

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita > Sent: Tuesday, September 29, 2015 4:36 PM > To: Kaigai Kouhei(海外 浩平); Robert Haas > Cc: PostgreSQL-development; 花田茂 > Subject: Re: [HACKERS] Foreign joi

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-29 Thread Dean Rasheed
On 28 September 2015 at 20:15, Stephen Frost wrote: > I listed out the various alternatives but didn't end up getting any > responses to it. I'm still of the opinion that the documentation is the > main thing which needs improving here, but we can also change CREATE > POLICY, et al, to require an

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Petr Jelinek
On 2015-09-29 05:05, Alvaro Herrera wrote: Petr Jelinek wrote: On 2015-09-02 16:14, Fujii Masao wrote: On Wed, Aug 5, 2015 at 2:16 AM, Robert Haas wrote: On Mon, Aug 3, 2015 at 10:31 AM, Fujii Masao wrote: track_commit_timestamp tracks COMMIT PREPARED as expected in standby server, but not

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-29 Thread Kyotaro HORIGUCHI
Hi, At Sat, 26 Sep 2015 18:00:33 +0200, Tomas Vondra wrote in <5606c121.10...@2ndquadrant.com> > Hi, > > On 09/26/2015 01:28 PM, Tomas Vondra wrote: > > > The patch does not change the check_index_only implementation - it > > still needs to check the clauses, just like in v1 of the patch. To >

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Shulgin, Oleksandr
On Tue, Sep 29, 2015 at 12:28 AM, Jim Nasby wrote: > On 9/18/15 5:05 AM, Shulgin, Oleksandr wrote: > >> >> So this has to be the responsibility of the reply sending backend in the >> end: to create and release the DSM *at some point*. >> > > What's wrong with just releasing it at the end of the s

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Andres Freund
On 2015-09-25 19:13:13 +0200, Shulgin, Oleksandr wrote: > the auto_explain contrib module. I now propose the most simple thing > possible in my opinion: a new GUC option for auto_explain. It will make > every backend, in which the module is loaded via *_preload_libraries > mechanism or using CREA

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Etsuro Fujita
On 2015/09/29 17:49, Kouhei Kaigai wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita RefetchForeignRow() does not take ForeignScanState as its argument, so it is not obvious to access its private field, isn't it? ExecRowMark

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Shulgin, Oleksandr
On Tue, Sep 29, 2015 at 12:57 PM, Andres Freund wrote: > On 2015-09-25 19:13:13 +0200, Shulgin, Oleksandr wrote: > > the auto_explain contrib module. I now propose the most simple thing > > possible in my opinion: a new GUC option for auto_explain. It will make > > every backend, in which the m

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Fujii Masao
On Tue, Sep 29, 2015 at 12:05 PM, Alvaro Herrera wrote: > Petr Jelinek wrote: >> On 2015-09-02 16:14, Fujii Masao wrote: >> >On Wed, Aug 5, 2015 at 2:16 AM, Robert Haas wrote: >> >>On Mon, Aug 3, 2015 at 10:31 AM, Fujii Masao wrote: >> >>>track_commit_timestamp tracks COMMIT PREPARED as expected

Re: [HACKERS] Comment update to pathnode.c

2015-09-29 Thread Robert Haas
On Tue, Sep 29, 2015 at 1:55 AM, Etsuro Fujita wrote: > Thanks for the comments! Attached is an updated version of the patch. Committed and back-patched to 9.5. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Robert Haas
On Mon, Sep 28, 2015 at 10:48 PM, Jim Nasby wrote: > Maybe I'm confused, but I thought the whole purpose of this was to get rid > of the risk associated with that calculation in favor of explicit truncation > boundaries in the WAL log. Yes. But if the master hasn't been updated yet, then we stil

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Joel Jacobson
On Tue, Sep 22, 2015 at 3:20 PM, Andres Freund wrote: > What I've tested is the following: > * continous burning of multis, both triggered via members and offsets > * a standby keeping up when the primary is old > * a standby keeping up when the primary is new > * basebackups made while a new prim

Re: [HACKERS] [Proposal] Table partition + join pushdown

2015-09-29 Thread Taiki Kondo
Hello, KaiGai-san Thank you for your introduction and comments. > * Suppose we focus on only HashJoin in the first version? > This patch also add support on NestLoop and MergeJoin, however, NestLoop > has no valuable scenario without parallel execution capability, and the > most valuable scenario

Re: [HACKERS] Improving test coverage of extensions with pg_dump

2015-09-29 Thread Michael Paquier
On Sat, Sep 26, 2015 at 10:22 PM, Andres Freund wrote: > On 2015-09-26 21:54:46 +0900, Michael Paquier wrote: >> On Wed, Sep 23, 2015 at 1:04 AM, Alvaro Herrera >> wrote: >> > We discussed this in some other thread, not long ago. I looked briefly >> > in the archives but couldn't find it. I thi

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Etsuro Fujita > Sent: Tuesday, September 29, 2015 8:00 PM > To: Kaigai Kouhei(海外 浩平); Robert Haas > Cc: PostgreSQL-development; 花田茂 > Subject: Re: [HACKERS] Foreign

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Petr Jelinek
On 2015-09-29 13:44, Fujii Masao wrote: On Tue, Sep 29, 2015 at 12:05 PM, Alvaro Herrera wrote: Petr Jelinek wrote: On 2015-09-02 16:14, Fujii Masao wrote: On Wed, Aug 5, 2015 at 2:16 AM, Robert Haas wrote: On Mon, Aug 3, 2015 at 10:31 AM, Fujii Masao wrote: track_commit_timestamp tracks

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Steve Crawford
On Mon, Sep 28, 2015 at 4:41 PM, Jim Nasby wrote: > Note that since they also offer a hosted solution we should use that to > play with instead of trying to install it at this point. > > Integrating the issue tracker looks like it's just a call to this API: > http://doc.gitlab.com/ce/api/issues.h

Re: [HACKERS] Rework the way multixact truncations work

2015-09-29 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Sep 28, 2015 at 10:48 PM, Jim Nasby wrote: > > Maybe I'm confused, but I thought the whole purpose of this was to get rid > > of the risk associated with that calculation in favor of explicit truncation > > boundaries in the WAL log. > > Yes. But if the master hasn't

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread David Fetter
On Tue, Sep 29, 2015 at 07:01:16AM -0700, Steve Crawford wrote: > On Mon, Sep 28, 2015 at 4:41 PM, Jim Nasby wrote: > > > Note that since they also offer a hosted solution we should use that to > > play with instead of trying to install it at this point. > > > > Integrating the issue tracker look

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Merlin Moncure
On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater wrote: > Hello, > > Last night I heard that Postgres had no issue/bug tracker. At first I > thought the guy was trolling me. Seriously, how could this be. Certainly a > mature open source project that is the database for a measurable percentage > of the

Re: [HACKERS] pageinspect patch, for showing tuple data

2015-09-29 Thread Nikolay Shaplov
В письме от 26 сентября 2015 20:57:25 пользователь Michael Paquier написал: > > So I would consider two options: Either move t_infomask/t_infomask2 > > details > > into storage.sgml or leave as it is. > > The documentation redirects the reader to look at htup_details.h (the > documentation is act

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Alvaro Herrera
Petr Jelinek wrote: > On 2015-09-29 13:44, Fujii Masao wrote: > >On Tue, Sep 29, 2015 at 12:05 PM, Alvaro Herrera > > wrote: > >-#define RecoveryRequiresBoolParameter(param_name, currValue, masterValue) \ > >-do { \ > >-bool _currValue = (currValue); \ > >-bool _masterValue = (masterValue)

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-29 Thread Asif Naeem
Thank you Tom. The issue seems not reproducible anymore with latest PG95 source code (commit 60fcee9e5e77dc748a9787fae34328917683b95e) Windows build i.e. C:\PG\postgresql\pg95_with_openssl>bin\psql.exe -d postgres -h > 172.16.141.232 > psql (9.5alpha2) > WARNING: Console code page (437) differs fr

Re: [HACKERS] [PATCH v2] GSSAPI encryption support

2015-09-29 Thread Robbie Harwood
Robbie Harwood writes: Michael Paquier writes: > Well, the issue is still here: login through gssapi fails with > your patch, not with HEAD. This patch is next on my review list by > the way so I'll see what I can do about it soon even if I am in > the US for Postgres O

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Steve Crawford
On Tue, Sep 29, 2015 at 7:16 AM, David Fetter wrote: > ...What we're not fine with is depending on a proprietary system, no > matter what type of license, as infrastructure... > > Exactly. Which is why I was warning about latching onto features only available in the closed enterprise version. Ch

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-29 Thread Adam Brightwell
> I've attached a patch to implement it. It's not fully polished but it's > sufficient for comment, I believe. Additional comments, documentation > and regression tests are to be added, if we have agreement on the > grammer and implementation approach. I have given the first patch a quick review

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-29 Thread Tomas Vondra
Hello, On 09/29/2015 12:27 PM, Kyotaro HORIGUCHI wrote: Hi, At Sat, 26 Sep 2015 18:00:33 +0200, Tomas Vondra wrote in <5606c121.10...@2ndquadrant.com> Hi, On 09/26/2015 01:28 PM, Tomas Vondra wrote: The patch does not change the check_index_only implementation - it still needs to check th

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-29 Thread Tom Lane
Asif Naeem writes: > Thank you Tom. The issue seems not reproducible anymore with latest PG95 > source code (commit 60fcee9e5e77dc748a9787fae34328917683b95e) Windows build Thanks for testing! I've marked this issue resolved in the 9.5 open-items list. regards, tom lane

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-29 Thread Adam Brightwell
> My vote would be to keep it as-is. Same for me. > It feels perfectly natural to me. USING clauses add to the query's > WHERE clause controlling which existing rows you can SELECT, UPDATE or > DELETE. WITH CHECK clauses control what new data you can add via > INSERT or UPDATE. UPDATE allows both

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Andres Freund
On 2015-09-24 17:25:21 +0200, Andres Freund wrote: > Stuff I want to fix by tomorrow: > * Whole row var references to exclude > * wrong offsets for columns after dropped ones > * INSTEAD DO UPDATE for tables with oids > > Do you know of anything else? So, took a bit longer than "tomorrow. I fought

Re: [HACKERS] Box type equality

2015-09-29 Thread Tom Lane
Stanislav Kelvich writes: > I've faced an issue with Box type comparison that exists almost for a five > years. Try twenty-five years. The code's been like that since Berkeley. > That can be fixed by b-tree equality for boxes, but we need some > decisions there. The problem with inventing

[HACKERS] Box type equality

2015-09-29 Thread Stanislav Kelvich
Hi. I've faced an issue with Box type comparison that exists almost for a five years. > create table t (b box); CREATE TABLE > select count(*), b from t group by b; ERROR: could not identify an equality operator for type box As mentioned in http://www.postgresql.org/message-id/pine.lnx.4.64.1

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Andrew Dunstan
On 09/29/2015 10:55 AM, Steve Crawford wrote: On Tue, Sep 29, 2015 at 7:16 AM, David Fetter > wrote: ...What we're not fine with is depending on a proprietary system, no matter what type of license, as infrastructure... Exactly. Which is why I was warning abo

Re: [HACKERS] GinPageIs* don't actually return a boolean

2015-09-29 Thread Alvaro Herrera
Andres Freund wrote: > I went through all headers in src/include and checked for macros > containing [^&]&[^&] and checked whether they have this hazard. Found a > fair number. > > That patch also changes !! tests into != 0 style. I don't think you pushed this, did you? -- Álvaro Herrera

Re: [HACKERS] Box type equality

2015-09-29 Thread Jeff Anton
The Box type is the oldest non-linear type in the Postgres system. We used it as the template for extensibility in the original system about thirty years ago. We had R-trees for box indexing. If you want fuzzy box matching, that seems possible with R-trees and some creativity by say matching

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-29 Thread Tomas Vondra
On 09/29/2015 04:57 PM, Tomas Vondra wrote: Hello, On 09/29/2015 12:27 PM, Kyotaro HORIGUCHI wrote: ... cost_index() seems to need to be fixed. It would count excluded clauses in estimate. Hmm, good point. The problem is that extract_nonindex_conditions uses baserel->baserestrictinfo agai

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 25 September 2015 at 12:13, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > I now believe that use of ProcessInterrupts() in the recently proposed > design as well as manipulation of proc latch due to use of shared memory > queue are major blockers. > > In order to simplify things

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Corey Huinker
> > > And they'd sure love to be in charge of our code repo. >> > > Mh - i'm not a native speaker. I didn't understand this line. > > Tom was saying that the JIRA/Atlassian people would happily volunteer to host our code repository for no cost (in money) to us. The implication being that they have

Re: [HACKERS] Parallel Seq Scan

2015-09-29 Thread Robert Haas
On Tue, Sep 29, 2015 at 12:39 AM, Amit Kapila wrote: > Attached patch is a rebased patch based on latest commit (d1b7c1ff) > for Gather node. > > - I have to reorganize the defines in execParallel.h and .c. To keep > ParallelExecutorInfo, in GatherState node, we need to include execParallel.h > in

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Corey Huinker writes: >>> And they'd sure love to be in charge of our code repo. >> Mh - i'm not a native speaker. I didn't understand this line. > Tom was saying that the JIRA/Atlassian people would happily volunteer to > host our code repository for no cost (in money) to us. Not that so much

Re: [HACKERS] CustomScan support on readfuncs.c

2015-09-29 Thread Robert Haas
On Mon, Sep 28, 2015 at 8:31 PM, Kouhei Kaigai wrote: >> Instead of doing this: >> >> +/* Dump library and symbol name instead of raw pointer */ >> appendStringInfoString(str, " :methods "); >> -_outToken(str, node->methods->CustomName); >> +_outToken(str, node->methods->methods_l

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Jeff Anton
Seems to me that there are a bunch of agendas here. I read about not wanting to be trapped into a proprietary system. You can be trapped in any software you depend upon. Compilers, Platforms, SCM, issue tracking are all places to be trapped. Postgres and Postgresql have been around a very l

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Andres Freund
On 2015-09-29 13:27:15 -0400, Tom Lane wrote: > Corey Huinker writes: > >>> And they'd sure love to be in charge of our code repo. > > >> Mh - i'm not a native speaker. I didn't understand this line. > > > Tom was saying that the JIRA/Atlassian people would happily volunteer to > > host our code

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Andres Freund writes: > On 2015-09-29 13:27:15 -0400, Tom Lane wrote: >> Not that so much as that the gitlab code really wants to be connected up >> to our code repo. That makes complete sense in terms of its goals as >> stated by Torsten upthread, namely to be a git repo manager. But it's >> no

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Joshua D. Drake
On 09/29/2015 07:25 AM, Merlin Moncure wrote: On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater wrote: Hello, Last night I heard that Postgres had no issue/bug tracker. At first I thought the guy was trolling me. Seriously, how could this be. Certainly a mature open source project that is the datab

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Andres Freund
On 2015-09-29 13:40:28 -0400, Tom Lane wrote: > Andres Freund writes: > > I don't have any opinion WRT gitlab, but I'm fairly certain it'd be > > unproblematic to configure automatic mirroring into it from > > gitmaster. > > I think you missed my point: gitlab would then believe it's in charge of

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Andres Freund writes: > On 2015-09-29 13:40:28 -0400, Tom Lane wrote: >> I think you missed my point: gitlab would then believe it's in charge of, >> eg, granting write access to that repo. We could perhaps whack it over >> the head till it only does what we want and not ten other things, but >>

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Robert Haas
On Tue, Sep 29, 2015 at 4:49 AM, Kouhei Kaigai wrote: > Also note that EvalPlanQualFetchRowMarks() will raise an error > if RefetchForeignRow callback returned NULL tuple. > Is it right or expected behavior? That's not how I read the code. If RefetchForeignRow returns NULL, we just ignore the ro

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Shulgin, Oleksandr
On Tue, Sep 29, 2015 at 7:01 PM, Simon Riggs wrote: > On 25 September 2015 at 12:13, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> I now believe that use of ProcessInterrupts() in the recently proposed >> design as well as manipulation of proc latch due to use of shared memor

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Alvaro Herrera
Hi Fujii, thanks for the review. I have pushed the patch to 9.5 and master. Fujii Masao wrote: > @@ -5321,7 +5333,7 @@ xact_redo_commit(xl_xact_parsed_commit *parsed, > /* Set the transaction commit timestamp and metadata */ > TransactionTreeSetCommitTsData(xid, parsed->nsubxacts, pars

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-29 Thread Jeremy Harris
On 27/09/15 21:58, Gavin Wahl wrote: > Somewhat harder but still possible would be using BRIN indexes to > accelerate ORDER BY. This would require a sorting algorithm that can take > advantage of mostly-sorted inputs. You would sort the page ranges by their > minimum or maximum value, then feed the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 12:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > Hitting a process with a signal and hoping it will produce a meaningful > response in all circumstances without disrupting its current task was way > too naive. > Hmm, I would have to disagree, sorry. For

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 13:20, Jeremy Harris wrote: > On 27/09/15 21:58, Gavin Wahl wrote: > > Somewhat harder but still possible would be using BRIN indexes to > > accelerate ORDER BY. This would require a sorting algorithm that can take > > advantage of mostly-sorted inputs. You would sort the

[HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
A problem the buildfarm has had for a long time is that if for some reason the scripts fail to stop a test postmaster, the postmaster process will hang around and cause subsequent runs to fail because of socket conflicts. This seems to have gotten a lot worse lately due to the influx of very slow b

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Peter Geoghegan
On Tue, Sep 29, 2015 at 8:24 AM, Andres Freund wrote: > So, took a bit longer than "tomorrow. I fought for a long while with a > mysterious issue, which turned out to be separate bug: The excluded > relation was affected by row level security policies, which doesn't make > sense. Why? You certain

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > But today I thought of another way: suppose that we teach the postmaster > to commit hara-kiri if the $PGDATA directory goes away. Since the > buildfarm script definitely does remove all the temporary data directories > it creates, this ought to get the job

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 11:48 AM, Tom Lane wrote: > But today I thought of another way: suppose that we teach the postmaster > to commit hara-kiri if the $PGDATA directory goes away. Since the > buildfarm script definitely does remove all the temporary data directories > it creates, this ought to get the jo

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I wouldn't want to do this every time through the postmaster's main loop, >> but we could do this once an hour for no added cost by adding the check >> where it does TouchSocketLockFiles; or once every few minutes if we >> carried a

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Andrew Dunstan
On 09/29/2015 02:48 PM, Tom Lane wrote: A problem the buildfarm has had for a long time is that if for some reason the scripts fail to stop a test postmaster, the postmaster process will hang around and cause subsequent runs to fail because of socket conflicts. This seems to have gotten a lot w

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I wouldn't want to do this every time through the postmaster's main loop, > >> but we could do this once an hour for no added cost by adding the check > >> where it does TouchSocketLockFi

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Andrew Dunstan writes: > On 09/29/2015 02:48 PM, Tom Lane wrote: >> Also, perhaps we'd only enable this behavior in --enable-cassert builds, >> to avoid any risk of a postmaster incorrectly choosing to suicide in a >> production scenario. Or maybe that's overly conservative. > Not every buildfar

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 12:18 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 09/29/2015 02:48 PM, Tom Lane wrote: >>> Also, perhaps we'd only enable this behavior in --enable-cassert builds, >>> to avoid any risk of a postmaster incorrectly choosing to suicide in a >>> production scenario. Or maybe tha

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Andres Freund
On September 29, 2015 8:52:14 PM GMT+02:00, Peter Geoghegan wrote: >On Tue, Sep 29, 2015 at 8:24 AM, Andres Freund >wrote: >> So, took a bit longer than "tomorrow. I fought for a long while with >a >> mysterious issue, which turned out to be separate bug: The excluded >> relation was affected by

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Josh Berkus writes: > On 09/29/2015 11:48 AM, Tom Lane wrote: >> But today I thought of another way: suppose that we teach the postmaster >> to commit hara-kiri if the $PGDATA directory goes away. Since the >> buildfarm script definitely does remove all the temporary data directories >> it create

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Tue, Sep 29, 2015 at 8:24 AM, Andres Freund wrote: > > So, took a bit longer than "tomorrow. I fought for a long while with a > > mysterious issue, which turned out to be separate bug: The excluded > > relation was affected by row level security poli

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Shulgin, Oleksandr
On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs wrote: > On 29 September 2015 at 12:52, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> Hitting a process with a signal and hoping it will produce a meaningful >> response in all circumstances without disrupting its current task was

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Alvaro Herrera
Tom Lane wrote: > Testing accessibility of "global/pg_control" would be enough to catch this > case, but only if we do it before you create a new one. So that seems > like an argument for making the test relatively often. The once-a-minute > option is sounding better and better. If we weren't a

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Oleg Bartunov
On Tue, Sep 29, 2015 at 6:36 PM, Andrew Dunstan wrote: > > > On 09/29/2015 10:55 AM, Steve Crawford wrote: > >> On Tue, Sep 29, 2015 at 7:16 AM, David Fetter > da...@fetter.org>> wrote: >> >> ...What we're not fine with is depending on a proprietary system, no >> matter what type of licen

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Joe Conway
On 09/29/2015 12:47 PM, Tom Lane wrote: > We could possibly add additional checks, like trying to verify that > pg_control has the same inode number it used to. But I'm afraid that > would add portability issues and false-positive hazards that would > outweigh the value. Not sure you remember the

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Alvaro Herrera
Oleg Bartunov wrote: > me too, but I have to mention one problem we should have in mind - it's > independency from political games (sanctions). Think about developers from > Russia, who one day may be blocked by Github, for example. That's a very good point. I think Github and other sites are

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Alvaro Herrera
Joe Conway wrote: > On 09/29/2015 12:47 PM, Tom Lane wrote: > > We could possibly add additional checks, like trying to verify that > > pg_control has the same inode number it used to. But I'm afraid that > > would add portability issues and false-positive hazards that would > > outweigh the value

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Joe Conway
On 09/29/2015 01:48 PM, Alvaro Herrera wrote: > Joe Conway wrote: >> On 09/29/2015 12:47 PM, Tom Lane wrote: >>> We could possibly add additional checks, like trying to verify that >>> pg_control has the same inode number it used to. But I'm afraid that >>> would add portability issues and false-p

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-09-29 Thread Peter Geoghegan
On Fri, Sep 25, 2015 at 8:51 AM, Marti Raudsepp wrote: > I've also been seeing lots of log messages saying "LOG: out of > memory" on a server that's hosting development databases. I put off > debugging this until now because it didn't seem to have any adverse > effects on the system. That's unfo

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 20:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs > wrote: > >> On 29 September 2015 at 12:52, Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de> wrote: >> >> >>> Hitting a process with a signal and hoping it

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Alvaro Herrera
Joe Conway wrote: > On 09/29/2015 01:48 PM, Alvaro Herrera wrote: > > I remember it, but I'm not sure it would have helped you. As I recall, > > your trouble was that after a reboot the init script decided to initdb > > the mount point -- postmaster wouldn't have been running at all ... > > Righ

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Adam Brightwell
On Mon, Aug 3, 2015 at 6:21 PM, Peter Geoghegan wrote: > On Mon, Aug 3, 2015 at 3:07 PM, Stephen Frost wrote: >> Thoughts? Trying to keep it straight-forward and provide a simple >> solution for users to be able to address the issue, if they're worried >> about it. Perhaps this, plus an additio

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Josh Berkus writes: > Give me source with the change, and I'll put it on a cheap, low-bandwith > AWS instance and hammer the heck out of it. That should raise any false > positives we can expect. Here's a draft patch against HEAD (looks like it will work on 9.5 or 9.4 without modifications, too)

[HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Josh Elsasser
Hi. I have a need to pipe the output from pg_basebackup for a multi-tablespace cluster into another program without spooling to disk. Seeing as the current -F tar output format can't do that, I've made an attempt at implementing that myself. As a side effect I've refactored the some of the pg_base

Re: [HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Andres Freund
Hi Josh, On 2015-09-29 14:38:11 -0700, Josh Elsasser wrote: > As a side effect I've refactored the some of the pg_basebackup code > for readability and reusability Cool, that's desperately needed. I've been trying to bug Magnus into doing that for a bunch of conferences now ;) > I've put my chan

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Robert Haas
On Mon, Sep 28, 2015 at 11:15 PM, Etsuro Fujita wrote: > I thought the same thing [1]. While I thought it was relatively easy to > make changes to RefetchForeignRow that way for the foreign table case > (scanrelid>0), I was not sure how hard it would be to do so for the foreign > join case (scanr

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Stephen Frost
* Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > On Mon, Aug 3, 2015 at 6:21 PM, Peter Geoghegan wrote: > > On Mon, Aug 3, 2015 at 3:07 PM, Stephen Frost wrote: > >> Thoughts? Trying to keep it straight-forward and provide a simple > >> solution for users to be able to addre

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Merlin Moncure
On Tue, Sep 29, 2015 at 12:42 PM, Joshua D. Drake wrote: > On 09/29/2015 07:25 AM, Merlin Moncure wrote: >> >> On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater wrote: >>> >>> Hello, >>> >>> Last night I heard that Postgres had no issue/bug tracker. At first I >>> thought the guy was trolling me. Ser

[HACKERS] [PATCH 3/6] pg_basebackup: factor out code to add a recovery.conf file to the tar file.

2015-09-29 Thread Joshua Elsasser
This is just a simple refactor for readability and reusability. --- src/bin/pg_basebackup/pg_basebackup.c | 46 --- 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c index f

[HACKERS] [PATCH 1/6] Add support for longer filenames in tar headers (up to 254 bytes).

2015-09-29 Thread Joshua Elsasser
New functions tarHeaderRename() and tarHeaderGetName() are exposed to store and retrieve the longer filenames. tarCreateHeader() continues to limit filenames to 99 bytes to preserve compatability with existing consumers. --- src/include/pgtar.h | 2 + src/port/tar.c | 134 +

[HACKERS] [PATCH 5/6] pg_basebackup: allow GetConnection() to make non-replication connections.

2015-09-29 Thread Joshua Elsasser
--- src/bin/pg_basebackup/pg_basebackup.c | 4 ++-- src/bin/pg_basebackup/pg_receivexlog.c | 4 ++-- src/bin/pg_basebackup/pg_recvlogical.c | 4 ++-- src/bin/pg_basebackup/streamutil.c | 6 +++--- src/bin/pg_basebackup/streamutil.h | 2 +- 5 files changed, 10 insertions(+), 10 deletions(-

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-29 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 28 September 2015 at 20:15, Stephen Frost wrote: > > I listed out the various alternatives but didn't end up getting any > > responses to it. I'm still of the opinion that the documentation is the > > main thing which needs improving here, but

[HACKERS] [PATCH 4/6] pg_basebackup: don't lose a zero-length file at the end of a tablespace.

2015-09-29 Thread Joshua Elsasser
After a complete tar header was buffered it would only be processed during the next iteration of the read loop. A zero-length file such as a directory had no data to read, so the loop would exit without ever having processed the file. --- src/bin/pg_basebackup/pg_basebackup.c | 238 +++

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-09-29 Thread David Rowley
On 29 September 2015 at 01:59, Tomas Vondra wrote: > > CREATE TABLE f (id1 INT, id2 INT, PRIMARY KEY (id1, id2)); > > CREATE TABLE d1 (id1 INT, id2 INT, FOREIGN KEY (id1, id2) REFERENCES > f(id1, id2)); > CREATE TABLE d2 (id1 INT, id2 INT, FOREIGN KEY (id1, id2) REFERENCES > f(id1, id2)); > > INS

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
I wrote: > Josh Berkus writes: >> Give me source with the change, and I'll put it on a cheap, low-bandwith >> AWS instance and hammer the heck out of it. That should raise any false >> positives we can expect. > Here's a draft patch against HEAD (looks like it will work on 9.5 or > 9.4 without m

[HACKERS] [PATCH 2/6] pg_basebackup: factor out tar open/close code for reusability.

2015-09-29 Thread Joshua Elsasser
This adds a simple struct and open and close functions to abstract and isolate the zlib vs. stdio output logic and allow it to be reused. --- src/bin/pg_basebackup/pg_basebackup.c | 300 +- 1 file changed, 154 insertions(+), 146 deletions(-) diff --git a/src/bin/pg

Re: [HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Joshua Elsasser
On Tue, Sep 29, 2015 at 11:50:37PM +0200, Andres Freund wrote: > On 2015-09-29 14:38:11 -0700, Josh Elsasser wrote: > > I've put my changes up as a series of relatively small commits on this > > branch of a github fork: > > > > https://github.com/jre/postgres/commits/single-tar > > > > Comments a

Re: [HACKERS] CustomScan support on readfuncs.c

2015-09-29 Thread Kouhei Kaigai
> On Mon, Sep 28, 2015 at 8:31 PM, Kouhei Kaigai wrote: > >> Instead of doing this: > >> > >> +/* Dump library and symbol name instead of raw pointer */ > >> appendStringInfoString(str, " :methods "); > >> -_outToken(str, node->methods->CustomName); > >> +_outToken(str, node->meth

[HACKERS] [PATCH 6/6] pg_basebackup: add a single-tar output format.

2015-09-29 Thread Joshua Elsasser
This will write one single tar file containing all tablespaces, and can be written to stdout. --- src/bin/pg_basebackup/pg_basebackup.c | 282 -- 1 file changed, 269 insertions(+), 13 deletions(-) diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_bas

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Alvaro Herrera
Merlin Moncure wrote: > I've read this email about three times now and it's not clear at all > to me what a issue/bug tracker brings to the table. In my opinion, this thread is about a bug tracker, not a patch tracker. We already have a patch tracking system which works very well. (We are not ab

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Josh Berkus
On 09/29/2015 03:08 PM, Merlin Moncure wrote: > I've read this email about three times now and it's not clear at all > to me what a issue/bug tracker brings to the table. Here are the problems I'd like to solve: 1. "Was this issue fixed in a Postgres update? Which one?" 2. Not losing track of m

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 12:47 PM, Tom Lane wrote: > Josh Berkus writes: >> In general, having the postmaster survive deletion of PGDATA is >> suboptimal. In rare cases of having it survive installation of a new >> PGDATA (via PITR restore, for example), I've even seen the zombie >> postmaster corrupt the d

  1   2   >