Re: [HACKERS] WIP: Rework access method interface

2015-10-02 Thread Amit Kapila
On Fri, Oct 2, 2015 at 8:14 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > > > I agree about staying with one SQL-visible function. > > Other changes: > * Documentation reflects interface changes. > * IndexAmRoutine moved from CacheMemoryContext to indexcxt. > * Various minor forma

Re: [HACKERS] pgbench stats per script & other stuff

2015-10-02 Thread Fabien COELHO
Here is a v10, which is a rebase because of the "--progress-timestamp" option addition. Here is a v11, which is a rebase after some recent changes committed to pgbench. -- Fabien.diff --git a/doc/src/sgml/ref/pgbench.sgml b/doc/src/sgml/ref/pgbench.sgml index 0ac40f1..e3a90e5 100644 --- a/d

Re: [HACKERS] Idea for improving buildfarm robustness

2015-10-02 Thread Michael Paquier
On Sat, Oct 3, 2015 at 1:39 PM, Tom Lane wrote: > I wrote: > > Here's a rewritten patch that looks at postmaster.pid instead of > > pg_control. It should be effectively the same as the prior patch in > terms > > of response to directory-removal cases, and it should also catch many > > overwrite

Re: [HACKERS] Proposing COPY .. WITH PERMISSIVE

2015-10-02 Thread dinesh kumar
On Wed, Sep 2, 2015 at 1:19 PM, Stefan Kaltenbrunner < ste...@kaltenbrunner.cc> wrote: > On 09/02/2015 10:10 PM, dinesh kumar wrote: > > On Tue, Sep 1, 2015 at 10:58 PM, Stefan Kaltenbrunner > > mailto:ste...@kaltenbrunner.cc>> wrote: > > > > On 07/25/2015 03:38 AM, dinesh kumar wrote: > >

Re: [HACKERS] Idea for improving buildfarm robustness

2015-10-02 Thread Tom Lane
I wrote: > Here's a rewritten patch that looks at postmaster.pid instead of > pg_control. It should be effectively the same as the prior patch in terms > of response to directory-removal cases, and it should also catch many > overwrite cases. BTW, my thought at the moment is to wait till after ne

Re: [HACKERS] Parallel Seq Scan

2015-10-02 Thread Amit Kapila
On Thu, Oct 1, 2015 at 7:41 PM, Robert Haas wrote: > > > Does heap_parallelscan_nextpage really need a pscan_finished output > parameter, or can it just return InvalidBlockNumber to indicate end of > scan? I think the latter can be done and would be cleaner. > I think we can do that way as well,

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 4:27 PM, Tom Lane wrote: > ... do you want to produce an updated patch? I'm not planning to look at > it until tomorrow anyway. I'll post a new patch by about midday tomorrow west coast time. Hopefully that works for you. -- Peter Geoghegan -- Sent via pgsql-hackers m

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Tom Lane
... do you want to produce an updated patch? I'm not planning to look at it until tomorrow anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hack

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 4:11 PM, Peter Geoghegan wrote: > Actually, isn't that another bug? The fact that we don't do the same > from within gc_qtexts() in normal cases, even with an exclusive lock > held? We do this: Ah, no. We check pgss->gc_count in any case, so it should be fine. That will als

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 3:57 PM, Peter Geoghegan wrote: > The spinlock acquisition above is actually necessary despite the > n_writers trick, because that's only used by qtext_store(). Actually, isn't that another bug? The fact that we don't do the same from within gc_qtexts() in normal cases, eve

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 3:57 PM, Peter Geoghegan wrote: > (void) AllocateFile(PGSS_TEXT_FILE, PG_BINARY_W); Naturally, a FreeFile() call will also be required for any successfully allocated file. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 2:42 PM, Tom Lane wrote: > Peter Geoghegan writes: >> I think that SIZE_MAX should be replaced by MaxAllocHugeSize before >> the patch is committed. That should be perfectly portable. > > Hmm ... only back to 9.4, but I guess that's far enough. I just realized that the exi

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Joshua D. Drake
On 10/02/2015 01:26 PM, Alvaro Herrera wrote: However, the contact surface between these two options wasn't really well polished. Formatting would be lost very frequently: I could write a nice email, and the customer would get a nice email, but if you looked at it in the web, it was very ugly.

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Tom Lane
Peter Geoghegan writes: > I think that SIZE_MAX should be replaced by MaxAllocHugeSize before > the patch is committed. That should be perfectly portable. Hmm ... only back to 9.4, but I guess that's far enough. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 2:29 PM, Tom Lane wrote: > I'm not too impressed with this bit: > > /* Allocate buffer; beware that off_t might be wider than size_t */ > - if (stat.st_size <= MaxAllocSize) > + if (stat.st_size <= SIZE_MAX) > buf = (char *) malloc(stat.st

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Tom Lane
Peter Geoghegan writes: > It would be nice to get this committed before the next point releases > are tagged, since I've now heard a handful of complaints like this. I'm not too impressed with this bit: /* Allocate buffer; beware that off_t might be wider than size_t */ - if (stat

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 2:04 PM, Peter Geoghegan wrote: > It would be nice to get this committed before the next point releases > are tagged, since I've now heard a handful of complaints like this. I grep'd for SIZE_MAX to make sure that that was something that is available on all supported platfo

Re: [HACKERS] pg_stat_statements query jumbling question

2015-10-02 Thread Peter Geoghegan
On Wed, Sep 2, 2015 at 7:41 PM, Satoshi Nagayasu wrote: > I know this still needs to be discussed, but I would like to submit > a patch for further discussion at the next CF, 2015-11. I think I already expressed this in my explanation of the current behavior, but to be clear: -1 from me to this p

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-10-02 Thread Peter Geoghegan
On Tue, Sep 22, 2015 at 6:01 PM, Peter Geoghegan wrote: > On Tue, Sep 22, 2015 at 5:01 PM, Peter Geoghegan wrote: >> My guess is that this very large query involved a very large number of >> constants, possibly contained inside an " IN ( )". Slight variants of >> the same query, that a human woul

Re: [HACKERS] Confusing remark about UPSERT in fdwhandler.sgml

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 4:04 AM, Peter Geoghegan wrote: > On Fri, Oct 2, 2015 at 1:00 AM, Etsuro Fujita > wrote: >> ISTM that the sentence "as remote constraints are not locally known" is >> somewhat confusing, because check constrains on remote tables can be >> defined locally in 9.5. How about

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 4:26 PM, Alvaro Herrera wrote: > Joshua D. Drake wrote: >> Put succinctly, I am willing to put resources into testing Redmine for our >> needs. I will need help to do so because I am not a committer/hacker. Andres >> thinks that isn't worth while. I think he is wrong. If he

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-10-02 Thread Robert Haas
On Thu, Oct 1, 2015 at 11:10 PM, Noah Misch wrote: > On Mon, Sep 28, 2015 at 05:13:56PM -0400, Stephen Frost wrote: >> * Noah Misch (n...@leadboat.com) wrote: >> > In schema reviews, I will raise a red flag for use of this feature; the >> > best >> > designs will instead use additional roles. I

Re: [HACKERS] Potential GIN vacuum bug

2015-10-02 Thread Robert Haas
On Thu, Oct 1, 2015 at 4:44 PM, Jeff Janes wrote: > Is the locking around indexes summarized someplace? Not to my knowledge. :-( -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) T

Re: [HACKERS] CustomScan support on readfuncs.c

2015-10-02 Thread Robert Haas
On Tue, Sep 29, 2015 at 6:19 PM, Kouhei Kaigai wrote: >> On Mon, Sep 28, 2015 at 8:31 PM, Kouhei Kaigai wrote: >> >> Instead of doing this: >> >> >> >> +/* Dump library and symbol name instead of raw pointer */ >> >> appendStringInfoString(str, " :methods "); >> >> -_outToken(str, no

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Alvaro Herrera
Joshua D. Drake wrote: > Put succinctly, I am willing to put resources into testing Redmine for our > needs. I will need help to do so because I am not a committer/hacker. Andres > thinks that isn't worth while. I think he is wrong. If he doesn't want to > help, he doesn't have to, thus the call f

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-10-02 Thread Robert Haas
On Tue, Sep 29, 2015 at 1:52 PM, Shulgin, Oleksandr wrote: > This is not a change of the direction, but rather of the approach. Hitting > a process with a signal and hoping it will produce a meaningful response in > all circumstances without disrupting its current task was way too naive. I think

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 3:14 AM, Amit Langote wrote: > On 2015/10/02 15:38, Fujii Masao wrote: >> >> +uint32 progress_param[N_PROGRESS_PARAM]; >> >> Why did you use an array to store the progress information of VACUUM? >> I think that it's better to use separate specific variables for them for

Re: [HACKERS] Parallel Seq Scan

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 4:27 AM, Kouhei Kaigai wrote: >> On Thu, Oct 1, 2015 at 2:35 AM, Kouhei Kaigai wrote: >> > Gather node was oversight by readfunc.c, even though it shall not be >> > serialized actually. >> > Also, it used incompatible WRITE_xxx_FIELD() macro on outfuncs.c. >> > >> > The att

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 2:59 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> The standby can have the feature enabled even though the master has it >> disabled? That seems like it can only lead to heartache. > > Can you elaborate? Sort of. Our rule up until now has always been that the standby

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Joshua D. Drake
On 10/02/2015 12:52 PM, Robert Haas wrote: OK. My reading of the thread is that the hackers who expressed an opinion mostly preferred debbugs and the people less involved in the project wanted something more like GitHub/GitLab. Some people also argued for and against Bugzilla and JIRA. I didn

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 2:43 PM, Joshua D. Drake wrote: > I am sorry if I wasn't clear. > > Put succinctly, I am willing to put resources into testing Redmine for our > needs. I will need help to do so because I am not a committer/hacker. Andres > thinks that isn't worth while. I think he is wrong.

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-10-02 Thread Alvaro Herrera
Robert Haas wrote: > The standby can have the feature enabled even though the master has it > disabled? That seems like it can only lead to heartache. Can you elaborate? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Se

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Joshua D. Drake
On 10/02/2015 11:36 AM, Robert Haas wrote: I don't know what this has to do with anything Andres said. I am sorry if I wasn't clear. Put succinctly, I am willing to put resources into testing Redmine for our needs. I will need help to do so because I am not a committer/hacker. Andres think

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 10:58 AM, Alvaro Herrera wrote: > Fujii Masao wrote: > >> What happens if pg_xact_commit_timestamp() is called in standby after >> track_commit_timestamp is disabled in master, DeactivateCommitTs() is >> called and all commit_ts files are removed in standby? I tried that cas

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 11:23 AM, Alvaro Herrera wrote: >> + /* all-frozen information is also cleared at the same time */ >> PageClearAllVisible(page); >> + PageClearAllFrozen(page); > > I wonder if it makes sense to have a macro to clear both in unison, > whi

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Robert Haas
On Fri, Oct 2, 2015 at 12:50 PM, Joshua D. Drake wrote: > I once drove a manual all the time. I swore by the manual. I loved the > control, the feeling (ridiculously) that I was a race car driver on the > Urban track. > > Then I got an automatic and realized how damn nice it was to not be in > con

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-10-02 Thread Jeff Janes
On Thu, Oct 1, 2015 at 7:48 AM, Magnus Hagander wrote: > On Thu, Oct 1, 2015 at 4:35 PM, Robert Haas wrote: > >> >> - Bug numbers are sometimes preserved in commit messages, but they >> never make it into release notes. This actually seems like something >> we could improve pretty easily and wi

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-02 Thread Masahiko Sawada
On Sat, Oct 3, 2015 at 12:23 AM, Alvaro Herrera wrote: > Masahiko Sawada wrote: > Thank you for taking time to review this feature. Attached the latest version patch (v15). >> @@ -2972,10 +2981,15 @@ l1: >>*/ >> PageSetPrunable(page, xid); >> >> + /* clear PD_ALL_VISIBLE and P

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Joshua D. Drake
On 10/02/2015 09:41 AM, Andres Freund wrote: Hi, On 2015-10-02 09:28:02 -0700, Joshua D. Drake wrote: I am proposing to setup a redmine instance on a VM. I am happy to do a lot of the legwork and should be able to get most of it done before EU. This is what I think I need from my fellows: -1.

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Andres Freund
Hi, On 2015-10-02 09:28:02 -0700, Joshua D. Drake wrote: > I am proposing to setup a redmine instance on a VM. I am happy to do a lot > of the legwork and should be able to get most of it done before EU. This is > what I think I need from my fellows: -1. This thread has already cost disproportion

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Joshua D. Drake
On 10/02/2015 09:34 AM, Dave Page wrote: Thoughts? Volunteers? I swore to myself that I'd stay out of this bikeshed, but... we already have a Redmine instance. I know but I didn't want to dogfood in an installation that was production. I am not going to put up vanilla redmine, I actually p

Re: Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Dave Page
> On 2 Oct 2015, at 17:28, Joshua D. Drake wrote: > > Hello, > > I believe it is pretty obvious we are moving in the direction of having a > tracker at this point. The problem is exactly which direction. Stephen has > offered some resources for his ideas and now I am offering some resources

Request for dogfood volunteers (was [HACKERS] No Issue Tracker - Say it Ain't So!)

2015-10-02 Thread Joshua D. Drake
Hello, I believe it is pretty obvious we are moving in the direction of having a tracker at this point. The problem is exactly which direction. Stephen has offered some resources for his ideas and now I am offering some resources for mine. I am proposing to setup a redmine instance on a VM.

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-02 Thread Alvaro Herrera
Masahiko Sawada wrote: > @@ -2972,10 +2981,15 @@ l1: >*/ > PageSetPrunable(page, xid); > > + /* clear PD_ALL_VISIBLE and PD_ALL_FORZEN flags */ Typo "FORZEN". > if (PageIsAllVisible(page)) > { > all_visible_cleared = true; > + > + /* all-

Re: [HACKERS] What is the extent of FDW join pushdown support in 9.5?

2015-10-02 Thread Bruce Momjian
On Wed, Sep 16, 2015 at 04:32:51PM -0400, Peter Eisentraut wrote: > On 9/15/15 10:02 PM, Kouhei Kaigai wrote: > >> The 9.5 release notes contain this promising but cryptic item: > >> > >> - Allow foreign data wrappers and custom scans to implement join > >> pushdown (KaiGai Kohei) > >> > >> As phra

Re: [HACKERS] [DOCS] max_worker_processes on the standby

2015-10-02 Thread Alvaro Herrera
Fujii Masao wrote: > What happens if pg_xact_commit_timestamp() is called in standby after > track_commit_timestamp is disabled in master, DeactivateCommitTs() is > called and all commit_ts files are removed in standby? I tried that case > and got the following assertion failure. Ah. So the stan

[HACKERS] Connection string parameter 'replication' in documentation

2015-10-02 Thread Takashi Ohnishi
Hi, I found that it can be set like 'replication = true' in connection parameter as documentation says in "50.3. Streaming Replication Protocol", but this parameter is not denoted in "31.1.2. Parameter Key Words". How about adding notation about this paramter in 31.1.2.? Attached is a patch for

[HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2015-10-02 Thread Amir Rohan
On 10/02/2015 03:33 PM, Michael Paquier wrote: > Any server instances created during the tests should never use a > user-defined port for portability. Hence using those ports as keys > just made sense. We could have for example custom names, that have > port values assigned to them, but that's act

[HACKERS] Re: In-core regression tests for replication, cascading, archiving, PITR, etc.

2015-10-02 Thread Michael Paquier
On Sat, Sep 26, 2015 at 10:25 AM, Amir Rohan wrote: > On 09/25/2015 01:47 PM, Michael Paquier wrote: >> offer a way to set up a cluster ready for hacking in a single >> command, and being able to switch among them easily. I am not sure we >> would really find a cross-platform script generic enough

Re: [HACKERS] pageinspect patch, for showing tuple data

2015-10-02 Thread Nikolay Shaplov
В письме от 2 октября 2015 13:10:22 пользователь Michael Paquier написал: > >> + > >> + General idea about output columns: lp_* > >> attributes > >> + are about where tuple is located inside the page; > >> + t_xmin, t_xmax, > >> + t_field3, t_ctid are > > about > > >> +

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-02 Thread Masahiko Sawada
On Fri, Oct 2, 2015 at 7:30 AM, Josh Berkus wrote: > On 10/01/2015 07:43 AM, Robert Haas wrote: >> On Thu, Oct 1, 2015 at 9:44 AM, Fujii Masao wrote: >>> I wonder how much it's worth renaming only the file extension while >>> there are many places where "visibility map" and "vm" are used, >>> for

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-10-02 Thread Oleg Bartunov
On Tue, Sep 29, 2015 at 5:55 PM, Steve Crawford < scrawf...@pinpointresearch.com> wrote: > On Tue, Sep 29, 2015 at 7:16 AM, David Fetter wrote: > >> ...What we're not fine with is depending on a proprietary system, no >> matter what type of license, as infrastructure... >> >> > Exactly. Which is

Re: [HACKERS] pageinspect patch, for showing tuple data

2015-10-02 Thread Nikolay Shaplov
В письме от 2 октября 2015 13:10:22 Вы написали: > > There also was a bug in original pageinspect, that did not get t_bits > > right > > > when there was OID in the tuple. I've fixed it too. > > Aha. Good catch! By looking at HeapTupleHeaderGetOid if the tuple has an > OID it will be stored at

Re: [HACKERS] Parallel Seq Scan

2015-10-02 Thread Kouhei Kaigai
> On Thu, Oct 1, 2015 at 2:35 AM, Kouhei Kaigai wrote: > > Gather node was oversight by readfunc.c, even though it shall not be > > serialized actually. > > Also, it used incompatible WRITE_xxx_FIELD() macro on outfuncs.c. > > > > The attached patch fixes both of incomsistence. > > Thanks. You m

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-10-02 Thread Kyotaro HORIGUCHI
Hello, thank you for explanation. I understood the background. On the current planner implement, row marks are tightly bound to initial RTEs. This is quite natural for the purpose of row marks. During join search, a joinrel should be comptible between local joins and remote joins, of course targe

Re: [HACKERS] Confusing remark about UPSERT in fdwhandler.sgml

2015-10-02 Thread Peter Geoghegan
On Fri, Oct 2, 2015 at 1:00 AM, Etsuro Fujita wrote: > ISTM that the sentence "as remote constraints are not locally known" is > somewhat confusing, because check constrains on remote tables can be > defined locally in 9.5. How about "unique constraints or exclusion > constraints on remote tables

[HACKERS] Confusing remark about UPSERT in fdwhandler.sgml

2015-10-02 Thread Etsuro Fujita
The following is a remark about UPSERT in fdwhandler.sgml. INSERT with an ON CONFLICT clause does not support specifying the conflict target, as remote constraints are not locally known. This in turn implies that ON CONFLICT DO UPDATE is not supported, since the specification i

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-10-02 Thread Amit Langote
On 2015/10/02 15:38, Fujii Masao wrote: > > +uint32 progress_param[N_PROGRESS_PARAM]; > > Why did you use an array to store the progress information of VACUUM? > I think that it's better to use separate specific variables for them for > better code readability, for example, variables scanned_