Re: fixing more format truncation issues

2018-03-13 Thread Michael Paquier
On Wed, Feb 28, 2018 at 11:14:23PM -0500, Peter Eisentraut wrote: > AFAICT, the issues addressed here either can't really happen without > trying very hard, or would cause harmless output truncation. Still, it > seems good to clean this up properly and not rely on made-up buffer size > guesses tha

Re: inserts into partitioned table may cause crash

2018-03-13 Thread Amit Langote
Fujita-san, Thanks for the updates and sorry I couldn't reply sooner. On 2018/03/06 21:26, Etsuro Fujita wrote: > One thing I notice while working on this is this in ExecInsert/CopyFrom, > which I moved to ExecPrepareTupleRouting as-is for the former: > > /* > * If we're capturing transi

Re: remove pg_class.relhaspkey

2018-03-13 Thread Michael Paquier
On Sat, Mar 10, 2018 at 01:52:56PM +0100, Tomas Vondra wrote: > I agree with this sentiment - I don't think those flags are particularly > helpful for client applications, and would vote +1 for removal. OK, so I can see that we are moving to a consensus here. > For the other flags we would probab

Re: Faster inserts with mostly-monotonically increasing values

2018-03-13 Thread Pavan Deolasee
On Wed, Mar 14, 2018 at 10:58 AM, Claudio Freire wrote: > > > I'm thinking there could be contention on some lock somewhere. > > Can you attach the benchmark script you're using so I can try to reproduce > it? > I am using a very ad-hoc script.. But here it is.. It assumes a presence of a branc

Re: add queryEnv to ExplainOneQuery_hook

2018-03-13 Thread Tom Lane
Michael Paquier writes: > On Wed, Mar 14, 2018 at 05:36:26PM +1300, Thomas Munro wrote: >> Hmm. I suppose we could have invented a new extended hook with a >> different name and back-patched it so that PG10 would support both. >> Then binary compatibility with existing compiled extensions wouldn'

Re: planner bug regarding lateral and subquery?

2018-03-13 Thread Tatsuro Yamada
Hi David and Stephen, On 2018/03/14 12:59, David G. Johnston wrote: On Tuesday, March 13, 2018, Tatsuro Yamada mailto:yamada.tats...@lab.ntt.co.jp>> wrote: Hi Hackers, I found a bug, maybe. If it is able to get an explain command result from below query successfully, I think t

Re: add queryEnv to ExplainOneQuery_hook

2018-03-13 Thread Michael Paquier
On Wed, Mar 14, 2018 at 05:36:26PM +1300, Thomas Munro wrote: > Hmm. I suppose we could have invented a new extended hook with a > different name and back-patched it so that PG10 would support both. > Then binary compatibility with existing compiled extensions wouldn't > be affected AFAICS, but yo

Re: Faster inserts with mostly-monotonically increasing values

2018-03-13 Thread Claudio Freire
On Wed, Mar 14, 2018 at 1:36 AM, Pavan Deolasee wrote: > > > On Sun, Mar 11, 2018 at 9:18 PM, Claudio Freire > wrote: >> >> On Sun, Mar 11, 2018 at 2:27 AM, Pavan Deolasee >> >> > >> > Yes, I will try that next - it seems like a good idea. So the idea would >> > be: >> > check if the block is sti

Re: [bug fix] Cascaded standby cannot start after a clean shutdown

2018-03-13 Thread Michael Paquier
On Tue, Feb 27, 2018 at 05:15:29AM +, Tsunakawa, Takayuki wrote: > That was my first thought, and I gave it up. As you say, > XLogReadRecord() could allocate up to 1 GB of memory for a garbage. > That allocation can fail due to memory shortage, which prevents the > recovery from proceeding. E

Re: planner bug regarding lateral and subquery?

2018-03-13 Thread Tatsuro Yamada
Hi Stephen, On 2018/03/14 12:36, Stephen Frost wrote: Greetings, * Tatsuro Yamada (yamada.tats...@lab.ntt.co.jp) wrote: I found a bug, maybe. I don't think so... * Result of Select: failed # select subq_1.c0 from test as ref_0,

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
On Tue, Mar 13, 2018 at 4:32 PM, Narendra Pradeep U U wrote: > Hi, > Thanks everyone for your suggestions. I would like to add explain > analyze of both the plans so that we can have broader picture. > > I have a work_mem of 1000 MB. > > The Plan which we get regularly with table being anal

Re: [HACKERS] GUC for cleanup indexes threshold.

2018-03-13 Thread Masahiko Sawada
On Sat, Mar 10, 2018 at 3:40 AM, Alexander Korotkov wrote: > On Fri, Mar 9, 2018 at 3:12 PM, Masahiko Sawada > wrote: >> >> On Fri, Mar 9, 2018 at 8:43 AM, Alexander Korotkov >> wrote: >> > 2) These parameters are reset during btbulkdelete() and set during >> > btvacuumcleanup(). >> >> Can't we

Re: add queryEnv to ExplainOneQuery_hook

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 2:57 PM, Jim Finnerty wrote: > Passing NULL in place of queryEnv in PG10 causes a failure in installcheck > tests portals and plpgsql. > > For PG10, if you want a both an ExplainOneQuery hook and clean run of > installcheck, is there a better workaround than to (a) pass NUL

Re: Faster inserts with mostly-monotonically increasing values

2018-03-13 Thread Pavan Deolasee
On Sun, Mar 11, 2018 at 9:18 PM, Claudio Freire wrote: > On Sun, Mar 11, 2018 at 2:27 AM, Pavan Deolasee > > > > > Yes, I will try that next - it seems like a good idea. So the idea would > be: > > check if the block is still the rightmost block and the insertion-key is > > greater than the first

Re: planner bug regarding lateral and subquery?

2018-03-13 Thread David G. Johnston
On Tuesday, March 13, 2018, Tatsuro Yamada wrote: > Hi Hackers, > > I found a bug, maybe. > If it is able to get an explain command result from below query > successfully, > I think that it means the query is executable. > There is a difference between executable, compilable, and able to execute

Re: planner bug regarding lateral and subquery?

2018-03-13 Thread Stephen Frost
Greetings, * Tatsuro Yamada (yamada.tats...@lab.ntt.co.jp) wrote: > I found a bug, maybe. I don't think so... > * Result of Select: failed > > # select > subq_1.c0 > from > test as ref_0, > lateral (select subq_0.c0 as c0 >fr

Re:Re: Re: [GSOC 18] Performance Farm Project——Initialization Project

2018-03-13 Thread Hongyuan Ma
Hi Dave, I am willing to use React to re-create the front-end application. Since I plan to use separate front-end and back-end development methods, this means that there will be no html files in Django applications. Front-end and back-end applications will interact via restful api. I will use re

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread David Rowley
On 14 March 2018 at 06:54, Jesper Pedersen wrote: > * "Add more expression types..." -- Are you planning to add more of > these ? Otherwise change the comment Run-time pruning will deal with Param types here. The comment there might be just to remind us all that the function must remain gener

planner bug regarding lateral and subquery?

2018-03-13 Thread Tatsuro Yamada
Hi Hackers, I found a bug, maybe. If it is able to get an explain command result from below query successfully, I think that it means the query is executable. However, I got an error by executing the query without an explain command. I guess that planner makes a wrong plan. I share a reproducti

Re: Fixes for missing schema qualifications

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 07:30:23PM -0700, David G. Johnston wrote: > On Tue, Mar 13, 2018 at 6:50 PM, Michael Paquier > wrote: >> To simplify user's life, we >> could also recommend just to users to issue a ALTER FUNCTION SET >> search_path to fix the problem for all functions, that's easier to >>

Re: Changing the autovacuum launcher scheduling; oldest table first algorithm

2018-03-13 Thread Masahiko Sawada
On Tue, Mar 6, 2018 at 11:27 PM, Alvaro Herrera wrote: > Hello > > I haven't read your respective patches yet, but both these threads > brought to memory a patch I proposed a few years ago that I never > completed: > > https://www.postgresql.org/message-id/flat/20130124215715.GE4528%40alvh.no-ip.o

Re: Fixes for missing schema qualifications

2018-03-13 Thread David G. Johnston
On Tue, Mar 13, 2018 at 6:50 PM, Michael Paquier wrote: > On Sat, Mar 10, 2018 at 08:36:34AM +, Noah Misch wrote: > > This qualifies some functions, but it leaves plenty of unqualified > operators. > > Yeah, I know that, and i don't have a perfect reply to offer to you. > There are a couple o

Re: User defined data types in Logical Replication

2018-03-13 Thread Masahiko Sawada
On Wed, Mar 7, 2018 at 9:19 PM, Masahiko Sawada wrote: > On Wed, Mar 7, 2018 at 2:52 AM, Alvaro Herrera > wrote: >> Masahiko Sawada wrote: >>> On Tue, Mar 6, 2018 at 8:35 AM, Alvaro Herrera >>> wrote: >> >>> > Therefore, I'm inclined to make this function raise a warning, then >>> > return a s

Re: PQHost() undefined behavior if connecting string contains both host and hostaddr types

2018-03-13 Thread Michael Paquier
On Fri, Mar 09, 2018 at 04:42:30PM -0500, Peter Eisentraut wrote: > It seems, however, that PGhost() has always been broken for hostaddr > use. In 9.6 (before the multiple-hosts stuff was introduced), when > connecting to "hostaddr=127.0.0.1", PGhost() returns "/tmp". Urgh. > > I think we should

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread Andrew Dunstan
> On Mar 14, 2018, at 10:58 AM, David Rowley > wrote: > > On 14 March 2018 at 11:36, Andrew Dunstan > wrote: >> Here are the benchmark results from the v15 patch. Fairly similar to >> previous results. I'm going to run some profiling again to see if I >> can identify any glaring hotspots. I

Re: Fixes for missing schema qualifications

2018-03-13 Thread Michael Paquier
On Wed, Mar 14, 2018 at 10:50:38AM +0900, Michael Paquier wrote: > For the rest, which basically concerns psql, I have been thinking that > actually using 2) would be the most painful approach, still something > which does not impact the user experience, while 3) is easier to > back-patch by minimi

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Jeff Janes
On Tue, Mar 13, 2018 at 4:57 PM, Thomas Munro wrote: > On Wed, Mar 14, 2018 at 12:29 PM, Tom Lane wrote: > > Thomas Munro writes: > >> There is a fundamental and complicated estimation problem lurking here > >> of course and I'm not sure what to think about that yet. Maybe there > >> is a very

Re: Fixes for missing schema qualifications

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 05:19:50PM -0700, David G. Johnston wrote: > On Tue, Mar 13, 2018 at 5:11 PM, Tatsuo Ishii wrote: >> select pg_catalog.count(*) from pg_catalog.pg_namespace where >> pg_catalog.nameeq(nspname, '%s'); >> >> > ​I'd rather write that: > > select [...] where nspname operator(p

Re: Fixes for missing schema qualifications

2018-03-13 Thread Michael Paquier
On Sat, Mar 10, 2018 at 08:36:34AM +, Noah Misch wrote: > This qualifies some functions, but it leaves plenty of unqualified operators. Yeah, I know that, and i don't have a perfect reply to offer to you. There are a couple of methods that we could use to tackle that: 1) For functions, enforce

Re: ExplainProperty* and units

2018-03-13 Thread David Rowley
On 14 March 2018 at 13:27, Andres Freund wrote: > In the attached *POC* patch I've added a 'unit' parameter to the numeric > ExplainProperty* functions, which EXPLAIN_FORMAT_TEXT adds to the > output. This can avoid the above and other similar branches (of which > the JIT patch would add a number

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 12:19:07PM -0400, David Steele wrote: > I'll attach new patches in a reply to [1] once I have made the changes > Tom requested. Cool, thanks for your patience. Looking forward to seeing those. I'll spend time on 0003 at the same time. Let's also put the additional umask

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 01:28:17PM -0400, Tom Lane wrote: > David Steele writes: >> On 3/12/18 3:28 AM, Michael Paquier wrote: >>> In pg_rewind and pg_resetwal, isn't that also a portion which is not >>> necessary without the group access feature? > >> These seem like a good idea to me with or wi

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread David Gould
On Tue, 13 Mar 2018 11:29:03 -0400 Tom Lane wrote: > David Gould writes: > > I have attached the patch we are currently using. It applies to 9.6.8. I > > have versions for older releases in 9.4, 9.5, 9.6. I fails to apply to 10, > > and presumably head but I can update it if there is any interes

Re: Fixes for missing schema qualifications

2018-03-13 Thread David G. Johnston
On Tue, Mar 13, 2018 at 5:26 PM, Tatsuo Ishii wrote: > Next question is, should we update the manual? There are bunch of > places where example queries are shown without schema qualifications. > > I hope that isn't deemed necessary. David J.

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread Andres Freund
Hi, On 2018-03-14 09:06:54 +1030, Andrew Dunstan wrote: > I do suspect that the "physical tlist" optimization sometimes turns > out not to be one. It seems perverse to be able to improve a query's > performance by dropping a column. Yea, it's definitely not always a boon. Especially w/ the newer

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread David Rowley
On 14 March 2018 at 11:36, Andrew Dunstan wrote: > Here are the benchmark results from the v15 patch. Fairly similar to > previous results. I'm going to run some profiling again to see if I > can identify any glaring hotspots. I do suspect that the "physical > tlist" optimization sometimes turns o

ExplainProperty* and units

2018-03-13 Thread Andres Freund
Hi, while adding EXPLAIN support for JITing (displaying time spent etc), I got annoyed by the amount of duplication required. There's a fair amount of if (es->format == EXPLAIN_FORMAT_TEXT) appendStringInfo(es->str, "Execution time: %.3f ms\n",

Re: Fixes for missing schema qualifications

2018-03-13 Thread Tatsuo Ishii
>> select pg_catalog.count(*) from pg_catalog.pg_namespace where >> pg_catalog.nameeq(nspname, '%s'); >> >> > ​I'd rather write that: > > select [...] where nspname operator(pg_catalog.=) '%s'​ > > Introducing undocumented implementation functions to these queries is > undesirable; and besides, i

Re: Fixes for missing schema qualifications

2018-03-13 Thread David G. Johnston
On Tue, Mar 13, 2018 at 5:11 PM, Tatsuo Ishii wrote: > >>> +"select pg_catalog.count(*) " > >>> +"from pg_catalog.pg_namespace > where nspname = '%s'", > >> > >> This qualifies some functions, but it leaves plenty of unqualif

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2018-03-13 Thread David Rowley
On 14 March 2018 at 09:25, Robert Haas wrote: > What do you think about the idea of using a projection path as a proxy > path instead of inventing a new method? It seems simple enough to do: > > new_path = (Path *) create_projection_path(root, new_rel, old_path, > old_path->pathtarget); > > ...wh

Re: Fixes for missing schema qualifications

2018-03-13 Thread Tatsuo Ishii
>>> +"select pg_catalog.count(*) " >>> +"from pg_catalog.pg_namespace where >>> nspname = '%s'", >> >> This qualifies some functions, but it leaves plenty of unqualified operators. > > So this should go something like this?

Re: Fixes for missing schema qualifications

2018-03-13 Thread Tatsuo Ishii
>> + "select pg_catalog.count(*) " >> + "from pg_catalog.pg_namespace where >> nspname = '%s'", > > This qualifies some functions, but it leaves plenty of unqualified operators. So this should go something like this? select

Re: SQL/JSON: functions

2018-03-13 Thread Oleg Bartunov
On 14 Mar 2018 01:54, "Michael Paquier" wrote: On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > The docs are here > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in more > friendly

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 12:29 PM, Tom Lane wrote: > Thomas Munro writes: >> There is a fundamental and complicated estimation problem lurking here >> of course and I'm not sure what to think about that yet. Maybe there >> is a very simple fix for this particular problem: > > Ah, I see you though

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > I'm not advocating the Sisyphean task of having PG incorporate > knowledge of all those possibilities. I'm advocating the conservative > approach: have PG be that well-behaved application that those extended > semantics are generally all de

JIT compiling with LLVM v12

2018-03-13 Thread Andres Freund
Hi, I've pushed a revised and rebased version of my JIT patchset. The git tree is at https://git.postgresql.org/git/users/andresfreund/postgres.git in the jit branch https://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/jit There's nothing hugely exciti

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Alvaro Herrera
By the way, I checked whether patch 0002 (additional tests) had an effect on coverage, and couldn't detect any changes in terms of lines/functions. Were you able to find any bugs in your code thanks to the new tests that would not have been covered by existing tests? -- Álvaro Herrera

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Tom Lane
Thomas Munro writes: > There is a fundamental and complicated estimation problem lurking here > of course and I'm not sure what to think about that yet. Maybe there > is a very simple fix for this particular problem: Ah, I see you thought of the same hack I did. I think this may actually be a g

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Tom Lane
Thomas Munro writes: > This looks like an invisible correlation problem. Yeah --- the planner has no idea that the join rows satisfying newdata.* *= newdata2.* are likely to be exactly the ones not satisfying newdata.ctid <> newdata2.ctid. It's very accidental that we got a good plan before. I'

Re: SQL/JSON: functions

2018-03-13 Thread Andres Freund
On 2018-03-14 07:54:35 +0900, Michael Paquier wrote: > On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > > The docs are here > > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 11:34 AM, Thomas Munro wrote: > LOG: duration: 26101.612 ms plan: > Query Text: SELECT newdata FROM pg_temp_3.pg_temp_16452 newdata WHERE > newdata IS NOT NULL AND EXISTS (SELECT * FROM pg_temp_3.pg_temp_16452 > newdata2 WHERE newdata2 IS NOT NULL AND newdata2 > OPERATOR(

Re: SQL/JSON: functions

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > The docs are here > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in more > friendly way. We'll have time to convert it to postgres f

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Alvaro Herrera
Amit Langote wrote: > I will continue working on improving the comments / cleaning things up and > post a revised version soon, but until then please look at the attached. I tried to give this a read. It looks pretty neat stuff -- as far as I can tell, it follows Robert's sketch for how this sho

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread Andrew Dunstan
On Tue, Mar 13, 2018 at 10:58 PM, Andrew Dunstan wrote: > On Tue, Mar 13, 2018 at 2:40 PM, Andrew Dunstan > wrote: > >>> >>> Going by the commitfest app, the patch still does appear to be waiting >>> on Author. Never-the-less, I've made another pass over it and found a >>> few mistakes and a coup

Re: [submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread Euler Taveira
2018-03-13 12:19 GMT-03:00 leap : > I develop a tool for pgsql, I want to submit it on pgsql. > how can I submit it? > As Laurenz said a tool doesn't necessarily have to be part of PostgreSQL. Sometimes a lot of people ask for a tool, someone write it and the community decide to maintain it. I'm no

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 8:07 AM, Jeff Janes wrote: > The following commit has caused a devastating performance regression > in concurrent refresh of MV: > > commit 7ca25b7de6aefa5537e0dbe56541bc41c0464f97 > Author: Tom Lane > Date: Wed Nov 29 22:00:29 2017 -0500 > > Fix neqjoinsel's behavio

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
Hi, On 2018-03-14 10:32:40 +1300, Thomas Munro wrote: > I decided to try this on a CentOS 7.2 box. It has LLVM 3.9 in the > 'epel' package repo, but unfortunately it only has clang 3.4. That's a bit odd, given llvm and clang really live in the same repo... > clang: error: unknown argument: '-f

Re: JIT compiling with LLVM v11

2018-03-13 Thread Thomas Munro
On Thu, Mar 1, 2018 at 9:02 PM, Andres Freund wrote: > Biggest changes: > - LLVM 3.9 - master are now supported. This includes a good chunk of > work by Pierre Ducroquet. I decided to try this on a CentOS 7.2 box. It has LLVM 3.9 in the 'epel' package repo, but unfortunately it only has clang

Re: parallel append vs. simple UNION ALL

2018-03-13 Thread Robert Haas
On Tue, Mar 13, 2018 at 12:35 AM, Ashutosh Bapat wrote: > It looks like it was not changed in all the places. make falied. I > have fixed all the instances of these two functions in the attached > patchset (only 0003 changes). Please check. Oops. Thanks. I'm going to go ahead and commit 0001 he

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 02:47 PM, Tom Lane wrote: > Well, to be blunt, what we target is POSIX-compatible systems. If > you're telling us to worry about non-POSIX filesystem semantics, > and your name is not Microsoft, it's going to be a hard sell. > We have enough to do just keeping up with that scope of t

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2018-03-13 Thread Robert Haas
On Mon, Feb 19, 2018 at 4:02 AM, David Rowley wrote: > On 19 February 2018 at 18:01, David Rowley > wrote: >> On 19 February 2018 at 15:11, Tomas Vondra >> wrote: >>> and perhaps we should do s/isproxy/is_proxy/ which seems like the usual >>> naming for boolean variables. >> >> You're right. I

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > On 03/13/2018 01:50 PM, Stephen Frost wrote: > > PG will stat PGDATA and conclude that the system is saying that group > > access has been granted on PGDATA and will do the same for subsequent > > files created later on. ... The o

Re: [submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread Laurenz Albe
leap wrote: > I develop a tool for pgsql, I want to submit it on pgsql. > how can I submit it? > > address: https://github.com/leapking/pgcheck I would leave it on Github and add some documentation. A lot of great tools for PostgreSQL are not part of the core distribution; that doesn't mean that

Re: [HACKERS] taking stdbool.h into use

2018-03-13 Thread Peter Eisentraut
On 3/1/18 23:34, Michael Paquier wrote: > Indeed, this is wrong. Peter, why did you switch suddendly this patch > as ready for committer? The patch is waiting for your input as you > mentioned that the GIN portion of this patch series is not completely > baked yet. So I have switched the patch i

Re: JIT compiling with LLVM v11

2018-03-13 Thread Tom Lane
Andres Freund writes: > On 2018-03-13 14:36:44 -0400, Robert Haas wrote: >> I realize that EXPLAIN (JIT OFF) may sound like it's intended to >> disable JIT itself > Yea, that's what I'm concerned about. >> , but I think it's pretty clear that EXPLAIN (BUFFERS OFF) does not >> disable the use of

neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Jeff Janes
The following commit has caused a devastating performance regression in concurrent refresh of MV: commit 7ca25b7de6aefa5537e0dbe56541bc41c0464f97 Author: Tom Lane Date: Wed Nov 29 22:00:29 2017 -0500 Fix neqjoinsel's behavior for semi/anti join cases. The below reproduction goes from tak

Re: Fix error in ECPG while connection handling

2018-03-13 Thread Jeevan Ladhe
> Thanks for spotting and fixing. I will push the patch as soon as I'm > online again. > Thanks Michael for taking care of this. Regards, Jeevan Ladhe.

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
Chapman Flack writes: > On 03/13/2018 01:50 PM, Stephen Frost wrote: >> I'll point out that PG does run on quite a few other OS's beyond Linux. > I'll second that, as I think it is making my argument. When I can > supply three or four examples of semantic subtleties that undermine > PG's assumpti

Re: proposal: schema variables

2018-03-13 Thread Pavel Stehule
2018-03-13 10:54 GMT+01:00 Pavel Luzanov : > Pavel Stehule wrote > > Now, there is one important question - storage - Postgres stores all > > objects to files - only memory storage is not designed yet. This is part, > > where I need a help. > > O, I do not feel confident in such questions. > May b

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
Hi, On 2018-03-13 14:36:44 -0400, Robert Haas wrote: > On Mon, Mar 12, 2018 at 5:04 PM, Andres Freund wrote: > > Currently a handful of explain outputs in the regression tests change > > output when compiled with JITing. Therefore I'm thinking of adding > > JITINFO or such option, which can be se

Re: JIT compiling with LLVM v11

2018-03-13 Thread Robert Haas
On Mon, Mar 12, 2018 at 5:04 PM, Andres Freund wrote: > Currently a handful of explain outputs in the regression tests change > output when compiled with JITing. Therefore I'm thinking of adding > JITINFO or such option, which can be set to false for those tests? Can we spell that JIT or at least

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-13 Thread Robert Haas
On Wed, Feb 14, 2018 at 5:37 AM, Amit Kapila wrote: > Your concern is valid, but isn't the same problem exists in another > approach as well, because in that also we can call > adjust_paths_for_srfs after generating gather path which means that we > might lose some opportunity to reduce the relati

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
On 2018-03-13 10:25:49 -0400, Peter Eisentraut wrote: > On 3/12/18 17:04, Andres Freund wrote: > > │ JIT: > > │ > > │ Functions: 4 > >

Re: TupleTableSlot abstraction

2018-03-13 Thread Andres Freund
On 2018-03-13 15:18:34 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > I've recently been discussing with Robert how to abstract > > TupleTableSlots in the light of upcoming developments around abstracting > > storage away. Besides that aspect I think we also need to make them

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 01:50 PM, Stephen Frost wrote: > Yes, that's true, but PG has never paid attention to POSIX ACLs or Linux > capabilities. Changing it to do so is quite a bit beyond the scope... I think we're largely in agreement here, as my aim was not to advocate that PG should work harder to und

Re: INOUT parameters in procedures

2018-03-13 Thread Pavel Stehule
2018-03-13 14:14 GMT+01:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > On 3/8/18 02:25, Pavel Stehule wrote: > > It looks like some error in this concept. The rules for enabling > > overwriting procedures should modified, so this collision should not be > > done. > > > > When I using p

Re: TupleTableSlot abstraction

2018-03-13 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > I've recently been discussing with Robert how to abstract > TupleTableSlots in the light of upcoming developments around abstracting > storage away. Besides that aspect I think we also need to make them > more abstract to later deal with vectorized excution, but I'm

Inquiry regarding the projects under GSOC

2018-03-13 Thread Dr K.G Yadav
Hello Postgres Team I am Ankit 3rd year B.tech student from India I am really interested to work along with your organisation as an intern to rectify the issues and bugs as I love challenges and stated in the docs that some features might be not possible to implement so I would love to try. My i

[submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread leap
hello! I develop a tool for pgsql, I want to submit it on pgsql. how can I submit it? address: https://github.com/leapking/pgcheck

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Jesper Pedersen
Hi Amit, On 03/13/2018 07:37 AM, Amit Langote wrote: I will continue working on improving the comments / cleaning things up and post a revised version soon, but until then please look at the attached. Passes check-world. Some minor comments: 0001: Ok 0002: Ok 0003: * Trailing white space

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > On 03/13/2018 10:40 AM, Stephen Frost wrote: > > ... Ultimately, the default which makes sense here isn't a > > one-size-fits-all but is system dependent and the administrator should > > be able to choose what permissions they wa

Re: Additional Statistics Hooks

2018-03-13 Thread Tom Lane
Mat Arye writes: > An example, of a case a protransform type system would not be able to > optimize is mathematical operator expressions like bucketing integers by > decile --- (integer / 10) * 10. Uh, why not? An estimation function that is specific to integer divide shouldn't have much trouble

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 10:40 AM, Stephen Frost wrote: > * Michael Paquier (mich...@paquier.xyz) wrote: >> Well, one thing is that the current checks in the postmaster make sure >> that a data folder is never using anything else than 0700. From a >> security point of view, making it possible to allow a post

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
David Steele writes: > On 3/12/18 3:28 AM, Michael Paquier wrote: >> In pg_rewind and pg_resetwal, isn't that also a portion which is not >> necessary without the group access feature? > These seem like a good idea to me with or without patch 03. Some of our > front-end tools (initdb, pg_upgrade

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-13 Thread Michail Nikolaev
Hello. Tom, thanks a lot for your thorough review. > What you've done to > IndexNext() is a complete disaster from a modularity standpoint: it now > knows all about the interactions between index_getnext, index_getnext_tid, > and index_fetch_heap. I was looking into the current IndexOnlyNext imp

Re: PATCH: Unlogged tables re-initialization tests

2018-03-13 Thread Alvaro Herrera
Dagfinn Ilmari Mannsåker wrote: > $SIG{__DIE__} gets called even for exceptions that would be caught by a > surrunding eval block, so this should at the very least be: > > $SIG{__DIE__} = sub { BAIL_OUT(@_) unless $^S }; > > However, that is still wrong, because die() and BAIL_OUT() mean > d

Re: PATCH: Exclude temp relations from base backup

2018-03-13 Thread David Steele
Hi, On 2/28/18 10:55 AM, David Steele wrote: > This is a follow-up patch from the exclude unlogged relations discussion > [1]. > > The patch excludes temporary relations during a base backup using the > existing looks_like_temp_rel_name() function for identification. > > It shares code to identi

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-13 Thread Arthur Zakirov
On Mon, Mar 12, 2018 at 02:11:42PM +0100, Julian Markwort wrote: > > In same manner pg_stat_statements_good_plan_reset() and > > pg_stat_statements_bad_plan_reset() functions can be combined in > > function: > > > pg_stat_statements_plan_reset(in queryid bigint, in good boolean, in bad > > boolean

Re: Additional Statistics Hooks

2018-03-13 Thread Mat Arye
On Tue, Mar 13, 2018 at 6:31 AM, David Rowley wrote: > On 13 March 2018 at 11:44, Tom Lane wrote: > > While it would certainly be nice to have better behavior for that, > > "add a hook so users who can write C can fix it by hand" doesn't seem > > like a great solution. On top of the sheer diffi

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
Hi Michael, On 3/12/18 3:28 AM, Michael Paquier wrote: > On Fri, Mar 09, 2018 at 01:51:14PM -0500, David Steele wrote: >> How about a GUC that enforces one mode or the other on startup? Default >> would be 700. The GUC can be set automatically by initdb based on the >> -g option. We had this GU

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > On 3/13/18 11:00 AM, Tom Lane wrote: > > Stephen Frost writes: > >> * Michael Paquier (mich...@paquier.xyz) wrote: > >>> If the problem is parsing, it could as well be more portable to put that > >>> in the control file, no? > > > >> Then

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Stephen Frost
Greetings, * Aleksander Alekseev (a.aleks...@postgrespro.ru) wrote: > Craig, I believe you probably did something wrong if you had to work > with some library directly. Actually you generate classes from text > description and just use them. I worked with Thrift some time ago, in > 2015 [1]. I wou

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread Tom Lane
David Gould writes: > I have attached the patch we are currently using. It applies to 9.6.8. I > have versions for older releases in 9.4, 9.5, 9.6. I fails to apply to 10, > and presumably head but I can update it if there is any interest. > The patch has three main features: > - Impose an orderi

Re: Additional Statistics Hooks

2018-03-13 Thread Mat Arye
On Tue, Mar 13, 2018 at 6:56 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Mar 13, 2018 at 4:14 AM, Tom Lane wrote: > > Mat Arye writes: > >> So the use-case is an analytical query like > > > >> SELECT date_trunc('hour', time) AS MetricMinuteTs, AVG(value) as avg > >> FR

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
On 3/13/18 11:00 AM, Tom Lane wrote: > Stephen Frost writes: >> * Michael Paquier (mich...@paquier.xyz) wrote: >>> If the problem is parsing, it could as well be more portable to put that >>> in the control file, no? > >> Then we'd need a tool to allow changing it for people who do want to >> cha

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread Tom Lane
David Gould writes: > I also thought about the theory and am confident that there really is no way > to trick it. Basically if there are enough pages that are different to affect > the overall density, say 10% empty or so, there is no way a random sample > larger than a few hundred probes can miss

Re: committing inside cursor loop

2018-03-13 Thread Peter Eisentraut
On 3/6/18 07:48, Ildus Kurbangaliev wrote: > Although as was discussed before it seems inconsistent without ROLLBACK > support. There was a little discussion about it, but no replies. Maybe > the status of the patch should be changed to 'Waiting on author' until > the end of discussion. I'm wonder

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-13 Thread Stephen Frost
Greetings, * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > On Mon, Mar 12, 2018 at 1:25 PM, Etsuro Fujita > wrote: > > (2018/03/09 20:55), Etsuro Fujita wrote: > >> (2018/03/08 14:24), Ashutosh Bapat wrote: > >>> For local constraints to be enforced, we use remote > >>> constraints. F

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Andrey Borodin
Thanks, Aleksander!SP- > 13 марта 2018 г., в 19:03, Aleksander Alekseev > написал(а): > >> Do you have any project related insights as to what I should put in >> there? > Christos, as far as I remember, good proposal must have schedule, implementation details and deliverables. Also, it is goo

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
Stephen Frost writes: > * Michael Paquier (mich...@paquier.xyz) wrote: >> If the problem is parsing, it could as well be more portable to put that >> in the control file, no? > Then we'd need a tool to allow changing it for people who do want to > change it. There's no reason we should have to h

  1   2   >