Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Ashutosh Bapat
On Mon, Mar 12, 2018 at 10:02 PM, Narendra Pradeep U U wrote: > Hi , > > Recently I came across a case where the planner choose larger table on > hash side. I am not sure whether it is an intended behavior or we are > missing something. > > I have two tables (a and b) each with single

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread David Gould
On Mon, 12 Mar 2018 12:21:34 -0400 Tom Lane wrote: > I wrote: > > Maybe this type of situation is an argument for trusting an ANALYZE-based > > estimate more than the VACUUM-based estimate. I remain uncomfortable with > > that in cases where VACUUM looked at much more of the table than ANALYZE >

Re: Problem while setting the fpw with SIGHUP

2018-03-13 Thread Michael Paquier
On Fri, Mar 09, 2018 at 01:42:04PM +0530, Dilip Kumar wrote: > While setting the full_page_write with SIGHUP I hit an assert in checkpoint > process. And, that is because inside a CRITICAL section we are calling > RecoveryInProgress which intern allocates memory. So I have moved > RecoveryInProgre

RE: Temporary tables prevent autovacuum, leading to XID wraparound

2018-03-13 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > So the correct fix is to improve autovacuum's check to discover whether > an old temp table is orphaned, so that it isn't fooled by putative owner > processes that are connected to some other DB. Step 2 of the proposed patch > tries to do that, but it's

Re: MCV lists for highly skewed distributions

2018-03-13 Thread John Naylor
Hi Dean, I've looked over your patch briefly, but not tested it yet. > [construction of a pathological data set] > So the table has around 31 million rows, and the stats target is maxed > out -- we're sampling around 10% of the table, and it's not possible > to sample more. Looking at the value a

Re: proposal: schema variables

2018-03-13 Thread Pavel Luzanov
Pavel Stehule wrote > Now, there is one important question - storage - Postgres stores all > objects to files - only memory storage is not designed yet. This is part, > where I need a help. O, I do not feel confident in such questions. May be some ideas you can get from extension with similar func

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Aleksander Alekseev
Hello everyone, > > I am mostly interested in anything that requires C/C++ implementation and > > AlgoDS. > > > > For that reason I would love to work in any of the following (in that > > order of preference): > > > >1. Sorting algorithms benchmark and implementation > >2. Enhancing amchec

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-13 Thread Anastasia Lubennikova
05.02.2018 10:10, Michael Paquier: So the patch set attached is made of the following: - 0001, which refactors all hardcoded system paths into pg_paths.h. This modifies only initdb.c and basebackup.c to ease reviews. - 0002 spreads the path changes and the use of pg_paths.h across the core code.

Re: Additional Statistics Hooks

2018-03-13 Thread David Rowley
On 13 March 2018 at 11:44, Tom Lane wrote: > While it would certainly be nice to have better behavior for that, > "add a hook so users who can write C can fix it by hand" doesn't seem > like a great solution. On top of the sheer difficulty of writing a > hook function, you'd have the problem that

Re: Additional Statistics Hooks

2018-03-13 Thread Ashutosh Bapat
On Tue, Mar 13, 2018 at 4:14 AM, Tom Lane wrote: > Mat Arye writes: >> So the use-case is an analytical query like > >> SELECT date_trunc('hour', time) AS MetricMinuteTs, AVG(value) as avg >> FROM hyper >> WHERE time >= '2001-01-04T00:00:00' AND time <= '2001-01-05T01:00:00' >> GROUP BY MetricMin

Re: Ambigous Plan - Larger Table on Hash Side

2018-03-13 Thread Narendra Pradeep U U
Hi, Thanks everyone for your suggestions. I would like to add explain analyze of both the plans so that we can have broader picture. I have a work_mem of 1000 MB. The Plan which we get regularly with table being analyzed . tpch=# explain analyze select b from tab2 left join tab1 o

Re: SQL/JSON: functions

2018-03-13 Thread Simon Riggs
On 7 March 2018 at 14:34, Nikita Glukhov wrote: > Attached 12th version of SQL/JSON patches rebased onto the latest master. Please write some docs or notes to go with this. If you drop a big pile of code with no explanation it will just be ignored. I think many people want SQL/JSON, but the pu

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-13 Thread Alexander Korotkov
On Mon, Mar 12, 2018 at 9:47 AM, Andrey Borodin wrote: > > 12 марта 2018 г., в 1:54, Alexander Korotkov > написал(а): > > > > On Wed, Mar 7, 2018 at 8:30 PM, Alvaro Herrera > wrote: > > I suggest to create a new function GinPredicateLockPage() that checks > > whether fast update is enabled for

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-13 Thread Alvaro Herrera
Alexander Korotkov wrote: > And what happen if somebody concurrently set (fastupdate = on)? > Can we miss conflicts because of that? I think it'd be better to have that option require AccessExclusive lock, so that it can never be changed concurrently with readers. Seems to me that penalizing eve

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-03-13 Thread Andrey Borodin
> 13 марта 2018 г., в 17:02, Alexander Korotkov > написал(а): > > BTW to BTW. I think we should check pending list size with > GinGetPendingListCleanupSize() here > + > + /* > +* If fast update is enabled, we acquire a predicate lock on the > entire > +* relation as fas

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Christos Maris
Hi all, At first, I appreciate your insights Craig, but I think I will stick with AlgoDS ;) Aleksander, I am mostly interested on the sorting algos benchmark and implementation one. I will start writing a proposal soon enough. Do you have any project related insights as to what I should put in th

Re: Using base backup exclusion filters to reduce data transferred with pg_rewind

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 01:16:27PM +0300, Anastasia Lubennikova wrote: > Since these patches contain mostly cosmetic changes and do not break > anything, I think it's fine to mark this thread as Ready For Committer > without long discussion. Thanks Anastasia for the review. The refactoring is qui

Re: Re: [HACKERS] make async slave to wait for lsn to be replayed

2018-03-13 Thread David Steele
Hi Ivan, On 3/6/18 9:25 PM, Michael Paquier wrote: > On Tue, Mar 06, 2018 at 02:24:24PM +0300, Ivan Kartyshov wrote: >> Hello, I now is preparing the patch over syntax that Simon offered. And in >> few day I will update the patch. >> Thank you for your interest in thread. > > It has been more tha

Re: SQL/JSON: functions

2018-03-13 Thread Oleg Bartunov
On Tue, Mar 13, 2018 at 2:04 PM, Simon Riggs wrote: > On 7 March 2018 at 14:34, Nikita Glukhov wrote: > >> Attached 12th version of SQL/JSON patches rebased onto the latest master. > > Please write some docs or notes to go with this. > > If you drop a big pile of code with no explanation it will

Re: INOUT parameters in procedures

2018-03-13 Thread Peter Eisentraut
On 3/8/18 02:25, Pavel Stehule wrote: > It looks like some error in this concept. The rules for enabling > overwriting procedures should modified, so this collision should not be > done. > > When I using procedure from PL/pgSQL, then it is clear, so I place on > *OUT position variables. But when I

Re: Fix error in ECPG while connection handling

2018-03-13 Thread Michael Meskes
> Now, ideally the connection would have been null here, but, as the > 'ClosePortalStmt' > rule freed the connection but did not set it to NULL, it still sees > that there > is a connection(which is actually having garbage in it) and throws an > error. Thanks for spotting and fixing. I will push t

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-13 Thread Amit Kapila
On Mon, Mar 12, 2018 at 7:18 PM, Amit Kapila wrote: > On Mon, Mar 12, 2018 at 12:18 AM, Alexander Korotkov > wrote: >> On Sat, Mar 3, 2018 at 2:53 PM, Amit Kapila wrote: >>> >>> On Fri, Mar 2, 2018 at 9:27 AM, Thomas Munro >>> > If that is indeed a race, could it be fixed by >>> > calling Predi

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Aleksander Alekseev
Hello Christos, > Do you have any project related insights as to what I should put in > there? Nope :) I believe Andrey Borodin and Atri Sharma have (added to CC). -- Best regards, Aleksander Alekseev signature.asc Description: PGP signature

Re: PATCH: Unlogged tables re-initialization tests

2018-03-13 Thread David Steele
Thanks for reviewing, Peter. On 3/9/18 5:23 PM, Peter Eisentraut wrote: > This seems like a useful test. > > On 3/5/18 12:35, David Steele wrote: >> +mkdir($tablespaceDir) >> +or die "unable to mkdir \"$tablespaceDir\""; > > Use BAIL_OUT instead of die in tests. Done. >> +$ts1UnloggedP

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-13 Thread Jeevan Chalke
Hi, I have resolved all the comments/issues reported in this new patch-set. Changes done by Ashutosh Bapat for splitting out create_grouping_paths() into separate functions so that partitionwise aggregate code will use them were based on my partitionwise aggregate changes. Those were like refacto

Re: JIT compiling with LLVM v11

2018-03-13 Thread Peter Eisentraut
On 3/12/18 17:04, Andres Freund wrote: > │ JIT: >│ > │ Functions: 4 >│ > │ Inlining: false

Re: [HACKERS] Proposal: generic WAL compression

2018-03-13 Thread Oleg Ivanov
On 02/07/2018 09:02 PM, Markus Nullmeier wrote: One general comment I can already make is that enabling compression should be made optional, which appears to be a small and easy addition to the generic WAL API. The new version of the patch is attached. In order to control generic WAL compressi

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Michael, * Michael Paquier (mich...@paquier.xyz) wrote: > On Mon, Mar 12, 2018 at 03:14:13PM -0400, Stephen Frost wrote: > > We already had a discussion about having a GUC for this and concluded, > > rightly in my view, that it's not sensible to have since we don't want > > all of the various tool

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
On 3/13/18 2:46 AM, Michael Paquier wrote: > On Mon, Mar 12, 2018 at 03:14:13PM -0400, Stephen Frost wrote: >> We already had a discussion about having a GUC for this and concluded, >> rightly in my view, that it's not sensible to have since we don't want >> all of the various tools having to read

Re: Transform for pl/perl

2018-03-13 Thread Nikita Glukhov
Hi. I have reviewed this patch too. Attached new version with v8-v9 delta-patch. Here is my changes: * HV_ToJsonbValue(): - addded missing hv_iterinit() - used hv_iternextsv() instead of hv_iternext(), HeSVKEY_force(), HeVAL() * SV_ToJsonbValue(): - added recursive dereferencing

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
Stephen Frost writes: > * Michael Paquier (mich...@paquier.xyz) wrote: >> If the problem is parsing, it could as well be more portable to put that >> in the control file, no? > Then we'd need a tool to allow changing it for people who do want to > change it. There's no reason we should have to h

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Andrey Borodin
Thanks, Aleksander!SP- > 13 марта 2018 г., в 19:03, Aleksander Alekseev > написал(а): > >> Do you have any project related insights as to what I should put in >> there? > Christos, as far as I remember, good proposal must have schedule, implementation details and deliverables. Also, it is goo

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

2018-03-13 Thread Stephen Frost
Greetings, * Ashutosh Bapat (ashutosh.ba...@enterprisedb.com) wrote: > On Mon, Mar 12, 2018 at 1:25 PM, Etsuro Fujita > wrote: > > (2018/03/09 20:55), Etsuro Fujita wrote: > >> (2018/03/08 14:24), Ashutosh Bapat wrote: > >>> For local constraints to be enforced, we use remote > >>> constraints. F

Re: committing inside cursor loop

2018-03-13 Thread Peter Eisentraut
On 3/6/18 07:48, Ildus Kurbangaliev wrote: > Although as was discussed before it seems inconsistent without ROLLBACK > support. There was a little discussion about it, but no replies. Maybe > the status of the patch should be changed to 'Waiting on author' until > the end of discussion. I'm wonder

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread Tom Lane
David Gould writes: > I also thought about the theory and am confident that there really is no way > to trick it. Basically if there are enough pages that are different to affect > the overall density, say 10% empty or so, there is no way a random sample > larger than a few hundred probes can miss

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
On 3/13/18 11:00 AM, Tom Lane wrote: > Stephen Frost writes: >> * Michael Paquier (mich...@paquier.xyz) wrote: >>> If the problem is parsing, it could as well be more portable to put that >>> in the control file, no? > >> Then we'd need a tool to allow changing it for people who do want to >> cha

Re: Additional Statistics Hooks

2018-03-13 Thread Mat Arye
On Tue, Mar 13, 2018 at 6:56 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Mar 13, 2018 at 4:14 AM, Tom Lane wrote: > > Mat Arye writes: > >> So the use-case is an analytical query like > > > >> SELECT date_trunc('hour', time) AS MetricMinuteTs, AVG(value) as avg > >> FR

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread Tom Lane
David Gould writes: > I have attached the patch we are currently using. It applies to 9.6.8. I > have versions for older releases in 9.4, 9.5, 9.6. I fails to apply to 10, > and presumably head but I can update it if there is any interest. > The patch has three main features: > - Impose an orderi

Re: Google Summer of Code: Potential Applicant

2018-03-13 Thread Stephen Frost
Greetings, * Aleksander Alekseev (a.aleks...@postgrespro.ru) wrote: > Craig, I believe you probably did something wrong if you had to work > with some library directly. Actually you generate classes from text > description and just use them. I worked with Thrift some time ago, in > 2015 [1]. I wou

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > On 3/13/18 11:00 AM, Tom Lane wrote: > > Stephen Frost writes: > >> * Michael Paquier (mich...@paquier.xyz) wrote: > >>> If the problem is parsing, it could as well be more portable to put that > >>> in the control file, no? > > > >> Then

Re: PATCH: Configurable file mode mask

2018-03-13 Thread David Steele
Hi Michael, On 3/12/18 3:28 AM, Michael Paquier wrote: > On Fri, Mar 09, 2018 at 01:51:14PM -0500, David Steele wrote: >> How about a GUC that enforces one mode or the other on startup? Default >> would be 700. The GUC can be set automatically by initdb based on the >> -g option. We had this GU

Re: Additional Statistics Hooks

2018-03-13 Thread Mat Arye
On Tue, Mar 13, 2018 at 6:31 AM, David Rowley wrote: > On 13 March 2018 at 11:44, Tom Lane wrote: > > While it would certainly be nice to have better behavior for that, > > "add a hook so users who can write C can fix it by hand" doesn't seem > > like a great solution. On top of the sheer diffi

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-13 Thread Arthur Zakirov
On Mon, Mar 12, 2018 at 02:11:42PM +0100, Julian Markwort wrote: > > In same manner pg_stat_statements_good_plan_reset() and > > pg_stat_statements_bad_plan_reset() functions can be combined in > > function: > > > pg_stat_statements_plan_reset(in queryid bigint, in good boolean, in bad > > boolean

Re: PATCH: Exclude temp relations from base backup

2018-03-13 Thread David Steele
Hi, On 2/28/18 10:55 AM, David Steele wrote: > This is a follow-up patch from the exclude unlogged relations discussion > [1]. > > The patch excludes temporary relations during a base backup using the > existing looks_like_temp_rel_name() function for identification. > > It shares code to identi

Re: PATCH: Unlogged tables re-initialization tests

2018-03-13 Thread Alvaro Herrera
Dagfinn Ilmari Mannsåker wrote: > $SIG{__DIE__} gets called even for exceptions that would be caught by a > surrunding eval block, so this should at the very least be: > > $SIG{__DIE__} = sub { BAIL_OUT(@_) unless $^S }; > > However, that is still wrong, because die() and BAIL_OUT() mean > d

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-03-13 Thread Michail Nikolaev
Hello. Tom, thanks a lot for your thorough review. > What you've done to > IndexNext() is a complete disaster from a modularity standpoint: it now > knows all about the interactions between index_getnext, index_getnext_tid, > and index_fetch_heap. I was looking into the current IndexOnlyNext imp

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
David Steele writes: > On 3/12/18 3:28 AM, Michael Paquier wrote: >> In pg_rewind and pg_resetwal, isn't that also a portion which is not >> necessary without the group access feature? > These seem like a good idea to me with or without patch 03. Some of our > front-end tools (initdb, pg_upgrade

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 10:40 AM, Stephen Frost wrote: > * Michael Paquier (mich...@paquier.xyz) wrote: >> Well, one thing is that the current checks in the postmaster make sure >> that a data folder is never using anything else than 0700. From a >> security point of view, making it possible to allow a post

Re: Additional Statistics Hooks

2018-03-13 Thread Tom Lane
Mat Arye writes: > An example, of a case a protransform type system would not be able to > optimize is mathematical operator expressions like bucketing integers by > decile --- (integer / 10) * 10. Uh, why not? An estimation function that is specific to integer divide shouldn't have much trouble

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > On 03/13/2018 10:40 AM, Stephen Frost wrote: > > ... Ultimately, the default which makes sense here isn't a > > one-size-fits-all but is system dependent and the administrator should > > be able to choose what permissions they wa

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Jesper Pedersen
Hi Amit, On 03/13/2018 07:37 AM, Amit Langote wrote: I will continue working on improving the comments / cleaning things up and post a revised version soon, but until then please look at the attached. Passes check-world. Some minor comments: 0001: Ok 0002: Ok 0003: * Trailing white space

[submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread leap
hello! I develop a tool for pgsql, I want to submit it on pgsql. how can I submit it? address: https://github.com/leapking/pgcheck

Inquiry regarding the projects under GSOC

2018-03-13 Thread Dr K.G Yadav
Hello Postgres Team I am Ankit 3rd year B.tech student from India I am really interested to work along with your organisation as an intern to rectify the issues and bugs as I love challenges and stated in the docs that some features might be not possible to implement so I would love to try. My i

Re: TupleTableSlot abstraction

2018-03-13 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > I've recently been discussing with Robert how to abstract > TupleTableSlots in the light of upcoming developments around abstracting > storage away. Besides that aspect I think we also need to make them > more abstract to later deal with vectorized excution, but I'm

Re: INOUT parameters in procedures

2018-03-13 Thread Pavel Stehule
2018-03-13 14:14 GMT+01:00 Peter Eisentraut < peter.eisentr...@2ndquadrant.com>: > On 3/8/18 02:25, Pavel Stehule wrote: > > It looks like some error in this concept. The rules for enabling > > overwriting procedures should modified, so this collision should not be > > done. > > > > When I using p

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 01:50 PM, Stephen Frost wrote: > Yes, that's true, but PG has never paid attention to POSIX ACLs or Linux > capabilities. Changing it to do so is quite a bit beyond the scope... I think we're largely in agreement here, as my aim was not to advocate that PG should work harder to und

Re: TupleTableSlot abstraction

2018-03-13 Thread Andres Freund
On 2018-03-13 15:18:34 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > I've recently been discussing with Robert how to abstract > > TupleTableSlots in the light of upcoming developments around abstracting > > storage away. Besides that aspect I think we also need to make them

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
On 2018-03-13 10:25:49 -0400, Peter Eisentraut wrote: > On 3/12/18 17:04, Andres Freund wrote: > > │ JIT: > > │ > > │ Functions: 4 > >

Re: [HACKERS] why not parallel seq scan for slow functions

2018-03-13 Thread Robert Haas
On Wed, Feb 14, 2018 at 5:37 AM, Amit Kapila wrote: > Your concern is valid, but isn't the same problem exists in another > approach as well, because in that also we can call > adjust_paths_for_srfs after generating gather path which means that we > might lose some opportunity to reduce the relati

Re: JIT compiling with LLVM v11

2018-03-13 Thread Robert Haas
On Mon, Mar 12, 2018 at 5:04 PM, Andres Freund wrote: > Currently a handful of explain outputs in the regression tests change > output when compiled with JITing. Therefore I'm thinking of adding > JITINFO or such option, which can be set to false for those tests? Can we spell that JIT or at least

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
Hi, On 2018-03-13 14:36:44 -0400, Robert Haas wrote: > On Mon, Mar 12, 2018 at 5:04 PM, Andres Freund wrote: > > Currently a handful of explain outputs in the regression tests change > > output when compiled with JITing. Therefore I'm thinking of adding > > JITINFO or such option, which can be se

Re: proposal: schema variables

2018-03-13 Thread Pavel Stehule
2018-03-13 10:54 GMT+01:00 Pavel Luzanov : > Pavel Stehule wrote > > Now, there is one important question - storage - Postgres stores all > > objects to files - only memory storage is not designed yet. This is part, > > where I need a help. > > O, I do not feel confident in such questions. > May b

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Tom Lane
Chapman Flack writes: > On 03/13/2018 01:50 PM, Stephen Frost wrote: >> I'll point out that PG does run on quite a few other OS's beyond Linux. > I'll second that, as I think it is making my argument. When I can > supply three or four examples of semantic subtleties that undermine > PG's assumpti

Re: Fix error in ECPG while connection handling

2018-03-13 Thread Jeevan Ladhe
> Thanks for spotting and fixing. I will push the patch as soon as I'm > online again. > Thanks Michael for taking care of this. Regards, Jeevan Ladhe.

neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Jeff Janes
The following commit has caused a devastating performance regression in concurrent refresh of MV: commit 7ca25b7de6aefa5537e0dbe56541bc41c0464f97 Author: Tom Lane Date: Wed Nov 29 22:00:29 2017 -0500 Fix neqjoinsel's behavior for semi/anti join cases. The below reproduction goes from tak

Re: JIT compiling with LLVM v11

2018-03-13 Thread Tom Lane
Andres Freund writes: > On 2018-03-13 14:36:44 -0400, Robert Haas wrote: >> I realize that EXPLAIN (JIT OFF) may sound like it's intended to >> disable JIT itself > Yea, that's what I'm concerned about. >> , but I think it's pretty clear that EXPLAIN (BUFFERS OFF) does not >> disable the use of

Re: [HACKERS] taking stdbool.h into use

2018-03-13 Thread Peter Eisentraut
On 3/1/18 23:34, Michael Paquier wrote: > Indeed, this is wrong. Peter, why did you switch suddendly this patch > as ready for committer? The patch is waiting for your input as you > mentioned that the GIN portion of this patch series is not completely > baked yet. So I have switched the patch i

Re: [submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread Laurenz Albe
leap wrote: > I develop a tool for pgsql, I want to submit it on pgsql. > how can I submit it? > > address: https://github.com/leapking/pgcheck I would leave it on Github and add some documentation. A lot of great tools for PostgreSQL are not part of the core distribution; that doesn't mean that

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Greetings Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > On 03/13/2018 01:50 PM, Stephen Frost wrote: > > PG will stat PGDATA and conclude that the system is saying that group > > access has been granted on PGDATA and will do the same for subsequent > > files created later on. ... The o

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2018-03-13 Thread Robert Haas
On Mon, Feb 19, 2018 at 4:02 AM, David Rowley wrote: > On 19 February 2018 at 18:01, David Rowley > wrote: >> On 19 February 2018 at 15:11, Tomas Vondra >> wrote: >>> and perhaps we should do s/isproxy/is_proxy/ which seems like the usual >>> naming for boolean variables. >> >> You're right. I

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Chapman Flack
On 03/13/2018 02:47 PM, Tom Lane wrote: > Well, to be blunt, what we target is POSIX-compatible systems. If > you're telling us to worry about non-POSIX filesystem semantics, > and your name is not Microsoft, it's going to be a hard sell. > We have enough to do just keeping up with that scope of t

Re: parallel append vs. simple UNION ALL

2018-03-13 Thread Robert Haas
On Tue, Mar 13, 2018 at 12:35 AM, Ashutosh Bapat wrote: > It looks like it was not changed in all the places. make falied. I > have fixed all the instances of these two functions in the attached > patchset (only 0003 changes). Please check. Oops. Thanks. I'm going to go ahead and commit 0001 he

Re: JIT compiling with LLVM v11

2018-03-13 Thread Thomas Munro
On Thu, Mar 1, 2018 at 9:02 PM, Andres Freund wrote: > Biggest changes: > - LLVM 3.9 - master are now supported. This includes a good chunk of > work by Pierre Ducroquet. I decided to try this on a CentOS 7.2 box. It has LLVM 3.9 in the 'epel' package repo, but unfortunately it only has clang

Re: JIT compiling with LLVM v11

2018-03-13 Thread Andres Freund
Hi, On 2018-03-14 10:32:40 +1300, Thomas Munro wrote: > I decided to try this on a CentOS 7.2 box. It has LLVM 3.9 in the > 'epel' package repo, but unfortunately it only has clang 3.4. That's a bit odd, given llvm and clang really live in the same repo... > clang: error: unknown argument: '-f

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 8:07 AM, Jeff Janes wrote: > The following commit has caused a devastating performance regression > in concurrent refresh of MV: > > commit 7ca25b7de6aefa5537e0dbe56541bc41c0464f97 > Author: Tom Lane > Date: Wed Nov 29 22:00:29 2017 -0500 > > Fix neqjoinsel's behavio

Re: [submit code] I develop a tool for pgsql, how can I submit it

2018-03-13 Thread Euler Taveira
2018-03-13 12:19 GMT-03:00 leap : > I develop a tool for pgsql, I want to submit it on pgsql. > how can I submit it? > As Laurenz said a tool doesn't necessarily have to be part of PostgreSQL. Sometimes a lot of people ask for a tool, someone write it and the community decide to maintain it. I'm no

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread Andrew Dunstan
On Tue, Mar 13, 2018 at 10:58 PM, Andrew Dunstan wrote: > On Tue, Mar 13, 2018 at 2:40 PM, Andrew Dunstan > wrote: > >>> >>> Going by the commitfest app, the patch still does appear to be waiting >>> on Author. Never-the-less, I've made another pass over it and found a >>> few mistakes and a coup

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Alvaro Herrera
Amit Langote wrote: > I will continue working on improving the comments / cleaning things up and > post a revised version soon, but until then please look at the attached. I tried to give this a read. It looks pretty neat stuff -- as far as I can tell, it follows Robert's sketch for how this sho

Re: SQL/JSON: functions

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > The docs are here > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in more > friendly way. We'll have time to convert it to postgres f

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 11:34 AM, Thomas Munro wrote: > LOG: duration: 26101.612 ms plan: > Query Text: SELECT newdata FROM pg_temp_3.pg_temp_16452 newdata WHERE > newdata IS NOT NULL AND EXISTS (SELECT * FROM pg_temp_3.pg_temp_16452 > newdata2 WHERE newdata2 IS NOT NULL AND newdata2 > OPERATOR(

Re: SQL/JSON: functions

2018-03-13 Thread Andres Freund
On 2018-03-14 07:54:35 +0900, Michael Paquier wrote: > On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > > The docs are here > > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Tom Lane
Thomas Munro writes: > This looks like an invisible correlation problem. Yeah --- the planner has no idea that the join rows satisfying newdata.* *= newdata2.* are likely to be exactly the ones not satisfying newdata.ctid <> newdata2.ctid. It's very accidental that we got a good plan before. I'

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Tom Lane
Thomas Munro writes: > There is a fundamental and complicated estimation problem lurking here > of course and I'm not sure what to think about that yet. Maybe there > is a very simple fix for this particular problem: Ah, I see you thought of the same hack I did. I think this may actually be a g

Re: [HACKERS] path toward faster partition pruning

2018-03-13 Thread Alvaro Herrera
By the way, I checked whether patch 0002 (additional tests) had an effect on coverage, and couldn't detect any changes in terms of lines/functions. Were you able to find any bugs in your code thanks to the new tests that would not have been covered by existing tests? -- Álvaro Herrera

JIT compiling with LLVM v12

2018-03-13 Thread Andres Freund
Hi, I've pushed a revised and rebased version of my JIT patchset. The git tree is at https://git.postgresql.org/git/users/andresfreund/postgres.git in the jit branch https://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/jit There's nothing hugely exciti

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Stephen Frost
Chapman, * Chapman Flack (c...@anastigmatix.net) wrote: > I'm not advocating the Sisyphean task of having PG incorporate > knowledge of all those possibilities. I'm advocating the conservative > approach: have PG be that well-behaved application that those extended > semantics are generally all de

Re: neqjoinsel versus "refresh materialized view concurrently"

2018-03-13 Thread Thomas Munro
On Wed, Mar 14, 2018 at 12:29 PM, Tom Lane wrote: > Thomas Munro writes: >> There is a fundamental and complicated estimation problem lurking here >> of course and I'm not sure what to think about that yet. Maybe there >> is a very simple fix for this particular problem: > > Ah, I see you though

Re: SQL/JSON: functions

2018-03-13 Thread Oleg Bartunov
On 14 Mar 2018 01:54, "Michael Paquier" wrote: On Tue, Mar 13, 2018 at 04:08:01PM +0300, Oleg Bartunov wrote: > The docs are here > https://github.com/obartunov/sqljsondoc/blob/master/README.jsonpath.md > > It's not easy to write docs for SQL/JSON in xml, so I decided to write in more > friendly

Re: Fixes for missing schema qualifications

2018-03-13 Thread Tatsuo Ishii
>> + "select pg_catalog.count(*) " >> + "from pg_catalog.pg_namespace where >> nspname = '%s'", > > This qualifies some functions, but it leaves plenty of unqualified operators. So this should go something like this? select

Re: Fixes for missing schema qualifications

2018-03-13 Thread Tatsuo Ishii
>>> +"select pg_catalog.count(*) " >>> +"from pg_catalog.pg_namespace where >>> nspname = '%s'", >> >> This qualifies some functions, but it leaves plenty of unqualified operators. > > So this should go something like this?

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2018-03-13 Thread David Rowley
On 14 March 2018 at 09:25, Robert Haas wrote: > What do you think about the idea of using a projection path as a proxy > path instead of inventing a new method? It seems simple enough to do: > > new_path = (Path *) create_projection_path(root, new_rel, old_path, > old_path->pathtarget); > > ...wh

Re: Fixes for missing schema qualifications

2018-03-13 Thread David G. Johnston
On Tue, Mar 13, 2018 at 5:11 PM, Tatsuo Ishii wrote: > >>> +"select pg_catalog.count(*) " > >>> +"from pg_catalog.pg_namespace > where nspname = '%s'", > >> > >> This qualifies some functions, but it leaves plenty of unqualif

Re: Fixes for missing schema qualifications

2018-03-13 Thread Tatsuo Ishii
>> select pg_catalog.count(*) from pg_catalog.pg_namespace where >> pg_catalog.nameeq(nspname, '%s'); >> >> > ​I'd rather write that: > > select [...] where nspname operator(pg_catalog.=) '%s'​ > > Introducing undocumented implementation functions to these queries is > undesirable; and besides, i

ExplainProperty* and units

2018-03-13 Thread Andres Freund
Hi, while adding EXPLAIN support for JITing (displaying time spent etc), I got annoyed by the amount of duplication required. There's a fair amount of if (es->format == EXPLAIN_FORMAT_TEXT) appendStringInfo(es->str, "Execution time: %.3f ms\n",

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread David Rowley
On 14 March 2018 at 11:36, Andrew Dunstan wrote: > Here are the benchmark results from the v15 patch. Fairly similar to > previous results. I'm going to run some profiling again to see if I > can identify any glaring hotspots. I do suspect that the "physical > tlist" optimization sometimes turns o

Re: ALTER TABLE ADD COLUMN fast default

2018-03-13 Thread Andres Freund
Hi, On 2018-03-14 09:06:54 +1030, Andrew Dunstan wrote: > I do suspect that the "physical tlist" optimization sometimes turns > out not to be one. It seems perverse to be able to improve a query's > performance by dropping a column. Yea, it's definitely not always a boon. Especially w/ the newer

Re: Fixes for missing schema qualifications

2018-03-13 Thread David G. Johnston
On Tue, Mar 13, 2018 at 5:26 PM, Tatsuo Ishii wrote: > Next question is, should we update the manual? There are bunch of > places where example queries are shown without schema qualifications. > > I hope that isn't deemed necessary. David J.

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-13 Thread David Gould
On Tue, 13 Mar 2018 11:29:03 -0400 Tom Lane wrote: > David Gould writes: > > I have attached the patch we are currently using. It applies to 9.6.8. I > > have versions for older releases in 9.4, 9.5, 9.6. I fails to apply to 10, > > and presumably head but I can update it if there is any interes

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 01:28:17PM -0400, Tom Lane wrote: > David Steele writes: >> On 3/12/18 3:28 AM, Michael Paquier wrote: >>> In pg_rewind and pg_resetwal, isn't that also a portion which is not >>> necessary without the group access feature? > >> These seem like a good idea to me with or wi

Re: PATCH: Configurable file mode mask

2018-03-13 Thread Michael Paquier
On Tue, Mar 13, 2018 at 12:19:07PM -0400, David Steele wrote: > I'll attach new patches in a reply to [1] once I have made the changes > Tom requested. Cool, thanks for your patience. Looking forward to seeing those. I'll spend time on 0003 at the same time. Let's also put the additional umask

  1   2   >