Re: [HACKERS] JIT compiling - v4.0

On 2017-10-04 11:56:47 +0300, Ants Aasma wrote: > On Wed, Oct 4, 2017 at 9:48 AM, Andres Freund wrote: > > Here's an updated version of the patchset. There's some substantial > > changes here, but it's still very obviously very far from committable as > > a whole. There's some helper commmits tha

Re: [HACKERS] Parallel Hash take II

v20 patch set (I was trying 0008, 0009 patch) not getting cleanly apply on latest commit also getting compilation error due to refactor in below commit. commit 0c5803b450e0cc29b3527df3f352e6f18a038cc6 Author: Peter Eisentraut Date: Sat Sep 23 09:49:22 2017 -0400 Refactor new file permissi

Re: [HACKERS] Additional logging for VACUUM and ANALYZE

On Thu, Oct 5, 2017 at 1:23 AM, Bossart, Nathan wrote: > Presently, there are a few edge cases in vacuum_rel() and analyze_rel() that I > believe do not have sufficient logging. This was discussed a bit in the > vacuum-multiple-relations thread [0], but it was ultimately decided that any > loggin

Re: [HACKERS] Logging idle checkpoints

At Tue, 3 Oct 2017 08:22:27 -0400, Stephen Frost wrote in <2017100317.gj4...@tamriel.snowman.net> > Greetings, > > * Kyotaro HORIGUCHI (horiguchi.kyot...@lab.ntt.co.jp) wrote: > > At Tue, 3 Oct 2017 10:23:08 +0900, Michael Paquier > > wrote in > > > > > On Tue, Oct 3, 2017 at 12:01 AM, S

[HACKERS] Re: [COMMITTERS] pgsql: Allow multiple tables to be specified in one VACUUM or ANALYZE c

On Wed, Oct 4, 2017 at 7:53 AM, Tom Lane wrote: > Allow multiple tables to be specified in one VACUUM or ANALYZE command. > > Not much to say about this; does what it says on the tin. > > However, formerly, if there was a column list then the ANALYZE action was > implied; now it must be specified,

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Thu, Oct 5, 2017 at 10:39 AM, Wood, Dan wrote: > Whatever you do make sure to also test 250 clients running lock.sql. Even > with the communities fix plus YiWen’s fix I can still get duplicate rows. > What works for “in-block” hot chains may not work when spanning blocks. Interesting. Whic

Re: [HACKERS] document and use SPI_result_code_string()

On 10/2/17 03:28, Daniel Gustafsson wrote: >> On 06 Sep 2017, at 14:25, Tom Lane wrote: >> >> Michael Paquier writes: >>> Fine for 0002. This reminds me of LockGXact and RemoveGXact in >>> twophase.c, as well as _hash_squeezebucket that have some code paths >>> that cannot return... Any thoughts

Re: [HACKERS] Block level parallel vacuum WIP

On Tue, Sep 19, 2017 at 4:31 PM, Masahiko Sawada wrote: > On Tue, Sep 19, 2017 at 3:33 PM, Thomas Munro > wrote: >> On Fri, Sep 8, 2017 at 10:37 PM, Masahiko Sawada >> wrote: >>> Since v4 patch conflicts with current HEAD I attached the latest version >>> patch. >> >> Hi Sawada-san, >> >> Here

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Whatever you do make sure to also test 250 clients running lock.sql. Even with the communities fix plus YiWen’s fix I can still get duplicate rows. What works for “in-block” hot chains may not work when spanning blocks. Once nearly all 250 clients have done their updates and everybody is waiti

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Wed, Oct 4, 2017 at 10:46 PM, Alvaro Herrera wrote: > Wong, Yi Wen wrote: >> My interpretation of README.HOT is the check is just to ensure the chain is >> continuous; in which case the condition should be: >> >> > if (TransactionIdIsValid(priorXmax) && >> >

Re: [HACKERS] 64-bit queryId?

On Thu, Oct 5, 2017 at 4:12 AM, Robert Haas wrote: > On Wed, Oct 4, 2017 at 10:11 AM, Michael Paquier > wrote: >> On Wed, Oct 4, 2017 at 11:04 PM, Robert Haas wrote: >>> Not really; dynahash won't merge two keys just because their hash >>> codes come out the same. But you're right; that's proba

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On Mon, Oct 2, 2017 at 8:31 PM, Daniel Gustafsson wrote: > > On 13 Sep 2017, at 07:44, Vaishnavi Prabakaran < > vaishnaviprabaka...@gmail.com> wrote: > > > > On Wed, Sep 13, 2017 at 3:33 PM, Craig Ringer > wrote: > > > > I really do not like calling it "commit" as t

Re: [HACKERS] Re: protocol version negotiation (Re: Libpq PGRES_COPY_BOTH - version compatibility)

Okay, I will add a mechanism to try connecting with 3.0 if 3.1 fails- that should be a few lines of code fe-connect.c; this will eliminate the need for a back-patch. What do you think of the rest of the change? Thanks, Badrul -Original Message- From: Robert Haas [mailto:robertmh...@gma

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Wed, Oct 4, 2017 at 6:46 AM, Alvaro Herrera wrote: > Wong, Yi Wen wrote: >> My interpretation of README.HOT is the check is just to ensure the chain is >> continuous; in which case the condition should be: >> >> > if (TransactionIdIsValid(priorXmax) && >> >

Re: [HACKERS] postgres_fdw super user checks

On Thu, Sep 14, 2017 at 1:08 PM, Robert Haas wrote: > On Thu, Sep 14, 2017 at 2:33 PM, Jeff Janes wrote: > > I think that foreign tables ought to behave as views do, where they run > as > > the owner rather than the invoker. No one has talked me out of it, but > no > > one has supported me on i

Re: [HACKERS] Possible SSL improvements for a newcomer to tackle

On Tue, Oct 3, 2017 at 6:44 AM, Tom Lane wrote: > Magnus Hagander writes: > > On Tue, Oct 3, 2017 at 6:33 AM, Tom Lane wrote: > >> I'm not an SSL expert, so insert appropriate grain of salt, but AIUI the > >> question is what are you going to verify against? > > > One way to do it would be to d

Re: [HACKERS] [sqlsmith] crash in RestoreLibraryState during low-memory testing

On Tue, Oct 3, 2017 at 3:04 AM, Andreas Seltenreich wrote: > Tom Lane writes: >> Presumably somebody could dig into the libc source code and prove or >> disprove this, though it would sure help to know exactly what platform >> and version Andreas is testing on. > > This is the code in glibc-2.24 a

Re: [HACKERS] why subplan is 10x faster then function?

2017-10-01 12:45 GMT+02:00 Sokolov Yura : > 1 октября 2017 г. 12:42:14 GMT+03:00, Pavel Stehule < > pavel.steh...@gmail.com> пишет: > >2017-09-30 23:23 GMT+02:00 Pavel Stehule : > > > >> Hi > >> > >> I have some strange slow queries based on usage "view" functions > >> > >> one function looks like

Re: [HACKERS] 64-bit queryId?

On Wed, Oct 4, 2017 at 10:11 AM, Michael Paquier wrote: > On Wed, Oct 4, 2017 at 11:04 PM, Robert Haas wrote: >> Not really; dynahash won't merge two keys just because their hash >> codes come out the same. But you're right; that's probably not the >> best way to do it. TBH, why do we even hav

Re: [HACKERS] Possible SSL improvements for a newcomer to tackle

On Wed, Oct 04, 2017 at 11:47:45AM -0700, Jeff Janes wrote: > On Mon, Oct 2, 2017 at 9:33 PM, Tom Lane wrote: > > It's possible that we could adopt some policy like "if the root.crt file > > exists then default to verify" ... but that seems messy and unreliable, > > so I'm not sure it would really

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Wed, Oct 4, 2017 at 8:23 AM, Ashutosh Bapat wrote: > I am not sure whether your assumption that expression with no Vars > would have em_relids empty is correct. I wonder whether we will add > any em_is_child members with empty em_relids; looking at > process_equivalence() those come from Restri

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Wed, Oct 4, 2017 at 11:00 AM, Wood, Dan wrote: > The early “break;” here is likely the xmin frozen reason as I found in the > other loop. It looks that way. Since we're already very defensive when it comes to this xmin/xmax matching business, and we're defensive while following an update cha

Re: [HACKERS] Possible SSL improvements for a newcomer to tackle

On Mon, Oct 2, 2017 at 9:33 PM, Tom Lane wrote: > > It's possible that we could adopt some policy like "if the root.crt file > exists then default to verify" ... but that seems messy and unreliable, > so I'm not sure it would really add any security. > That is what we do. If root.crt exists, we

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

Ah, David Fetter points out that I should also update tabe completion for psql. I'll do that at some point. I notice there's no table completion for column constraint attributes... If it's obvious enough I'll try to fix that too. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] [PATCH] Add ALWAYS DEFERRED option for constraints

Ay, NOT WIP -- I left that in the Subject: by accident. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] [PATCH] WIP Add ALWAYS DEFERRED option for constraints

[make check-world passes. Tests and docs included. Should be ready for code review.] Attached are patches to add an ALWAYS DEFERRED option to CONSTRAINTs and CONSTRAINT TRIGGERs, meaning: SET CONSTRAINTS .. IMMEDIATE will not make immediate any constraint/trigger that is declared as ALWAYS DEFE

[HACKERS] Re: [BUGS] Re: [PATCH] BUG #13416: Postgres >= 9.3 doesn't use optimized shared memory on Solaris anymore

Hi, On 2017-10-04 10:47:06 -0700, Sean Chittenden wrote: > Hello.  We identified the same problem.  Sam Gwydir and Josh Clulow were able > to put together the appropriate fix after. > > The breakage in src/backend/port/sysv_shmem.c and  > src/include/storage/dsm_impl.h should be back ported to a

[HACKERS] Re: [PATCH] BUG #13416: Postgres >= 9.3 doesn't use optimized shared memory on Solaris anymore

Hello.  We identified the same problem.  Sam Gwydir and Josh Clulow were able to put together the appropriate fix after. The breakage in src/backend/port/sysv_shmem.c and  src/include/storage/dsm_impl.h should be back ported to all supported versions (the regression was introduced between the 9.

Re: [HACKERS] Horrible CREATE DATABASE Performance in High Sierra

Brent Dearth writes: > Is there an issue tracker I could be looking at to follow along on the > progress on this issue? This email thread is pretty much it ... Current status is that I've filed a bug report with Apple and am waiting to see their response before deciding what to do next. If they

Re: [HACKERS] Horrible CREATE DATABASE Performance in High Sierra

Tom, Andres - Is there an issue tracker I could be looking at to follow along on the progress on this issue? Thanks so much! On Mon, Oct 2, 2017 at 9:06 PM, Tom Lane wrote: > Andres Freund writes: > > On 2017-10-02 19:50:51 -0400, Tom Lane wrote: > >> What I saw was that the backend process w

Re: [HACKERS] [PATCH] Improve geometric types

> Now, it's also not clear that anything in PG really cares. But if we > do care, I think we should keep pg_hypot() ... and maybe clarify the > comment a bit more. I am not sure how useful NaNs are in geometric types context, but we allow them, so inconsistent hypot() would be a problem. I will

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Wed, Oct 4, 2017 at 11:34 AM, Robert Haas wrote: > +Enables or disables the query planner's use of partition-wise join > +plans. When enabled, it spends time in creating paths for joins > between > +partitions and consumes memory to construct expression nodes to be > u

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

On Tue, Oct 3, 2017 at 8:43 PM, Wong, Yi Wen wrote: > My interpretation of README.HOT is the check is just to ensure the chain is > continuous; in which case the condition should be: > >> if (TransactionIdIsValid(priorXmax) && >> !TransactionIdEquals(priorX

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Tue, Oct 3, 2017 at 3:27 PM, Robert Haas wrote: > I decided to skip over 0001 for today and spend some time looking at > 0002-0006. Back to 0001. +Enables or disables the query planner's use of partition-wise join +plans. When enabled, it spends time in creating paths for join

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Thu, Sep 21, 2017 at 8:52 AM, Robert Haas wrote: > On Thu, Sep 21, 2017 at 8:21 AM, Ashutosh Bapat > wrote: >> About your earlier comment of making build_joinrel_partition_info() >> simpler. Right now, the code assumes that partexprs or >> nullable_partexpr can be NULL when either of them is n

Re: [HACKERS] datetime.h defines like PM conflict with external libraries

> Isn't pgtypeslib/*.h exposed to ecpg-using applications? No, the public interface is is include/*.h, pgtypeslib/*.h is only internal. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Meskes at (Debian|Postgresql) dot Org Jabber: michael at xmpp dot

Re: [HACKERS] 64-bit queryId?

On Wed, Oct 4, 2017 at 11:04 PM, Robert Haas wrote: > Not really; dynahash won't merge two keys just because their hash > codes come out the same. But you're right; that's probably not the > best way to do it. TBH, why do we even have pgss_hash_fn? It seems > like using tag_hash would be super

Re: [HACKERS] datetime.h defines like PM conflict with external libraries

Michael Meskes writes: >> Maybe it'd be good idea to unify some of that stuff so that ecpg can >> use it, too? Having a second copy of the same stuff in >> src/interfaces/ecpg/pgtypeslib/dt.h is pretty terrible. Even if not, >> let's make sure they don't diverge. > Please let's unify whatever w

Re: [HACKERS] 64-bit queryId?

On Wed, Oct 4, 2017 at 9:49 AM, Michael Paquier wrote: > I am still on the learning curve with pg_stat_statements... This still > does not look complete to me. pgss_hash_fn only makes use of the last > four bytes of the query ID. What about computing the hash using as > also the first four bytes?

Re: [HACKERS] 64-bit queryId?

On Wed, Oct 4, 2017 at 9:08 PM, Robert Haas wrote: > On Tue, Oct 3, 2017 at 9:39 PM, Michael Paquier > wrote: >>> I'm sorry, but I don't understand this comment. >> >> I just mean that your patch is correct here. I don't always complain :) > > Oh, OK. I'm all right with my patch being correct. >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Wong, Yi Wen wrote: > My interpretation of README.HOT is the check is just to ensure the chain is > continuous; in which case the condition should be: > > > if (TransactionIdIsValid(priorXmax) && > > !TransactionIdEquals(priorXmax, > > HeapTupleHeaderGetRa

Re: [HACKERS] Possible SSL improvements for a newcomer to tackle

On Tue, Oct 3, 2017 at 11:39 AM, Nico Williams wrote: > On Tue, Oct 03, 2017 at 12:33:00AM -0400, Tom Lane wrote: > > So to default to verification would be to default to failing to > > connect at all until user has created a ~/.postgresql/root.crt file with > > valid, relevant entries. That see

Re: [HACKERS] PoC: full merge join on comparison clause

As discussed earlier, I changed the way we work with mergeopfamilies. I use the "is_equality" flag to indicate whether the clause is an equality one, and fill mergeopfamilies for both equality and inequality operators. The updated patch is attached (rebased to 20b6552242). -- Alexander Kuzmenko

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

On Wed, Oct 4, 2017 at 5:32 PM, Robert Haas wrote: > On Wed, Oct 4, 2017 at 6:40 AM, Ashutosh Bapat > wrote: >> Just like the local constraints on a foreign table are not ensured on >> remote table (unless user takes steps to make that sure), WCO defined >> locally need not be (and probably can n

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Wed, Oct 4, 2017 at 12:57 AM, Robert Haas wrote: > > 0003: > > The commit message mentions estimate_num_groups but the patch doesn't touch > it. This was fixed when we converted many rel->reloptkind == RELOPT_BASEREL to IS_SIMPLE_REL(). I have removed this section from the commit message. >

Re: [HACKERS] [PATCH] WIP Add ALWAYS DEFERRED option for constraints

På onsdag 04. oktober 2017 kl. 00:24:19, skrev Vik Fearing < vik.fear...@2ndquadrant.com >: On 10/03/2017 10:10 PM, Andreas Joseph Krogh wrote: > While we're in deferrable constraints land...; > I even more often need deferrable /conditional /unique-indexes. >

Re: [HACKERS] 64-bit queryId?

On Tue, Oct 3, 2017 at 9:39 PM, Michael Paquier wrote: >> I'm sorry, but I don't understand this comment. > > I just mean that your patch is correct here. I don't always complain :) Oh, OK. I'm all right with my patch being correct. Here's a new version that hopefully fixes the things that you

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Alvaro Herrera wrote: > Peter Geoghegan wrote: > > > I thought that we no longer store FrozenTransactionId (xid 2) as our > > "raw" xmin while freezing, and yet that's what we see here. > > I'm doing this in 9.3. I can't tell if the new tuple freezing stuff > broke things more thoroughly, but it

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

On Wed, Oct 4, 2017 at 6:40 AM, Ashutosh Bapat wrote: > Just like the local constraints on a foreign table are not ensured on > remote table (unless user takes steps to make that sure), WCO defined > locally need not be (and probably can not be) ensured remotely. We can > check whether a row being

Re: [HACKERS] Warnings in objectaddress.c

On Tue, Oct 3, 2017 at 9:41 PM, Tom Lane wrote: > Robert Haas writes: >> Perhaps we should apply some glorified version of this: > >> +if (list_length(object) < 2) >> +elog(ERROR, "fail"); > >> However, I'm not 100% sure that would be sufficient to suppress these >> warnings, because

Re: [HACKERS] Re: protocol version negotiation (Re: Libpq PGRES_COPY_BOTH - version compatibility)

On Tue, Oct 3, 2017 at 9:46 PM, Tom Lane wrote: > Badrul Chowdhury writes: >> 1. Pgwire protocol v3.0 with negotiation is called v3.1. >> 2. There are 2 patches for the change: a BE-specific patch that will be >> backported and a FE-specific patch that is only for pg10 and above. > > TBH, anythi

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

On Wed, Oct 4, 2017 at 3:45 PM, Etsuro Fujita wrote: > On 2017/10/03 18:16, Ashutosh Bapat wrote: >> >> Enforcing WCO constraints imposed by the local server on the row/DML >> being passed to the foreign server is fine, but trying to impose them >> on the row being inserted/updated at the foreign

Re: [HACKERS] Another oddity in handling of WCO constraints in postgres_fdw

On 2017/10/03 18:16, Ashutosh Bapat wrote: Enforcing WCO constraints imposed by the local server on the row/DML being passed to the foreign server is fine, but trying to impose them on the row being inserted/updated at the foreign server looks odd. May be we should just leave this case as it is.

Re: [HACKERS] datetime.h defines like PM conflict with external libraries

> Maybe it'd be good idea to unify some of that stuff so that ecpg can > use > it, too? Having a second copy of the same stuff in > src/interfaces/ecpg/pgtypeslib/dt.h is pretty terrible. Even if not, > let's make sure they don't diverge. Please let's unify whatever we can. The fewer manual sync

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Wood, Dan wrote: > One minor side note… Is it weird for xmin/xmax to go backwards in a hot row > chain? > > lp | t_ctid | lp_off | t_infomask | t_infomask2 | t_xmin | t_xmax > ++++-++ > 1 | (0,1) | 8152 | 2818 | 3

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Wood, Dan wrote: > There is a tangled web of issues here. With the community fix we get a > corrupted page(invalid redirect ptr from indexed item). The cause of that is: > pruneheap.c: > > /* >* Check the tuple XMIN against prior XMAX, if any >

Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

Peter Geoghegan wrote: > I thought that we no longer store FrozenTransactionId (xid 2) as our > "raw" xmin while freezing, and yet that's what we see here. I'm doing this in 9.3. I can't tell if the new tuple freezing stuff broke things more thoroughly, but it is already broken in earlier releas

Re: [HACKERS] datetime.h defines like PM conflict with external libraries

Andrew Dunstan wrote: > On 10/03/2017 04:43 PM, Tom Lane wrote: > > I like the new-header-file idea because it will result in minimal code > > churn and thus minimal back-patching hazards. > > > > I do *not* like "PG_PM". For our own purposes that adds no uniqueness > > at all. If we're to touc

Re: [HACKERS] list of credits for release notes

On Wed, Oct 4, 2017 at 8:51 AM, Laurenz Albe wrote: > Peter Eisentraut wrote: > > At the PGCon Developer Meeting it was agreed[0] to add a list of credits > > to the release notes, including everyone who was mentioned in a commit > > message. I have now completed that list. > > > > Attached is t

Re: [HACKERS] Add TOAST to system tables with ACL?

On Tue, Oct 3, 2017 at 9:19 PM, Tom Lane wrote: > For pg_database, you'd have to make sure that the startup-time check of > database CONNECT privilege still works if the ACL's been pushed out of > line. > I've tried this case. At first, make database temp with no connect privilege from public a

Re: [HACKERS] JIT compiling - v4.0

On Wed, Oct 4, 2017 at 9:48 AM, Andres Freund wrote: > Here's an updated version of the patchset. There's some substantial > changes here, but it's still very obviously very far from committable as > a whole. There's some helper commmits that are simple and independent > enough to be committable

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

On Sat, Sep 30, 2017 at 3:31 AM, Robert Haas wrote: > On Fri, Sep 29, 2017 at 12:44 AM, Vaishnavi Prabakaran > wrote: > > Option name "--enable-pgdumpall-behaviour" is very generic > > Yeah, that's a terrible name, at least in my opinion. > OK. I will use a new name based on the discussion.

Re: [HACKERS] Binary search in fmgr_isbuiltin() is a bottleneck.

On 2017-10-02 15:01:36 -0700, Andres Freund wrote: > On 2017-10-02 17:57:51 -0400, Tom Lane wrote: > > Andres Freund writes: > > > Done that way. It's a bit annoying, because we've to take care to > > > initialize the "unused" part of the array with a valid signalling it's > > > an unused mapping.

Re: [HACKERS] parallelize queries containing initplans

On Wed, Oct 4, 2017 at 12:55 PM, Amit Kapila wrote: > On Wed, Oct 4, 2017 at 3:40 AM, Robert Haas wrote: >> On Tue, Oct 3, 2017 at 7:33 AM, Amit Kapila wrote: >> >> Having said all that, I think that this patch only wants to handle the >> subset of cases (2) and (4) where the relevant InitPlan i

Re: [HACKERS] parallelize queries containing initplans

On Wed, Oct 4, 2017 at 3:40 AM, Robert Haas wrote: > On Tue, Oct 3, 2017 at 7:33 AM, Amit Kapila wrote: > > Having said all that, I think that this patch only wants to handle the > subset of cases (2) and (4) where the relevant InitPlan is attached > ABOVE the Gather node -- which seems very reas