Re: [HACKERS] Causal reads take II

2017-08-09 Thread Thomas Munro
On Mon, Jul 31, 2017 at 5:49 PM, Thomas Munro wrote: > Here is a version to put it back. Rebased after conflicting commit 030273b7. Now using format-patch with a commit message to keep track of review/discussion history. -- Thomas Munro http://www.enterprisedb.com synchronous-replay

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-08-09 Thread Thomas Munro
On Thu, Aug 10, 2017 at 1:39 AM, Thomas Munro wrote: > On my computer it took ~1.5 seconds to plan a 1000 partition join, > ~7.1 seconds to plan a 2000 partition join, and ~50 seconds to plan a > 4000 partition join. I poked around in a profiler a bit and saw that > for the 2000 part

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-08-10 Thread Thomas Munro
n", or something else? If so, would that be a special case, or just the logical extreme case for 0014-WIP-Partition-wise-join-for-1-1-1-0-0-1-partition-ma.patch, where one single "partition" on the non-partitioned side maps to all the partitions on the partitioned size? -- Thoma

[HACKERS] Thoughts on unit testing?

2017-08-10 Thread Thomas Munro
inside an FMGR function it'd be always in a transaction, which has implications. There are probably better ways to do it. Thoughts, better ideas? [1] https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C [2] https://github.com/google/googletest/blob/master/googletest/samples/sample1_uni

Re: [HACKERS] Thoughts on unit testing?

2017-08-10 Thread Thomas Munro
On Fri, Aug 11, 2017 at 10:35 AM, Andres Freund wrote: > On 2017-08-11 09:53:23 +1200, Thomas Munro wrote: >> One idea that keeps coming back to me is that we could probably extend >> our existing regression tests to cover C tests with automatic >> discovery/minimal boilerp

Re: [HACKERS] Thoughts on unit testing?

2017-08-10 Thread Thomas Munro
On Fri, Aug 11, 2017 at 11:13 AM, Thomas Munro wrote: > Just create a .test.c file and type "TEST(my_math, > factorial) { EXPECT_EQ(6, factorial(3)); }" ... Of course that would really need to #include "something/test_macros.h" and "something/factorial.h",

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-11 Thread Thomas Munro
SharedRecordTypmodRegistry? */ > + if (CurrentSharedRecordTypmodRegistry.shared != NULL) > > Why do we want to do lookups in both? I don't think it's a good idea to > have a chance that you could have the same typmod in both the local > registry (because it'd been created before the shared one) and in the > shared (because it was created in a worker). Ah, that's for caching > purposes? If so, see my above point that we shouldn't have a serialized > version of typdesc (yesyes, constraints will be a bit ugly). Right, that's what I've now done. It's basically a write-through cache: we'll try to find it in the backend local structures and then fall back to the shared one. But if we find it in shared memory, we'll just copy the pointer it into our local data structures. In the last version I'd build a new TupleDesc from the serialized form, but now there is no serialized form, just TupleDesc objects which are now shmem-friendly (except for constraints, which do not survive the matter transfer into shmem; see TupleDescCopy). > + /* > +* While we still hold the atts_index entry locked, add this to > +* typmod_index. That's important because we don't want anyone to be > able > +* to find a typmod via the former that can't yet be looked up in the > +* latter. > +*/ > + typmod_index_entry = > + > dht_find_or_insert(CurrentSharedRecordTypmodRegistry.typmod_index, > + &typmod, &found); > + if (found) > + elog(ERROR, "cannot create duplicate shared record typmod"); > + typmod_index_entry->typmod = typmod; > + typmod_index_entry->serialized_tupdesc = serialized_dp; > + dht_release(CurrentSharedRecordTypmodRegistry.typmod_index, > + typmod_index_entry); > > What if we fail to allocate memory for the entry in typmod_index? Well, I was careful to make sure that it was only pushed onto the list in the atts_index entry until after we'd successfully allocated entries in both indexes, so there was no way to exit from this function leaving a TupleDesc in one index but not the other. In other words, it might have created an atts_index entry but it'd have an empty list. But yeah, on reflection we shouldn't leak shared_dp in that case. In this version I added PG_CATCH() to dsa_free() it and PG_RETHROW(). More testing and review needed. -- Thomas Munro http://www.enterprisedb.com shared-record-typmods-v4.patchset.tgz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-12 Thread Thomas Munro
ble entries should be a small struct that has a union { dsa_pointer, TupleDesc } and a discriminator field to say which it is, and the hash + eq functions should be wrappers that follow dsa_pointer if needed and then forward to hashTupleDesc() (a function that does hash_combine() over the Oids) and equalTupleDescs(). (That complication might not exist if tupleDesc were fixed size and could be directly in the hash table entry, but in the process of flattening it (= holding the attributes in it) I made it variable size, so we have to use a pointer to it in the hash table since both DynaHash and DHT work with fixed size entries). Thoughts? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patches I'm thinking of pushing shortly

2017-08-12 Thread Thomas Munro
e wasn't really anyone > saying not to commit it, and no new ideas have surfaced. +1 I'd vote for including this in v10. There doesn't seem to be any downside to this: it's a no brainer to avoid our exploding hash table case when we can see it coming. -- Thomas Munr

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-14 Thread Thomas Munro
scope guards. I have had some ideas about some kind of destructor mechanism that might replace what we're doing with DSM detach hooks in various places and also work in containers like hash tables (ie entries could have destructors), but doing it with the stack is another leve

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-15 Thread Thomas Munro
On Tue, Aug 15, 2017 at 5:44 PM, Thomas Munro wrote: > On Mon, Aug 14, 2017 at 12:32 PM, Andres Freund wrote: >> But architecturally I'm still not sure I quite like the a bit ad-hoc >> manner session state is defined here. I think we much more should go >> towards a P

Re: [HACKERS] Thoughts on unit testing?

2017-08-15 Thread Thomas Munro
ioned earlier amounts to doing what you've been doing, except with more tooling and conventions to lower the barrier to use it. A set of standard test macros, automatic discovery of your tests, collocation of tests alongside the module under test in the source tree, good test result reporting etc. I&#x

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-15 Thread Thomas Munro
Will respond to the actionable code review points separately with a new patch set, but first: On Wed, Aug 16, 2017 at 10:06 AM, Andres Freund wrote: > On 2017-08-15 17:44:55 +1200, Thomas Munro wrote: >> > @@ -99,12 +72,9 @@ CreateTemplateTupleDesc(int natts,

Re: [HACKERS] proposal: psql: check env variable PSQL_PAGER

2017-08-15 Thread Thomas Munro
he first that is set is used. + Stray blank line. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] taking stdbool.h into use

2017-08-15 Thread Thomas Munro
gt; 0006-Add-bool8-typedef-for-system-catalog-structs.patch Make sense. > 0007-Avoid-use-of-bool-in-thread_test.c.patch Looks good. > 0008-Use-stdbool.h-if-available.patch I'm afraid this autoconf stuff is gibberish, and I can't personally tell if it's the right gi

Re: [HACKERS] proposal: psql: check env variable PSQL_PAGER

2017-08-16 Thread Thomas Munro
On Wed, Aug 16, 2017 at 9:56 PM, Pavel Stehule wrote: > 2017-08-16 6:58 GMT+02:00 Thomas Munro : >> Stray blank line. > > I am not sure if I see last issue well. I meant this: + ... but it doesn't really matter. > Sending updated patch Thanks, looks good.

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2017-08-16 Thread Thomas Munro
On Thu, Jun 29, 2017 at 12:24 PM, Thomas Munro wrote: > fallocate-v5.patch Added to commitfest so we don't lose track of this. I'm mainly concerned about the fact that we have a way for PostgreSQL to die that looks exactly like a bug, when really it's masking an out-of-memor

Re: [HACKERS] Fix number skipping in to_number

2017-08-16 Thread Thomas Munro
ws) SELECT '' AS to_char_3, to_char( (q1 * -1), 'PR'), to_char( (q2 * -1), '99999999.999PR') FROM INT8_TBL; to_char_3 | to_char |to_char ---++

Re: [HACKERS] Supporting huge pages on Windows

2017-08-16 Thread Thomas Munro
u please post a rebased version? Thanks! -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-16 Thread Thomas Munro
il.com . -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix number skipping in to_number

2017-08-17 Thread Thomas Munro
continue; } }<--- this guy here might need to move after "noadd = true"? + if (strncmp(Np->in

Re: [HACKERS] Inadequate infrastructure for NextValueExpr

2017-08-17 Thread Thomas Munro
On Fri, Jul 14, 2017 at 1:46 PM, Thomas Munro wrote: > [...] > T_NamedTuplestoreScan can be produced by outfuncs.c with tagname > NAMEDTUPLESTORESCAN but that tagname is not recognized by readfuncs.c > [...] > > That revealed a defect in commit > 18ce3a4ab22d2984f8540ab4809

Re: [HACKERS] Parallel Hash take II

2017-08-18 Thread Thomas Munro
On Wed, Aug 2, 2017 at 10:06 PM, Thomas Munro wrote: > On Tue, Aug 1, 2017 at 1:11 PM, Andres Freund wrote: >> WRT the main patch: > > Thanks for the review. I will respond soon, but for now I just wanted > to post a rebased version (no changes) because v16 no longer appl

Re: [HACKERS] [PATCH] Pattern based listeners for asynchronous messaging (LISTEN/NOTIFY)

2017-08-18 Thread Thomas Munro
rMain [inlined] BackendRun at postmaster.c:4357 frame #11: postgres`PostmasterMain [inlined] BackendStartup at postmaster.c:4029 frame #12: postgres`PostmasterMain at postmaster.c:1753 frame #13: postgres`PostmasterMain(argc=, argv=0x0008018d11c0) at postmaster.c:1361 frame #14: postgres`ma

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2017-08-19 Thread Thomas Munro
.c guc.c:3309:3: error: ‘SSL_LIBRARY’ undeclared here (not in a function) SSL_LIBRARY, ^~~ I guess it should have a fallback definition, though I don't know what it should be. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2017-08-19 Thread Thomas Munro
On Sun, Aug 20, 2017 at 8:10 AM, Thomas Munro wrote: > On Fri, Aug 18, 2017 at 2:14 AM, Daniel Gustafsson wrote: >> Attached is an updated set of patches, rebased on top of master, with bug >> fixes >> and additional features missing in the first set. While not complete

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-20 Thread Thomas Munro
e suggesting that this is likely to get in the way when we try to tackle this area more thoroughly. Andres is suggesting that this is a good time to take steps in this direction. 2. Andres didn't like what I did to DecrTupleDescRefCount, namely allowing to run when there is no ResourceOwner. I

Re: [HACKERS] POC: Sharing record typmods between backends

2017-08-20 Thread Thomas Munro
rite code that compiles against PG11 and also earlier releases without having to do ugly conditional macros stuff. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-04-21 Thread Thomas Munro
ike it's run that test four times and failed twice, > so far. So, not 100% reproducible, but there's something rotten there. > Timing-dependent, maybe? Thanks. I'm away from my computer right now but will investigate this and send a fix later today. -- T

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-04-22 Thread Thomas Munro
On Sat, Apr 22, 2017 at 9:13 PM, Simon Riggs wrote: > On 22 April 2017 at 06:45, Thomas Munro wrote: > >> Thanks. I'm away from my computer right now but will investigate this >> and send a fix later today. > > Thanks. I'll review later today. The assert

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-04-22 Thread Thomas Munro
On Sun, Apr 23, 2017 at 3:41 AM, Tom Lane wrote: > Thomas Munro writes: >> The assertion fails reliably for me, because standby2's reported write >> LSN jumps backwards after the timeline changes: for example I see >> 302 then 3028470 then 302 follow

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-04-23 Thread Thomas Munro
On Sun, Apr 23, 2017 at 6:01 PM, Tom Lane wrote: > Thomas Munro writes: >> On Sun, Apr 23, 2017 at 3:41 AM, Tom Lane wrote: >>> As for this patch itself, is it reasonable to try to assert that the >>> timeline has in fact changed? > >> The protocol doesn'

[HACKERS] Transition tables for triggers on foreign tables and views

2017-04-25 Thread Thomas Munro
east in certain cases, as future feature enhancements. For now, do you agree that we should reject such triggers as unsupported? See attached. Separately, I noticed an obsolete sentence in the trigger documentation. See doc.patch. -- Thomas Munro http://www.enterprisedb.com prevent-unsupporte

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-04-26 Thread Thomas Munro
eter Geoghegan's parallel tuple sort patch, by trying it (I've made a start, more soon). 4. Figure out how the costing model needs to be tweaked, probably based on experimentation. I'm taking a short break to work on other things right now but will post a version with those chang

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

2017-04-27 Thread Thomas Munro
On Mon, Apr 24, 2017 at 2:53 PM, Tom Lane wrote: > Thomas Munro writes: >> Every machine sees the LSN moving backwards, but the code path that >> had the assertion only reached if it decides to interpolate, which is >> timing dependent: there needs to be a future sample i

[HACKERS] A misconception about the meaning of 'volatile' in GetNewTransactionId?

2017-04-29 Thread Thomas Munro
can xids concurrently, such as TransactionIdIsInProgress(). This is almost exactly the example from the section "Avoiding Memory Order Bugs" in src/backend/storage/lmgr/README.barrier. Thoughts? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] Shared Memory hash tables only at startup

2017-05-02 Thread Thomas Munro
kooky pointers, which falls out of Postgres's policies of avoiding multithreading and embracing parallelism. [1] https://www.postgresql.org/message-id/flat/CAEepm%3D3d8o8XdVwYT6O%3DbHKsKAM2pu2D6sV1S_%3D4d%2BjStVCE7w%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com -- Sen

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-02 Thread Thomas Munro
ichplan == 0) without rerouting via a partitioned table (mt_partition_dispatch_info == NULL). See attached patch for discussion (it lacks tests and needs better comments). Does this make sense? Do you see a better way? [1] https://www.postgresql.org/message-id/CACjxUsNhdm4ZCgaVreLK5kAwHTZUkqJAV

Re: [HACKERS] Time based lag tracking for logical replication

2017-05-02 Thread Thomas Munro
thinko will affect logical rep with Petr's patch more. I had been meaning to post the improvement but got sidetracked by that recovery test failure problem. I'll post that in the next few days. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-04 Thread Thomas Munro
is involved there. On the other hand, doing that with inheritance hierarchies would be an incompatible behavioural change, which I guess people don't want -- am I right? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] delta relations in AFTER triggers

2017-05-04 Thread Thomas Munro
40) + 81 at nodeNamedtuplestorescan.c:197 frame #6: 0x00010eca46a6 postgres`ExecReScan(node=0x7ff21d007840) + 822 at execAmi.c:216 frame #7: 0x00010ece7eca postgres`ExecNestLoop(node=0x7ff21d006310) + 538 at nodeNestloop.c:148 I think the problem is that the tuplestore

Re: [HACKERS] delta relations in AFTER triggers

2017-05-04 Thread Thomas Munro
, so we should reject transition tables on STATEMENT triggers for TRUNCATE at creation time too. See attached. Thoughts? > Log file and core dump attached for reference. Thanks! Just by the way, it'd be better to post just an interesting stack trace fragment rather than a core file

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-04 Thread Thomas Munro
On Fri, May 5, 2017 at 2:40 AM, Robert Haas wrote: > On Thu, May 4, 2017 at 4:46 AM, Thomas Munro > wrote: >> On Thu, May 4, 2017 at 4:02 AM, Alvaro Herrera >> wrote: >>> Robert Haas wrote: >>>> I suspect that most users would find it more useful t

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-04 Thread Thomas Munro
, but when it was there Postgres thought it was the same cost as a plan where every worker hammers your system building the same hash table, whereas XPRS would have considered such a plan ludicrously expensive (depending on his 'w' term, see page 28, which determines whether you care more abo

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-05 Thread Thomas Munro
iple open descriptors on a single file, but they all point to a single inode structure." [6] https://github.com/freebsd/freebsd/blob/7e6cabd06e6caa6a02eeb86308dc0cb3f27e10da/sys/sys/file.h#L180 [7] https://github.com/freebsd/freebsd/blob/7e6cabd06e6caa6a02eeb86308dc0cb3f27e10da/sys/kern/vfs_vnop

Re: [HACKERS] Declarative partitioning - another take

2017-05-07 Thread Thomas Munro
or partitioning +hierarchy does not cause the statement-level triggers of affected child +tables to be fired; only the root table's statement-level triggers are +fired. However, row-level triggers of any affected child tables will be +fired. + + + Why talk specifically about t

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-07 Thread Thomas Munro
On Mon, May 8, 2017 at 1:39 PM, David Rowley wrote: > On 6 May 2017 at 13:44, Thomas Munro wrote: >> Experimentation required... > > Indeed. I do remember long discussions on this before Parallel seq > scan went in, but I don't recall if anyone checked any OS kernels

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-08 Thread Thomas Munro
On Fri, May 5, 2017 at 8:29 AM, Thomas Munro wrote: > On Fri, May 5, 2017 at 2:40 AM, Robert Haas wrote: >> On Thu, May 4, 2017 at 4:46 AM, Thomas Munro >> wrote: >>> On Thu, May 4, 2017 at 4:02 AM, Alvaro Herrera >>> wrote: >>>> Robert Haas wrot

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-09 Thread Thomas Munro
On Mon, May 8, 2017 at 7:09 PM, Thomas Munro wrote: > On Fri, May 5, 2017 at 8:29 AM, Thomas Munro > wrote: >> On Fri, May 5, 2017 at 2:40 AM, Robert Haas wrote: >>> On Thu, May 4, 2017 at 4:46 AM, Thomas Munro >>> wrote: >>>> On Thu, May 4, 20

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-09 Thread Thomas Munro
On Tue, May 9, 2017 at 10:29 PM, Thomas Munro wrote: > In master, the decision to populate transition tables happens in > AfterTriggerSaveEvent (called by the ExecAR* functions) in trigger.c. > It does that if it sees that there are triggers on the > relation-being-modified that hav

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-09 Thread Thomas Munro
On Wed, May 10, 2017 at 9:57 AM, Alvaro Herrera wrote: > Thomas Munro wrote: > >> Recall that transition tables can be specified for statement-level >> triggers AND row-level triggers. If you specify them for row-level >> triggers, then they can see all rows changed so fa

Re: [HACKERS] Transition tables for triggers on foreign tables and views

2017-05-09 Thread Thomas Munro
Hi, Since the last one conflicted with recent changes, here's a rebased patch to prevent transition tables on views and foreign tables. -- Thomas Munro http://www.enterprisedb.com prevent-unsupported-transition-tables-v2.patch Description: Binary data -- Sent via pgsql-hackers mailing

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-09 Thread Thomas Munro
On Wed, May 10, 2017 at 2:31 PM, Amit Langote wrote: > On 2017/05/10 6:51, Thomas Munro wrote: >> No such problem exists for partition hierarchies since the tables all >> appear as the same type to user code (though conversions may be >> happening for technical reasons).

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-09 Thread Thomas Munro
On Wed, May 10, 2017 at 11:22 AM, Thomas Munro wrote: > On Wed, May 10, 2017 at 9:57 AM, Alvaro Herrera > wrote: >> Thomas Munro wrote: >> >>> Recall that transition tables can be specified for statement-level >>> triggers AND row-level triggers. If you spe

Re: [HACKERS] Declarative partitioning - another take

2017-05-09 Thread Thomas Munro
; attached updated patch. > > LGTM. Committed. +A statement that targets a parent table in a inheritance or partitioning A tiny typo: s/a inheritance/an inheritance/ -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-10 Thread Thomas Munro
On Wed, May 10, 2017 at 3:55 PM, Robert Haas wrote: > On Tue, May 9, 2017 at 11:48 PM, Thomas Munro > wrote: >> Hmm. DB2 has transition tables (invented them maybe?) and it allows >> OLD/NEW TABLE on row-level triggers: >> >> https://www.ibm.com/support/kn

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-10 Thread Thomas Munro
On Wed, May 10, 2017 at 11:10 PM, Thomas Munro wrote: > 2. If you attach a row-level trigger with transition tables to any > inheritance child, it will see transition tuples from all tables in > the inheritance hierarchy at or below the directly named table that > were modified

Re: [HACKERS] Hash Functions

2017-05-14 Thread Thomas Munro
oats, but they show up as different types in C so not relevant.) [1] https://en.wikipedia.org/wiki/IBM_Floating_Point_Architecture [2] https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.cbcux01/flotcop.htm#flotcop [3] https://www.postgresql.org/message-id/flat/BLU437-SMTP4B3FF36

Re: [HACKERS] Hash Functions

2017-05-14 Thread Thomas Munro
On Mon, May 15, 2017 at 10:08 AM, Thomas Munro wrote: > [2] > https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.cbcux01/flotcop.htm#flotcop Though looking more closely I see that the default is IEEE in 64 bit builds, which seems like a good way to kill the

[HACKERS] COPY FROM STDIN behaviour on end-of-file

2017-05-15 Thread Thomas Munro
Hi hackers, I you hit ^d while COPY FROM STDIN is reading then subsequent COPY FROM STDIN commands return immediately. That's because we never clear the end-of-file state on the libc FILE. Shouldn't we do that, perhaps with something like the attached? -- Thomas

[HACKERS] Relcache leak when row triggers on partitions are fired by COPY

2017-05-15 Thread Thomas Munro
there be an ExecCleanupTriggerState(estate) used by all places? -- Thomas Munro http://www.enterprisedb.com fix-relcache-leak-in-copy-with-triggers.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Relcache leak when row triggers on partitions are fired by COPY

2017-05-15 Thread Thomas Munro
ike that. The call to ExecCloseIndices() may technically be redundant (we never opened them). -- Thomas Munro http://www.enterprisedb.com fix-relcache-leak-in-copy-with-triggers-v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] COPY FROM STDIN behaviour on end-of-file

2017-05-16 Thread Thomas Munro
On Tue, May 16, 2017 at 6:29 PM, Vaishnavi Prabakaran wrote: > On Tue, May 16, 2017 at 8:40 AM, Thomas Munro > wrote: >> >> >> I you hit ^d while COPY FROM STDIN is reading then subsequent COPY >> FROM STDIN commands return immediately. > > > Hi, I could no

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-16 Thread Thomas Munro
On Wed, May 17, 2017 at 9:20 AM, Kevin Grittner wrote: > On Wed, May 10, 2017 at 7:02 AM, Thomas Munro > wrote: >> On Wed, May 10, 2017 at 11:10 PM, Thomas Munro >> wrote: >>> 2. If you attach a row-level trigger with transition tables to any >>> inher

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-16 Thread Thomas Munro
On Wed, May 17, 2017 at 10:13 AM, Kevin Grittner wrote: > On Tue, May 16, 2017 at 4:50 PM, Thomas Munro > wrote: >> >> I'm about to post a much simpler patch that collects uniform tuples >> from children, addressing the reported bug, and simply rejects >> t

Re: [HACKERS] COPY FROM STDIN behaviour on end-of-file

2017-05-16 Thread Thomas Munro
outer" stream, such as a file that contains the COPY ... FROM STDIN command, but doesn't it seem like a general enough statement to cover ^d in the interactive case too? Here's a version incorporating your other suggestions and a comment to explain. [1] https://github.com/freebsd/f

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-17 Thread Thomas Munro
On Wed, May 17, 2017 at 6:04 PM, Amit Langote wrote: > On 2017/05/17 11:22, Thomas Munro wrote: >> Here is that patch. Thoughts? > > I looked at the patch and noticed that there might be some confusion about > what's in the EState.es_root_result_relations array. Tha

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-17 Thread Thomas Munro
On Wed, May 17, 2017 at 7:42 PM, Thomas Munro wrote: > On Wed, May 17, 2017 at 6:04 PM, Amit Langote > wrote: >> targetRelInfo should instead be set to mtstate->rootResultRelInfo that was >> set in ExecInitModifyTable() as described above, IOW, as follows: >>

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-05-17 Thread Thomas Munro
over in syncrep.c. I think that file could use a few words of explanation for why it's OK to access pid, state and flush without synchronisation. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] different column orders in regression test database

2017-05-18 Thread Thomas Munro
Then "c" is dumped as: CREATE TABLE c PARTITION OF p FOR VALUES IN (42); If you wanted to preserve column orders for partitions I guess you'd have to teach to to detect the difference (ignoring dropped columns?) and generate the two step create-and-attach commands. -- Thomas M

Re: [HACKERS] different column orders in regression test database

2017-05-18 Thread Thomas Munro
On Fri, May 19, 2017 at 10:53 AM, Peter Eisentraut wrote: > On 5/18/17 16:21, Thomas Munro wrote: >> That's because if you attach a partition with a different column >> ordering, > > Is it intentional and sensible to allow that in the first place? Or was > it ju

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-18 Thread Thomas Munro
(BBB,42), (CCC,1066) On Thu, May 18, 2017 at 10:16 PM, Amit Langote wrote: > +typedef struct TriggerTransitionState > +{ > ... > +boolttf_delete_old_table; > > Just curious: why ttf_? TriggerTransition field? Oops. Changed to "tts_". I had renamed this

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-18 Thread Thomas Munro
e members. > > Ah. BTW, maybe it's not a problem, but the existing TupleTableSlot's > member names are prefixed with tts_, too. :) Would TransitionCaptureState be a better name for this struct? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-19 Thread Thomas Munro
On Fri, May 19, 2017 at 6:35 PM, Amit Langote wrote: > On 2017/05/19 15:16, Thomas Munro wrote: >> Would TransitionCaptureState be a better name for this struct? > > Yes. Although, losing the Trigger prefix might make it sound a bit > ambiguous though. Right above its d

Re: [HACKERS] transition table behavior with inheritance appears broken (was: Declarative partitioning - another take)

2017-05-19 Thread Thomas Munro
On Sat, May 20, 2017 at 10:43 AM, Thomas Munro wrote: > On Fri, May 19, 2017 at 6:35 PM, Amit Langote > wrote: >> On 2017/05/19 15:16, Thomas Munro wrote: >>> Would TransitionCaptureState be a better name for this struct? >> >> Yes. Although, losing the Trigger

Re: [HACKERS] Causal reads take II

2017-05-21 Thread Thomas Munro
On Mon, May 22, 2017 at 4:10 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: > I'm wondering about status of this patch and how can I try it out? Hi Dmitry, thanks for your interest. >> On 3 January 2017 at 02:43, Thomas Munro >> wrote: >> The replay lag trac

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-05-21 Thread Thomas Munro
On Thu, Apr 27, 2017 at 11:03 AM, Thomas Munro wrote: > On Thu, Apr 27, 2017 at 5:13 AM, Oleg Golovanov wrote: >> Can you actualize your patch set? The error got from >> 0010-hj-parallel-v12.patch. > > I really should get around to setting up a cron job to tell me about >

Re: [HACKERS] Causal reads take II

2017-05-23 Thread Thomas Munro
On Mon, May 22, 2017 at 6:32 AM, Thomas Munro wrote: > On Mon, May 22, 2017 at 4:10 AM, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> I'm wondering about status of this patch and how can I try it out? > > Hi Dmitry, thanks for your interest. > >>> On

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-05-26 Thread Thomas Munro
and a diacritical marker 0301 (COMBINING ACCENT ACUTE). So we need to teach it to be recursive. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-05-26 Thread Thomas Munro
if combining_ids[0] is a plain letter, it looks like it should also check if combining_ids[0] itself is a letter with marks. Also get_plain_letter would need to be able to recurse to extract the "a". I hope that helps! -- Thomas Munro http://www.enterprisedb.com -- Sent v

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

2017-05-27 Thread Thomas Munro
o use a small bit of named traditional shmem for discovery purposes, and it's slightly against the rules to do that when you haven't called RequestAddinShmemSpace, and it's too late to do that. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Extra Vietnamese unaccent rules

2017-05-28 Thread Thomas Munro
like that could be useful for runtime use (I'm sure there is a whole world of Unicode support we could add), but here we're only trying to generate a mapping file to add to the source tree, so I'm not sure how it's relevant. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-29 Thread Thomas Munro
On Fri, Apr 7, 2017 at 5:21 PM, Thomas Munro wrote: > * It would be nice for the SharedRecordTypeRegistry to be able to > survive longer than a single parallel query, perhaps in a per-session > DSM segment. Perhaps eventually we will want to consider a > query-scoped area, a transa

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-05-30 Thread Thomas Munro
On Fri, May 19, 2017 at 2:05 PM, Michael Paquier wrote: > On Thu, May 18, 2017 at 1:43 PM, Thomas Munro > wrote: >> On Thu, May 11, 2017 at 1:48 PM, Michael Paquier >> wrote: >>> I had my eyes on the WAL sender code this morning, and I have noticed >>>

Re: [HACKERS] strcmp() tie-breaker for identical ICU-collated strings

2017-06-01 Thread Thomas Munro
first we use the collation-aware comparison, and then as a tie breaker, we use a binary comparison. If we didn't do a binary comparison as a tie-breaker, wouldn't the result be logically incompatible with the = operator, which does a binary comparison? Put another way, if we didn't use binary order tie-breaking, we'd have to teach texteq to understand collations (ie be defined as not (a < b) and not (b > a)) otherwise we'd permit contradictions like a != b and not (a < b) and not (b > a). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] strcmp() tie-breaker for identical ICU-collated strings

2017-06-01 Thread Thomas Munro
On Fri, Jun 2, 2017 at 9:27 AM, Peter Geoghegan wrote: > On Thu, Jun 1, 2017 at 2:24 PM, Thomas Munro > wrote: >> Why should ICU be any different than the system provider in this >> respect? In both cases, we have a two-level comparison: first we use >> the collation-aw

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-02 Thread Thomas Munro
create trigger trig1 after insert on table1 referencing new table as oo for each statement execute procedure trigfunc(); create trigger trig2 after insert on table2 referencing new table as oo for each statement execute procedure trigfunc(); with t as (insert into table1 values (1)) insert int

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-02 Thread Thomas Munro
On Sat, Jun 3, 2017 at 12:10 AM, Thomas Munro wrote: > I would have expected that to be handled by separate transition tables > at different query levels, but evidently it isn't. I am wondering if we need to wrap the execution of a modifying CTE in AfterTriggerBegin

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-02 Thread Thomas Munro
On Sat, Jun 3, 2017 at 1:20 AM, Thomas Munro wrote: > On Sat, Jun 3, 2017 at 12:10 AM, Thomas Munro > wrote: >> I would have expected that to be handled by separate transition tables >> at different query levels, but evidently it isn't. > > I am wondering if we ne

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-03 Thread Thomas Munro
On Sun, Jun 4, 2017 at 10:41 AM, Kevin Grittner wrote: > On Fri, Jun 2, 2017 at 8:46 PM, Thomas Munro > wrote: > >> So, afterTriggers.query_stack is used to handle the reentrancy that >> results from triggers running further statements that might fire >> triggers. It

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Thomas Munro
On Tue, Jun 6, 2017 at 2:15 AM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jun 3, 2017 at 10:39 PM, Thomas Munro >> wrote: >>> In the meantime, it seems like you agree that rejecting wCTEs that >>> affect tables with triggers with transition tables is

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Thomas Munro
On Tue, Jun 6, 2017 at 10:11 AM, Tom Lane wrote: > Thomas Munro writes: >> On Tue, Jun 6, 2017 at 2:15 AM, Tom Lane wrote: >>> FWIW, parse analysis is surely NOT the time for such a check. Triggers >>> might get added to a table between analysis and execution. I thi

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-05 Thread Thomas Munro
On Tue, Jun 6, 2017 at 12:58 PM, Craig Ringer wrote: > On 4 June 2017 at 06:41, Kevin Grittner wrote: >> On Fri, Jun 2, 2017 at 8:46 PM, Thomas Munro >> wrote: >> >>> So, afterTriggers.query_stack is used to handle the reentrancy that >>> results from t

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-06 Thread Thomas Munro
that needs to use MakeTransitionCaptureState() -- but it seems to lack a place to keep that around, and I ran out of time thinking about that today. Thoughts? [1] https://www.postgresql.org/message-id/CAEepm%3D1dGNzh98Gt21fn_Ed6k20sVB-NuAARE1EF693itK6%3DLg%40mail.gmail.com -- Thomas Munr

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-06 Thread Thomas Munro
d OLD, I'll bet you one internet point that they say we should apply the same access restrictions as the underlying table, and we don't. Am I wrong? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Thomas Munro
On Wed, Jun 7, 2017 at 10:47 AM, Peter Geoghegan wrote: > On Mon, Jun 5, 2017 at 6:40 PM, Thomas Munro > wrote: >> After sleeping on it, I don't think we need to make that decision here >> though. I think it's better to just move the tuplestores into >> Modi

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Thomas Munro
On Wed, Jun 7, 2017 at 7:27 PM, Thomas Munro wrote: > On Wed, Jun 7, 2017 at 10:47 AM, Peter Geoghegan wrote: >> I suppose you'll need two tuplestores for the ON CONFLICT DO UPDATE >> case -- one for updated tuples, and the other for inserted tuples. > > Hmm. Right.

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Thomas Munro
On Thu, Jun 8, 2017 at 9:19 AM, Kevin Grittner wrote: > On Wed, Jun 7, 2017 at 3:42 AM, Thomas Munro > wrote: >> On Wed, Jun 7, 2017 at 7:27 PM, Thomas Munro >> wrote: >>> On Wed, Jun 7, 2017 at 10:47 AM, Peter Geoghegan wrote: >>>> I suppose you'll

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Thomas Munro
re of updates and inserts, but only if they explicitly invite such problems into their lives with that incantation. I think the code change for 2 is not enormous. Split the tuplestore in two, route tuples appropriately, and teach NamedTuplestoreScan to scan both if a union is needed. It do

Re: [HACKERS] PG10 transition tables, wCTEs and multiple operations on the same table

2017-06-07 Thread Thomas Munro
On Thu, Jun 8, 2017 at 11:05 AM, Thomas Munro wrote: > 1. Keep the current behaviour. [...] > > 2. Make a code change that would split the 'new table' tuplestore in > two: an insert tuplestore and an update tuplestore (for new images; > old images could remain in the ol

<    1   2   3   4   5   6   7   8   9   >