Re: [HACKERS] Proposal: Local indexes for partitioned table

2018-01-12 Thread Peter Eisentraut
7;t compare indexes' operator classes and collations. Also, some new test cases for pg_dump would be nice. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-12 Thread Peter Geoghegan
GetOldestXmin()'s second argument is true in the patch, rather than PROCARRAY_FLAGS_VACUUM. That's due to bitrot that was not caught during some previous rebase (commit af4b1a08 changed the signature). Will fix. You've given me a lot more to work through in your most recent mail, Rober

Re: WIP: a way forward on bootstrap data

2018-01-12 Thread Peter Eisentraut
c sample we have "n" which stands for proname, because we have > this line: >+ NameDataproname BKI_ABBREV(n); I'm afraid a key value system would invite writing the attributes in random order and create a mess over time. But if we want to do it, I think we could also add it to the current BKI format. The same goes for defining default values for some columns. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-13 Thread Peter Geoghegan
ine with killing force_parallel_mode, though, because it will be possible to force the use of parallelism by using the existing parallel_workers table storage param in the next version of the patch, regardless of how small the table is. Thanks for the review. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-15 Thread Peter Geoghegan
On Sun, Jan 14, 2018 at 8:25 PM, Amit Kapila wrote: > On Sun, Jan 14, 2018 at 1:43 AM, Peter Geoghegan wrote: >> On Sat, Jan 13, 2018 at 4:32 AM, Amit Kapila wrote: >>> Yeah, but this would mean that now with parallel create index, it is >>> possible that some tuples

Re: [HACKERS] generated columns

2018-01-16 Thread Peter Eisentraut
On 1/16/18 01:35, Michael Paquier wrote: > On Wed, Dec 27, 2017 at 12:31:05PM -0500, Peter Eisentraut wrote: >> On 9/12/17 15:35, Jaime Casanova wrote: >>> On 10 September 2017 at 00:08, Jaime Casanova >>> wrote: >>>> >>>> During my own tes

Re: [HACKERS] Transaction control in procedures

2018-01-16 Thread Peter Eisentraut
)? This is all changed now after the patch for portal pinning in PL/Perl and PL/Python has been committed. The attached patch behaves better. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From 84f35266cc63d

Re: [HACKERS] taking stdbool.h into use

2018-01-16 Thread Peter Eisentraut
On 1/11/18 17:01, Peter Eisentraut wrote: > Looking around where else they are used, the uses in numeric.c sure seem > like noops: > > #if SIZEOF_DATUM == 8 > #define NumericAbbrevGetDatum(X) ((Datum) SET_8_BYTES(X)) > #define DatumGetNumericAbbrev(X) ((int64) GET_8_B

Re: let's not complain about harmless patch-apply failures

2018-01-16 Thread Peter Geoghegan
g problems mechanically. Rebasing a patch without conflicts (including seeing a warning about offsets) does not mean that your patch didn't become broken in some subtle, harmful way. Mechanical detection is only useful to the extent that it guides and augments human oversight. -- Peter Geoghegan

Re: let's not complain about harmless patch-apply failures

2018-01-16 Thread Peter Geoghegan
erally not helpful unless used thoughtfully. -- Peter Geoghegan

Re: pg_(total_)relation_size and partitioned tables

2018-01-17 Thread Peter Eisentraut
I'm setting this patch to Returned with feedback. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Package version in PG_VERSION and version()

2018-01-17 Thread Peter Eisentraut
ht solution for packagers, so I'm tempted to close this commit fest item. There is some speculation that using it could break third-party tools, but (a) we would need more concrete evidence, (b) we should fix *that* then, and (c) it's likely unavoidable in general. -- Peter Eisentrau

Re: [HACKERS] [Patch] Log SSL certificate verification errors

2018-01-17 Thread Peter Eisentraut
Graham, will you be able to respond to my questions or provide an updated patch within the next week or so? On 1/2/18 09:17, Peter Eisentraut wrote: > The server-side changes look pretty reasonable. > > On the client side, I'd like to see some comments explaining the >

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-17 Thread Peter Geoghegan
ogress, to make sure that what I add is comparable to what ultimately gets committed for parallel query. -- Peter Geoghegan

Re: [HACKERS] GnuTLS support

2018-01-17 Thread Peter Eisentraut
On 1/2/18 10:35, Peter Eisentraut wrote: > On 11/26/17 20:05, Andreas Karlsson wrote: >> I have now implemented this in the attached patch (plus added support >> for channel binding and rebased it) but I ran into one issue which I >> have not yet solved. The script for the

Re: Package version in PG_VERSION and version()

2018-01-17 Thread Peter Eisentraut
identifier in "psql -version" specifically would actually be the biggest win in this, because that's where a lot of issues arise (e.g., wrong libedit/readline library, different compiled-in socket location). -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Dev

Re: [HACKERS] GnuTLS support

2018-01-17 Thread Peter Eisentraut
ementation, but is that one? The proposed GnuTLS patch does make use of ssl_dh_params_file. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: PostgreSQL crashes with SIGSEGV

2018-01-17 Thread Peter Geoghegan
On Wed, Jan 17, 2018 at 11:23 AM, Tom Lane wrote: > Aleksandr Parfenov writes: >> The new status of this patch is: Ready for Committer > > I don't feel particularly comfortable committing a patch that > was clearly labeled as a rushed draft by its author. > Peter, where

Re: PostgreSQL crashes with SIGSEGV

2018-01-17 Thread Peter Geoghegan
tuplesort_getdatum() subset fix might still be a good idea. I wonder where you stand on this. -- Peter Geoghegan

Re: [HACKERS] GnuTLS support

2018-01-17 Thread Peter Eisentraut
e the case for all TLS implementations ever, this is a pretty good indicator to me that it is an implementation-independent concept. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] GnuTLS support

2018-01-17 Thread Peter Eisentraut
tring" is easy to look up and well documented. If instead we chop it up into something that is more like the OpenSSL settings, I think we are not helping anyone. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] replace GrantObjectType with ObjectType

2018-01-17 Thread Peter Eisentraut
ign is > wrong. I would have thought that roles should be contained within a > single file. And the role is dropped at the end of alter_table.sql as > your patch does. So perhaps something crashed during your own tests and > you added this DROP ROLE to ease things? I just copied this from alter_ge

Re: PostgreSQL crashes with SIGSEGV

2018-01-17 Thread Peter Geoghegan
hat grouping sets is right (and that mode_final() is wrong). Do you? -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-17 Thread Peter Geoghegan
On Wed, Jan 17, 2018 at 10:27 AM, Robert Haas wrote: > On Wed, Jan 17, 2018 at 12:27 PM, Peter Geoghegan wrote: >> I think that both problems (the live _bt_parallel_scan_and_sort() bug, >> as well as the general issue with needing to account for parallel >> worker fork(

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-17 Thread Peter Geoghegan
etic data distributions/types. >> WFM. Also added documentation for the wait events to monitoring.sgml, >> which I somehow missed the first time around. > > But you forgot to update the preceding "morerows" line, so the > formatting will be all messed up. Fixed. >> I removed "really". The point of the comment is that we've already set >> up temp tablespaces for the shared fileset in the parallel case. >> Shared filesets figure out which tablespaces will be used up-front -- >> see SharedFileSetInit(). > > So why not say it that way? i.e. For parallel sorts, this should have > been done already, but it doesn't matter if it gets done twice. Okay. > I don't see any reason not to make those contingent only on > trace_sort. The user can puzzle apart which messages are which from > the PIDs in the logfile. Okay. I have removed anything that restrains the verbosity of trace_sort for the WORKER() case. I think that you were right about it the first time, but I now think that this is going too far. I'm letting it go, though. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-17 Thread Peter Geoghegan
oesn't really > matter: if that comes along later, it will be trivial to adjust the > code to take advantage of it. Okay. I'll work on adopting dynamic barriers in the way you described. I just wanted to make sure that we're all on the same page about what that looks like. -- Peter Geoghegan

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-01-18 Thread Peter Eisentraut
free any memory in the operating system, that's no good. Can we get some usage statistics out of the memory context? It seems like we need to keep serializing transactions until we actually see the memory context size drop. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH] session_replication_role = replica with TRUNCATE

2018-01-18 Thread Peter Eisentraut
umentation, so I'm wondering if we should document that they are the >> same. >> > It's for use by 3rd party tools (afaik both londiste and slony use it) > to differentiate between replicated and not replicated changes. I have committed some documentation updat

Re: [PATCH] session_replication_role = replica with TRUNCATE

2018-01-18 Thread Peter Eisentraut
iolates constraints. It's also weird that you can violate deferred unique constraints but not immediate ones (I think, not tested). So I'm proposing the attached alternative patch, which creates constraint triggers to be TRIGGER_FIRES_ALWAYS by default. Thoughts? -- Peter Eis

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
parallel_leader_participation=off allow a degenerate parallel CREATE INDEX in the next version. I think that it will make parallel_leader_participation less useful, with no upside, but there doesn't seem to be much more that I can do about that. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
out the question of ripping out parallel_leader_participation entirely. > Can you please elaborate what part of optimizer are you talking about > where without leader participation partial path will always lose to a > serial sequential scan path? See my remarks to Robert just now. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
On Thu, Jan 18, 2018 at 10:27 AM, Robert Haas wrote: > On Thu, Jan 18, 2018 at 1:14 PM, Peter Geoghegan wrote: >> That seems pretty far fetched. > > I don't think it is, and there are plenty of other examples. All you > need is a query plan that involves significant

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
nt within InitializeParallelDSM() should instead say something like "Serialize indexes-pending-reindex state". Other than that, looks good to me. It's a simple patch with a clear purpose. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Peter Geoghegan
everything within nbtsort.c. However, tuplesort_leader_wait() seems kind of like something that we should have on general principle. And, more importantly, it would be tricky to use a barrier even for this, because we still have that baked-in assumption that nworkers_launched is the single source of truth about the number of participants. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-19 Thread Peter Geoghegan
to either want me to remove it from consideration entirely, or to remove the code that specifically disallows a "degenerate parallel CREATE INDEX". I need a final answer on that. -- Peter Geoghegan

Re: [HACKERS] GnuTLS support

2018-01-19 Thread Peter Eisentraut
known well enough nowadays that we can just remove the explanation. 0004-Move-SSL-API-comments-to-header-files.patch 0005-Extract-common-bits-from-OpenSSL-implementation.patch Move copy-and-paste avoidance. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Su

Re: [HACKERS] replace GrantObjectType with ObjectType

2018-01-19 Thread Peter Eisentraut
On 1/17/18 23:52, Michael Paquier wrote: > On Wed, Jan 17, 2018 at 05:23:25PM -0500, Peter Eisentraut wrote: >> On 1/16/18 23:38, Michael Paquier wrote: >>> + if (prop->objtype == OBJECT_TABLE) >>> + /* >>> +* If the property data says it

Re: PATCH: Configurable file mode mask

2018-01-19 Thread Peter Eisentraut
erm testing. > > Does anyone have suggestions on tests for pg_resetwal and pg_upgrade? > Should I start from scratch? A test suite for pg_resetwal would be nice. TAP tests for pg_upgrade will create problems with the build farm. There was a recent thread about that. -- Peter Eis

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-19 Thread Peter Geoghegan
arameter for this. When the storage param has been set, we don't care about the amount of memory available to each worker. You can stress-test the implementation as needed. (The storage param does care about max_parallel_maintenance_workers, but you can set that as high as you like.) -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-19 Thread Peter Geoghegan
a worker, but that workers will be used? If so, yes, you can get that. The only issue is that you may not be able to launch parallel workers due to hitting a limit like max_parallel_workers, in which case you'll get a serial index build despite everything. Nothing we can do about that, thou

Re: improve type conversion of SPI_processed in Python

2018-01-20 Thread Peter Eisentraut
In any case > my main point is let's keep these two functions using similar approaches. I ran extensive tests on this and went ahead with the simplified versions. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-20 Thread Peter Geoghegan
I had no idea that there is this issue with nworkers_launched until very recently. But then, that field has absolutely no comments. -- Peter Geoghegan

Re: [HACKERS] WIP: Covering + unique indexes.

2018-01-20 Thread Peter Geoghegan
> Also, I've checked bt_index_parent_check() on master. I fixed that recently. It should be fine now. -- Peter Geoghegan

Re: Use of term hostaddrs for multiple hostaddr values

2018-01-20 Thread Peter Eisentraut
your input! > > Applying the error message portion only on HEAD is a good plan, there is > no point to make the life of translaters unnecessary painful. I would backpatch both. The updated error message is arguably easier to translate. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-21 Thread Peter Geoghegan
efore can be expected to show up. Is there actually a meaningful difference between the way nworkers_launched is depended upon in each case, though? -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-21 Thread Peter Geoghegan
est in this problem, for example by calling WaitForParallelWorkerToAttach() itself. I may have missed something, but right now ISTM that it would be risky to make the API anything other than what both nodeGather.c and nbtsort.c already expect (that they'll either have nworkers_launched w

Re: [HACKERS] Transaction control in procedures

2018-01-22 Thread Peter Eisentraut
On 1/16/18 15:24, Andrew Dunstan wrote: > Looks good. Marking ready for committer. committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Remove PARTIAL_LINKING?

2018-01-22 Thread Peter Eisentraut
t; Any objections to removing that? Seems that's largely just removing a > bit of logic from common.mk. Yeah, I think it's OK to remove that. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Add %r substitution for psql prompts to show recovery status

2018-01-22 Thread Peter Eisentraut
On 1/9/18 15:36, Peter Eisentraut wrote: > On 12/7/17 19:54, Tatsuo Ishii wrote: >>> On Wed, Dec 6, 2017 at 9:19 PM, Ian Barwick >>> wrote: >>>> Note this substitution sends a "pg_is_in_recovery()" query to the server >>>> each time it&#x

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-22 Thread Peter Geoghegan
think that that's only because it's normal for all output to become available all at once for nbtsort.c workers. The startup cost is close to or actually the same as the total cost, as it *always* is for sort nodes. -- Peter Geoghegan

Re: [HACKERS] A design for amcheck heapam verification

2018-01-22 Thread Peter Geoghegan
ich is actually what happens in the vast majority of real world cases. When it does find a problem, there should be some new lesson to be learned. If there isn't some new insight, then somebody somewhere is doing a bad job. -- Peter Geoghegan

Re: [HACKERS] A design for amcheck heapam verification

2018-01-22 Thread Peter Geoghegan
; Manolios as much as possible. > That's all what I've found. I do not know did the patch had all necessary > reviewers attention. Please, feel free to change status if you think that > patch is ready. From my point of view, the patch is in a good shape. Michael said he'd do more review. I generally feel this is close, though. Thanks for the review -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-22 Thread Peter Geoghegan
logtape.c. The fact that randomAccess support *would* be straightforward is a sign that I haven't accidentally introduced some other assumption, or special case. -- Peter Geoghegan

Re: [PATCH] session_replication_role = replica with TRUNCATE

2018-01-22 Thread Peter Eisentraut
he subscription cannot continue, so > executing the a constraint trigger does not solve the problem. But that is already the case for any other kind of constraint or type violation. Just that certain kinds of constraints are apparently ignored for no clear reason. -- Peter Eisentraut

Re: [PATCH] session_replication_role = replica with TRUNCATE

2018-01-22 Thread Peter Eisentraut
On 1/18/18 12:41, Tom Lane wrote: > Peter Eisentraut writes: >> So I'm proposing the attached alternative patch, which creates >> constraint triggers to be TRIGGER_FIRES_ALWAYS by default. >> Thoughts? > > Hm, the general idea seems attractive, but I'm not

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-22 Thread Peter Geoghegan
_wait_internal(). It's not obvious that that will work in all cases ("Note that if handle == NULL, and the process fails to attach, we'll potentially get stuck here forever"). Also, I don't see how this addresses the parallel_leader_participation issue I raised. -- Peter Geoghegan

Re: [HACKERS] taking stdbool.h into use

2018-01-23 Thread Peter Eisentraut
tely >> there seems to be no contemporaneous mailing list discussion, so >> it's not clear what Kevin based this change on. > > This was a hotfix after a buildfarm breakage, the base commit being > f566515. > >> Want to try reverting the GET_X_BYTES() parts of it and see if

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Peter Eisentraut
f the method doesn't exist yet. As a > solution, let's just live with some run duplication for a period until > the machines are upgraded to a future buildfarm client code. Well, people protested against that approach. -- Peter Eisentraut http://www.2ndQuadrant.com/

Re: PATCH: Configurable file mode mask

2018-01-23 Thread Peter Eisentraut
nsistent with how we currently test pg_upgrade. Basically, what you'd need is some way to pass some options to the initdb invocations in the pg_upgrade test script, right? That would seem reasonable, and also useful for testing upgrading with other initdb options. -- Peter Eisentraut h

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-23 Thread Peter Eisentraut
a table. So I think this change is not desirable. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Test-cases for exclusion constraints is missing in alter_table.sql file

2018-01-23 Thread Peter Eisentraut
files which i mentioned in my earlier > email. Thanks. There is ALTER TABLE partitioned ADD EXCLUDE USING gist (a WITH &&); in alter_table.sql. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Handling better supported channel binding types for SSL implementations

2018-01-23 Thread Peter Eisentraut
ng the channel binding type, so a boolean result should be enough. In which case we wouldn't actually need this for GnuTLS yet. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-01-23 Thread Peter Eisentraut
On 1/21/18 18:08, Daniel Gustafsson wrote: > As per before, my patch for running tests against another set of binaries is > included as well as a fix for connstrings with spaces, but with the recent > hacking by Peter I assume this is superfluous. It was handy for development > so &g

Re: pgsql: Improve scripting language in pgbench

2018-01-23 Thread Peter Eisentraut
ines which are ignored by perl execution > anyway. It may be some leftovers from debugging which got past everyone. > If so, I think that it is better removed, see the attached cleanup patch. fixed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Peter Geoghegan
der_participation=off (or some way of getting the same behavior through a #define) is to be retained, then an artificial wait is required as a substitute for the leader's participation as a worker. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 10:50 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 10:36 AM, Robert Haas wrote: >> I am going to repeat my previous suggest that we use a Barrier here. >> Given the discussion subsequent to my original proposal, this can be a >> lot simpler

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-01-23 Thread Peter Eisentraut
', "SELECT \$\$connected with $connstr\$\$", So the problem must have been a single quote in the connstr. That can surely happen, but then so can having a $$. So without a concrete example, I'm not sure how to proceed. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
assumed that nworkers_launched is a reliable indicator of how many workers will show up tells us something. Moreover, parallel_leader_participation=off is always going to be wonky under a regime based on "see who shows up", because failure becomes indistinguishable from high latency -- I'd rather avoid having to think about nbtsort.c in distributed systems terms. -- Peter Geoghegan

Re: [HACKERS] MERGE SQL Statement for PG11

2018-01-23 Thread Peter Geoghegan
ry I had hoped to see in your regression test. > I'm expecting to commit this and then come back for the Partitioning & > RLS later, but will wait a few days for comments and other reviews. I don't think that it's okay to defer RLS or partitioning support till after an initial commit. While it's probably true that MERGE can just follow ON CONFLICT's example when it comes to column-level privileges, this won't be true with RLS. -- Peter Geoghegan

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
SIGUSR1) and after GetBackgroundWorkerPid() is > guaranteed to return BGWH_STOPPED after that, and as long as we only > ever use latch/CFI loops to wait, and as long as we try to read from a > shm_mq, then I don't see a failure mode that hangs. What about the parallel_leader_participation=off case? -- Peter Geoghegan

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
dd a WaitForParallelWorkersToAttach() function, then the performance hit would probably not be noticeable in practice. The parallel_leader_participates case would still work without special care (that's the main hazard that makes using a barrier seem unappealing). -- Peter Geoghegan

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Peter Geoghegan
On Tue, Jan 23, 2018 at 9:38 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 10:38 AM, Peter Geoghegan wrote: >> The leader can go ahead and sort before calling something like a new >> WaitForParallelWorkersToAttach() function (or even >> WaitForParallelWorkersToFinis

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-24 Thread Peter Eisentraut
ndent events. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: documentation is now XML

2018-01-24 Thread Peter Eisentraut
essional publishing. That's where things are now. The next wave in this area will be doing the whole thing with HTML and CSS. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-24 Thread Peter Geoghegan
stgr.es/m/CAH2-Wzm6dF=g9lywthgcqzrc4dzbe-8tv28yvg0xj8q6e4+...@mail.gmail.com [2] https://postgr.es/m/CAA4eK1LEFd28p1kw2Fst9LzgBgfMbDEq9wPh9jWFC0ye6ce62A%40mail.gmail.com [3] https://postgr.es/m/caa4ek1+a0of4m231vbgpr_0ygg_bnmrgzlib7wqde-fybsy...@mail.gmail.com -- Peter Geoghegan

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-24 Thread Peter Geoghegan
we should pursue this approach. I wouldn't put too much weight on keeping this simple for users of the parallel infrastructure, though, because something like Amit's WaitForParallelWorkersToAttach() idea still seems acceptable. "Call this function before trusting the finality of nworkers_launched" isn't too onerous a rule to have to follow. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-24 Thread Peter Geoghegan
On Wed, Jan 24, 2018 at 12:13 PM, Thomas Munro wrote: > On Thu, Jan 25, 2018 at 8:54 AM, Peter Geoghegan wrote: >> I have used Thomas' chaos-monkey-fork-process.patch to verify: >> >> 1. The problem of fork failure causing nbtsort.c to wait forever is a >> real

Re: documentation is now XML

2018-01-24 Thread Peter Eisentraut
On 1/24/18 09:45, Bruce Momjian wrote: > So we are not > using TeX anymore for PG11+ docs? as of PG10 -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-24 Thread Peter Geoghegan
I think that we should implement "some other technique", instead of using a barrier. As I've said, Amit's WaitForParallelWorkersToAttach() idea seems like a promising "other technique". [1] https://www.postgresql.org/message-id/caa4ek1+a0of4m231vbgpr_0ygg_bnmrgzlib7wqde-fybsy...@mail.gmail.com [2] https://www.postgresql.org/message-id/CA+TgmoaF8UA8v8hP=ccoquc50pucpc8abj-_yyc++ygggjw...@mail.gmail.com -- Peter Geoghegan

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-24 Thread Peter Geoghegan
hrough, as part of a future enhancement to the parallel infrastructure? Surely every caller to CreateParallelContext() should not need to invent their own way of doing this. All I want is to be able to rely on nworkers_launched. That's not in tension with this other goal/aspiration, and actually seems to complement it. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-24 Thread Peter Geoghegan
with this today only due to using a tuple queue, such as nodeGather.c, but may not always get away with it in the future. I've run out of time to assess what you've done here in any real depth. For now, I will say that this approach seems interesting to me. I'll take a closer look tomorrow. -- Peter Geoghegan

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Peter Eisentraut
On 1/19/18 13:43, Peter Eisentraut wrote: > Comparing the existing {be,fe}-secure-openssl.c with the proposed > {be,fe}-secure-gnutls.c, and with half an eye on the previously proposed > Apple Secure Transport implementation, I have identified a few more > areas of refactoring that sh

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Peter Eisentraut
On 1/23/18 13:39, Robert Haas wrote: > On Tue, Jan 23, 2018 at 12:03 PM, Peter Eisentraut > wrote: >> It might also be useful to set these reloptions before you add a new >> column to a table. > > I don't understand. AAUI, it is currently the case that if you set &

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-01-25 Thread Peter Eisentraut
;, '-c', "SELECT \$\$connected with $connstr\$\$", >> >> So the problem must have been a single quote in the connstr. > Right, looking back at b5e2b87d-3e8a-4597-9a7f-8489b3b67...@yesql.se I > realized > I was misremembering, the issue was that I h

Re: Doc tweak for huge_pages?

2018-01-25 Thread Peter Eisentraut
otten a bit out of order with repeated patching. I also softened the advice against THP just a bit, since that is apparently still changing all the time. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Subscription code improvements

2018-01-25 Thread Peter Eisentraut
pears around subscriptions, I might > propose it again. I have looked at the patches again. They seem generally reasonable, but I don't see quite why we want or need them. More details would help review them. Do they fix any bugs, or are they just rearranging code? -- Peter Eisentraut

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-25 Thread Peter Eisentraut
Since we currently don't handle sequences at all, we'll omit the six sequence changes for now.) -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Peter Eisentraut
On 1/25/18 09:40, Tom Lane wrote: > Peter Eisentraut writes: >> On 1/23/18 13:39, Robert Haas wrote: >>> I don't understand. AAUI, it is currently the case that if you set >>> the options before the TOAST table exists, they are lost. > >> Well, that

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Peter Eisentraut
On 1/25/18 20:10, Michael Paquier wrote: > Peter, could you change ssl_version() and ssl_cipher() in sslinfo at the > same time please? I think that those should use the generic backend-side > APIs as well. sslinfo depends heavily on OpenSSL, OK, but if possible > getting this code

Re: [HACKERS] generated columns

2018-01-25 Thread Peter Eisentraut
m attributes > (xmax, xmin, etc.). Some tests could be added. Hard to test that, because the results would be nondeterministic. > - if (tab->relkind == RELKIND_RELATION || > - tab->relkind == RELKIND_PARTITIONED_TABLE) > + if ((tab->relkind == RELKIND_RELATION || > +

Re: Redefining inet_net_ntop

2018-01-25 Thread Peter Eisentraut
x27;m thinking of adding a > test to configure and omitting our own version if configure finds it. ISTR that ours is hacked to produce specific output. Do the regression tests still pass if you use the one from the OS? -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQ

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-25 Thread Peter Geoghegan
hare this feeling. > Just to conclude, following are proposed ways to solve this problem: > > 1. Implement a new API WaitForParallelWorkersToAttach and use that to > solve this problem. Peter G. and Amit thinks, this is a good way to > solve this problem. > 2. Use existing API W

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
produces the same output as R NORMALIZE (S > NORMALIZE T), perhaps? It is not commutative and also not associative. Assume relations that contain one tuple each as follows: R={[1, 100)}, S={[50, 100)}, and T={[10, 20)} (R NORMALIZE S) NORMALIZE T gives {[1, 10), [10, 20), [20,50), [50, 100)} while R NORMALIZE (S NORMALIZE T) gives {[1, 50), [50, 100)} Best regards, Anton, Johann, Michael, Peter

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
te result tuples, that do not contribute to the final result would be emmitted, and thus the performance would suffer. Best regards, Anton, Johann, Michael, Peter

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
ERE NOT EXISTS ( SELECT * FROM (budg s ALIGN budg r ON s.amnt > r.amnt WITH (t,t)) s WHERE s.amnt > r.amnt AND r.t = s.t ); -- name | amnt |t -- --+--+- -- Joe |5 | [2012-02-01,2012-05-01) -- Ann |7 | [2012-05-01,2012-09-01) -- Per |3 | [2012-09-01,2012-10-01) Best regards, Anton, Johann, Michael, Peter

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
with a lot of money have already implemented; it's unlikely > that they are going to adopt something totally different from any > existing system but inconveniently similar to ours. We agree with you. Best regards, Anton, Johann, Michael, Peter

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-26 Thread Peter Geoghegan
On Fri, Jan 26, 2018 at 10:01 AM, Robert Haas wrote: > On Fri, Jan 26, 2018 at 1:30 AM, Peter Geoghegan wrote: >> I had imagined that WaitForParallelWorkersToAttach() would give me an >> error in the style of WaitForParallelWorkersToFinish(), without >> actually waiting for

Re: unique indexes on partitioned tables

2018-01-26 Thread Peter Eisentraut
t fail. Throw some null values in, to check PK behavior as well. It should be trivial, but seems kind of useful. Other than that, this looks pretty good to me. A logical extension of the previous partitioned index patch. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-26 Thread Peter Geoghegan
ly seems like we'd want to preserve that at a minimum. -- Peter Geoghegan

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-26 Thread Peter Geoghegan
On Fri, Jan 26, 2018 at 11:17 AM, Robert Haas wrote: > Hmm, I like the idea of making it a #define instead of having it > depend on parallel_leader_participation. Let's do that. If the > consensus is later that it was the wrong decision, it'll be easy to > change it

<    17   18   19   20   21   22   23   24   25   26   >