Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-16 Thread Thomas Munro
On Tue, Oct 8, 2019 at 11:09 PM Amit Kapila wrote: > On Sat, Oct 5, 2019 at 3:10 AM Andres Freund wrote: > > On 2019-10-04 17:08:29 -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2019-10-04 16:31:29 -0400, Bruce Momjian wrote: > > > >> Yeah, it is certainly weird that you have to

Re: ICU for global collation

2019-10-17 Thread Thomas Munro
On Thu, Oct 17, 2019 at 3:52 PM Thomas Munro wrote: > I rebased this patch, and tweaked get_collation_action_version() very > slightly so that you get collation version change detection (of the > ersatz kind provided by commit d5ac14f9) for the default collation > even when not using

Re: "pg_ctl: the PID file ... is empty" at end of make check

2019-10-17 Thread Thomas Munro
On Fri, Oct 18, 2019 at 1:21 AM Tom Lane wrote: > Thomas Munro writes: > > On Tue, Oct 15, 2019 at 1:55 PM Tom Lane wrote: > >> and now prairiedog has shown it too: > >> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog&dt=2019-10-14%2021%3A4

Re: Tweaking DSM and DSA limits

2019-10-20 Thread Thomas Munro
On Fri, Jun 21, 2019 at 6:52 AM Andres Freund wrote: > On 2019-06-20 14:20:27 -0400, Robert Haas wrote: > > On Tue, Jun 18, 2019 at 9:08 PM Thomas Munro wrote: > > > Perhaps also the number of slots per backend should be dynamic, so > > > that you have the option to i

Parallel leader process info in EXPLAIN

2019-10-23 Thread Thomas Munro
Hi, While working on some slides explaining EXPLAIN, I couldn't resist the urge to add the missing $SUBJECT. The attached 0001 patch gives the following: Gather ... time=0.146..33.077 rows=1 loops=1) Workers Planned: 2 Workers Launched: 2 Buffers: shared hit=4425 -> Parallel Seq Scan o

Drop type "smgr"?

2019-02-27 Thread Thomas Munro
ojects propose to add new smgr implementations alongside md.c in order to use bufmgr.c for more kinds of files, but it seems entirely bogus to extend the unused smgr type to cover those. -- Thomas Munro https://enterprisedb.com 0001-Remove-the-vestigial-smgr-type.patch Description: Binary data

Re: Drop type "smgr"?

2019-02-27 Thread Thomas Munro
On Thu, Feb 28, 2019 at 7:08 PM Tom Lane wrote: > Thomas Munro writes: > > Motivation: A couple of projects propose to add new smgr > > implementations alongside md.c in order to use bufmgr.c for more kinds > > of files, but it seems entirely bogus to extend the unused s

Re: Drop type "smgr"?

2019-02-27 Thread Thomas Munro
On Thu, Feb 28, 2019 at 7:37 PM Tom Lane wrote: > Thomas Munro writes: > > On Thu, Feb 28, 2019 at 7:08 PM Tom Lane wrote: > >> I agree that smgrtype as it stands is pretty pointless, but what > >> will we be using instead to get to those other implementations? &g

Re: Drop type "smgr"?

2019-02-28 Thread Thomas Munro
ithout too much drama; that was one of the things that made me feel that the magic database OID approach was acceptable at least in the short term. -- Thomas Munro https://enterprisedb.com

Re: Drop type "smgr"?

2019-02-28 Thread Thomas Munro
On Fri, Mar 1, 2019 at 4:09 AM Tom Lane wrote: > Thomas Munro writes: > > On Thu, Feb 28, 2019 at 7:37 PM Tom Lane wrote: > >> Thomas Munro writes: > >>> Our current thinking is that smgropen() should know how to map a small > >>> number

Re: Drop type "smgr"?

2019-02-28 Thread Thomas Munro
On Fri, Mar 1, 2019 at 10:41 AM Shawn Debnath wrote: > On Fri, Mar 01, 2019 at 10:33:06AM +1300, Thomas Munro wrote: > > It doesn't make any sense to put things like clog or any other SLRU in > > a non-default tablespace though. It's perfectly OK if not all smgr > &

Re: Drop type "smgr"?

2019-02-28 Thread Thomas Munro
On Fri, Mar 1, 2019 at 11:31 AM Shawn Debnath wrote: > On Thu, Feb 28, 2019 at 02:08:49PM -0800, Andres Freund wrote: > > On 2019-03-01 09:48:33 +1300, Thomas Munro wrote: > > > On Fri, Mar 1, 2019 at 7:24 AM Andres Freund wrote: > > > > On 2019-02-28

Re: Index Skip Scan

2019-02-28 Thread Thomas Munro
athnodes. >> >> I had a look on this. >> >> The name "index skip scan" is a different feature from the >> feature with the name on other prodcuts, which means "index scan >> with postfix key (of mainly of multi column key) that scans >> ignorin

Re: Refactoring the checkpointer's fsync request queue

2019-03-01 Thread Thomas Munro
On Thu, Feb 28, 2019 at 10:43 AM Thomas Munro wrote: > On Thu, Feb 28, 2019 at 10:27 AM Shawn Debnath wrote: > > We had a quick offline discussion to get on the same page and we agreed > > to move forward with Andres' approach above. Attached is patch v10. > > Here&#

Re: Refactoring the checkpointer's fsync request queue

2019-03-01 Thread Thomas Munro
th 128 MB shared_buffers, we will use an extra 16kB (one extra > byte to represent the owner). I am hesitant to change this right now > unless folks feel strongly about it. > > If so, I would combine the type and owner by splitting it up in 4 bit > chunks, allowing for 16 request types and 16 smgrs. This change would > only apply for the in-memory queue. The code and functions would > continue using the enums. +1 -- Thomas Munro https://enterprisedb.com

Re: Refactoring the checkpointer's fsync request queue

2019-03-01 Thread Thomas Munro
n the forseeable future). Now perhaps the single enum idea was going to involve explicit values that encode the two values SYNC_REQ_CANCEL_MD = 0x1 | (0x04 << 4) so you could still extract the requester part, but that's just the same thing with uglier code. -- Thomas Munro https://enterprisedb.com

Re: Drop type "smgr"?

2019-03-01 Thread Thomas Munro
te your own pseudo-filesystem that stores a whole PostgreSQL cluster inside big data files or raw partitions. Luckily we don't need to tackle such mountainous terrain to avoid the page cache, today. -- Thomas Munro https://enterprisedb.com

Re: [HACKERS] SERIALIZABLE with parallel query

2019-03-03 Thread Thomas Munro
rallelism (TPC-H etc) but don't do any writes, or good for testing writes (TPC-B etc) but don't do any parallelism. I'm going to experiment with the "SIBENCH" approach from the Cahill paper and see where that leads. -- Thomas Munro https://enterprisedb.com

Fsync-before-close thought experiment

2019-03-03 Thread Thomas Munro
Hash pathnames and map to an array of lwlocks + error flags. Any process trying to sync a file must hold the lock and check for a pre-existing error flag. Now a checkpoint cannot succeed if any backend has recently decided to panic. You could skip that if data_sync_retry = on. -- Thomas Munro https://enterprisedb.com

Re: Rare SSL failures on eelpout

2019-03-03 Thread Thomas Munro
On Wed, Jan 23, 2019 at 11:23 AM Thomas Munro wrote: > On Wed, Jan 23, 2019 at 4:07 AM Tom Lane wrote: > > The whole thing reminds me of the recent bug #15598: > > > > https://www.postgresql.org/message-id/87k1iy44fd.fsf%40news-spur.riddles.org.uk > > Yeah, if errors g

Re: Rare SSL failures on eelpout

2019-03-04 Thread Thomas Munro
On Tue, Mar 5, 2019 at 10:08 AM Tom Lane wrote: > I wrote: > > Thomas Munro writes: > >> That suggests that we could perhaps handle ECONNRESET both at startup > >> packet send time (for certificate rejection, eelpout's case) and at > >> initial query send

Re: Rare SSL failures on eelpout

2019-03-04 Thread Thomas Munro
On Tue, Mar 5, 2019 at 11:35 AM Tom Lane wrote: > Thomas Munro writes: > > OK, here's something. I can reproduce it quite easily on this > > machine, and I can fix it like this: > > >libpq_gettext("could not send startup packet: %s\n"), > >

Re: Refactoring the checkpointer's fsync request queue

2019-03-05 Thread Thomas Munro
reeBSD system (note that fsync is disabled so it's not that -- it must be bogus queue-related CPU?) -- Thomas Munro https://enterprisedb.com

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
ng on master and REL_11_STABLE: https://buildfarm.postgresql.org/cgi-bin/show_failures.pl?max_days=200&branch=&stage=sslCheck&filter=Submit Disappointingly, that turned out to be just because 10 and earlier didn't care what the error message said. -- Thomas Munro https://enterprisedb.com

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
On Wed, Mar 6, 2019 at 3:33 AM Tom Lane wrote: > Thomas Munro writes: > > Disappointingly, that turned out to be just because 10 and earlier > > didn't care what the error message said. > > That is, you can reproduce the failure on old branches? That lets > out a h

Re: Refactoring the checkpointer's fsync request queue

2019-03-05 Thread Thomas Munro
are added we don't leave them uninitialised. I like the syntax "FileTagData tag = {{0}}". (Unfortunately extra nesting required here because first member is a struct, and C99 doesn't allow us to use empty {} like C++, even though some versions of GCC accept it. Rats.) -- Thomas Munro https://enterprisedb.com

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
On Wed, Mar 6, 2019 at 6:07 AM Tom Lane wrote: > Thomas Munro writes: > > You can see that poll() already knew the other end had closed the > > socket. Since this is clearly timing... let's see, yeah, I can make > > it fail every time by adding sleep(1) before the com

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
On Wed, Mar 6, 2019 at 7:05 AM Thomas Munro wrote: > On Wed, Mar 6, 2019 at 6:07 AM Tom Lane wrote: > > Annoying. I'd be happier about writing code to fix this if I could > > reproduce it :-( > > Hmm. Note that eelpout only started doing it with OpenSSL 1.1.1. Ble

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
.7 ./test.py --client Sending A... 2 Sending B... 2 Sending C... 2 This is the server saying goodbye So... can anyone tell us what happens on Windows? (A secondary question might be what happens if the server and client are on different machines since I guess it could be different?) -- Thomas

Re: few more wait events to add to docs

2019-03-05 Thread Thomas Munro
e missing documentation. Hmm, yeah. I wish these were alphabetised, I wish there was an automated warning about this, I wish these tranches were declared a better way that by adding code in RegisterLWLockTranches(). -- Thomas Munro https://enterprisedb.com missing-tranches.patch Description: Binary data

Re: Rare SSL failures on eelpout

2019-03-05 Thread Thomas Munro
On Wed, Mar 6, 2019 at 4:07 PM Shawn Debnath wrote: > On Wed, Mar 06, 2019 at 11:13:31AM +1300, Thomas Munro wrote: > > So... can anyone tell us what happens on Windows? > C:\Users\Shawn Debnath\Desktop>c:\Python27\python.exe tmunro-ssl-test.py > --client > Sending

Re: Drop type "smgr"?

2019-03-06 Thread Thomas Munro
On Thu, Feb 28, 2019 at 7:02 PM Thomas Munro wrote: > The type smgr has only one value 'magnetic disk'. ~15 years ago it > also had a value 'main memory', and in Berkeley POSTGRES 4.2 there was > a third value 'sony jukebox'. Back then, all tables had an

Re: Refactoring the checkpointer's fsync request queue

2019-03-06 Thread Thomas Munro
On Wed, Mar 6, 2019 at 6:16 AM Shawn Debnath wrote: > On Tue, Mar 05, 2019 at 11:53:16AM -0500, Tom Lane wrote: > > Thomas Munro writes: > > > Why do we need to include fmgr.h in md.h? > > > > More generally, any massive increase in an include file's inclusi

Re: patch to allow disable of WAL recycling

2019-03-07 Thread Thomas Munro
it's for, rather than when to set it? To make clear that it's a performance setting, not a safety one. "Setting this option to off may increase performance on copy-on-write filesystems." -- Thomas Munro https://enterprisedb.com

Re: patch to allow disable of WAL recycling

2019-03-07 Thread Thomas Munro
On Fri, Mar 8, 2019 at 12:35 PM Jerry Jelinek wrote: > On Thu, Mar 7, 2019 at 3:09 PM Thomas Munro wrote: >> My understanding is that it's not really the COW-ness that makes it >> not necessary, it's the fact that fdatasync() doesn't do anything >> different

Re: dropdb --force

2019-03-08 Thread Thomas Munro
/* without "force" flag raise exception immediately, or after 5 minutes */ Normally we call it an "error", not an "exception". -- Thomas Munro https://enterprisedb.com

Re: Re: proposal: make NOTIFY list de-duplication optional

2019-03-08 Thread Thomas Munro
id NOTIFY/LISTEN/UNLISTEN commands NOTIFY notify_async2; NOTIFY notify_async2, ''; -- Thomas Munro https://enterprisedb.com

Re: I have some troubles to run test_shm_mq;

2019-03-11 Thread Thomas Munro
Try this first: cd src/test/modules/test_shm_mq make install -- Thomas Munro https://enterprisedb.com

Re: I have some troubles to run test_shm_mq;

2019-03-11 Thread Thomas Munro
ith exit code 1 Hmm, I don't know actually know why test_shm_mq_main() ends with proc_exit(1) instead of 0. It's possible that it was written when the meaning of bgworker exit codes was still being figured out, but I'm not sure... >> Works, thank you Thomas! I have spent m

Using condition variables to wait for checkpoints

2019-03-12 Thread Thomas Munro
ECKPOINT Time: 101.848 ms Patched: postgres=# checkpoint; CHECKPOINT Time: 1.851 ms -- Thomas Munro https://enterprisedb.com 0001-Use-condition-variables-to-wait-for-checkpoints.patch Description: Binary data

Re: Introduce timeout capability for ConditionVariableSleep

2019-03-12 Thread Thomas Munro
x27;re restarting the timeout clock every time through the loop without adjustment, and I think that's not a good choice: spurious wake-ups cause bonus waiting. -- Thomas Munro https://enterprisedb.com

Re: Refactoring the checkpointer's fsync request queue

2019-03-12 Thread Thomas Munro
s patch is to force the checkpointer to open and close the files every time which seems OK to me. I know Andres wants to make a pass through it too. [1] https://www.postgresql.org/message-id/CA%2BhUKGKN42jB%2BubCKru716HPtMbahdia39GwG5pLgWLMZ_y1ng%40mail.gmail.com -- Thomas Munro https://enterprisedb.com

Re: Using condition variables to wait for checkpoints

2019-03-13 Thread Thomas Munro
ted by ckpt_lck, and pushed. There are some other things like this in the tree (grepping for poll/pg_usleep loops finds examples in xlog.c, standby.c, ...). That might be worth looking into. -- Thomas Munro https://enterprisedb.com

Re: [HACKERS] SERIALIZABLE with parallel query

2019-03-14 Thread Thomas Munro
On Mon, Mar 4, 2019 at 10:17 AM Thomas Munro wrote: > On Thu, Oct 11, 2018 at 10:15 AM Kevin Grittner wrote: > > It applies and builds clean, it passed make world with cassert and TAP > > tests, and I can't see any remaining flaws. This is true both of just > > the 00

Re: Rare SSL failures on eelpout

2019-03-16 Thread Thomas Munro
On Wed, Mar 6, 2019 at 9:21 AM Tom Lane wrote: > Thomas Munro writes: > > Bleugh. I think this OpenSSL package might just be buggy on ARM. On > > x86, apparently the same version of OpenSSL and all other details of > > the test the same, I can see that SSL_connect() retu

Re: Rare SSL failures on eelpout

2019-03-17 Thread Thomas Munro
On Sun, Mar 17, 2019 at 2:00 AM Thomas Munro wrote: > On Wed, Mar 6, 2019 at 9:21 AM Tom Lane wrote: > > Yeah, I've still been unable to reproduce even with the sleep idea, > > so eelpout is definitely looking like a special snowflake from here. > > In any case, the

Re: Parallel query vs smart shutdown and Postmaster death

2019-03-17 Thread Thomas Munro
On Sun, Mar 17, 2019 at 5:53 PM Arseny Sher wrote: > Thomas Munro writes: > > > 1. Why does pmdie()'s SIGTERM case terminate parallel workers > > immediately? That breaks aborts running parallel queries, so they > > don't get to end their work normally. > &

Re: Rare SSL failures on eelpout

2019-03-18 Thread Thomas Munro
the question is: is there a way for send() to report an error, but recv() to report EAGAIN so we finish up blocked? I can't immediately see how that could happen. > My current feeling is that this is OK to put in HEAD but I think the > risk-reward ratio isn't very good for the back branches. Even with > an OpenSSL version where this makes a difference, the problematic > behavior is pretty hard to hit. So I'm a bit inclined to do nothing > in the back branches. Shouldn't we also back-patch the one-line change adding pqHandleSendFailure()? Then eelpout would be green for REL_11_STABLE, and any future buildfarm animals (probably underpowered ones with wonky scheduling) when they are eventually upgraded to OpenSSL 1.1.1. -- Thomas Munro https://enterprisedb.com

Re: Rare SSL failures on eelpout

2019-03-18 Thread Thomas Munro
On Tue, Mar 19, 2019 at 12:25 PM Thomas Munro wrote: > 2. Linux, FreeBSD and Darwin gave slightly different error sequences > when writing after the remote connection was closed (though I suspect > they'd behave much the same way for a connection to a remote host), > but all all

Re: Rare SSL failures on eelpout

2019-03-18 Thread Thomas Munro
On Tue, Mar 19, 2019 at 12:44 PM Tom Lane wrote: > Thomas Munro writes: > > Shouldn't we also back-patch the one-line change adding > > pqHandleSendFailure()? > > As I said before, I don't like that patch: at best it's an abuse of > pqHandleSendFailure, be

Re: DNS SRV support for LDAP authentication

2019-03-19 Thread Thomas Munro
On Sat, Feb 16, 2019 at 10:57 PM Thomas Munro wrote: > Yeah. This coding is ugly and StringInfo would be much nicer. > Thinking about that made me realise that the proposed SRV case should > also handle multiple SRV records by building a multi-URL string too > (instead of just taki

Re: Rare SSL failures on eelpout

2019-03-19 Thread Thomas Munro
ta is over. This plan looks good to me. -- Thomas Munro https://enterprisedb.com

Re: Re: A separate table level option to control compression

2019-03-20 Thread Shaun Thomas
on HOT updates. Still, those are just minor nitpicks, and I don't expect that to affect the quality of the patch implementation. Good show, gents! -- Shaun M Thomas - 2ndQuadrant PostgreSQL Training, Services and Support shaun.tho...@2ndquadrant.com | www.2ndQuadrant.com

MacPorts support for "extra" tests

2019-03-20 Thread Thomas Munro
a" test is ssl, but that was already working. -- Thomas Munro https://enterprisedb.com ldap-on-macports.patch Description: Binary data

Re: DNS SRV support for LDAP authentication

2019-03-20 Thread Thomas Munro
On Tue, Mar 19, 2019 at 9:01 PM Thomas Munro wrote: > I'd like to commit this soon. Done, after some more comment adjustments. Thanks Daniel and Graham for your feedback! -- Thomas Munro https://enterprisedb.com

Re: Usage of epoch in txid_current

2019-03-24 Thread Thomas Munro
On Mon, Feb 4, 2019 at 8:41 PM Andres Freund wrote: > On 2018-09-19 13:58:36 +1200, Thomas Munro wrote: > > > +/* > > + * Advance nextFullXid to the value after a given xid. The epoch is > > inferred. > > + * If lock_free_check is true, then the caller must

Re: Usage of epoch in txid_current

2019-03-25 Thread Thomas Munro
On Mon, Mar 25, 2019 at 5:01 PM Thomas Munro wrote: > New version attached. I'd like to commit this for PG12. Here is a follow-up sketch patch that shows FullTransactionId being used in the transaction stack, so you can call eg GetCurrentFullTransactionId(). A table access method c

Re: MacPorts support for "extra" tests

2019-03-25 Thread Thomas Munro
On Thu, Mar 21, 2019 at 4:39 PM Tom Lane wrote: > Thomas Munro writes: > > Peter E added some nice tests for LDAP and Kerberos, but they assume > > you have Homebrew when testing on a Mac. Here's a patch to make them > > work with MacPorts too (a competin

Re: Usage of epoch in txid_current

2019-03-25 Thread Thomas Munro
a Deathstation 9000 might allocate 8 billion more xids with all the necessary auto-vacuuming to allow that before scheduling you back onto the CPU. Admittedly, I haven't though about this very deeply at all, there may be a simple and correct way to do it. -- Thomas Munro https://enterprisedb.com

Re: Refactoring the checkpointer's fsync request queue

2019-03-25 Thread Thomas Munro
On Wed, Mar 13, 2019 at 2:27 PM Thomas Munro wrote: > [...] Aside from some refactoring which I > think looks good anyway and prepares for future patches, the main > effect of this patch is to force the checkpointer to open and close > the files every time which seems OK to me. I&#x

Re: Usage of epoch in txid_current

2019-03-26 Thread Thomas Munro
On Tue, Mar 26, 2019 at 12:58 PM Thomas Munro wrote: > ... I think you could probably reclaim that space by > using a more compact representation of vacuumFlags, overflowed, > delayChkpt, nxids (it's funny, the comment says "as tightly as > possible", which clearly

txid_status() off-by-one error

2019-03-26 Thread Thomas Munro
ch == now_epoch && xid > now_epoch_last_xid)) + || (xid_epoch == now_epoch && xid >= now_epoch_last_xid)) ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE), errmsg("transaction

Re: txid_status() off-by-one error

2019-03-27 Thread Thomas Munro
On Wed, Mar 27, 2019 at 7:55 PM Thomas Munro wrote: > If you keep asking for txid_status(txid_current() + 1) in new > transactions, you eventually hit: > > ERROR: could not access status of transaction 32768 > DETAIL: Could not read from file "pg_xact/" at

Re: Usage of epoch in txid_current

2019-03-27 Thread Thomas Munro
On Tue, Mar 26, 2019 at 12:58 PM Thomas Munro wrote: > On Tue, Mar 26, 2019 at 3:23 AM Heikki Linnakangas wrote: > > Looks good. I did some testing and proof-reading and made a few minor changes: * I tidied up the code that serialises transaction state. It was already hammering r

Re: Re: A separate table level option to control compression

2019-03-27 Thread Shaun Thomas
concrete reason not to abuse the parameter. :shipit: Vaya con Queso -- Shaun M Thomas - 2ndQuadrant PostgreSQL Training, Services and Support shaun.tho...@2ndquadrant.com | www.2ndQuadrant.com

Re: Usage of epoch in txid_current

2019-03-27 Thread Thomas Munro
o weird new problem in heap_compute_xid_horizon_for_tuples() which I assumed to be somehow my fault due to the mention of xid horizons, but I eventually realised that master was broken on that machine and followed that up elsewhere. Phew. Thanks for the reviews! Pushed. -- Thomas Munro https://enterprisedb.com

Re: patch to allow disable of WAL recycling

2019-03-28 Thread Thomas Munro
On Fri, Mar 29, 2019 at 8:59 AM Robert Haas wrote: > On Tue, Mar 26, 2019 at 3:24 PM Jerry Jelinek > wrote: > > The latest patch is rebased, builds clean, and passes some basic testing. > > Please let me know if there is anything else I could do on this. > > I agree wit

Re: patch to allow disable of WAL recycling

2019-03-28 Thread Thomas Munro
On Fri, Mar 29, 2019 at 10:47 AM Thomas Munro wrote: > On Fri, Mar 29, 2019 at 8:59 AM Robert Haas wrote: > > On Tue, Mar 26, 2019 at 3:24 PM Jerry Jelinek > > wrote: > > > The latest patch is rebased, builds clean, and passes some basic testing. > > > Please

Re: pg_ctl on windows can't open postmaster.pid: Permission denied

2019-03-29 Thread Thomas Munro
On Wed, Dec 13, 2017 at 5:01 PM Thomas Munro wrote: > On Wed, Dec 13, 2017 at 4:24 PM, Andres Freund wrote: > > Hi, > > > > Buildfarm animal thrips just failed with a curious error: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thrips&a

Re: [PATCH] Incremental sort (was: PoC: Partial sort)

2019-03-29 Thread Shaun Thomas
know so someone else can pick it up. -- Shaun M Thomas - 2ndQuadrant PostgreSQL Training, Services and Support shaun.tho...@2ndquadrant.com | www.2ndQuadrant.com

Removing a few more lseek() calls

2019-03-30 Thread Thomas Munro
Hello, Patch 0001 gets rid of the unconditional lseek() calls for SLRU I/O, as a small follow-up to commit c24dcd0c. Patch 0002 gets rid of a few places that usually do a good job of avoiding lseek() calls while reading and writing WAL, but it seems better to have no code at all. -- Thomas

Re: pgsql: Compute XID horizon for page level index vacuum on primary.

2019-03-30 Thread Thomas Munro
no IO concurrency at all (a single old school spindle), just because the OS will chew on that queue in the background while we're also doing stuff, which is probably what that "+ 10" is expressing. But that seems to apply to bitmap heapscan too, doesn't it? -- Thomas Munro https://enterprisedb.com

DWIM mode for psql

2019-03-31 Thread Thomas Munro
override errors automatically as follows, though that is proving trickier to get working. Example: postgres=# SUDO DROP TABLE PG_DATABASS; NO CARRIER -- Thomas Munro https://enterprisedb.com 0001-Add-do-what-I-mean-mode-to-psql.patch Description: Binary data

Re: C_C_A animal on HEAD gets stuck in initdb

2019-04-01 Thread Thomas Munro
LueN9rofAA3HHSYikW-Zw%40mail.gmail.com I see that its first failure was after commit 558a9165e0 (along with others). -- Thomas Munro https://enterprisedb.com

Re: errbacktrace

2019-07-22 Thread Thomas Munro
anups, documentation, and automatic back traces from assertion failures. Now works out of the box on FreeBSD. The assertion thing is a nice touch. I wonder if it'd make sense to have a log_min_backtrace GUC that you could set to error/fatal/panicwhatever (perhaps in a later patch). -- Thomas Munro https://enterprisedb.com

Re: SegFault on 9.6.14

2019-07-22 Thread Thomas Munro
On Fri, Jul 19, 2019 at 3:00 PM Amit Kapila wrote: > On Thu, Jul 18, 2019 at 7:15 PM Tom Lane wrote: > > Thomas Munro writes: > > > Hmm, so something like a new argument "bool final" added to the > > > ExecXXXShutdown() functions, which receives false in

Re: Introduce timeout capability for ConditionVariableSleep

2019-07-22 Thread Thomas Munro
On Tue, Jul 16, 2019 at 1:11 AM Robert Haas wrote: > On Fri, Jul 12, 2019 at 11:03 PM Thomas Munro wrote: > > I pushed this too. It's a separate commit, because I think there is > > at least a theoretical argument that it should be back-patched. I'm > > no

Re: stress test for parallel workers

2019-07-23 Thread Thomas Munro
ell_201907_unique_idx ON > child.unused0_huawei_umts_nodeb_locell_201907 USING btree ... > ... I've set > max_parallel_workers_per_gather=0, ... Just by the way, parallelism in CREATE INDEX is controlled by max_parallel_maintenance_workers, not max_parallel_workers_per_gather. -- Thomas Munro https://enterprisedb.com

Re: stress test for parallel workers

2019-07-23 Thread Thomas Munro
aster dies while they are waiting for workers; most other places (probably every other backend in your cluster) just quietly exit. That tells us something about what's happening, but on its own doesn't tell us that parallelism plays an important role in the failure mode. -- Thomas Munro https://enterprisedb.com

Re: stress test for parallel workers

2019-07-23 Thread Thomas Munro
On Wed, Jul 24, 2019 at 10:03 AM Thomas Munro wrote: > > edata = > If you have that core, it might be interesting to go to frame 2 and > print *edata or edata->saved_errno. ... Rats. We already saw that it's optimised out so unless we can find that somewhere else

Re: stress test for parallel workers

2019-07-23 Thread Thomas Munro
On Wed, Jul 24, 2019 at 10:42 AM Justin Pryzby wrote: > On Wed, Jul 24, 2019 at 10:03:25AM +1200, Thomas Munro wrote: > > On Wed, Jul 24, 2019 at 5:42 AM Justin Pryzby wrote: > > > #2 0x0085ddff in errfinish (dummy=) at > > > elog.c:555 > > >

Re: stress test for parallel workers

2019-07-23 Thread Thomas Munro
y a panic if a checkpoint happens to run exactly at a moment where the create index hasn't yet written the byte that breaks the camel's back, but the checkpoint pushes it over edge in one of these places where it panics on failure. -- Thomas Munro https://enterprisedb.com

Re: stress test for parallel workers

2019-07-23 Thread Thomas Munro
On Wed, Jul 24, 2019 at 10:11 AM Tom Lane wrote: > Thomas Munro writes: > > *I suspect that the only thing implicating parallelism in this failure > > is that parallel leaders happen to print out that message if the > > postmaster dies while they are waiting for worke

Re: Parallel Append subplan order instability on aye-aye

2019-07-23 Thread Thomas Munro
animals on one kernel, and need to increase net.core.rmem_max and net.core.rmem_default (or maybe the write side variants, or both, or something like that). -- Thomas Munro https://enterprisedb.com

Re: On the stability of TAP tests for LDAP

2019-07-23 Thread Thomas Munro
should do that too. I don't know how to write Perl but I'll try... -- Thomas Munro https://enterprisedb.com

Re: On the stability of TAP tests for LDAP

2019-07-23 Thread Thomas Munro
On Wed, Jul 24, 2019 at 3:52 PM Thomas Munro wrote: > I guess we should do that too. I don't know how to write Perl but I'll try... Does this look about right? -- Thomas Munro https://enterprisedb.com wait-for-slapd.patch Description: Binary data

Re: On the stability of TAP tests for LDAP

2019-07-23 Thread Thomas Munro
, and the > sleep phases make use of Time::HiRes::usleep. Thanks, here's v2. -- Thomas Munro https://enterprisedb.com wait-for-slapd-v2.patch Description: Binary data

Re: On the stability of TAP tests for LDAP

2019-07-24 Thread Thomas Munro
version mentioned in tools/pgindent/README). Fixed. -- Thomas Munro https://enterprisedb.com wait-for-slapd-v3.patch Description: Binary data

Re: Should we add xid_current() or a int8->xid cast?

2019-07-24 Thread Thomas Munro
is unsigned. If we had a SQL type for 64 bit xids, it should be convertible to xid, and the reverse conversion should require a more complicated dance. Of course we can't casually change txid_current() without annoying people who are using it, so perhaps if we invent a new SQL

Re: Should we add xid_current() or a int8->xid cast?

2019-07-24 Thread Thomas Munro
the past > >> and it looked like a mess. > > > I assume Thomas was thinking more of another bespoke type like xid, just > > wider. There's some notational advantage in not being able to > > immediately do math etc on xids. > > Well, we could invent an x

Re: double free in ExecHashJoin, 9.6.12

2019-07-24 Thread Thomas Munro
postgresql-9.6 and postgresql-9.6-dbg installed you could run "gdb /usr/lib/postgresql/9.6/bin/postgres" and then "disassemble ExecHashJoin". The code at "<+1442>" (0x5a2) is presumably calling free or some other libc thing (though I'm surprised not to see an intervening palloc thing). -- Thomas Munro https://enterprisedb.com

Re: POC: Cleaning up orphaned files using undo logs

2019-07-24 Thread Thomas Munro
until a checkpoint says they existed at that point in the WAL stream. Does this make sense? BTW we could probably use posix_fallocate() instead of writing zeroes; I think Andres mentioned that recently. I see also that someone tried that for WAL and it got reverted back in 2013 (commit b1892aaeaaf34d8d1637221fc1cbda82ac3fcd71, I didn't try to hunt down the discussion). [1] https://lkml.org/lkml/2012/9/3/83 [2] https://github.com/torvalds/linux/commit/b71fc079b5d8f42b2a52743c8d2f1d35d655b1c5 -- Thomas Munro https://enterprisedb.com

Re: On the stability of TAP tests for LDAP

2019-07-25 Thread Thomas Munro
! Pushed, thanks. -- Thomas Munro https://enterprisedb.com

Re: SegFault on 9.6.14

2019-07-26 Thread Thomas Munro
laint[2]), but I ran out of time for debugging adventures today. [1] https://www.postgresql.org/message-id/CA%2BhUKGJyqDp9FZSHLTjiNMcz-c6%3DRdStB%2BUjVZsR8wfHnJXy8Q%40mail.gmail.com [2] https://www.postgresql.org/message-id/flat/86137f17-1dfb-42f9-7421-82fd786b04a1%40anayrat.info -- Thomas Munro https://enterprisedb.com

Re: LLVM compile failing in seawasp

2019-07-26 Thread Thomas Munro
reconsider. FYI This is now on LLVM's release_90 branch, due out on August 28. -- Thomas Munro https://enterprisedb.com

Re: LLVM compile failing in seawasp

2019-07-27 Thread Thomas Munro
ot;, > less than 400 hundred instances, and backpatch to all supported > versions:-( I would just #undef Min for our small number of .cpp files that include LLVM headers. It's not as though you need it in C++, which has std::min() from . -- Thomas Munro https://enterprisedb.com

Re: LLVM compile failing in seawasp

2019-07-27 Thread Thomas Munro
On Sat, Jul 27, 2019 at 7:12 PM Thomas Munro wrote: > On Sat, Jul 27, 2019 at 7:06 PM Fabien COELHO wrote: > > Maybe we should consider doing an explicit bug report, but I would not bet > > that they are going to fold… or fixing the issue pg side, eg "pg_Min", > >

Re: Built-in connection pooler

2019-07-27 Thread Thomas Munro
ake check. You can see it on Linux with ./configure ... CFLAGS="-DWAIT_USE_POLL". -- Thomas Munro https://enterprisedb.com

Re: LLVM compile failing in seawasp

2019-07-28 Thread Thomas Munro
ing ready to tell us if something else breaks. Personally, I don't see any reason why should entertain a request to change their variable names to avoid our short common word macros that aren't even all-caps, but if someone asks them and they agree to do that before the final 9.0 rel

Re: LLVM compile failing in seawasp

2019-07-28 Thread Thomas Munro
On Mon, Jul 29, 2019 at 9:55 AM Tom Lane wrote: > Thomas Munro writes: > > Let's just commit the #undef so that seawasp is green and back to > > being ready to tell us if something else breaks. > > +1. I was afraid that working around this would be impossibly > pai

<    3   4   5   6   7   8   9   10   11   12   >