Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-25 Thread Thomas Munro
000\210\035\000H!\027\000\210\216\000T\vB\017\304\t\027\000\210\a\000\206\b\033\030.\"", > '\000' ..., force_align_d = 0, force_align_i64 = 0}} > > (gdb) p innerFile->files[0] > $8 = 2397 Hmm. That all looks pretty sane from here, but tells us nothing about whether it was already freed and if so where. -- Thomas Munro https://enterprisedb.com

Re: pg11.5: ExecHashJoinNewBatch: glibc detected...double free or corruption (!prev)

2019-08-25 Thread Thomas Munro
On Mon, Aug 26, 2019 at 1:44 PM Justin Pryzby wrote: > On Mon, Aug 26, 2019 at 01:09:19PM +1200, Thomas Munro wrote: > > On Sun, Aug 25, 2019 at 3:15 PM Peter Geoghegan wrote: > > > I was reminded of this issue from last year, which also appeared to > > > involve BufFi

Re: old_snapshot_threshold vs indexes

2019-08-26 Thread Thomas Munro
we figure out what to do about the other problem and what needs to be back-patched. -- Thomas Munro https://enterprisedb.com

Re: old_snapshot_threshold vs indexes

2019-08-26 Thread Thomas Munro
don't have. If we add support for that in some future release, we can figure out how to do it properly then, no? -- Thomas Munro https://enterprisedb.com

gharial segfaulting on REL_12_STABLE only

2019-08-26 Thread Thomas Munro
d371e2, the only non-doc commit listed on the first failure, could have anything to do with that. -- Thomas Munro https://enterprisedb.com

Re: gharial segfaulting on REL_12_STABLE only

2019-08-26 Thread Thomas Munro
On Tue, Aug 27, 2019 at 1:48 PM Tom Lane wrote: > A stack trace would likely be really useful right about now. Yeah. Looking into how to get that. -- Thomas Munro https://enterprisedb.com

Re: old_snapshot_threshold vs indexes

2019-08-26 Thread Thomas Munro
On Tue, Aug 27, 2019 at 1:54 PM Tom Lane wrote: > Thomas Munro writes: > > OK I started writing a patch and realised there were a few ugly > > problems that I was about to report here... but now I wonder if, based > > on the comment for RelationHasUnloggedIndex(), we should

Re: PostgreSQL and Real Application Testing (RAT)

2019-08-27 Thread Thomas Kellerer
ROS Didier schrieb am 27.08.2019 um 12:47: > In my business, one of the things blocking the migration from Oracle > to PostgreSQL is not having the equivalent of Oracle Real Application > Testing . > > This product captures a charge in production and replay it in a test > environment. > > this allo

Re: old_snapshot_threshold vs indexes

2019-08-28 Thread Thomas Munro
workloads like the one generated by test-indexscan.sql (from my earlier message) from many backends at the same time on a large system. That's just an observation I'm leaving here, I'm not planning to chase that any further for now. -- Thomas Munro https://enterprisedb.com

Re: no mailing list hits in google

2019-08-28 Thread Thomas Kellerer
Merlin Moncure schrieb am 28.08.2019 um 18:22: My test case here is the query: pgsql-hackers That search term is the first hit on DuckDuckGo: https://duckduckgo.com/?q=pgsql-hackers+ExecHashJoinNewBatch&t=h_&ia=web Searching for "postgres ExecHashJoinNewBatch" returns that ot position 4 https:

Re: RFC: seccomp-bpf support

2019-08-28 Thread Thomas Munro
p whitelists currently cause PostgreSQL to panic for users of Docker, Nspawn etc on POWER and ARM because of that. That's a bug being fixed elsewhere, but it reveals another thing to be careful of if you're trying to build your own whitelist by guessing what libc needs to call. [1] https://ww

Re: gharial segfaulting on REL_12_STABLE only

2019-08-28 Thread Thomas Munro
On Tue, Aug 27, 2019 at 2:09 PM Thomas Munro wrote: > On Tue, Aug 27, 2019 at 1:48 PM Tom Lane wrote: > > A stack trace would likely be really useful right about now. > > Yeah. Looking into how to get that. Erm. I heard the system was in a very unhappy state and couldn&#

Re: Consolidate 'unique array values' logic into a reusable function?

2019-08-29 Thread Thomas Munro
Hello, I'm reviving a thread from 2016, because I wanted this thing again today. Tom Lane wrote: > Thomas Munro writes: > > Here's a sketch patch that creates a function array_unique which takes > > the same arguments as qsort or qsort_arg and returns the new length. &

Re: Should we add xid_current() or a int8->xid cast?

2019-08-31 Thread Thomas Munro
On Fri, Aug 2, 2019 at 10:42 PM Thomas Munro wrote: > On Thu, Jul 25, 2019 at 1:11 PM Thomas Munro wrote: > > On Thu, Jul 25, 2019 at 12:42 PM Tom Lane wrote: > > > Andres Freund writes: > > > > On 2019-07-24 20:34:39 -0400, Tom Lane wrote: > > > >>

Re: XPRS

2019-09-01 Thread Thomas Munro
sort of fudging together of ideas from conversations with Kevin Grittner (who talked about admission control a few years back), Peter Geoghegan (who mentioned opportunistically asking for more), and things I've heard of on SQL Server ("memory grants"). I think it would provide some r

moonjelly vs tsearch

2019-09-01 Thread Thomas Munro
Hi, News from Fabien's bleeding edge compiler zoo: apparently GCC 10 20190831 thinks the tsearch test should produce different output than 20190824 did. It looks like the parsing of question marks change... I can't wait for next week's installment. -- Thomas Munro https://enterprisedb.com

Re: POC: Cleaning up orphaned files using undo logs

2019-09-01 Thread Thomas Munro
t; > I found that we don't emit an WAL record when we update the > slot->meta.status as UNDO_LOG_STATUS_FULL. If we don't that, after > crash recovery, some new transaction may use that undo log which is > wrong, IMHO. Am I missing something? Thanks, right, t

Re: XPRS

2019-09-02 Thread Thomas Munro
opose a concrete rule, but maybe something based on session_max_work_mem / njoins, with various fudge factors, and some accounting for parallel workers; it's probably good to low-ball it and rely on session_extra_work_mem). Yeah, I think you'd want to be able to set session_XXX on databases and roles so that you can say your regular users can't eat more than 10% of memory each, but a big reporting thing is allowed more. -- Thomas Munro https://enterprisedb.com

Re: Unix-domain socket support on Windows

2019-09-02 Thread Thomas Munro
ct/postgresql-cfbot/postgresql/build/1.0.55034?fullLog=true -- Thomas Munro https://enterprisedb.com

Re: pgbench - rework variable management

2019-09-02 Thread Thomas Munro
On Wed, Aug 14, 2019 at 3:54 AM Fabien COELHO wrote: > Some windows-specific hacks are note tested. Somehow this macro hackery has upset the Windows socket headers: https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.55019 -- Thomas Munro https://enterprisedb.com

Re: SIGQUIT on archiver child processes maybe not such a hot idea?

2019-09-02 Thread Thomas Munro
t all sounds like a lot of work though, and it might be easier to just make an exception and use SIGTERM to shut down the archiver, as I think Tom was suggesting. Unfortunately we have the same problem elsewhere, where we use popen(). I just wrote a C program that does just "sleep(60)",

Re: pgbench - rework variable management

2019-09-02 Thread Thomas Munro
7;ll build and test every time you push a new tweak. Takes a few minutes to get the answer each time you try something, but I have managed to get things working on Windows that way. -- Thomas Munro https://enterprisedb.com

Re: Unimpressed with pg_attribute_always_inline

2018-01-01 Thread Thomas Munro
debugger. > > Therefore, I think that pg_attribute_always_inline is not merely > useless but actively bad, and should be removed. My intention was to make sure it really did get inlined at higher optimisation levels even though the compiler wouldn't otherwise choose to do that in a couple of special cases, not to force inlining even at -O0. Not sure how to achieve the first of those things without the second. I wonder if there is a better way. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] LDAPS

2018-01-02 Thread Thomas Munro
her "note" call to introduce the LDAPS > tests section. I realised that I should probably also include a new test for ldaptls=1, so that we can see that both ways of doing TLS are working. I added that test, and added a "note" to label the whole section as "TLS"

Re: Ensuring hash tuples are properly maxaligned

2018-01-02 Thread Thomas Munro
e *narrower* if you don't have atomics or ask for it with USE_SMALL_DSA_POINTER -- Thomas Munro http://www.enterprisedb.com

Re: TupleDescCopy doesn't clear atthasdef, attnotnull, attidentity

2018-01-03 Thread Thomas Munro
On Thu, Jan 4, 2018 at 12:01 PM, Tom Lane wrote: > Thomas Munro writes: >> Andrew Gierth complained off-list that TupleDescCopy() doesn't clear >> atthasdef. Yeah, that's an oversight. The function is new in commit >> cc5f81366c36 and was written by me to support

Re: TupleDescCopy doesn't clear atthasdef, attnotnull, attidentity

2018-01-03 Thread Thomas Munro
On Thu, Jan 4, 2018 at 12:29 PM, Thomas Munro wrote: > If there is any system where sizeof(FormData_pg_attribute) != > (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid)), won't > load_relcache_init_file() get upset? Oh, I see it would just go to > read_failed and the

Re: Unimpressed with pg_attribute_always_inline

2018-01-04 Thread Thomas Munro
On Tue, Jan 2, 2018 at 4:58 PM, Thomas Munro wrote: > On Tue, Jan 2, 2018 at 4:17 PM, Tom Lane wrote: >> gaur | nodeHashjoin.c:167: warning: `always_inline' attribute >> directive ignored >> mastodon | .\src\backend\executor\nodeHashjoin.c(165): warning C4141

Re: Condition variable live lock

2018-01-04 Thread Thomas Munro
leep(), and it won't miss any signals that it can't afford to miss because the first call to ConditionVariableSleep() will return immediately so the caller will recheck its condition. -- Thomas Munro http://www.enterprisedb.com

Re: Condition variable live lock

2018-01-04 Thread Thomas Munro
ed anyone. Maybe the only reason we have that return code is so that ConditionVariableBroadcast() can use it the way it does in master... An alternative would be to mark sentinel entries somehow so that signallers can detect them and signal again, but that's not backpatchable. -- Thomas Munro http://www.enterprisedb.com

Re: Condition variable live lock

2018-01-05 Thread Thomas Munro
d pop the first entry at the same time, so that we're not adding an extra spinlock acquire/release? -- Thomas Munro http://www.enterprisedb.com

Re: Condition variable live lock

2018-01-05 Thread Thomas Munro
g I'd still like to see some effort put into reducing that. Will do. -- Thomas Munro http://www.enterprisedb.com

Intermittent crashes on brolga in join.sql test

2018-01-07 Thread Thomas Munro
attached script? I don't know anything about Cygwin myself but I see that it's possible to get a core file that gdb can load with "dumper": https://stackoverflow.com/questions/320001/using-a-stackdump-from-cygwin-executable https://cygwin.com/cygwin-ug-net/dumper.h

Re: heads up: Fix for intel hardware bug will lead to performance regressions

2018-01-07 Thread Thomas Munro
ontrolled by arbitrary buffer sizes? Examples: 8kB BufFile buffers and 128kB replication stream buffers. Just an idea, not sure if it's worth looking into; maybe we already spend enough time filling those buffers that a 50% syscall markup won't hurt. -- Thomas Munro http://www.enterprisedb.com

Re: Condition variable live lock

2018-01-07 Thread Thomas Munro
; so why not just get rid of it? > > Concretely, as per attached. +1 for the idea. Haven't looked at the code yet but I'll review this and the proclist patch shortly. -- Thomas Munro http://www.enterprisedb.com

Re: Condition variable live lock

2018-01-07 Thread Thomas Munro
It's okay to do this > * because whenever control does return to the other test-and-sleep loop, > * its ConditionVariableSleep call will just re-establish that sleep as > * the prepared one. > */ +1 [1] https://www.postgresql.org/message-id/flat/CAEepm%3D0qQ6DO-u%3D25ny5EJAUbWeHbAQgJj1UJFAL1NWJNxC%2Bgg%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com

Re: heads up: Fix for intel hardware bug will lead to performance regressions

2018-01-07 Thread Thomas Munro
On Mon, Jan 8, 2018 at 2:38 PM, Thomas Munro wrote: > On Fri, Jan 5, 2018 at 6:28 AM, Robert Haas wrote: >> On Tue, Jan 2, 2018 at 5:23 PM, Andres Freund wrote: >>> Note that real-world scenarios probably will see somewhat smaller >>> impact, as this was measured o

Re: Condition variable live lock

2018-01-08 Thread Thomas Munro
On Mon, Jan 8, 2018 at 5:25 PM, Thomas Munro wrote: > On Sun, Jan 7, 2018 at 10:43 AM, Tom Lane wrote: >> Actually ... perhaps a better design would be to have >> ConditionVariable[PrepareTo]Sleep auto-cancel any prepared sleep for >> a different condition variable, analog

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-09 Thread Thomas Munro
restriction within buffile.c > ever was removed, everything would work fine. +1 > * Added new call to BufFileExportShared() when freezing tape in logtape.c. +1 > * Polished the stuff that is added to buffile.c. Mostly comments that > clarify its reason for existing. Also added Assert()s

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-10 Thread Thomas Munro
On Thu, Jan 11, 2018 at 11:42 AM, Peter Geoghegan wrote: > "Peter Geoghegan, Rushabh Lathia" seems right. Thomas did write a very > small amount of the actual code, but I think it was more of a review > thing (he is already credited as a reviewer). +1 --

Re: [HACKERS] Planning counters in pg_stat_statements

2018-01-11 Thread Thomas Munro
Total time spent executing the statement, in milliseconds ... and likewise for the min, max, avg, stddev columns. > + > /* > * PlannedStmt node > * > > Extra change. Fixed. Thanks! -- Thomas Munro http://www.enterprisedb.com pg-stat-statements-planning-v2.patch Description: Binary data

Re: add queryEnv to ExplainOneQuery_hook

2018-01-11 Thread Thomas Munro
re relations). [1] https://www.postgresql.org/message-id/20161113222804.o2uw7c2fish67iej%40alap3.anarazel.de -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2018-01-11 Thread Thomas Munro
Tom Lane wrote: > Perhaps serialize the contents into an array in DSM, then rebuild a hash > table from that in the worker. Robert might have a better idea though. I'd happily volunteer to write or review a patch to do that. Is there a rebase of the stuff that got reverted, to build on? -- Thomas Munro http://www.enterprisedb.com

Re: Doc tweak for huge_pages?

2018-01-11 Thread Thomas Munro
On Tue, Jan 9, 2018 at 6:24 AM, Catalin Iacob wrote: > On Fri, Dec 1, 2017 at 10:09 PM, Thomas Munro > wrote: >>> On 11/30/17 23:35, Thomas Munro wrote: >>>> Hmm. Yeah, it does, but apparently it's not so transparent. So if we >>>> mention that we&

Re: Jsonb transform for pl/python

2018-01-11 Thread Thomas Munro
r error : Entity 'datatype' not defined &datatype; ^ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Cached plans and statement generalization

2018-01-11 Thread Thomas Munro
defined reference to `PortalGetHeapMemory' That's because commit 0f7c49e85518dd846ccd0a044d49a922b9132983 killed PortalGetHeapMemory. Looks like it needs to be replaced with portal->portalContext. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Cache lookup errors with functions manipulation object addresses

2018-01-11 Thread Thomas Munro
unc.sgml:20721: parser error : chunk is not well balanced postgres.sgml:105: parser error : Failure to process entity func &func; ^ postgres.sgml:105: parser error : Entity 'func' not defined &func; ^ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-01-11 Thread Thomas Munro
Failure to process entity reference &reference; ^ postgres.sgml:231: parser error : Entity 'reference' not defined &reference; ^ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Support to COMMENT ON DATABASE CURRENT_DATABASE

2018-01-11 Thread Thomas Munro
here: gram.y:1160:19: error: incompatible pointer types assigning to 'char *' from 'Node *' (aka 'struct Node *') [-Werror,-Wincompatible-pointer-types] { (yyval.str) = (yyvsp[0].node); } ^ ~~~ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2018-01-11 Thread Thomas Munro
Dbs(NULL, false); ^ In file included from startup.c:21:0: describe.h:68:13: note: declared here extern bool listAllDbs(const char *pattern, bool verbose, sortby_type sortby, bool sort_desc); ^ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] postgres_fdw bug in 9.6

2018-01-11 Thread Thomas Munro
he new fields in > JoinPathExtraData to the end of that struct. FYI this doesn't compile anymore, because initial_cost_hashjoin() and create_hashjoin_path() changed in master. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Bug in to_timestamp().

2018-01-11 Thread Thomas Munro
imestamp('2011-12-18 11:38 -05:20', '-MM-DD HH12:MI TZH:TZM'); to_timestamp -- ! Sat Dec 17 22:18:00 2011 PST (1 row) SELECT to_timestamp('2011-12-18 11:38 20', '-MM-DD HH12:MI TZM'); == -- Thomas Munro http://www.enterprisedb.com

Re: Add default role 'pg_access_server_files'

2018-01-11 Thread Thomas Munro
ILED Because: ! ERROR: only superuser can change options of a file_fdw foreign table ... ! ERROR: only superuser or a member of the pg_access_server_files role can change options of a file_fdw foreign table -- Thomas Munro http://www.enterprisedb.com

Re: Transform for pl/perl

2018-01-11 Thread Thomas Munro
ECT testSVToJsonb2('1E+131071'); ! ERROR: invalid input syntax for type numeric: "inf" CONTEXT: PL/Perl function "testsvtojsonb2" SELECT testSVToJsonb2('-1'); testsvtojsonb2 == -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] [BUGS] BUG #14825: enum type: unsafe use?

2018-01-11 Thread Thomas Munro
On Fri, Jan 12, 2018 at 4:19 PM, Robert Haas wrote: > On Thu, Jan 11, 2018 at 6:01 PM, Thomas Munro > wrote: >> [ the data isn't session lifetime ] >> >> So I agree with Tom's suggestion: >> >> On Wed, Oct 4, 2017 at 2:29 PM, Tom Lane wrote: >&

Re: [HACKERS] Crash on promotion when recovery.conf is renamed

2018-01-13 Thread Thomas Munro
On Fri, Jan 12, 2018 at 4:35 PM, Stephen Frost wrote: > Appears to compile and pass make check-world still (not sure why Thomas' > bot currently thinks the build fails, since it worked here..). It looks good now. There was a brief window when all Travis CI builds said this whi

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-18 Thread Thomas Munro
occur to me that that was kind of weird, and I >>> struggled with it on my own for a little while, I ultimately agreed >>> with Thomas that it added something to have ltsConcatWorkerTapes() >>> call some buffile function in every iteration of its loop. >>> (Buf

Re: [HACKERS] Planning counters in pg_stat_statements

2018-01-18 Thread Thomas Munro
t seems a bit strange to have to go through the locking twice, but I don't have a better idea. First attempt seems to be working. I'll post an updated patch in a couple of days when I'm back from travelling and have a tidier version with a new GUC and have thought about the nested_level question. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-19 Thread Thomas Munro
ut I'm not entirely sure.Rather > than making an executive decision immediately, I'd like to wait a few > days to give others a chance to comment. I am hoping that we might get > some other opinions, especially from Thomas who implemented > parallel_leader_participation, or

Re: [HACKERS] Supporting huge pages on Windows

2018-01-21 Thread Thomas Munro
Control (UAC) must be disabled. When the UAC is enabled, the normal +command prompt revokes the user right Lock Pages in Memory when started. Is that first sentence missing the word "and" after "administrator"? -- Thomas Munro http://www.enterprisedb.com

Re: Doc tweak for huge_pages?

2018-01-21 Thread Thomas Munro
On Fri, Jan 12, 2018 at 1:12 PM, Thomas Munro wrote: > On Tue, Jan 9, 2018 at 6:24 AM, Catalin Iacob wrote: >> So I tried to redo the second paragraph and ended up with the >> attached. Rationale for the changes: >> * changed "this feature" to "explicitly r

Re: Doc tweak for huge_pages?

2018-01-21 Thread Thomas Munro
On Mon, Jan 22, 2018 at 6:30 PM, Justin Pryzby wrote: > On Mon, Jan 22, 2018 at 03:54:26PM +1300, Thomas Munro wrote: >> On Fri, Jan 12, 2018 at 1:12 PM, Thomas Munro >> wrote: >> > On Tue, Jan 9, 2018 at 6:24 AM, Catalin Iacob >> > wrote: >> > I d

Re: [HACKERS] Adding column_constraint description in ALTER TABLE synopsis

2018-01-22 Thread Thomas Munro
y line 6 and synopsis ^ ref/alter_table.sgml:1534: parser error : chunk is not well balanced ^ ref/alter_table.sgml:1534: parser error : chunk is not well balanced ^ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Adding column_constraint description in ALTER TABLE synopsis

2018-01-22 Thread Thomas Munro
On Tue, Jan 23, 2018 at 12:41 PM, Thomas Munro wrote: > On Mon, Jan 15, 2018 at 2:32 PM, Stephen Frost wrote: >> If someone else would like to review it, that'd be great, otherwise I'll >> probably get it committed soon. > > FYI the v2 doesn't build: > &

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
eiling; that would be awful.) (After talking this through with Robert off-list). Right, the CHECK_FOR_INTERRUPTS() in ConditionVariableSleep() handles errors from parallel workers. There is no problem here. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Planning counters in pg_stat_statements

2018-01-23 Thread Thomas Munro
On Fri, Jan 19, 2018 at 12:14 AM, Thomas Munro wrote: > On Sat, Jan 13, 2018 at 2:16 PM, Tom Lane wrote: >> What we could/should do instead, I think, is have pgss_planner_hook >> make its own pgss_store() call to log the planning time results >> (which would mean we

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Thomas Munro
fixed a warning from antique GCC which didn't understand this magic. 2. You also defined it as plain old "inline" for unknown compilers. Perhaps we should combine these? -- Thomas Munro http://www.enterprisedb.com

Re: pgsql: Add parallel-aware hash joins.

2018-01-23 Thread Thomas Munro
test case was excessively expensive for > what it covered, and I'm now thinking the same about hash joins. Does join-test-shrink.patch (from my earlier message) help much, on prairiedog? It cut "check" time

Re: [PATCH] fix for C4141 warning on MSVC

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 1:42 PM, Tom Lane wrote: > Thomas Munro writes: >> On Wed, Jan 24, 2018 at 1:16 PM, Michail Nikolaev >> wrote: >>> Just very small fix for C4141 warning > >> Thanks. This is similar to the fix I proposed over here: >> https

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Thomas Munro
0 (1 row) postgres=# select count(*) from foox; ERROR: lost connection to parallel worker postgres=# select count(*) from foox; ERROR: lost connection to parallel worker postgres=# select count(*) from foox; ERROR: lost connection to parallel worker postgres=# select count(*) from foox; ERROR: lost connection to parallel worker -- Thomas Munro http://www.enterprisedb.com chaos-monkey-fork-process.patch Description: Binary data

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-23 Thread Thomas Munro
m to attach you can detect unexpected dead workers (via shm_mq_receive), at the cost of having the leader wasting time waiting around for forked processes to say hello when it could instead be sorting tuples. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
Wait() to hang forever. If you attach and raise an error, the leader will receive an error message via CFI() and will then raise an error itself and terminate all workers during cleanup. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-23 Thread Thomas Munro
On Wed, Jan 24, 2018 at 6:43 PM, Amit Kapila wrote: > On Wed, Jan 24, 2018 at 10:36 AM, Thomas Munro > wrote: >> On Wed, Jan 24, 2018 at 5:59 PM, Amit Kapila wrote: >>>>> I am going to repeat my previous suggest that we use a Barrier here. >>>>> Give

Re: Help needed in using 'on_dsm_detach' callback

2018-01-24 Thread Thomas Munro
h some free shared space, and in this capacity they can be used to own a DSA area (or shared files, or interprocess queues, or ..). Hope that helps. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2018-01-24 Thread Thomas Munro
On Wed, Jan 24, 2018 at 5:25 PM, Thomas Munro wrote: > If there were some way for the postmaster to cause reason > PROCSIG_PARALLEL_MESSAGE to be set in the leader process instead of > just notification via kill(SIGUSR1) when it fails to fork a parallel > worker, we'd get (1

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-24 Thread Thomas Munro
On Thu, Jan 25, 2018 at 8:54 AM, Peter Geoghegan wrote: > I have used Thomas' chaos-monkey-fork-process.patch to verify: > > 1. The problem of fork failure causing nbtsort.c to wait forever is a > real problem. Sure enough, the coding pattern within > _bt_leader_heapscan()

Re: copy.c allocation constant

2018-01-24 Thread Thomas Munro
for Windows) including my current favourite we waste a bunch of memory. -- Thomas Munro http://www.enterprisedb.com

Re: copy.c allocation constant

2018-01-24 Thread Thomas Munro
MiB, 48 MiB, 56 MiB, 64 MiB] | | ... ... | | 512 PiB [2560 PiB, 3 EiB, 3584 PiB, 4 EiB] | | 1 EiB [5 EiB, 6 EiB, 7 EiB]| +------+ -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] UPDATE of partition key

2018-01-24 Thread Thomas Munro
bly need to improve the scheduling logic a bit to make it useful first (prioritising new patches or something, since otherwise it'll take up to multiple days to get around to valgrind-testing any given patch...). -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] SERIALIZABLE with parallel query

2018-01-24 Thread Thomas Munro
On Wed, Dec 13, 2017 at 5:30 PM, Thomas Munro wrote: > On Wed, Dec 13, 2017 at 2:09 PM, Haribabu Kommi > wrote: >> Thanks for explaining the problem in generating an isolation test to >> test the serialize parallel query. >> >> Committer can decide whether existi

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-24 Thread Thomas Munro
On Thu, Jan 25, 2018 at 9:28 AM, Peter Geoghegan wrote: > On Wed, Jan 24, 2018 at 12:13 PM, Thomas Munro > wrote: >> On Thu, Jan 25, 2018 at 8:54 AM, Peter Geoghegan wrote: >>> I have used Thomas' chaos-monkey-fork-process.patch to verify: >>> >>

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2018-01-24 Thread Thomas Munro
ion_size -- ! 8192 (1 row) SELECT count(*) FROM vactst; ====== -- Thomas Munro http://www.enterprisedb.com

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-24 Thread Thomas Munro
&cascade, &restart_seqs); ^ -- Thomas Munro http://www.enterprisedb.com

Re: ALTER TABLE ADD COLUMN fast default

2018-01-24 Thread Thomas Munro
tty sure it just needs NULL in the third argument. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] [PATCH] Lockable views

2018-01-24 Thread Thomas Munro
uild: you now need to say instead of , and you need to say . -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2018-01-25 Thread Thomas Munro
e weird timing effect that you don't see on your development system. -- Thomas Munro http://www.enterprisedb.com

Documentation of pgcrypto AES key sizes

2018-01-25 Thread Thomas Munro
Hi, I noticed that the documentation for encrypt()/decrypt() says "aes — AES (Rijndael-128)", but in fact 192 and 256 bit keys are also supported, whether you build --with-openssl or --without-openssl. Should that say "AES (Rijndael-128, -192 or -256)" instead? -

Re: WIP: Covering + unique indexes.

2018-01-25 Thread Thomas Munro
etimes reduce the size of the index while retaining the possibility of using a faster index-only scan. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-26 Thread Thomas Munro
On Fri, Jan 26, 2018 at 7:30 PM, Peter Geoghegan wrote: > On Thu, Jan 25, 2018 at 10:00 PM, Amit Kapila wrote: >> However, now I see that you and Thomas are trying to find a different >> way to overcome this problem differently, so not sure if I should go >> ahead or not.

Failed test 'pg_recvlogical acknowledged changes, nothing pending on slot'

2018-01-28 Thread Thomas Munro
as anything to do with it because it doesn't touch anything outside contrib/pg_stat_statements. -- Thomas Munro http://www.enterprisedb.com

Failed test 'psql query died successfully after SIGQUIT'

2018-01-28 Thread Thomas Munro
es=13, Tests=134, 198 wallclock secs ( 0.10 usr 0.01 sys + 35.98 cusr 10.35 csys = 46.44 CPU) Result: FAIL Makefile:19: recipe for target 'check' failed -- Thomas Munro http://www.enterprisedb.com

Re: Transform for pl/perl

2018-01-28 Thread Thomas Munro
On Fri, Jan 12, 2018 at 9:47 PM, Anthony Bykov wrote: > On Fri, 12 Jan 2018 15:19:26 +1300 > Thomas Munro wrote: > Hello, thank you for your message. > The problem was that different perl compilers uses different infinity > representations. Some of them use "Inf" o

Re: [HACKERS] Restricting maximum keep segments by repslots

2018-01-28 Thread Thomas Munro
On Thu, Jan 11, 2018 at 7:59 PM, Kyotaro HORIGUCHI wrote: > [new patch set] FYI this is still broken: test ddl ... FAILED You could see that like this: cd contrib/test_decoding make check -- Thomas Munro http://www.enterprisedb.com

Re: Documentation of pgcrypto AES key sizes

2018-01-28 Thread Thomas Munro
On Sat, Jan 27, 2018 at 4:56 AM, Robert Haas wrote: > On Thu, Jan 25, 2018 at 8:19 PM, Michael Paquier > wrote: >> On Fri, Jan 26, 2018 at 12:33:41PM +1300, Thomas Munro wrote: >>> I noticed that the documentation for encrypt()/decrypt() says "aes — >>> AES (

Re: [HACKERS] Secondary index access optimizations

2018-01-28 Thread Thomas Munro
e_eq_internal(typcache, pred_range, clause_range)) + { + return true; + } Unnecessary braces. +/* + * Get range type for the corresprent scalar type. + * Returns NULl if such range type is not found. + * This function performs sequential scan in pg_range table, + * but since number of range

Re: Failed test 'psql query died successfully after SIGQUIT'

2018-01-28 Thread Thomas Munro
On Mon, Jan 29, 2018 at 5:35 PM, Tom Lane wrote: > Thomas Munro writes: >> Does anyone have any thoughts on why Travis CI might be failing >> occasionally as shown below, but the build farm apparently doesn't? >> It's not alien technology, it's just Ubuntu L

Re: Failed test 'psql query died successfully after SIGQUIT'

2018-01-28 Thread Thomas Munro
On Mon, Jan 29, 2018 at 5:48 PM, Thomas Munro wrote: > On Mon, Jan 29, 2018 at 5:35 PM, Tom Lane wrote: >> Thomas Munro writes: >>> Does anyone have any thoughts on why Travis CI might be failing >>> occasionally as shown below, but the build farm apparently

Re: [HACKERS] [PATCH] Lockable views

2018-01-29 Thread Thomas Munro
to me. If there's no objection, especially from Thomas > Munro, I will mark this as "ready for committer". About the idea: it makes some kind of sense to me that we should lock the underlying table, in all the same cases that you could do DML on the view automatically. I wond

Re: JIT compiling with LLVM v9.0

2018-01-30 Thread Thomas Munro
00/final 297347) (based on LLVM 4.0.0) This seems to be a valid complaint. I don't think you should be (indirectly) wrapping Types.h in extern "C". At a guess, your llvmjit.h should be doing its own #ifdef __cplusplus'd linkage specifiers, so you can use it from C or C++, but making sure that you don't #include LLVM's headers from a bizarro context where __cplusplus is defined but the linkage is unexpectedly already "C"? -- Thomas Munro http://www.enterprisedb.com

Re: JIT compiling with LLVM v9.0

2018-01-30 Thread Thomas Munro
On Wed, Jan 31, 2018 at 3:05 PM, Andres Freund wrote: > On 2018-01-31 14:42:26 +1300, Thomas Munro wrote: >> I'm just starting to look at this (amazing) work, and I don't have a >> strong opinion yet. But certainly, making it easy for packagers to >> put the -ji

<    5   6   7   8   9   10   11   12   13   14   >