Re: Comment fix and question about dshash.c

2018-10-26 Thread Thomas Munro
tries in ~53 buckets, but with 1.0 you expect to have 100 entries in ~64 buckets. It'd be a fair criticism that that's arbitrarily different to the choice made for hash joins, and dynahash's default is 1 (though it's a run-time parameter). [1] https://math.stackexchange.com/questions/470662/average-number-of-bins-occupied-when-throwing-n-balls-into-n-bins -- Thomas Munro http://www.enterprisedb.com

Re: Comment fix and question about dshash.c

2018-10-26 Thread Thomas Munro
On Sat, Oct 27, 2018 at 10:03 AM Thomas Munro wrote: > space-vs-collision trade-off. If I have my maths right[1], with 0.75 > you expect to have 75 entries in ~53 buckets, but with 1.0 you expect > to have 100 entries in ~64 buckets. It'd be a fair criticism that Forgot to add

Re: Comment fix and question about dshash.c

2018-10-27 Thread Thomas Munro
On Sun, Oct 28, 2018 at 1:22 AM Tomas Vondra wrote: > On 10/27/2018 01:51 PM, Antonin Houska wrote: > > Antonin Houska wrote: > >> Thomas Munro wrote: > >>> On Sat, Oct 27, 2018 at 12:30 AM Antonin Houska wrote: > >>> Are you saying there is a b

Re: PostgreSQL vs SQL/XML Standards

2018-10-29 Thread Thomas Kellerer
>> I have a access to too old 11.2 Oracle. There I had to modify query >> because there is not boolean type. I replaced bool by int, but I got a >> error >> ORA-19224:XPTY-004 .. expected node()*, got xs:string - it doesn't work >> with/without string() wrappings. >> >The problem is in last line -

Re: Hash Joins vs. Bloom Filters / take 2

2018-11-01 Thread Thomas Munro
) > It is currently in the list of patches for the current commitfest, but > based on your previous post I'm not sure if you're planning to get back to > this patch just now. If you plan to resume work on it, I'll sign up as a > reviewer. I'm also signed up to review. -- Thomas Munro http://www.enterprisedb.com

Re: pread() and pwrite()

2018-11-03 Thread Thomas Munro
rms that has pread/pwrite. Here's a patch to add Windows support by supplying src/backend/port/win32/pread.c. Thoughts? -- Thomas Munro http://www.enterprisedb.com 0001-Use-pread-pwrite-instead-of-lseek-read-write-v8.patch Description: Binary data 0002-Supply-pread-pwrite-implem

Re: pread() and pwrite()

2018-11-05 Thread Thomas Munro
On Sun, Nov 4, 2018 at 12:03 AM Thomas Munro wrote: > On Sat, Nov 3, 2018 at 2:07 AM Jesper Pedersen > wrote: > > This still applies, and passes make check-world. > > > > I wonder what the commit policy is on this, if the Windows part isn't > > included. I re

Re: pread() and pwrite()

2018-11-05 Thread Thomas Munro
t; On 2018-Nov-04, Thomas Munro wrote: > >> Here's a patch to add Windows support by supplying > >> src/backend/port/win32/pread.c. Thoughts? > > > Hmm, so how easy is to detect that somebody runs read/write on fds where > > pread/pwrite have occurred? I g

Re: Strange failure in LWLock on skink in REL9_5_STABLE

2018-11-05 Thread Thomas Munro
x27;m not very > optimistic about this any more. > > But ripping it out in the back branches seems unnecessary. I'd just > leave the bugs unfixed there. Most likely nobody is using that stuff, > but if they are, let's not break it. Thanks. Pushed to master only. -- Thomas Munro http://www.enterprisedb.com

Re: Strange failure in LWLock on skink in REL9_5_STABLE

2018-11-05 Thread Thomas Munro
On Tue, Nov 6, 2018 at 6:15 PM Michael Paquier wrote: > On Tue, Nov 06, 2018 at 05:29:36PM +1300, Thomas Munro wrote: > > Thanks. Pushed to master only. > > Just a wild idea while this thread is hot: could we add in the > description of the broken APIs or in their header

Re: Support custom socket directory in pg_upgrade

2018-11-06 Thread Thomas Munro
in the docs part from v1. I think PGSOCKETDIR should be mentioned in the documentation like the other environment variables, and also I'm wondering if it actually works: you set it to the current working directory first, then parse the command line option if present, and then read the env var

Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

2018-11-06 Thread Thomas Munro
other from COPY ... FROM PROGRAM. The differences were: PIPE, TTIN, TTOU and USR2. For the first and last of those, the default action would be to terminate the process, but the COPY PROGRAM child ignored them; for TTIN and TTOU, the default action would be to stop the process, but again they are ignored. Why do bgwriter.c, startup.c, ... set SIGTTIN and SIGTTOU back to SIG_DFL, but not regular backends? -- Thomas Munro http://www.enterprisedb.com

Re: pread() and pwrite()

2018-11-06 Thread Thomas Munro
On Wed, Nov 7, 2018 at 4:42 AM Jesper Pedersen wrote: > Passes check-world, and includes the feedback on this thread. > > New status: Ready for Committer Thanks! Pushed. I'll keep an eye on the build farm to see if anything breaks on Cygwin or some other frankenOS. -- Tho

Cache relation sizes?

2018-11-06 Thread Thomas Munro
better idea for how to do this. It occurred to me that I might be missing something or this would have been done already... -- Thomas Munro http://www.enterprisedb.com

Re: DNS SRV support for LDAP authentication

2018-11-06 Thread Thomas Munro
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro wrote: > Some people like to use DNS SRV records to advertise LDAP servers on > their network. Microsoft Active Directory is usually (always?) set up > that way. Here is a patch to allow our LDAP auth module to support > that kind of di

Re: Copy data to DSA area

2018-11-06 Thread Thomas Munro
f that address space as necessary (instead of doing that in dsa_get_address()). AFAIK that would work, but it seems to be a bit weird to go to such lengths. It would be a kind of home-made simulation of threads. On the other hand, that is what we're already doing in dsa.c, except more slowly due to extra software address translation from funky pseudo-addresses. 3. Something something threads. -- Thomas Munro http://www.enterprisedb.com

Re: pread() and pwrite()

2018-11-07 Thread Thomas Munro
there's no objection I will commit this. Thanks for adjusting that. I suppose I would have known about this if cfbot checked every patch with valgrind, which I might look into. I'm a little confused about how an uninitialised value originating in an OID list finishes up in an xlog buffer, considering that OIDs don't have padding. -- Thomas Munro http://www.enterprisedb.com

Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

2018-11-07 Thread Thomas Munro
that to be expected. I tried to think of a scenario where the already-received output is truly invalidated by a later error that happens after we close the pipe. It could be something involving a program that uses something optimistic like serializable snapshot isolation that can later de

Re: Postgres, fsync, and OSs (specifically linux)

2018-11-07 Thread Thomas Munro
On Fri, Oct 19, 2018 at 6:42 PM Craig Ringer wrote: > On Fri, 19 Oct 2018 at 07:27, Thomas Munro > wrote: >> 2. I am +1 on back-patching Craig's PANIC-on-failure logic. Doing >> nothing is not an option I like. I have some feedback and changes to >> propose tho

Re: POC: Cleaning up orphaned files using undo logs

2018-11-07 Thread Thomas Munro
On Tue, Nov 6, 2018 at 12:42 AM Kuntal Ghosh wrote: > On Thu, Nov 1, 2018 at 8:53 AM Thomas Munro > wrote: > > It passes make check on Unix and Windows, though currently it's > > failing some of the TAP tests for reasons I'm looking into (possibly > > due to bu

Re: Postgres, fsync, and OSs (specifically linux)

2018-11-08 Thread Thomas Munro
On Fri, Nov 9, 2018 at 7:07 AM Robert Haas wrote: > On Wed, Nov 7, 2018 at 9:41 PM Thomas Munro > wrote: > > My plan is do a round of testing and review of this stuff next week > > once the dust is settled on the current minor releases (including > > fixing a few typos

Strange corruption in psql output on mereswine

2018-11-08 Thread Thomas Munro
011%3A58%3A07 -- Thomas Munro http://www.enterprisedb.com

Re: Copy data to DSA area

2018-11-08 Thread Thomas Munro
On Fri, Nov 9, 2018 at 2:19 PM Ideriha, Takeshi wrote: > From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > >I know of 3 ideas that would make your idea C work, so that you could share > >something as complicated as a query plan directly without having to > >deseri

Re: BUG #15449: file_fdw using program cause exit code error when using LIMIT

2018-11-08 Thread Thomas Munro
is not enough... due to buffering, it works. Try 100: postgres=# create foreign table ft5 (a text) server svf1 options (program 'ruby -e "for i in 1..100 do puts i; end"'); CREATE FOREIGN TABLE postgres=# select * from ft5 limit 5; ERROR: program "ruby -e "for i in 1..100 do puts i; end"" failed DETAIL: child process exited with exit code 1 -- Thomas Munro http://www.enterprisedb.com

Re: Skylake-S warning

2018-11-10 Thread Thomas Munro
k > explicitly using barriers is much saner these days. > > Comments? +1 I said the same over here: https://www.postgresql.org/message-id/flat/CAEepm%3D1nff0x%3D7i3YQO16jLA2qw-F9O39YmUew4oq-xcBQBs0g%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com

Re: DSM segment handle generation in background workers

2018-11-12 Thread Thomas Munro
On Mon, Nov 12, 2018 at 9:34 PM Noah Misch wrote: > On Sat, Oct 13, 2018 at 11:45:17PM +1300, Thomas Munro wrote: > > + /* Set a different seed for random() in every backend. */ > > + srandom((unsigned int) MyProcPid ^ (unsigned int) MyStartTimestamp); > > > -

Re: [HACKERS] Decimal64 and Decimal128

2018-11-12 Thread Thomas Munro
On Fri, Jun 16, 2017 at 9:42 AM Thomas Munro wrote: > On Fri, Sep 25, 2015 at 5:06 PM, Pavel Stehule > wrote: > > 2015-09-25 0:25 GMT+02:00 Jim Nasby : > >> On 9/24/15 3:35 PM, Peter Geoghegan wrote: > >>> > >>> I would worry about the

Re: Copy data to DSA area

2018-11-12 Thread Thomas Munro
On Tue, Nov 13, 2018 at 9:45 AM Robert Haas wrote: > On Thu, Nov 8, 2018 at 9:05 PM Thomas Munro > wrote: > > * I had some ideas about some kind of "allocation rollback" interface: > > you begin an "allocation transaction", allocate a bunch of stuff > &

Re: Possible buffer overrun in src/backend/libpq/hba.c gethba_options()

2018-11-12 Thread Thomas Munro
t's a false positive. > Is that the case, or should MAX_HBA_OPTIONS be increased? Right, thank you. It seems clear that MAX_HBA_OPTIONS should be increased and the comment near its definition is wrong. Will fix. -- Thomas Munro http://www.enterprisedb.com

Re: Possible buffer overrun in src/backend/libpq/hba.c gethba_options()

2018-11-12 Thread Thomas Munro
On Tue, Nov 13, 2018 at 3:42 PM Thomas Munro wrote: > On Tue, Nov 13, 2018 at 3:02 PM Julian Hsiao wrote: > > During a routine Coverity scan of our internal PostgreSQL fork, it > > issued a buffer overrun warning for src/backend/libpq/hba.c, > > gethba_options()[0

Re: Refactoring the checkpointer's fsync request queue

2018-11-13 Thread Thomas Munro
(Replies to a couple of different messages below) On Wed, Nov 14, 2018 at 6:04 AM Robert Haas wrote: > On Sun, Nov 11, 2018 at 9:59 PM Thomas Munro > wrote: > > There is one major problem with this patch > > If there's only one, you're doing great! Although admitt

Re: Copy data to DSA area

2018-11-13 Thread Thomas Munro
s to be a major subproblem to deal with early in the project. I will be very happy if we can come up with something :-) On Wed, Nov 14, 2018 at 12:40 AM Ideriha, Takeshi wrote: > From: Thomas Munro [mailto:thomas.mu...@enterprisedb.com] > >postgres=# call hoge_list(3); > >NOTICE

Re: DSM segment handle generation in background workers

2018-11-13 Thread Thomas Munro
On Wed, Nov 14, 2018 at 3:24 PM Noah Misch wrote: > On Mon, Nov 12, 2018 at 09:39:01AM -0500, Tom Lane wrote: > > Thomas Munro writes: > > > On Mon, Nov 12, 2018 at 9:34 PM Noah Misch wrote: > > >> Compared to the old code, the new code requires more wa

Re: DSM segment handle generation in background workers

2018-11-13 Thread Thomas Munro
On Wed, Nov 14, 2018 at 6:34 PM Noah Misch wrote: > On Wed, Nov 14, 2018 at 05:50:26PM +1300, Thomas Munro wrote: > > On Wed, Nov 14, 2018 at 3:24 PM Noah Misch wrote: > > > What counts is the ease of predicting a complete seed. HEAD's algorithm > > > has >

Re: DSM segment handle generation in background workers

2018-11-14 Thread Thomas Munro
On Wed, Nov 14, 2018 at 8:52 PM Noah Misch wrote: > On Wed, Nov 14, 2018 at 08:22:42PM +1300, Thomas Munro wrote: > > On Wed, Nov 14, 2018 at 6:34 PM Noah Misch wrote: > > > On Wed, Nov 14, 2018 at 05:50:26PM +1300, Thomas Munro wrote: > > > > On Wed, Nov 14, 201

Re: DSM segment handle generation in background workers

2018-11-14 Thread Thomas Munro
On Thu, Nov 15, 2018 at 4:25 AM Tom Lane wrote: > Thomas Munro writes: > > What do you think about the attached? > > I think you need to cast MyStartTimestamp to uint64 before shifting > to ensure portable behavior of the shifts. In principle it wouldn't > matter bec

Re: Cache relation sizes?

2018-11-15 Thread Thomas Munro
On Fri, Nov 9, 2018 at 4:42 PM David Rowley wrote: > On 7 November 2018 at 11:46, Andres Freund wrote: > > On 2018-11-07 11:40:22 +1300, Thomas Munro wrote: > >> PostgreSQL likes to probe the size of relations with lseek(SEEK_END) a > >> lot. For example, a

Re: DNS SRV support for LDAP authentication

2018-11-15 Thread Thomas Munro
On Wed, Nov 7, 2018 at 4:39 PM Thomas Munro wrote: > On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro > wrote: > > Some people like to use DNS SRV records to advertise LDAP servers on > > their network. Microsoft Active Directory is usually (always?) set up > > that way.

Re: Refactoring the checkpointer's fsync request queue

2018-11-16 Thread Thomas Munro
nt.event == WL_TIMEOUT. Oops. I will fix that when I post a new experimental version that uses the bgworker as discussed, and we can try to figure out if that design will fly. -- Thomas Munro http://www.enterprisedb.com

Re: Refactoring the checkpointer's fsync request queue

2018-11-16 Thread Thomas Munro
to put it another way, you can't be given a lower sequence number than another process that has already written, because that other process must have been given a sequence number before it wrote. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-11-16 Thread Thomas Munro
On Sat, Nov 17, 2018 at 5:56 AM Tom Lane wrote: > Thomas Munro writes: > > [ 0001-Add-WL_EXIT_ON_PM_DEATH-pseudo-event-v4.patch ] > > I took a quick look through this. I have no objection to the idea of > letting the latch infrastructure do the proc_exit(1), but I'm won

Re: Postgres, fsync, and OSs (specifically linux)

2018-11-17 Thread Thomas Munro
On Fri, Nov 9, 2018 at 9:06 AM Robert Haas wrote: > On Thu, Nov 8, 2018 at 3:04 PM Thomas Munro > wrote: > > My reasoning for choosing bms_join() is that it cannot fail, assuming > > the heap is not corrupted. It simply ORs the two bit-strings into > > whichever is the

Re: Postgres, fsync, and OSs (specifically linux)

2018-11-18 Thread Thomas Munro
On Fri, Nov 9, 2018 at 9:03 AM Thomas Munro wrote: > On Fri, Nov 9, 2018 at 7:07 AM Robert Haas wrote: > > On Wed, Nov 7, 2018 at 9:41 PM Thomas Munro > > wrote: > > > My plan is do a round of testing and review of this stuff next week > > > once the dus

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Thomas Munro
show them as different types and show the page size, but you'd have to check that while it's actually running a parallel query. For example, a large parallel hash join that runs for a while would do it, and in theory you might be able to see a small performance improvement for larger pag

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Thomas Munro
an't use large pages on Linux (because Linux's POSIX shm_open() really just opens files on /dev/shm, which has a 4k block size), but can use them on FreeBSD (because its shm_open() isn't bound to page sizes, it can and sometimes decides to use large pages). -- Thomas Munro http://www.enterprisedb.com

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-20 Thread Thomas Munro
On Wed, Nov 21, 2018 at 9:07 AM Andres Freund wrote: > On 2018-11-21 09:00:58 +1300, Thomas Munro wrote: > > On Wed, Nov 21, 2018 at 4:37 AM REIX, Tony wrote: > > > YES ! Reading this file, your suggestion should work ! Thx ! > > > > > > I've rebuilt an

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Thomas Munro
On Wed, Nov 21, 2018 at 1:08 PM Andres Freund wrote: > Let's see what I broke :/ Per rhinoceros, contrib/sepgsql. This patch fixes the build for me (on Debian with libselinux1-dev installed), though I don't know how to test that it works and make check does nothing useful in there

Wonky rename semantics on Windows

2018-11-20 Thread Thomas Munro
ostgresql.org/message-id/flat/CAPpHfds9trA6ipezK3BsuuOSQwEmESiqj8pkOxACFJpoLpcoNw%40mail.gmail.com#9b04576b717175e9dbf03cc991977d3f -- Thomas Munro http://www.enterprisedb.com

Re: Wonky rename semantics on Windows

2018-11-20 Thread Thomas Munro
On Wed, Nov 21, 2018 at 5:43 PM Thomas Munro wrote: > Googling led me to some recently posted clues on SO and elsewhere > about a flag FILE_RENAME_FLAG_POSIX_SEMANTICS which can be used with > SetFileInformationByHandle(), that wasn't mentioned in the earlier > -hackers thread

Re: incorrect xlog.c coverage report

2018-11-21 Thread Thomas Munro
now if it's really safe. If that is somehow magically OK, you'd probably also need the chdir() hack from proc_exit() to get per-pid files. -- Thomas Munro http://www.enterprisedb.com

Re: Accounting of zero-filled buffers in EXPLAIN (BUFFERS)

2018-11-22 Thread Thomas Munro
ure what is the suitable name. I think that might be interesting information in the future, but let's consider that for a later patch. -- Thomas Munro http://www.enterprisedb.com 0001-Don-t-count-zero-filled-buffers-as-read-in-EXPLAI-v2.patch Description: Binary data

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2018-11-22 Thread Thomas Munro
On Sat, Nov 17, 2018 at 2:27 PM Thomas Munro wrote: > Thanks for the review. Pushed. -- Thomas Munro http://www.enterprisedb.com

Re: Shared Memory: How to use SYSV rather than MMAP ?

2018-11-23 Thread Thomas Munro
ent variables or loader controls, since apparently other things like data segments etc apparently can, though I can't find any text that says that's the case and I have no AIX system). -- Thomas Munro http://www.enterprisedb.com

Re: pgsql: Add WL_EXIT_ON_PM_DEATH pseudo-event.

2018-11-24 Thread Thomas Munro
On Sun, Nov 25, 2018 at 3:38 AM Christoph Berg wrote: > Re: Thomas Munro 2018-11-23 > > Add WL_EXIT_ON_PM_DEATH pseudo-event. > > I think this broke something: > > TRAP: FailedAssertion(»!(!IsUnderPostmaster || (wakeEvents & (1 << 5)) || > (wakeEvents & (1

Re: pgsql: Add WL_EXIT_ON_PM_DEATH pseudo-event.

2018-11-25 Thread Thomas Munro
On Sun, Nov 25, 2018 at 12:59 PM Thomas Munro wrote: > On Sun, Nov 25, 2018 at 3:38 AM Christoph Berg wrote: > > TRAP: FailedAssertion(»!(!IsUnderPostmaster || (wakeEvents & (1 << 5)) || > > (wakeEvents & (1 << 4)))«, Datei: > > »/build/pos

Re: [HACKERS] [PATCH] Generic type subscripting

2018-11-25 Thread Thomas Munro
ticed on cfbot.cputube.org: pg_type.c:167:10: error: passing argument 3 of ‘GenerateTypeDependencies’ makes integer from pointer without a cast [-Werror] false); ^ In file included from pg_type.c:28:0: ../../../src/include/catalog/pg_type.h:335:13: note: expected ‘Oid’ but argument is of type ‘void *’ extern void GenerateTypeDependencies(Oid typeObjectId, ^ -- Thomas Munro http://www.enterprisedb.com

Re: pgsql: Add WL_EXIT_ON_PM_DEATH pseudo-event.

2018-11-25 Thread Thomas Munro
On Mon, Nov 26, 2018 at 6:56 AM Tom Lane wrote: > Thomas Munro writes: > > Fix pushed. > > By way of penance, I have now configured PG_TEST_EXTRA="ssl ldap > > kerberos" for my build farm animals elver and eelpout. elver should > > pass at the next build, a

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2018-11-25 Thread Thomas Munro
3619:43: error: ‘outer_part’ may be used uninitialized in this function [-Werror=maybe-uninitialized] merged_index = map_and_merge_partitions(outer_pmap, outer_mmap, ^ partbounds.c:3475:6: note: ‘outer_part’ was declared here int outer_part; ^ cc1: all warnings being treated as errors -- Tho

Re: New GUC to sample log queries

2018-11-25 Thread Thomas Munro
ion makes integer from pointer without a cast [-Werror] }, ^ guc.c:3328:3: error: (near initialization for ‘ConfigureNamesReal[19].gen.flags’) [-Werror] -- Thomas Munro http://www.enterprisedb.com

Re: SSL tests failing with "ee key too small" error on Debian SID

2018-11-25 Thread Thomas Munro
ith > any configuration. This would require regenerating the certs in the > tree. Any thoughts from others? I don't really have opinion here, but I wanted to point out that src/test/ldap/t/001_auth.pl creates new certs on the fly, which is a bit inconsistent with the SSL test&

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-11-26 Thread Thomas Munro
d here other_sel, ^ extended_stats.c:1227:6: error: ‘mcv_sel’ may be used uninitialized in this function [-Werror=maybe-uninitialized] sel = mcv_sel + other_sel; ^ extended_stats.c:1087:5: note: ‘mcv_sel’ was declared here mcv_sel, ^ -- Thomas Munr

Re: Refactoring the checkpointer's fsync request queue

2018-11-26 Thread Thomas Munro
On Fri, Nov 23, 2018 at 5:45 PM Thomas Munro wrote: > I do have a new plan though... Ugh. The plan in my previous email doesn't work, I was confused about the timing of the buffer header update. Back to the drawing board. -- Thomas Munro http://www.enterprisedb.com

Re: pg_stat_ssl additions

2018-11-26 Thread Thomas Munro
tch cipher names with \w+, but some cipher names sometimes (on some library versions?) have hyphens in them instead of underscores. https://travis-ci.org/postgresql-cfbot/postgresql/builds/459620336 -- Thomas Munro http://www.enterprisedb.com

Re: dsa_allocate() faliure

2018-11-26 Thread Thomas Munro
uld not find 7 free pages", which should not happen. This is on 10.6, so it has the commit "Fix segment_bins corruption in dsa.c.". Hmm. I will see if I can come up with a many-partition torture test reproducer for this. [1] https://www.postgresql.org/message-id/flat/CAJk1zg10iCNsxFvQ4pgKe1B0rdjNG9iELA7AzLXjXnQm5T%3DKzQ%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com

Re: dsa_allocate() faliure

2018-11-27 Thread Thomas Munro
On Tue, Nov 27, 2018 at 4:00 PM Thomas Munro wrote: > Hmm. I will see if I can come up with a many-partition torture test > reproducer for this. No luck. I suppose one theory that could link both failure modes would a buffer overrun, where in the non-shared case it trashes a pointer t

Re: Accounting of zero-filled buffers in EXPLAIN (BUFFERS)

2018-11-27 Thread Thomas Munro
On Tue, Nov 27, 2018 at 2:53 PM Haribabu Kommi wrote: > Thanks for the updated patch. It looks good. > I marked it in the commitfest as ready for committer. Pushed. Thanks for the reviews! -- Thomas Munro http://www.enterprisedb.com

"pg_ctl: the PID file ... is empty" at end of make check

2018-11-27 Thread Thomas Munro
Hello, Today I saw a one-off case of $SUBJECT, on macOS. I can't reproduce it, but I noticed exactly the same thing on longfin the other day: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=longfin&dt=2018-11-25%2005%3A39%3A04 Anyone know what that's about? -- Tho

Re: "pg_ctl: the PID file ... is empty" at end of make check

2018-11-27 Thread Thomas Munro
On Wed, Nov 28, 2018 at 4:10 PM Tom Lane wrote: > Thomas Munro writes: > > Today I saw a one-off case of $SUBJECT, on macOS. I can't reproduce > > it, but I noticed exactly the same thing on longfin the other day: > > https://buildfarm.postgresql.org/cgi-bin/show_log.p

Re: "pg_ctl: the PID file ... is empty" at end of make check

2018-11-27 Thread Thomas Munro
On Wed, Nov 28, 2018 at 5:28 PM Tom Lane wrote: > Thomas Munro writes: > > Today I saw a one-off case of $SUBJECT, on macOS. I can't reproduce > > it, but I noticed exactly the same thing on longfin the other day: > > https://buildfarm.postgresql.org/cgi-bin/show_log.p

Re: "pg_ctl: the PID file ... is empty" at end of make check

2018-11-28 Thread Thomas Munro
t;) > pg_ctl: reads file, gets zero bytes > > But that idea is in the OS-bug class, which we agree seems unlikely. I think you might be right. https://github.com/macdice/unlinktest -- Thomas Munro http://www.enterprisedb.com

Re: "pg_ctl: the PID file ... is empty" at end of make check

2018-11-28 Thread Thomas Munro
On Thu, Nov 29, 2018 at 3:30 AM Tom Lane wrote: > Thomas Munro writes: > > On Wed, Nov 28, 2018 at 6:47 PM Tom Lane wrote: > >> Is it possible that unlink() on APFS is not atomic? > > > I think you might be right. > > https://github.com/macdice/unlinktest >

Re: Python versions (was Re: RHEL 8.0 build)

2018-11-28 Thread Thomas Munro
" binary. Confirmed that if you pkg install python2 and python3 meta-packages on a FreeBSD box you get python2 and python3 commands (symlinks to python2.7, python3.6). -- Thomas Munro http://www.enterprisedb.com

Re: minor typo in dsa.c

2018-11-28 Thread Thomas Munro
On Thu, Nov 29, 2018 at 1:40 PM Ideriha, Takeshi wrote: > I found a minor typo in dsa.c. > > s/set_dsa_size_limit/dsa_set_size_limit/ Thanks! Pushed. -- Thomas Munro http://www.enterprisedb.com

Re: worker_spi shouldn't execute again on sigterm

2018-11-28 Thread Thomas Munro
DsT49RBaBJsaot_qNhPSCXBm9rzA%40mail.gmail.com The relevant TL;DR is that if you're calling anything non-trivial in the loop, you might get stuck in wait syscall if you use a half-baked SIGTERM handler that only knows how to exit your top level loop, but prevents CHECK_FOR_INTERRUPTS() from detecting shutdown. -- Thomas Munro http://www.enterprisedb.com

Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full

2018-11-29 Thread Thomas Munro
pass and show the feature working correctly. I think this is getting close to committable. I see that Magnus has signed up as committer. -- Thomas Munro http://www.enterprisedb.com

Re: Hash Joins vs. Bloom Filters / take 2

2018-02-20 Thread Thomas Munro
ls in the scan. In other words, I wonder if our left-deep plans stand to gain more from Bloom filter push down. [1] http://www.nus.edu.sg/nurop/2010/Proceedings/SoC/NUROP_Congress_Cheng%20Bin.pdf [2] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.73.8069&rep=rep1&type=pdf [3]

Typo in predicate.c comment

2018-02-21 Thread Thomas Munro
Hi, Here's a tiny patch to fix a typo. -- Thomas Munro http://www.enterprisedb.com typo.patch Description: Binary data

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
roup, the leader has gone away, so just exit quietly" but for various other similar things we spew errors (most commonly seen one being "ERROR: could not map dynamic shared memory segment"). Intentional? -- Thomas Munro http://www.enterprisedb.com ssi-parallel-v11.patch Description: Binary data

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Feb 23, 2018 at 1:54 AM, Thomas Munro wrote: > The attached is a draft patch only, needing some testing and polish. > Brickbats, better ideas? Note, that version is broken for multiple Gather nodes, but that's fixable. Comments on the general idea welcome. -- Thomas

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Feb 23, 2018 at 3:29 PM, Amit Kapila wrote: > On Thu, Feb 22, 2018 at 10:35 PM, Robert Haas wrote: >> On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro >>> PS I noticed that for BecomeLockGroupMember() we say "If we can't >>> join the lock group, t

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-22 Thread Thomas Munro
On Fri, Feb 23, 2018 at 7:56 PM, Amit Kapila wrote: > On Fri, Feb 23, 2018 at 8:48 AM, Thomas Munro > wrote: >> On Fri, Feb 23, 2018 at 3:29 PM, Amit Kapila wrote: >>> By the way, in which case leader can exit early? As of now, we do >>> wait for workers t

Isolation tester result formatting

2018-02-23 Thread Thomas Munro
avept ... ok Patch attached. I realise this isn't the most pressing problem faced by computer science... -- Thomas Munro http://www.enterprisedb.com test-name-inflation-adjustment.patch Description: Binary data

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-23 Thread Thomas Munro
On Fri, Feb 23, 2018 at 6:05 AM, Robert Haas wrote: > On Thu, Feb 22, 2018 at 7:54 AM, Thomas Munro > wrote:> >> The best solution I have come up with so far is to add a reference >> count to SERIALIZABLEXACT. I toyed with putting the refcount into the >> DSM i

Re: Treating work_mem as a shared resource (Was: Parallel Hash take II)

2018-02-23 Thread Thomas Munro
and no matter which of its strategies it picked (either it receives a budget of work_mem / participants, and then multiplies it by participants to create a no-partition hash table combining the participants' budgets, or it lets each participant chew on smaller hash tables adding up to at

Updating parallel.sgml's treatment of parallel joins

2018-02-23 Thread Thomas Munro
Hi hackers, Here is an attempt at updating parallel.sgml to cover Parallel Hash. I will be neither surprised nor offended if Robert would like to put it differently! -- Thomas Munro http://www.enterprisedb.com update-parallel-join-docs.patch Description: Binary data

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-25 Thread Thomas Munro
On Sat, Feb 24, 2018 at 12:04 AM, Thomas Munro wrote: > I'm testing another version that is a lot simpler: like v10, it relies > on the knowledge that the leader's transaction will always end after > the workers have finished, but it handles the RO_SAFE optimisati

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-02-26 Thread Thomas Munro
On Tue, Feb 13, 2018 at 7:47 PM, Thomas Munro wrote: > One way to avoid all that might be to use pseudo page numbers that > don't suffer from splits. I don't know how you'd choose the > constant, but it could be something like pseudo page number = hash > value % 1024.

Re: Unexpected behavior with transition tables in update statement trigger

2018-02-26 Thread Thomas Munro
On Tue, Feb 27, 2018 at 4:18 AM, Tom Kazimiers wrote: > On Mon, Feb 26, 2018 at 11:15:44PM +1300, Thomas Munro wrote: >> On Sat, Feb 24, 2018 at 4:47 PM, Tom Kazimiers >> wrote: >> Thanks for the reproducer. Yeah, that seems to be a bug. >> nodeNamedTuplestorescan.c a

Re: Kerberos test suite

2018-02-26 Thread Thomas Munro
27;; +} One thing that surprised me is that your test runs my system's installed psql command out of my $PATH, not the one under test. Is that expected? -- Thomas Munro http://www.enterprisedb.com

Re: Registering LWTRANCHE_PARALLEL_HASH_JOIN

2018-02-27 Thread Thomas Munro
On Wed, Feb 28, 2018 at 8:39 AM, Robert Haas wrote: > On Sat, Feb 10, 2018 at 6:07 PM, Thomas Munro > wrote: >> I forgot to register a display name for LWTRANCHE_PARALLEL_HASH_JOIN, >> the tranche ID used by the LWLock that Parallel Hash uses when handing >> out chunk

Re: Unexpected behavior with transition tables in update statement trigger

2018-02-27 Thread Thomas Munro
On Wed, Feb 28, 2018 at 9:58 AM, Tom Lane wrote: > Thomas Munro writes: >> Here's a new version with tuplestore_select_read_pointer() added in >> another place where it was lacking, and commit message. Moving to >> -hackers, where patches go. > > Pushed, along

Re: [HACKERS] GSoC 2017: weekly progress reports (week 6)

2018-02-27 Thread Thomas Munro
On Wed, Jan 3, 2018 at 4:31 AM, Shubham Barai wrote: > I have created new isolation tests. Please have a look at > updated patch. Hi Shubham, Could we please have a rebased version of the gin one? -- Thomas Munro http://www.enterprisedb.com

Synchronous replay take III

2018-02-28 Thread Thomas Munro
GGUQ%40mail.gmail.com [5] https://www.postgresql.org/message-id/CAEepm%3D352uctNiFoN84UN4gtunbeTK-PBLouVe8i_b8ZPcJQFQ%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com 0001-Synchronous-replay-mode-for-avoiding-stale-reads--v5.patch Description: Binary data

Re: [HACKERS] SERIALIZABLE with parallel query

2018-02-28 Thread Thomas Munro
On Mon, Feb 26, 2018 at 6:37 PM, Thomas Munro wrote: > I've now broken it into two patches. Rebased. -- Thomas Munro http://www.enterprisedb.com 0001-Enable-parallel-query-with-SERIALIZABLE-isolatio-v13.patch Description: Binary data 0002-Enable-the-read-only-SERIALIZABLE-optimi

Re: PATCH: Unlogged tables re-initialization tests

2018-02-28 Thread Thomas Munro
it, because it currently prevents me from automatically testing all Commitfest patches on the appveyor.com Windows build farm. I know that it passes as a non-administrative user. -- Thomas Munro http://www.enterprisedb.com

Re: Synchronous replay take III

2018-02-28 Thread Thomas Munro
On Thu, Mar 1, 2018 at 2:39 PM, Thomas Munro wrote: > I was pinged off-list by a fellow -hackers denizen interested in the > synchronous replay feature and wanting a rebased patch to test. Here > it goes, just in time for a Commitfest. Please skip to the bottom of > this message

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-03-01 Thread Thomas Munro
CB(crc32_c, *p_buf); +} >From the department of trivialities, our coding style has braces like this: if (length & sizeof(uint16)) { CRC32CH(crc32_c, *(uint16*)p_buf); p_buf += sizeof(uint16); } if (length & sizeof(uint8)) CRC32CB(crc32_c, *p_buf); -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] GSoC 2017: weekly progress reports (week 4) and patch for hash index

2018-03-01 Thread Thomas Munro
tion predicate locks at the cost of slightly higher collision rate then we should consider that for the whole system (including heap page predicate locking), not just hash indexes. Please consider those ideas parked for now. -- Thomas Munro http://www.enterprisedb.com

Re: Suspicious call of initial_cost_hashjoin()

2018-03-02 Thread Thomas Munro
ommitter? > > Done. Thanks. > Actually I think it'd be nice if the "parallel_hash" argument was mentioned in > the header comment of initial_cost_hashjoin() function, but not sure this is > worth returning the patch to the author. Done. -- Thomas Munro http://www.enterprise

Re: Kerberos test suite

2018-03-02 Thread Thomas Munro
On Thu, Mar 1, 2018 at 4:43 AM, Peter Eisentraut wrote: > On 2/27/18 00:56, Thomas Munro wrote: >> FWIW it passes for me if I add this: >> >> +elsif ($^O eq 'freebsd') >> +{ >> + $krb5_bin_dir = '/usr/local/bin'; >> + $krb5_sb

<    9   10   11   12   13   14   15   16   17   18   >