Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-02-29 Thread Thomas Munro
On Mon, Feb 29, 2016 at 9:05 PM, Amit Langote wrote: > > Hi Thomas, > > On 2016/02/29 15:20, Thomas Munro wrote: >> Thanks for looking at the patch! Here is a new version with the >> following changes: >> >> 1. Some draft user documentation has been added, a

Re: [HACKERS] GCC 6 warning fixes

2016-02-29 Thread Thomas Munro
uld be other ways address this, too. This way seems fine to me (you probably want the function to continue to exist rather than, say, becoming a macro evaluating to false on non-WIN32, if this gets backpatched). I got this warning from GCC6 and it went away with this patch. -- Thomas Mun

Re: [HACKERS] psql completion for ids in multibyte string

2016-02-29 Thread Thomas Munro
tgres=# drpo table "い The patch looks correct to me: it counts characters rather than bytes, which is the right thing to do because the value is passed to substr() which also works in characters rather than bytes. I tested with "éclair", and without the patch, tab completion doesn

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-02-29 Thread Thomas Munro
On Tue, Mar 1, 2016 at 2:46 PM, Amit Langote wrote: > > Hi, > > On 2016/02/29 18:05, Thomas Munro wrote: >> On Mon, Feb 29, 2016 at 9:05 PM, Amit Langote wrote: >>> + servers. A transaction that is run with >>> causal_reads set >>> +

Re: [HACKERS] psql completion for ids in multibyte string

2016-03-01 Thread Thomas Munro
On Wed, Mar 2, 2016 at 7:54 AM, Robert Haas wrote: > On Mon, Feb 29, 2016 at 6:32 PM, Thomas Munro > wrote: >> On Fri, Feb 26, 2016 at 6:34 PM, Kyotaro HORIGUCHI >> wrote: >>> Hello, this is the second patch plitted out. This allows >>> multibyte names to

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-03-02 Thread Thomas Munro
RepStandbyNanes/SyncRepStandbys/ ??? + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + (errmsg_internal("Invalid syntax. synchronous_standby_names parse returned %d", + parse_rc; Looking at other error messages I see that they always start with lower case and then put extra de

Re: [HACKERS] psql completion for ids in multibyte string

2016-03-02 Thread Thomas Munro
On Thu, Mar 3, 2016 at 2:07 PM, Kyotaro HORIGUCHI wrote: > Hello, thank you for the comments. > > At Wed, 2 Mar 2016 10:10:55 +1300, Thomas Munro > wrote in > >> On Wed, Mar 2, 2016 at 7:54 AM, Robert Haas wrote: >> > On Mon, Feb 29, 2016 at 6:32 PM, Thomas M

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-03-03 Thread Thomas Munro
inimum wait time is the roundtrip time between primary to standby. +The minimum wait time is the roundtrip time between the primary and the +almost synchronous standby. s/almost/slowest/ -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-h

[HACKERS] Typo in comment

2016-03-04 Thread Thomas Munro
Hi Here is a patch to fix a typo in a comment in timestamp.c. -- Thomas Munro http://www.enterprisedb.com typo.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-03-06 Thread Thomas Munro
On Thu, Mar 3, 2016 at 7:34 PM, Michael Paquier wrote: > On Tue, Mar 1, 2016 at 11:53 AM, Thomas Munro > wrote: >> On Tue, Mar 1, 2016 at 2:46 PM, Amit Langote >> wrote: >>> >>> Hi, >>> >>> On 2016/02/29 18:05, Thomas Munro wrote: &g

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-03-08 Thread Thomas Munro
On Thu, Mar 3, 2016 at 7:34 PM, Michael Paquier wrote: > WAL replay for 2PC should also > call XLogRequestWalReceiverReply() when needed. Ah yes, I missed this important sentence. I will address that in the next version after some testing. -- Thomas Munro http://www.enterprise

[HACKERS] Floating point timestamps

2016-03-10 Thread Thomas Munro
Hi, Is the plan to remove support for floating point timestamps at some stage? If so, what is that waiting on, and would it provide sufficient warning if (say) 9.6 were documented as the last major release to support that build option? Thanks! -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-03-10 Thread Thomas Munro
On Fri, Mar 11, 2016 at 8:50 AM, Simon Riggs wrote: > On 3 February 2016 at 23:12, Thomas Munro > wrote: > >> >> It quacks suspiciously like a bug. > > > Agreed > > What's more important is that is very publicly a bug in the eyes of others > and shoul

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-03-10 Thread Thomas Munro
On Fri, Mar 11, 2016 at 10:00 AM, Simon Riggs wrote: > On 10 March 2016 at 20:36, Thomas Munro > wrote: >> >> On Fri, Mar 11, 2016 at 8:50 AM, Simon Riggs >> wrote: >> > On 3 February 2016 at 23:12, Thomas Munro >> > >> > wrote:

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-03-10 Thread Thomas Munro
On Fri, Mar 11, 2016 at 6:31 PM, Thomas Munro wrote: > I'm not sure what to make of the pre-existing comment about following > HOT-chains and concurrent index builds (which I moved). Does it mean > there is some way that CREATE INDEX CONCURRENTLY could cause us to > consider th

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-03-11 Thread Thomas Munro
On Fri, Mar 11, 2016 at 6:31 PM, Thomas Munro wrote: > This patch introduces a drop-in replacement > check_unique_tuple_still_live to call instead of heap_hot_search. The > replacement function also calls heap_hot_search_buffer, but while it > has the buffer it takes the opportunity

Re: [HACKERS] Proposal: BSD Authentication support

2016-03-11 Thread Thomas Munro
is needs review from an OpenBSD user specifically. FreeBSD and NetBSD use PAM instead of BSD auth. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-03-13 Thread Thomas Munro
On Sat, Mar 12, 2016 at 1:25 AM, Kevin Grittner wrote: > On Thu, Mar 10, 2016 at 11:31 PM, Thomas Munro > wrote: > >> Here's a much simpler version with more comments > >> It handles the same set of isolation test specs. > > I'm impressed that you found a

[HACKERS] pg_stat_get_progress_info(NULL) blows up

2016-03-13 Thread Thomas Munro
Hi I guess pg_stat_get_progress_info should either be strict (see attached) or check for NULL. -- Thomas Munro http://www.enterprisedb.com pg_stat_get_progress_info-strict.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-03-14 Thread Thomas Munro
On Tue, Mar 15, 2016 at 6:58 AM, Robert Haas wrote: > On Sun, Mar 13, 2016 at 11:50 PM, Thomas Munro > wrote: >> The last patches I posted don't apply today due to changes in master, >> so here's a freshly merged patch series. > > +from the current

Re: [HACKERS] Proposal: BSD Authentication support

2016-03-14 Thread Thomas Munro
s required for BSD Authentication support'). I tried configuring BSD auth in pg_hba.conf on a system built without the new feature and it behaved sensibly ('invalid authentication method "bsd": not supported by this build'). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-03-15 Thread Thomas Munro
eed to implement to fully support >>> nested style at first version. >>> We have to carefully design this feature while considering >>> expandability, but overkill implementation could be cause of crash. >>> Consider remaining time for 9.6, I feel we could impl

Re: [HACKERS] Proposal: BSD Authentication support

2016-03-19 Thread Thomas Munro
On Fri, Mar 18, 2016 at 12:49 PM, Marisa Emerson wrote: > On 18/03/16 03:57, Thomas Munro wrote: >> >> You used one name in the docs and another in the code: >> >> +BSD Authentication on PostgreSQL uses the >> auth-postgres >> +login type and a

Re: [HACKERS] Proposal: BSD Authentication support

2016-03-19 Thread Thomas Munro
; I've so far only tested passwd authentication. I'd be interested to test some > of the other authentication styles, I think this would be a useful feature. Agreed. It looks like this is still very useful with the default, and maybe adding support for specifying the auth style in pg_hb

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Thomas Munro
* ModifyWaitEvent is frequently used to switch from waiting for reads to +* waiting on writes. +*/ s/were/where/ + /* +* Even though unused, we also poss epoll_ev as the data argument if +* EPOLL_CTL_DELETE is passed as action. There used to be an epol

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Thomas Munro
On Mon, Mar 21, 2016 at 3:46 AM, Andres Freund wrote: > On 2016-03-21 01:31:30 +1300, Thomas Munro wrote: >> I couldn't get the second patch to apply for some reason, > > Weird? Even efter appying the first one first? Ah, I was using patch -p1. I needed to use git am, whic

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Thomas Munro
On Mon, Mar 21, 2016 at 3:46 AM, Andres Freund wrote: > On 2016-03-21 01:31:30 +1300, Thomas Munro wrote: >> I couldn't get the second patch to apply for some reason, > > Weird? Even efter appying the first one first? > > >> but I have been trying out your &quo

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-21 Thread Thomas Munro
On Mon, Mar 21, 2016 at 6:09 PM, Andres Freund wrote: > On 2016-03-21 11:52:43 +1300, Thomas Munro wrote: >> * I would be interested in writing a kqueue implementation of this for >> *BSD (and MacOSX?) at some point if someone doesn't beat me to it. > > I hoped t

[HACKERS] Typo in comment

2016-03-26 Thread Thomas Munro
Hi, Here are a couple of patches to fix a typo in a comment in latch.c: - * The memory barrier has be to be placed here to ensure that any flag + * The memory barrier has to be placed here to ensure that any flag Thanks, -- Thomas Munro http://www.enterprisedb.com typo.patch Description

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-03-26 Thread Thomas Munro
On Sat, Mar 26, 2016 at 5:04 AM, David Steele wrote: > Hi Thomas, > > On 3/13/16 8:20 PM, Thomas Munro wrote: > >> <...> I will have another look at this in >> a few days but for now I need to do some other things, so I'm posting >> these observat

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-03-28 Thread Thomas Munro
On Tue, Mar 29, 2016 at 1:56 AM, Thomas Munro wrote: > On Mon, Mar 28, 2016 at 8:54 PM, Michael Paquier > wrote: >> I have been also thinking a lot about this patch, and the fact that >> the WAL receiver latch is being used within the internals of >> libpqwalreceiver h

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-03-29 Thread Thomas Munro
tandby). Also I fixed a silly bug in SyncRepWaitForLSN when capping the mode. I have also renamed XACT_COMPLETION_SYNC_APPLY_FEEDBACK to the more general XACT_COMPLETION_APPLY_FEEDBACK, because the later 0004 patch will use it for a more general purpose than synchronous_commit. -- Thomas Munro http://www.enterprisedb.com 0

[HACKERS] kqueue

2016-03-29 Thread Thomas Munro
dZ_mC+V3YtB79zf27280nign8MKOLxy2FKhvc1RzN=g...@mail.gmail.com [2] https://github.com/libevent/libevent/commit/5602e451ce872d7d60c640590113c5a81c3fc389 -- Thomas Munro http://www.enterprisedb.com kqueue-v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

[HACKERS] FATAL: could not send end-of-streaming message to primary: no COPY in progress

2016-03-30 Thread Thomas Munro
server (no error necessary), unclean shutdown of remote server, and timeline negotiation? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-03-30 Thread Thomas Munro
andby \"%s\" is now the synchronous standby with priority %u", + application_name, MyWalSnd->sync_standby_priority))); s/ the / a / offered by a transaction commit. This level of protection is referred -to as 2-safe replication in computer science theory. +t

[HACKERS] Truncation of identifiers

2016-01-13 Thread Thomas Munro
e a second long name that collides with the first after truncation? I suppose there could be a GUC truncate_oversized_identifiers defaulting to off, which could be turned on by those who really prefer the current truncate-but-raise-NOTICE behaviour. -- Thomas Munro http://www.enterprisedb.com -

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-01-19 Thread Thomas Munro
On Wed, Dec 30, 2015 at 5:15 PM, Thomas Munro wrote: > On Wed, Nov 18, 2015 at 11:50 PM, Thomas Munro > wrote: >> Here is a new version of the patch with a few small improvements: >> ... >> [causal-reads-v3.patch] > > That didn't apply after 6e7b3359 (which fix

Re: [HACKERS] proposal: add 'waiting for replication' to pg_stat_activity.state

2016-01-28 Thread Thomas Munro
should use "" instead of <> and should come after #include "miscadmin.h". Thinking of other patches in flight, I think I'd want the proposed N-sync standbys feature to be able to explain in more detail what it's waiting for (and likewise my causal reads proposal

[HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-01-31 Thread Thomas Munro
message-id/flat/20140221002001.29130.27...@wrigleys.postgresql.org -- Thomas Munro http://www.enterprisedb.com ssi-read-write-unique-wip.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-02-03 Thread Thomas Munro
On Wed, Feb 3, 2016 at 10:59 PM, Amit Langote wrote: > There seems to be a copy-pasto there - shouldn't that be: > > + if (walsndctl->lsn[SYNC_REP_WAIT_FLUSH] < MyWalSnd->flush) Indeed, thanks! New patch attached. -- Thomas Munro http://www.enterprisedb.com causal-rea

Re: [HACKERS] WIP: Detecting SSI conflicts before reporting constraint violations

2016-02-03 Thread Thomas Munro
On Thu, Feb 4, 2016 at 7:48 AM, Robert Haas wrote: > On Sun, Jan 31, 2016 at 5:19 PM, Thomas Munro > wrote: >> As described in a recent Reddit discussion[1] and bug report 9301[2], >> there are scenarios where overlapping concurrent read-write sequences >> produce serial

Re: [HACKERS] count_nulls(VARIADIC "any")

2016-02-08 Thread Thomas Munro
nk it would >> be better as "num_nonnulls", as I see Oleksandr suggested already. > > Not hearing any complaints, I pushed it with that change and some other > cosmetic adjustments. Would num_values be a better name than num_nonnulls? -- Thomas Munro http://www.

Re: [HACKERS] [ADMIN] 9.5 new setting "cluster name" and logging

2016-02-08 Thread Thomas Munro
(this was part of the cluster_name patch that didn't go it). Given that csvlog's output format is hardcoded in write_csvlog, how is it supposed to evolve without upsetting consumers of this data? Wouldn't we first need to add a GUC that lets you control the columns it outputs?

Re: [HACKERS] count_nulls(VARIADIC "any")

2016-02-08 Thread Thomas Munro
On Tue, Feb 9, 2016 at 9:26 AM, Tom Lane wrote: > Thomas Munro writes: >> Would num_values be a better name than num_nonnulls? > > If "value" is a term that excludes null values, it's news to me. Ah, right, I was thinking of null as the absence of a value. But

Re: [HACKERS] Proposal: "Causal reads" mode for load balancing reads without stale data

2016-02-21 Thread Thomas Munro
On Mon, Feb 22, 2016 at 2:10 AM, Thom Brown wrote: > On 3 February 2016 at 10:46, Thomas Munro > wrote: >> On Wed, Feb 3, 2016 at 10:59 PM, Amit Langote >> wrote: >>> There seems to be a copy-pasto there - shouldn't that be: >>> >>> + if (wal

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

2016-02-21 Thread Thomas Munro
arallelism of queries involving an FDW should not be allowed if your transaction has written through the FDW. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

2016-02-22 Thread Thomas Munro
On Tue, Feb 23, 2016 at 4:03 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Feb 22, 2016 at 11:02 AM, Thomas Munro >> wrote: >>> The postgres_fdw failure is a visibility-of-my-own-uncommitted-work >>> problem. The first command in a transaction updates a

Re: [HACKERS] [PATH] Correct negative/zero year in to_date/to_timestamp

2016-02-22 Thread Thomas Munro
| to_timestamp > ---+--- > 0010-01-01 00:00:00+00 BC | 0010-01-01 00:00:00+00 BC > (1 row) > > > Testings, complains, advice, comment improvements are very appreciated. This seems to be a messy topic. The usage of "AD" an

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

2016-02-23 Thread Thomas Munro
apshots and allow parallelism in limited cases if it can somehow prove there have been no uncommitted writes; and non-MVCC/snapshot RDBMSs might be OK in lower isolation levels if you haven't written anything or have explicitly opted in to uncommitted reads (otherwise you'd risk invisible

Re: [HACKERS] postgres_fdw vs. force_parallel_mode on ppc

2016-02-23 Thread Thomas Munro
On Wed, Feb 24, 2016 at 5:48 PM, Thomas Munro wrote: > Here is a first pass at that. [...] On Wed, Feb 24, 2016 at 1:23 AM, Robert Haas wrote: > file_fdw is parallel-safe, ... And here is a patch to apply on top of the last one, to make file_fdw return true. But does it really work cor

Re: [HACKERS] Random note of encouragement

2016-02-24 Thread Thomas Munro
this is a devel release, things may change, blah blah. But still, >> something has changed for the better here! > > Wow, that is cool. Can anyone suggest which commit improved this? Since it sums numerics, maybe integer transition functions from commit 959277a4f579da5243968c7500695

Re: [HACKERS] Random note of encouragement

2016-02-24 Thread Thomas Munro
On Thu, Feb 25, 2016 at 1:43 PM, Bruce Momjian wrote: > On Thu, Feb 25, 2016 at 12:50:06PM +1300, Thomas Munro wrote: >> On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian wrote: >> > On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote: >> >> Now when I r

Re: [HACKERS] WIP: Barriers

2016-08-17 Thread Thomas Munro
On Tue, Aug 16, 2016 at 1:55 AM, Robert Haas wrote: > On Sat, Aug 13, 2016 at 7:18 PM, Thomas Munro > wrote: >> My use case for this is coordinating the phases of parallel hash >> joins, but I strongly suspect there are other cases. Parallel sort >> springs to mind,

[HACKERS] synchronous_commit = remote_flush

2016-08-17 Thread Thomas Munro
t;on" to control syncrep, and also the people who use "off" vs "on" to control asynchronous commit on single-node systems. Is there any sensible way to do that, or is it not broken and I should pipe down, or is it just far too entrenched and never going to change? --

Re: [HACKERS] synchronous_commit = remote_flush

2016-08-21 Thread Thomas Munro
On Fri, Aug 19, 2016 at 7:32 PM, Masahiko Sawada wrote: > On Fri, Aug 19, 2016 at 5:25 AM, Robert Haas wrote: >> On Thu, Aug 18, 2016 at 12:22 AM, Thomas Munro >> wrote: >>> To do something about the confusion I keep seeing about what exactly >>> &qu

Re: [HACKERS] synchronous_commit = remote_flush

2016-08-21 Thread Thomas Munro
re likely you'd want some combination: 2-safe or group-safe on some subset of servers to satisfy your durability requirements, and applied on some other perhaps larger subset of servers for consistency. But this is just water cooler handwaving. [1] https://infoscience.epfl.ch/rec

Re: [HACKERS] dsm_unpin_segment

2016-08-21 Thread Thomas Munro
On Sat, Aug 20, 2016 at 11:37 PM, Amit Kapila wrote: > On Tue, Aug 9, 2016 at 10:07 AM, Thomas Munro > wrote: >> On Tue, Aug 9, 2016 at 12:53 PM, Tom Lane wrote: >>> The larger picture here is that Robert is exhibiting a touching but >>> unfounded faith that exte

Re: [HACKERS] dsm_unpin_segment

2016-08-22 Thread Thomas Munro
andle count of postmaster is incremented > by 1 and then by using dsm_demo_unpin_segment() unpinned the segment > which decrements the Handle count in Postmaster. Thanks! -- Thomas Munro http://www.enterprisedb.com dsm-unpin-segment-v4.patch Description: Binary data -- Sent via pgsql-ha

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2016-08-22 Thread Thomas Munro
On Tue, Aug 23, 2016 at 8:41 AM, Robert Haas wrote: > On Tue, Aug 16, 2016 at 7:41 PM, Thomas Munro > wrote: >> I still think it's worth thinking about something along these lines on >> Linux only, where holey Swiss tmpfs files can bite you. Otherwise >> disablin

Re: [HACKERS] Server crash due to SIGBUS(Bus Error) when trying to access the memory created using dsm_create().

2016-08-22 Thread Thomas Munro
0774us base + memset = 602925us base + fallocate + memset = 655433us -- Thomas Munro http://www.enterprisedb.com #include #include #include #include #include #include #include #define SEGMENT_NAME "/my_test_segment" int main(int argc, char *argv[]) { int loops, i; off_t size; bool

Re: [HACKERS] [RFC] Change the default of update_process_title to off

2016-08-24 Thread Thomas Munro
ing several GB of buffer pool. I wonder if that might help Postgres on Windows. This could be useful as a starting point to test that theory: https://www.postgresql.org/message-id/CAEepm%3D075-bgHi_VDt4SCAmt%2Bo_%2B1XaRap2zh7XwfZvT294oHA%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] kqueue

2016-09-09 Thread Thomas Munro
bsd-src/blob/master/sys/kern/kern_event.c [5] https://github.com/openbsd/src/blob/master/sys/kern/kern_event.c [6] https://github.com/opensource-apple/xnu/blob/master/bsd/kern/kern_event.c [7] http://marc.info/?l=freebsd-arch&m=98147346707952&w=2 -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Merge Join with an Index Optimization

2016-09-11 Thread Thomas Munro
trictly necessary, I don't know). There are a whole lot of interesting execution tricks that could be enabled by btree skipping (see Oracle, DB2, MySQL, SQLite). DISTINCT was the simplest thing I could think of where literally every other RDBMS beats us because we don't have it. But t

Re: [HACKERS] kqueue

2016-09-13 Thread Thomas Munro
51654 TPS -> 55739 TPS = 7.9% improvement GCC 6.1.0 from MacPorts: 52552 TPS -> 55143 TPS = 4.9% improvement I reran the tests under FreeBSD 10.3 on a 4 core laptop and again saw absolutely no measurable difference at 1, 4 or 24 clients. Maybe a big enough server could be made to contend on the p

Re: [HACKERS] kqueue

2016-09-14 Thread Thomas Munro
On Thu, Sep 15, 2016 at 10:48 AM, Keith Fiske wrote: > Thomas Munro brought up in #postgresql on freenode needing someone to test a > patch on a larger FreeBSD server. I've got a pretty decent machine (3.1Ghz > Quad Core Xeon E3-1220V3, 16GB ECC RAM, ZFS mirror on WD Red HDD) so off

Re: [HACKERS] make async slave to wait for lsn to be replayed

2016-09-14 Thread Thomas Munro
ng 'wait forever', and another meaning 'don't wait at all: if it's not applied yet, then timeout immediately'. In any case I'd consider using names for special wait times and using those for clarity: WAITLSN_INFINITE_WAIT, WAITLSN_NO_WAIT. Later I'll have feedba

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-15 Thread Thomas Munro
essage-id/flat/CAEepm%3D2i78TOJeV4O0-0meiihiRfVQ29ur7%3DMBHxsUKaPSWeAg%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] kqueue

2016-09-15 Thread Thomas Munro
On Thu, Sep 15, 2016 at 11:04 AM, Thomas Munro wrote: > On Thu, Sep 15, 2016 at 10:48 AM, Keith Fiske wrote: >> Thomas Munro brought up in #postgresql on freenode needing someone to test a >> patch on a larger FreeBSD server. I've got a pretty decent machine (3.1Ghz >>

Re: [HACKERS] recovery_min_apply-delay and remote_apply

2016-09-16 Thread Thomas Munro
dangerous > if someone doesn't take care enough. Yes, I missed that sentence. Thanks. > I think we need a doc patch for that at least, see attached patch against > master, but 9.6 should have a corrected one, too. +1 -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql

Re: [HACKERS] PATCH: Keep one postmaster monitoring pipe per process

2016-09-19 Thread Thomas Munro
code in basebackup.c also waits for WL_POSTMASTER_DEATH but doesn't check for it in the return value *or* call PostmasterIsAlive(). I'm not sure what to make of that. I didn't test it but it looks like maybe it would continue running after postmaster death but not honour the thr

Re: [HACKERS] Tracking wait event for latches

2016-09-20 Thread Thomas Munro
aps an extra boolean would be needed to record whether postmaster death is in there so we could deduce whether there are any sockets). It would be interesting specifically for the case of FDWs where it would be nice to be able to see clearly that it's waiting for a remote server ("Socket"). It may also be interesting to know if there is a timeout. Postmaster death doesn't seem newsworthy, we're nearly always also waiting for that exceptional event so it'd just be clutter to report it. [1] http://stackoverflow.com/questions/147267/easy-way-to-use-variables-of-enum-types-as-string-in-c -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Tracking wait event for latches

2016-09-20 Thread Thomas Munro
On Wed, Sep 21, 2016 at 3:40 PM, Michael Paquier wrote: > On Wed, Sep 21, 2016 at 8:13 AM, Thomas Munro > wrote: >> It looks like this array wants to be in alphabetical order, but it >> isn't quite. Also, perhaps a compile time assertion about the size of >> the a

Re: [HACKERS] Tracking wait event for latches

2016-09-21 Thread Thomas Munro
On Thu, Sep 22, 2016 at 1:23 AM, Robert Haas wrote: > On Tue, Sep 20, 2016 at 7:13 PM, Thomas Munro > wrote: >> This paragraph seems a bit confused. I suggest something more like >> this: "The server process is waiting for one or more sockets, a timer >> or an

Re: [HACKERS] Tracking wait event for latches

2016-09-22 Thread Thomas Munro
On Fri, Sep 23, 2016 at 1:49 AM, Robert Haas wrote: > On Wed, Sep 21, 2016 at 5:49 PM, Thomas Munro > wrote: >>> Moreover, it's pretty confusing that we have this general concept of >>> wait events in pg_stat_activity, and then here the specific type of >>&g

Re: [HACKERS] ICU integration

2016-09-22 Thread Thomas Munro
.out @@ -0,0 +1,1076 @@ +/* + * This test is for Linux/glibc systems and assumes that a full set of + * locales is installed. It must be run in a database with UTF-8 encoding, + * because other encodings don't support all the characters used. + */ Should say ICU. [1] https://www.postgresql.

[HACKERS] Refactor StartupXLOG?

2016-09-23 Thread Thomas Munro
itters who have to backpatch bug fixes? Is it a project goal to reduce the size of large complicated functions like StartupXLOG and heap_update? It seems like a good way for new players to learn how they work. -- Thomas Munro http://www.enterprisedb.com refactor-startupxlog-sketch.patch Descri

[HACKERS] Complete LOCK TABLE ... IN ACCESS|ROW|SHARE

2016-09-23 Thread Thomas Munro
Hi After LOCK TABLE ... IN ACCESS|ROW|SHARE we run out of completions. Here's a patch to improve that, for November. -- Thomas Munro http://www.enterprisedb.com complete-lock-table.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Refactor StartupXLOG?

2016-09-24 Thread Thomas Munro
On Sat, Sep 24, 2016 at 8:24 PM, Heikki Linnakangas wrote: > On 09/24/2016 05:01 AM, Thomas Munro wrote: >> >> What would the appetite be for that kind of refactoring work, >> considering the increased burden on committers who have to backpatch >> bug fixes? Is it a

Re: [HACKERS] ICU integration

2016-09-24 Thread Thomas Munro
On Sat, Sep 24, 2016 at 10:13 PM, Peter Geoghegan wrote: > On Fri, Sep 23, 2016 at 7:27 AM, Thomas Munro > wrote: >> It looks like varstr_abbrev_convert calls strxfrm unconditionally >> (assuming TRUST_STRXFRM is defined). This needs to >> use ucol_getSortKey instead whe

Re: [HACKERS] [GENERAL] C++ port of Postgres

2016-09-25 Thread Thomas Munro
* from a transaction that is not yet visible to snapshots; * compare the comments at the head of tqual.c. */ - *current = thisentry; + COPY_QUEUE_POS(*current, thisentry); reachedStop = true; break; } -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-25 Thread Thomas Munro
at a large company with many database users. I think experienced users probably initially felt mollycoddled when they first encountered the error but I'm sure that some were secretly glad of its existence from time to time... I think it's a useful feature for users who want it, and

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-25 Thread Thomas Munro
On Mon, Sep 26, 2016 at 1:18 PM, Thomas Munro wrote: > underscore to a minus in various places. That fixes these errors: Correction: s/these errors:/the above errors./ -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] VACUUM's ancillary tasks

2016-09-25 Thread Thomas Munro
ible? Alternatively, we should consider bumping changes_since_analyze in this case too, if you're going to do it in the rewrite case. -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-25 Thread Thomas Munro
On Mon, Sep 26, 2016 at 1:18 PM, Thomas Munro wrote: > On Mon, Sep 19, 2016 at 4:02 PM, David Fetter wrote: > > > > [training_wheels_004.patch] > > openjade:filelist.sgml:144:16:E: character "_" invalid: only parameter > literal, "CDATA",

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2016-09-25 Thread Thomas Munro
On Mon, Sep 26, 2016 at 5:11 PM, Thomas Munro wrote: > It seems that the version of docbook that you get if you follow the > instructions[1] ... > And I mean these instructions: [1] https://www.postgresql.org/docs/devel/static/docguide-toolsets.html -- Thomas Munro http://www.enterprisedb.com

Re: [HACKERS] Tracking wait event for latches

2016-09-25 Thread Thomas Munro
T, 0, WAIT_IPC, WE_MQ_RECEIVE); If the class really is strictly implied by the WaitEventIdentifier, then do we really need to supply it everywhere when calling the various wait functions? That's going to be quite a few functions: WaitLatch, WaitLatchOrSocket, WaitEventSetWait for no

Re: [HACKERS] VACUUM's ancillary tasks

2016-09-26 Thread Thomas Munro
On Tue, Sep 27, 2016 at 2:33 AM, Tom Lane wrote: > Thomas Munro writes: >> I noticed that ATExecAlterColumnType does this: >> * Drop any pg_statistic entry for the column, since it's now wrong type > >> What if there is no rewrite, because

Re: [HACKERS] Tracking wait event for latches

2016-09-27 Thread Thomas Munro
On Mon, Sep 26, 2016 at 7:07 PM, Michael Paquier wrote: > On Mon, Sep 26, 2016 at 1:46 PM, Thomas Munro > wrote: >> If the class really is strictly implied by the WaitEventIdentifier, >> then do we really need to supply it everywhere when calling the >> various wait fun

Re: [HACKERS] make async slave to wait for lsn to be replayed

2016-09-27 Thread Thomas Munro
On Thu, Sep 15, 2016 at 2:41 PM, Thomas Munro wrote: > On Thu, Sep 1, 2016 at 2:16 AM, Ivan Kartyshov > wrote: >> Hi hackers, >> >> Few days earlier I've finished my work on WAITLSN statement utility, so I’d >> like to share it. >> [...] >> Your

Re: [HACKERS] Supporting huge pages on Windows

2016-09-27 Thread Thomas Munro
red_buffers performance. [1] https://www.postgresql.org/message-id/CAEepm=075-bghi_vdt4scamt+o_+1xarap2zh7xwfzvt294...@mail.gmail.com [2] https://www.postgresql.org/message-id/0A3221C70F24FB45833433255569204D1F5EE995@G01JPEXMBYT05 -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing li

Re: [HACKERS] Tracking wait event for latches

2016-09-27 Thread Thomas Munro
ther patch, but it doesn't matter much because those processes won't show up yet anyway. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [sqlsmith] stuck spinlock in pg_stat_get_wal_receiver after OOM

2017-10-03 Thread Thomas Munro
just latches anyway, but it may be a better abstraction. On the other hand I'm not sure how waits on a ConditionVariable would be multiplexed with IO (a distinct wait event, or leaky abstraction where the caller relies on the fact that it's built on MyProc->latch, something else?). -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Hash take II

2017-10-05 Thread Thomas Munro
ble temporary tablespaces, but it's a terrible API, since you have to promise to use the same stripe number when opening the same name later... Maybe I should use a hash of the name for that instead. Thoughts? -- Thomas Munro http://www.enterprisedb.com 0006-Remove-BufFile-s-isTemp-flag.

Re: [HACKERS] oversight in EphemeralNamedRelation support

2017-10-09 Thread Thomas Munro
> PFA a simple patch to fix this issue, with updated regression test. Thanks! I suppose we could consider moving the schemaname check into getRTEForSpecialRelationType(), since otherwise both callers need to do that (and as you discovered, one forgot). -- Thomas Munro http://www.e

[HACKERS] Continuous integration on Windows?

2017-10-11 Thread Thomas Munro
he pointer). I wonder... has anyone here with Microsoft know-how ever tried to produce an appveyor.yml file that would do a MSVC build and check-world? -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] oversight in EphemeralNamedRelation support

2017-10-12 Thread Thomas Munro
On Fri, Oct 13, 2017 at 8:50 AM, Tom Lane wrote: > Julien Rouhaud writes: >> On Mon, Oct 9, 2017 at 10:43 PM, Thomas Munro >> wrote: >>> I suppose we could consider moving the schemaname check into >>> getRTEForSpecialRelationType(), since otherwise both caller

Re: [HACKERS] Continuous integration on Windows?

2017-10-12 Thread Thomas Munro
rent compilers, different word size, add valgrind, ...). I don't know if it would ever make sense to have standardised CI control files in the tree -- many projects do -- but it's very easy to carry a commit that adds them in development branches but drop it as part of the for

Re: [HACKERS] oversight in EphemeralNamedRelation support

2017-10-12 Thread Thomas Munro
On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane wrote: > Thomas Munro writes: >> Before that, CTE used as modify targets produced a different error message: > >> postgres=# WITH d AS (SELECT 42) INSERT INTO d VALUES (1); >> ERROR: relation "d" does not exist >&

Re: [HACKERS] oversight in EphemeralNamedRelation support

2017-10-12 Thread Thomas Munro
On Fri, Oct 13, 2017 at 12:46 PM, Tom Lane wrote: > Thomas Munro writes: >> On Fri, Oct 13, 2017 at 10:01 AM, Tom Lane wrote: >>> The CTE was simply not part of the available namespace for the INSERT's >>> target, so it found the regular table instead. v10 has

Re: [HACKERS] Log LDAP "diagnostic messages"?

2017-10-12 Thread Thomas Munro
On Fri, Oct 13, 2017 at 3:59 PM, Peter Eisentraut wrote: > On 9/24/17 07:00, Thomas Munro wrote: >> Fair point. In that case there are a few others we should consider >> moving down too for consistency, like in the attached. > >> Thanks, that is much tidier. Done

  1   2   3   4   5   6   7   8   9   >