On Tue, May 31, 2016 at 10:23 PM, Tom Lane wrote:
> Noah Misch writes:
>> On Tue, May 31, 2016 at 10:09:05PM -0400, Robert Haas wrote:
>>> What I *think* is going on here is:
>>> - ac1d794 lowered performance
>>> - backend_flush_after with a non-zero default lowered performance with
>>> a vengean
Noah Misch writes:
> On Tue, May 31, 2016 at 10:09:05PM -0400, Robert Haas wrote:
>> What I *think* is going on here is:
>> - ac1d794 lowered performance
>> - backend_flush_after with a non-zero default lowered performance with
>> a vengeance
>> - 98a64d0 repaired the damage done by ac1d794, or mu
On Tue, May 31, 2016 at 10:09:05PM -0400, Robert Haas wrote:
> On Sun, May 29, 2016 at 1:40 AM, Noah Misch wrote:
> > On Fri, Dec 25, 2015 at 08:08:15PM +0300, Васильев Дмитрий wrote:
> >> I suddenly found commit ac1d794 gives up to 3 times performance
> >> degradation.
> >>
> >> I tried to run p
On Sun, May 29, 2016 at 1:40 AM, Noah Misch wrote:
> On Fri, Dec 25, 2015 at 08:08:15PM +0300, Васильев Дмитрий wrote:
>> I suddenly found commit ac1d794 gives up to 3 times performance degradation.
>>
>> I tried to run pgbench -s 1000 -j 48 -c 48 -S -M prepared on 70 CPU-core
>> machine:
>> commi
On Fri, Dec 25, 2015 at 08:08:15PM +0300, Васильев Дмитрий wrote:
> I suddenly found commit ac1d794 gives up to 3 times performance degradation.
>
> I tried to run pgbench -s 1000 -j 48 -c 48 -S -M prepared on 70 CPU-core
> machine:
> commit ac1d794 gives me 363,474 tps
> and previous commit a05dc
On Mon, Mar 21, 2016 at 6:09 PM, Andres Freund wrote:
> On 2016-03-21 11:52:43 +1300, Thomas Munro wrote:
>> * I would be interested in writing a kqueue implementation of this for
>> *BSD (and MacOSX?) at some point if someone doesn't beat me to it.
>
> I hoped that somebody would do that - that'd
Hi,
I've pushed the new API. We might want to use it in more places...
On 2015-12-25 20:08:15 +0300, Васильев Дмитрий wrote:
> I suddenly found commit ac1d794 gives up to 3 times performance degradation.
>
> I tried to run pgbench -s 1000 -j 48 -c 48 -S -M prepared on 70 CPU-core
> machine:
> co
Hi,m
On 2016-03-21 11:26:43 +0530, Amit Kapila wrote:
> LOG: database system was shut down at 2016-03-21 11:17:13 IST
> LOG: MultiXact member wraparound protections are now enabled
> LOG: database system is ready to accept connections
> LOG: autovacuum launcher started
> FATAL: failed to set
On Mon, Mar 21, 2016 at 10:26 AM, Amit Kapila
wrote:
>
> On Mon, Mar 21, 2016 at 10:21 AM, Andres Freund
wrote:
>>
>>
>>
>> On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila <
amit.kapil...@gmail.com> wrote:
>>
>> >The article pointed by you justifies that the way ResetEvent is done by
>> >patc
Hi,
On 2016-03-21 11:52:43 +1300, Thomas Munro wrote:
> The contract that I invented here is that an async-aware parent node
> can ask any child node "are you ready?" and get back various answers
> including an fd which means please don't call ExecProcNode until this
> fd is ready to read. But a
On Mon, Mar 21, 2016 at 10:21 AM, Andres Freund wrote:
>
>
> On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila <
> amit.kapil...@gmail.com> wrote:
>
> >The article pointed by you justifies that the way ResetEvent is done by
> >patch is correct. I am not sure, but you can weigh, if there is a n
On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila
wrote:
>The article pointed by you justifies that the way ResetEvent is done by
>patch is correct. I am not sure, but you can weigh, if there is a need
>of
>comment so that if we want enhance this part of code (or want to write
>something si
On Sun, Mar 20, 2016 at 7:13 AM, Andres Freund wrote:
>
> On 2016-03-19 15:43:27 +0530, Amit Kapila wrote:
> > On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund
wrote:
> > >
> > > On March 18, 2016 11:52:08 PM PDT, Amit Kapila <
amit.kapil...@gmail.com>
> > wrote:
> > > >> >Won't the new code needs
On Mon, Mar 21, 2016 at 3:46 AM, Andres Freund wrote:
> On 2016-03-21 01:31:30 +1300, Thomas Munro wrote:
>> I couldn't get the second patch to apply for some reason,
>
> Weird? Even efter appying the first one first?
>
>
>> but I have been trying out your "latch" branch on some different OSs
>> a
On Mon, Mar 21, 2016 at 3:46 AM, Andres Freund wrote:
> On 2016-03-21 01:31:30 +1300, Thomas Munro wrote:
>> I couldn't get the second patch to apply for some reason,
>
> Weird? Even efter appying the first one first?
Ah, I was using patch -p1. I needed to use git am, which understands
how to re
On 2016-03-18 05:56:41 -0400, Robert Haas wrote:
> >> 0 at the top of the loop and skip it forthwith if so.
> >
> > You mean in WaitEventSetWait()? There's
> > else if (rc == 0)
> > {
> > break;
> > }
> > which is the timeout c
On 2016-03-21 01:31:30 +1300, Thomas Munro wrote:
> I couldn't get the second patch to apply for some reason,
Weird? Even efter appying the first one first?
> but I have been trying out your "latch" branch on some different OSs
> and porting some code that does a bunch of waiting on many sockets
On Sun, Mar 20, 2016 at 5:14 PM, Andres Freund wrote:
> On 2016-03-19 18:45:36 -0700, Andres Freund wrote:
>> On 2016-03-19 16:44:49 +0530, Amit Kapila wrote:
>> > On Fri, Mar 18, 2016 at 1:34 PM, Andres Freund wrote:
>> > >
>> > >
>> > > Attached is a significantly revised version of the earlier
On 2016-03-17 09:01:36 -0400, Robert Haas wrote:
> > * Right now the caller has to allocate the WaitEvents he's waiting for
> > locally (likely on the stack), but we also could allocate them as part
> > of the WaitEventSet. Not sure if that'd be a benefit.
>
> I'm not seeing this. What do you
On 2016-03-17 09:40:08 -0400, Robert Haas wrote:
> On Thu, Mar 17, 2016 at 9:01 AM, Robert Haas wrote:
> > I'll look at 0005 next, but thought I would send these comments along first.
>
> 0005: This is obviously very much WIP, but I think the overall
> direction of it is good.
> 0006: Same.
>
>
On 2016-03-19 18:45:36 -0700, Andres Freund wrote:
> On 2016-03-19 16:44:49 +0530, Amit Kapila wrote:
> > On Fri, Mar 18, 2016 at 1:34 PM, Andres Freund wrote:
> > >
> > >
> > > Attached is a significantly revised version of the earlier series. Most
> > > importantly I have:
> > > * Unified the wi
On 2016-03-16 15:08:07 -0400, Robert Haas wrote:
> On Wed, Mar 16, 2016 at 2:52 PM, Andres Freund wrote:
> > How about the following sketch of an API
> >
> > typedef struct LatchEvent
> > {
> > uint32 events; /* interesting events */
> > uint32 revents; /* returned events */
> >
On 2016-03-19 16:44:49 +0530, Amit Kapila wrote:
> On Fri, Mar 18, 2016 at 1:34 PM, Andres Freund wrote:
> >
> >
> > Attached is a significantly revised version of the earlier series. Most
> > importantly I have:
> > * Unified the window/unix latch implementation into one file (0004)
> >
>
> Afte
On 2016-03-19 15:43:27 +0530, Amit Kapila wrote:
> On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund wrote:
> >
> > On March 18, 2016 11:52:08 PM PDT, Amit Kapila
> wrote:
> > >> >Won't the new code needs to ensure that ResetEvent(latchevent)
> > >should
> > >> >get
> > >> >called in case WaitForMu
On 2016-02-08 17:49:18 +0900, Kyotaro HORIGUCHI wrote:
> How about allowing registration of a callback for every waiting
> socket. The signature of the callback function would be like
I don't think a a callback based API is going to serve us well. Most of
the current latch callers would get notice
On Thu, Mar 17, 2016 at 9:01 AM, Robert Haas wrote:
> I'll look at 0005 next, but thought I would send these comments along first.
0005: This is obviously very much WIP, but I think the overall
direction of it is good.
0006: Same.
I think you should use PGINVALID_SOCKET rather than -1 in various
On Wed, Mar 16, 2016 at 3:25 PM, Andres Freund wrote:
>> > - Given current users we don't need a large amount of events, so having
>> > to iterate through the registered events doesn't seem bothersome. We
>> > could however change the api to be something like
>> >
>> > int WaitLatchEventSet(
Hi,
On 2016-03-17 09:01:36 -0400, Robert Haas wrote:
> 0001: Looking at this again, I'm no longer sure this is a bug.
> Doesn't your patch just check the same conditions in the opposite
> order?
Yes, that's what's required
> 0004:
>
> + * drain it everytime WaitLatchOrSocket() is used.
Andres Freund wrote:
> From 1d444b0855dbf65d66d73beb647b772fff3404c8 Mon Sep 17 00:00:00 2001
> From: Andres Freund
> Date: Fri, 18 Mar 2016 00:52:07 -0700
> Subject: [PATCH 4/5] Combine win32 and unix latch implementations.
>
> Previously latches for windows and unix had been implemented in
> d
On 2016-01-14 12:07:23 -0500, Robert Haas wrote:
> On Thu, Jan 14, 2016 at 12:06 PM, Andres Freund wrote:
> > On 2016-01-14 11:31:03 -0500, Robert Haas wrote:
> >> On Thu, Jan 14, 2016 at 10:56 AM, Andres Freund wrote:
> >> I think your idea of a data structure the encapsulates a set of events
>
Hi,
On 2016-03-16 15:41:28 -0400, Robert Haas wrote:
> On Wed, Mar 16, 2016 at 3:25 PM, Andres Freund wrote:
> >> > - Given current users we don't need a large amount of events, so having
> >> > to iterate through the registered events doesn't seem bothersome. We
> >> > could however change t
On 2016-03-18 20:14:07 +0530, Amit Kapila wrote:
> + * from inside a signal handler in latch_sigusr1_handler().
>
> *
>
> * Note: we assume that the kernel calls involved in drainSelfPipe()
>
> * and SetLatch() will provide adequate synchronization on machines
>
> * with weak memory ord
On Fri, Mar 18, 2016 at 1:53 PM, Andres Freund wrote:
> On 2016-03-18 05:56:41 -0400, Robert Haas wrote:
>> >> 0 at the top of the loop and skip it forthwith if so.
>> >
>> > You mean in WaitEventSetWait()? There's
>> > else if (rc == 0)
>> > {
>> >
Hi,
On 2016-03-17 09:01:36 -0400, Robert Haas wrote:
> 0001: Looking at this again, I'm no longer sure this is a bug.
> Doesn't your patch just check the same conditions in the opposite
> order?
Which is important, because what's in what pfds[x] depends on
wakeEvents. Folded it into a later patch
On Wed, Mar 16, 2016 at 10:04 PM, Andres Freund wrote:
> Questions:
> * I'm kinda inclined to merge the win32 and unix latch
> implementations. There's already a fair bit in common, and this is
> just going to increase that amount.
Don't care either way.
> * Right now the caller has to alloc
On Fri, Mar 18, 2016 at 1:34 PM, Andres Freund wrote:
>
>
> Attached is a significantly revised version of the earlier series. Most
> importantly I have:
> * Unified the window/unix latch implementation into one file (0004)
>
After applying patch 0004* on HEAD, using command patch -p1 <
, I am ge
On Fri, Mar 18, 2016 at 1:34 PM, Andres Freund wrote:
>
> Hi,
>
> On 2016-03-17 09:01:36 -0400, Robert Haas wrote:
> > 0001: Looking at this again, I'm no longer sure this is a bug.
> > Doesn't your patch just check the same conditions in the opposite
> > order?
>
> Which is important, because wha
On Thu, Mar 17, 2016 at 10:53 AM, Andres Freund wrote:
>> I wonder if there's a way to refactor this code to avoid having so
>> much cut-and-paste duplication.
>
> I guess you mean WaitEventSetWait() and WaitEventAdjust*? I've tried,
> and my attempt ended up look nearly unreadable, because of th
On 2016-03-16 15:41:28 -0400, Robert Haas wrote:
> On Wed, Mar 16, 2016 at 3:25 PM, Andres Freund wrote:
> >> > - Given current users we don't need a large amount of events, so having
> >> > to iterate through the registered events doesn't seem bothersome. We
> >> > could however change the ap
On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund wrote:
>
> On March 18, 2016 11:52:08 PM PDT, Amit Kapila
wrote:
> >> >Won't the new code needs to ensure that ResetEvent(latchevent)
> >should
> >> >get
> >> >called in case WaitForMultipleObjects() comes out when both
> >> >pgwin32_signal_event an
On Thu, Mar 17, 2016 at 10:57 AM, Andres Freund wrote:
> On 2016-03-17 09:01:36 -0400, Robert Haas wrote:
>> 0001: Looking at this again, I'm no longer sure this is a bug.
>> Doesn't your patch just check the same conditions in the opposite
>> order?
>
> Yes, that's what's required
I mean, they a
On 2016-03-18 14:00:58 -0400, Robert Haas wrote:
> No, I mean it should be quite common for a particular fd to have no
> events reported. If we're polling on 100 fds and 1 of them is active
> and the other 99 are just sitting there, we want to skip over the
> other 99 as quickly as possible.
cur_
On Thu, Mar 17, 2016 at 7:34 AM, Andres Freund wrote:
> Hi,
>
> * I can do a blind rewrite of the windows implementation, but I'm
> obviously not going to get that entirely right. So I need some help
> from a windows person to test this.
>
I can help you verifying the windows implementation.
On March 18, 2016 11:52:08 PM PDT, Amit Kapila wrote:
>On Sat, Mar 19, 2016 at 12:14 PM, Andres Freund
>wrote:
>>
>>
>>
>> On March 18, 2016 11:32:53 PM PDT, Amit Kapila
>
>wrote:
>> >On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund
>> >wrote:
>> >>
>> >> On 2016-03-18 20:14:07 +0530, Amit Kapi
On Wed, Mar 16, 2016 at 2:52 PM, Andres Freund wrote:
> How about the following sketch of an API
>
> typedef struct LatchEvent
> {
> uint32 events; /* interesting events */
> uint32 revents; /* returned events */
> int fd; /* fd associated with event */
> } LatchEvent;
>
>
On Sat, Mar 19, 2016 at 12:14 PM, Andres Freund wrote:
>
>
>
> On March 18, 2016 11:32:53 PM PDT, Amit Kapila
wrote:
> >On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund
> >wrote:
> >>
> >> On 2016-03-18 20:14:07 +0530, Amit Kapila wrote:
> >>
> >> > I have done some
> >> > tests on Windows with 0
On March 18, 2016 11:32:53 PM PDT, Amit Kapila wrote:
>On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund
>wrote:
>>
>> On 2016-03-18 20:14:07 +0530, Amit Kapila wrote:
>>
>> > I have done some
>> > tests on Windows with 0003 patch which includes running the
>regressions
>> > (vcregress check) and
On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund wrote:
>
> On 2016-03-18 20:14:07 +0530, Amit Kapila wrote:
>
> > I have done some
> > tests on Windows with 0003 patch which includes running the regressions
> > (vcregress check) and it passes. Will look into it tomorrow once again
and
> > share i
On Thu, Mar 17, 2016 at 11:17 AM, Andres Freund wrote:
> Right now, do use a WaitEventSet you'd do something like
> WaitEvent event;
>
> ModifyWaitEvent(FeBeWaitSet, 0, waitfor, NULL);
>
> WaitEventSetWait(FeBeWaitSet, 0 /* no timeout */, &event, 1
On 2016-03-18 15:49:51 -0300, Alvaro Herrera wrote:
> This seems a reasonable change, but I think that the use of WIN32 vs.
> LATCH_USE_WIN32 is pretty confusing. In particular, LATCH_USE_WIN32
> isn't actually used for anything ... I suppose we don't care since this
> is a temporary state of affa
Hello, I don't see how ac1d794 will be dealt, but I tried an
example implement of multi-socket version of WaitLatchOrSocket
using callbacks on top of the current master where ac1d794 has
not been removed yet.
At Thu, 14 Jan 2016 13:46:44 -0500, Robert Haas wrote
in
> On Thu, Jan 14, 2016 at 12:
On Thu, Feb 11, 2016 at 1:41 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Thu, Feb 11, 2016 at 1:19 PM, Andres Freund wrote:
>>> Well, I can't do anything about that right now. I won't have the time to
>>> whip up the new/more complex API we discussed upthread in the next few
>>> days. So ei
Robert Haas writes:
> On Thu, Feb 11, 2016 at 1:19 PM, Andres Freund wrote:
>> Well, I can't do anything about that right now. I won't have the time to
>> whip up the new/more complex API we discussed upthread in the next few
>> days. So either we go with a simpler API (e.g. pretty much a cleane
On Thu, Feb 11, 2016 at 1:19 PM, Andres Freund wrote:
>> > Putting it on the open items list sounds good to me.
>>
>> Well, OK, I've done that then. I don't really agree that it's not a
>> problem; the OP said he saw a 3x regression, and some of my colleagues
>> doing benchmarking are complaining
On 2016-02-11 13:09:27 -0500, Robert Haas wrote:
> On Thu, Feb 11, 2016 at 12:53 PM, Andres Freund wrote:
> >> One problem is that it makes for misleading results if you try to
> >> benchmark 9.5 against 9.6.
> >
> > You need a really beefy box to show the problem. On a large/new 2 socket
> > mach
On Thu, Feb 11, 2016 at 12:53 PM, Andres Freund wrote:
>> One problem is that it makes for misleading results if you try to
>> benchmark 9.5 against 9.6.
>
> You need a really beefy box to show the problem. On a large/new 2 socket
> machine the performance regression in in the 1-3% range for a pgb
On 2016-02-11 12:50:58 -0500, Robert Haas wrote:
> On Thu, Feb 11, 2016 at 12:48 PM, Tom Lane wrote:
> > Robert Haas writes:
> >> I think we should either get this fixed RSN or revert the problematic
> >> commit until we get it fixed. I'd be rather disappointed about the
> >> latter because I th
On Thu, Feb 11, 2016 at 12:48 PM, Tom Lane wrote:
> Robert Haas writes:
>> I think we should either get this fixed RSN or revert the problematic
>> commit until we get it fixed. I'd be rather disappointed about the
>> latter because I think this was a very good thing on the merits, but
>> probab
Robert Haas writes:
> I think we should either get this fixed RSN or revert the problematic
> commit until we get it fixed. I'd be rather disappointed about the
> latter because I think this was a very good thing on the merits, but
> probably not good enough to justify taking the performance hit
On Thu, Jan 14, 2016 at 12:28 PM, Tom Lane wrote:
> Andres Freund writes:
>> This brings me to something related: I'm wondering if we shouldn't merge
>> unix/win32_latch.c.
>
> Well, it's duplicated code on the one hand versus maze-of-ifdefs on the
> other. Feel free to try it and see, but I'm u
Hello, I am one who wants waiting on many sockets at once.
At Thu, 14 Jan 2016 18:55:51 +0100, Andres Freund wrote in
<20160114175551.gm10...@awork2.anarazel.de>
> On 2016-01-14 18:14:21 +0100, Andres Freund wrote:
> > On 2016-01-14 12:07:23 -0500, Robert Haas wrote:
> > > > Do we want to provid
On Thu, Jan 14, 2016 at 12:14 PM, Andres Freund wrote:
> On 2016-01-14 12:07:23 -0500, Robert Haas wrote:
>> > Do we want to provide a backward compatible API for all this? I'm fine
>> > either way.
>>
>> How would that work?
>
> I'm thinking of something like;
>
> int WaitOnLatchSet(LatchEventSet
On 2016-01-14 18:14:21 +0100, Andres Freund wrote:
> On 2016-01-14 12:07:23 -0500, Robert Haas wrote:
> > > Do we want to provide a backward compatible API for all this? I'm fine
> > > either way.
> >
> > How would that work?
>
> I'm thinking of something like;
>
> int WaitOnLatchSet(LatchEventSe
Andres Freund writes:
> This brings me to something related: I'm wondering if we shouldn't merge
> unix/win32_latch.c.
Well, it's duplicated code on the one hand versus maze-of-ifdefs on the
other. Feel free to try it and see, but I'm unsure it'd be an improvement.
regar
On 2016-01-14 18:14:21 +0100, Andres Freund wrote:
> I'm thinking of something like;
>
> int WaitOnLatchSet(LatchEventSet *set, int wakeEvents, long timeout);
>
> int
> WaitLatchOrSocket(volatile Latch *latch, int wakeEvents, pgsocket sock,long
> timeout)
> {
> LatchEventSet set;
>
>
On 2016-01-14 12:07:23 -0500, Robert Haas wrote:
> > Do we want to provide a backward compatible API for all this? I'm fine
> > either way.
>
> How would that work?
I'm thinking of something like;
int WaitOnLatchSet(LatchEventSet *set, int wakeEvents, long timeout);
int
WaitLatchOrSocket(volatil
Robert Haas writes:
> On Thu, Jan 14, 2016 at 12:06 PM, Andres Freund wrote:
>> Do we want to provide a backward compatible API for all this? I'm fine
>> either way.
> How would that work?
I see no great need to be backwards-compatible on this, especially if it
would complicate matters at all.
On Thu, Jan 14, 2016 at 12:06 PM, Andres Freund wrote:
> On 2016-01-14 11:31:03 -0500, Robert Haas wrote:
>> On Thu, Jan 14, 2016 at 10:56 AM, Andres Freund wrote:
>> I think your idea of a data structure the encapsulates a set of events
>> for which to wait is probably a good one. WaitLatch doe
On 2016-01-14 11:31:03 -0500, Robert Haas wrote:
> On Thu, Jan 14, 2016 at 10:56 AM, Andres Freund wrote:
> I think your idea of a data structure the encapsulates a set of events
> for which to wait is probably a good one. WaitLatch doesn't seem like
> a great name. Maybe WaitEventSet, and then
On Thu, Jan 14, 2016 at 10:56 AM, Andres Freund wrote:
> So, I'm wondering how we'd exactly use a hyptothetical
> SetSocketToWaitOn, or SetSocketsToWaitOn (or whatever). I mean it can
> make a fair bit of sense to sometimes wait on MyLatch/port->sock and
> sometimes on MyLatch/fdw connections. The
On January 14, 2016 5:16:59 PM GMT+01:00, Robert Haas
wrote:
>Yeah. Although I think for now it would be fine to just error out if
>somebody tries to add a socket and there already is one. Then we
>could lift that limitation in a later commit. Of course if Andres
>wants to do the whole thing n
On Thu, Jan 14, 2016 at 10:46 AM, Tom Lane wrote:
> Robert Haas writes:
>> Incidentally, if we're going to whack around the latch API, it would
>> be nice to pick a design which wouldn't be too hard to extend to
>> waiting on multiple sockets. The application I have in mind is to
>> send of quer
On 2016-01-14 10:39:55 -0500, Robert Haas wrote:
> Doesn't this code make it possible for the self-pipe to fill up,
> self-deadlocking the process? Suppose we repeatedly enter
> WaitLatchOrSocket(). Each time we do, just after waiting = true is
> set, somebody sets the latch. We handle the signa
Robert Haas writes:
> Incidentally, if we're going to whack around the latch API, it would
> be nice to pick a design which wouldn't be too hard to extend to
> waiting on multiple sockets. The application I have in mind is to
> send of queries to several foreign servers at once and then wait unti
On Thu, Jan 14, 2016 at 9:39 AM, Andres Freund wrote:
> I finally got back to working on this. Attached is a WIP patch series
> implementing:
> 0001: Allow to easily choose between the readiness primitives in unix_latch.c
> Pretty helpful for testing, not useful for anything else.
Looks goo
Andres Freund writes:
> 0004 currently contains one debatable optimization, which I'd like to
> discuss: Currently the 'sock' passed to WaitLatchOrSocket is not
> removed/added to the epoll fd, if it's the numerically same as in the
> last call. That's good for performance, but would be wrong if t
On 2015-12-26 12:22:48 +0100, Andres Freund wrote:
> On 2015-12-25 16:29:53 -0500, Tom Lane wrote:
> > Andres Freund writes:
> > > There's a couple solutions I can think of to that problem:
> > > 1) Use epoll()/kqueue, or other similar interfaces that don't require
> > >re-registering fds at e
On Sat, Dec 26, 2015 at 5:41 PM, Andres Freund wrote:
> On 2015-12-26 12:22:48 +0100, Andres Freund wrote:
> > > > 3) Replace the postmaster_alive_fds socketpair by some other
> signalling
> > > >mechanism. E.g. sending a procsignal to each backend, which sets
> the
> > > >latch and a spe
Andres Freund writes:
> A unportable and easy version of this, actually making sense this time,
> would be to use prctl(PR_SET_PDEATHSIG, SIGQUIT). That'd send SIGQUIT to
> backends whenever postmaster dies. Obviously that's not portable
> either - doing this for linux only wouldn't be all that k
On 2015-12-26 12:22:48 +0100, Andres Freund wrote:
> > > 3) Replace the postmaster_alive_fds socketpair by some other signalling
> > >mechanism. E.g. sending a procsignal to each backend, which sets the
> > >latch and a special flag in the latch structure.
> >
> > And what would send the s
On 2015-12-25 16:29:53 -0500, Tom Lane wrote:
> Andres Freund writes:
> > There's a couple solutions I can think of to that problem:
> > 1) Use epoll()/kqueue, or other similar interfaces that don't require
> >re-registering fds at every invocation. My guess is that that'd be
> >desirable
Andres Freund writes:
> There's a couple solutions I can think of to that problem:
> 1) Use epoll()/kqueue, or other similar interfaces that don't require
>re-registering fds at every invocation. My guess is that that'd be
>desirable for performance anyway.
Portability, on the other hand,
On 2015-12-25 13:28:55 -0500, Tom Lane wrote:
> Hmm. And all those FDs point to the same pipe. I wonder if we're looking
> at contention for some pipe-related data structure inside the kernel.
Sounds fairly likely - and not too surprising. In this scenario we've a
couple 100k registrations/unreg
2015-12-25 22:42 GMT+03:00 Васильев Дмитрий :
>
>
>
> 2015-12-25 21:28 GMT+03:00 Tom Lane :
>
>> Andres Freund writes:
>> > On December 25, 2015 7:10:23 PM GMT+01:00, Tom Lane
>> wrote:
>> >> Seems like what you've got here is a kernel bug.
>>
>> > I wouldn't go as far as calling it a kernel
2015-12-25 21:28 GMT+03:00 Tom Lane :
> Andres Freund writes:
> > On December 25, 2015 7:10:23 PM GMT+01:00, Tom Lane
> wrote:
> >> Seems like what you've got here is a kernel bug.
>
> > I wouldn't go as far as calling it a kernel bug. Were still doing 300k
> tps. And were triggering the perfo
Andres Freund writes:
> On December 25, 2015 7:10:23 PM GMT+01:00, Tom Lane
> wrote:
>> Seems like what you've got here is a kernel bug.
> I wouldn't go as far as calling it a kernel bug. Were still doing 300k tps.
> And were triggering the performance degradation by adding another socket
> (
On December 25, 2015 7:10:23 PM GMT+01:00, Tom Lane wrote:
>=?UTF-8?B?0JLQsNGB0LjQu9GM0LXQsiDQlNC80LjRgtGA0LjQuQ==?=
> writes:
>> ��� Samples: 1M of event 'cycles', Event count (approx.):
>816922259995, UID:
>> pgpro
>> Overhead Shared Object Symbol
>
>> 69,72% [kernel][k]
=?UTF-8?B?0JLQsNGB0LjQu9GM0LXQsiDQlNC80LjRgtGA0LjQuQ==?=
writes:
> â Samples: 1M of event 'cycles', Event count (approx.): 816922259995, UID:
> pgpro
> Overhead Shared Object Symbol
> 69,72% [kernel][k] _raw_spin_lock_irqsave
> 1,43% postgres[.] _bt_compa
2015-12-25 20:44 GMT+03:00 Andres Freund :
> On December 25, 2015 6:27:06 PM GMT+01:00, "Васильев Дмитрий"
> >>If so, could you provide a hierarchical before/after profile?
> >
> >Performance | Git hash commit| Date
> >~ 360k tps | c3e7c24a1d60dc6ad56e2a0723399f1570c54224 | Thu Nov 12
> >0
On December 25, 2015 6:27:06 PM GMT+01:00, "Васильев Дмитрий"
>>If so, could you provide a hierarchical before/after profile?
>
>Performance | Git hash commit| Date
>~ 360k tps | c3e7c24a1d60dc6ad56e2a0723399f1570c54224 | Thu Nov 12
>09:12:18
>2015 -0500
>~ 360k tps | ac1d7945f866b1928c2554c
2015-12-25 20:27 GMT+03:00 Васильев Дмитрий :
>
> 2015-12-25 20:18 GMT+03:00 Andres Freund :
>
>> On December 25, 2015 6:08:15 PM GMT+01:00, "Васильев Дмитрий" <
>> d.vasil...@postgrespro.ru> wrote:
>> >Hello hackers!
>> >
>> >I suddenly found commit ac1d794 gives up to 3 times performance
>> >deg
2015-12-25 20:18 GMT+03:00 Andres Freund :
> On December 25, 2015 6:08:15 PM GMT+01:00, "Васильев Дмитрий" <
> d.vasil...@postgrespro.ru> wrote:
> >Hello hackers!
> >
> >I suddenly found commit ac1d794 gives up to 3 times performance
> >degradation.
> >
> >I tried to run pgbench -s 1000 -j 48 -c 4
On December 25, 2015 6:08:15 PM GMT+01:00, "Васильев Дмитрий"
wrote:
>Hello hackers!
>
>I suddenly found commit ac1d794 gives up to 3 times performance
>degradation.
>
>I tried to run pgbench -s 1000 -j 48 -c 48 -S -M prepared on 70
>CPU-core
>machine:
>commit ac1d794 gives me 363,474 tps
>and pr
Hello hackers!
I suddenly found commit ac1d794 gives up to 3 times performance degradation.
I tried to run pgbench -s 1000 -j 48 -c 48 -S -M prepared on 70 CPU-core
machine:
commit ac1d794 gives me 363,474 tps
and previous commit a05dc4d gives me 956,146
and master( 3d0c50f ) with revert ac1d794
94 matches
Mail list logo
