On Thu, Sep 3, 2015 at 11:07 PM, Robert Haas wrote:
> On Tue, Aug 25, 2015 at 7:51 AM, Amit Kapila
> wrote:
> > [ new patch ]
>
> Andres pinged me about this patch today. It contained a couple of
> very strange whitespace anomalies, but otherwise looked OK to me, so I
> committed it with after
On Tue, Aug 25, 2015 at 7:51 AM, Amit Kapila wrote:
> [ new patch ]
Andres pinged me about this patch today. It contained a couple of
very strange whitespace anomalies, but otherwise looked OK to me, so I
committed it with after fixing those issues and tweaking the comments
a bit. Hopefully we
On Tue, Aug 25, 2015 at 5:21 PM, Amit Kapila
wrote:
>
> On Thu, Aug 20, 2015 at 3:49 PM, Andres Freund wrote:
> > How hard did you try checking whether this causes regressions? This
> > increases the number of atomics in the commit path a fair bit. I doubt
> > it's really bad, but it seems like a
On Thu, Aug 20, 2015 at 3:49 PM, Andres Freund wrote:
>
> On 2015-08-20 15:38:36 +0530, Amit Kapila wrote:
> > On Wed, Aug 19, 2015 at 9:09 PM, Andres Freund
wrote:
> > > I spent some time today reviewing the commited patch. So far my only
> > > major complaint is that I think the comments are on
On Fri, Aug 21, 2015 at 2:31 PM, Andres Freund wrote:
> No, if it's paired like that, I don't think it's allowed to fail.
>
> But, as the code stands, there's absolutely no guarantee you're not
> seeing something like:
> P1: a = 0;
> P1: b = 0;
> P1: PGSemaphoreLock(&P1);
> P2: a = 1;
> P2: PGSema
On 2015-08-21 14:08:36 -0400, Robert Haas wrote:
> On Wed, Aug 19, 2015 at 11:39 AM, Andres Freund wrote:
> > There's a bunch of issues with those two blocks afaics:
> >
> > 1) The first block (in one backend) might read INVALID_PGPROCNO before
> >ever locking the semaphore if a second backend
On Wed, Aug 19, 2015 at 11:39 AM, Andres Freund wrote:
> There's a bunch of issues with those two blocks afaics:
>
> 1) The first block (in one backend) might read INVALID_PGPROCNO before
>ever locking the semaphore if a second backend quickly enough writes
>INVALID_PGPROCNO. That way the
On 2015-08-20 15:38:36 +0530, Amit Kapila wrote:
> On Wed, Aug 19, 2015 at 9:09 PM, Andres Freund wrote:
> > I spent some time today reviewing the commited patch. So far my only
> > major complaint is that I think the comments are only insufficiently
> > documenting the approach taken:
> > Stuff l
On Thu, Aug 20, 2015 at 3:38 PM, Amit Kapila
wrote:
>
> On Wed, Aug 19, 2015 at 9:09 PM, Andres Freund wrote:
> >
> >
> > How hard did you try checking whether this causes regressions? This
> > increases the number of atomics in the commit path a fair bit. I doubt
> > it's really bad, but it seem
On Wed, Aug 19, 2015 at 9:09 PM, Andres Freund wrote:
>
> Hi,
>
> > On Wed, Aug 5, 2015 at 10:59 AM, Amit Kapila
wrote:
> > OK, committed.
>
> I spent some time today reviewing the commited patch. So far my only
> major complaint is that I think the comments are only insufficiently
> documenting
Hi,
> On Wed, Aug 5, 2015 at 10:59 AM, Amit Kapila wrote:
> OK, committed.
I spent some time today reviewing the commited patch. So far my only
major complaint is that I think the comments are only insufficiently
documenting the approach taken:
Stuff like avoiding ABA type problems by clearling
On 2015-08-07 14:20:55 -0400, Jesper Pedersen wrote:
> On 08/07/2015 02:03 PM, Andres Freund wrote:
> Confirmed.
>
> Running w/o -P x and the problem goes away.
Pushed the fix. Thanks for pointing the problem out!
- Andres
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
On 08/07/2015 02:03 PM, Andres Freund wrote:
but you will have to use a 9.5 pgbench to see it, especially with higher
client counts.
Hm, you were using -P X, is that right?
This bisects down to 1bc90f7a7b7441a88e2c6d4a0e9b6f9c1499ad30 - "Remove
thread-emulation support from pgbench."
And th
Hi,
On 2015-08-07 19:30:46 +0200, Andres Freund wrote:
> On 2015-08-07 12:49:20 -0400, Jesper Pedersen wrote:
> > No, this patch helps on performance - there is an improvement in numbers
> > between
> >
> > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=253de7e1eb9abbcf57e6c229a8a3
On 2015-08-07 12:49:20 -0400, Jesper Pedersen wrote:
> On 08/07/2015 11:40 AM, Robert Haas wrote:
> >On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen
> > wrote:
> >>Just thought I would post it in this thread, because this change does help
> >>on the performance numbers compared to 9.5 :)
> >
> >So
On 08/07/2015 11:40 AM, Robert Haas wrote:
On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen
wrote:
Just thought I would post it in this thread, because this change does help
on the performance numbers compared to 9.5 :)
So are you saying that the performance was already worse before this
patc
On Fri, Aug 7, 2015 at 10:30 AM, Jesper Pedersen
wrote:
> Just thought I would post it in this thread, because this change does help
> on the performance numbers compared to 9.5 :)
So are you saying that the performance was already worse before this
patch landed, and then this patch made it somew
On 2015-08-07 20:17:28 +0530, Amit Kapila wrote:
> On Fri, Aug 7, 2015 at 8:00 PM, Jesper Pedersen
> wrote:
>
> > On 08/07/2015 12:41 AM, Amit Kapila wrote:
> >
> >> On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas
> >> wrote:
> >>
> >>>
> >>>
> >>> OK, committed.
> >>>
> >>>
> >> Thank you.
> >>
> >
On 08/07/2015 10:47 AM, Amit Kapila wrote:
Fyi, there is something in pgbench that has caused a testing regression -
havn't tracked down what yet.
Against 9.6 server (846f8c9483a8f31e45bf949db1721706a2765771)
9.6 pgbench:
progress: 10.0 s, 53525.0 tps, lat 1.485 ms stddev 0.523
pro
On Fri, Aug 7, 2015 at 8:00 PM, Jesper Pedersen
wrote:
> On 08/07/2015 12:41 AM, Amit Kapila wrote:
>
>> On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas
>> wrote:
>>
>>>
>>>
>>> OK, committed.
>>>
>>>
>> Thank you.
>>
>>
> Fyi, there is something in pgbench that has caused a testing regression -
> h
On 08/07/2015 12:41 AM, Amit Kapila wrote:
On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas wrote:
OK, committed.
Thank you.
Fyi, there is something in pgbench that has caused a testing regression
- havn't tracked down what yet.
Against 9.6 server (846f8c9483a8f31e45bf949db1721706a2765771
On Thu, Aug 6, 2015 at 9:36 PM, Robert Haas wrote:
>
>
> OK, committed.
>
Thank you.
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com
On Wed, Aug 5, 2015 at 10:59 AM, Amit Kapila wrote:
> On Tue, Aug 4, 2015 at 8:59 PM, Robert Haas wrote:
>>
>> I'm not entirely happy with the name "nextClearXidElem" but apart from
>> that I'm fairly happy with this version. We should probably test it
>> to make sure I haven't broken anything;
On Tue, Aug 4, 2015 at 8:59 PM, Robert Haas wrote:
>
>
> I'm not entirely happy with the name "nextClearXidElem" but apart from
> that I'm fairly happy with this version. We should probably test it
> to make sure I haven't broken anything;
I have verified the patch and it is fine. I have teste
On Wed, Aug 5, 2015 at 8:30 AM, Pavan Deolasee wrote:
> I actually even thought if we can define some macros or functions to work on
> atomic list of PGPROCs. What we need is:
>
> - Atomic operation to add a PGPROC to list list and return the head of the
> list at the time of addition
> - Atomic o
On Tue, Aug 4, 2015 at 8:59 PM, Robert Haas wrote:
>
>
> I spent some time looking at this patch today and found that a good
> deal of cleanup work seemed to be needed. Attached is a cleaned-up
> version which makes a number of changes:
>
>
> I'm not entirely happy with the name "nextClearXidEle
On 2015-08-04 21:20:20 +0530, Amit Kapila wrote:
> Note - The function header comments on pg_atomic_read_u32 and
> corresponding write call seems to be reversed, but that is something
> separate.
Fixed, thanks for noticing.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
T
On Tue, Aug 4, 2015 at 11:29 AM, Robert Haas wrote:
> 4. I factored out the actual XID-clearing logic into a new function
> ProcArrayEndTransactionInternal instead of repeating it twice. On the
> flip side, I merged PushProcAndWaitForXidClear with
> PopProcsAndClearXids and renamed the result to P
On Tue, Aug 4, 2015 at 11:50 AM, Amit Kapila wrote:
> I have kept barriers based on comments on top of atomic read, refer
> below code:
>
> * No barrier semantics.
> */
> STATIC_IF_INLINE uint32
> pg_atomic_read_u32(volatile pg_atomic_uint32 *ptr)
>
> Note - The function header comments on pg_at
On 2015-08-04 21:20:20 +0530, Amit Kapila wrote:
> I have kept barriers based on comments on top of atomic read, refer
> below code:
> * No barrier semantics.
> */
> STATIC_IF_INLINE uint32
> pg_atomic_read_u32(volatile pg_atomic_uint32 *ptr)
>
> Note - The function header comments on pg_atomi
On 2015-08-04 11:43:45 -0400, Robert Haas wrote:
> On Tue, Aug 4, 2015 at 11:33 AM, Andres Freund wrote:
> > Actually by far not all system calls are full barriers?
>
> How do we know which ones are and which ones are not?
Good question. Reading the source code of all implementations I suppose
:
On Tue, Aug 4, 2015 at 8:59 PM, Robert Haas wrote:
>
> On Mon, Aug 3, 2015 at 8:39 AM, Amit Kapila
wrote:
> >> I agree and modified the patch to use 32-bit atomics based on idea
> >> suggested by Robert and didn't modify lwlock.c.
> >
> > While looking at patch, I found that the way it was initia
On Tue, Aug 4, 2015 at 11:33 AM, Andres Freund wrote:
> On 2015-08-04 11:29:39 -0400, Robert Haas wrote:
>> On Mon, Aug 3, 2015 at 8:39 AM, Amit Kapila wrote:
>> 1. I got rid of all of the typecasts. You're supposed to treat
>> pg_atomic_u32 as a magic data type that is only manipulated via the
On 2015-08-04 11:29:39 -0400, Robert Haas wrote:
> On Mon, Aug 3, 2015 at 8:39 AM, Amit Kapila wrote:
> 1. I got rid of all of the typecasts. You're supposed to treat
> pg_atomic_u32 as a magic data type that is only manipulated via the
> primitives provided, not just cast back and forth between
On Mon, Aug 3, 2015 at 8:39 AM, Amit Kapila wrote:
>> I agree and modified the patch to use 32-bit atomics based on idea
>> suggested by Robert and didn't modify lwlock.c.
>
> While looking at patch, I found that the way it was initialising the list
> to be empty was wrong, it was using pgprocno a
On Fri, Jul 31, 2015 at 10:11 AM, Amit Kapila
wrote:
>
> On Wed, Jul 29, 2015 at 11:48 PM, Andres Freund
wrote:
> >
> > On 2015-07-29 12:54:59 -0400, Robert Haas wrote:
> > > I would try to avoid changing lwlock.c. It's pretty easy when so
> > > doing to create mechanisms that work now but make
On Wed, Jul 29, 2015 at 11:48 PM, Andres Freund wrote:
>
> On 2015-07-29 12:54:59 -0400, Robert Haas wrote:
> > I would try to avoid changing lwlock.c. It's pretty easy when so
> > doing to create mechanisms that work now but make further upgrades to
> > the general lwlock mechanism difficult. I
On Wed, Jul 29, 2015 at 2:18 PM, Andres Freund wrote:
> On 2015-07-29 12:54:59 -0400, Robert Haas wrote:
>> I would try to avoid changing lwlock.c. It's pretty easy when so
>> doing to create mechanisms that work now but make further upgrades to
>> the general lwlock mechanism difficult. I'd lik
On 2015-07-29 12:54:59 -0400, Robert Haas wrote:
> I would try to avoid changing lwlock.c. It's pretty easy when so
> doing to create mechanisms that work now but make further upgrades to
> the general lwlock mechanism difficult. I'd like to avoid that.
I'm massively doubtful that re-implementin
On Wed, Jul 29, 2015 at 10:54 AM, Amit Kapila wrote:
> On Mon, Jul 27, 2015 at 8:47 PM, Robert Haas wrote:
>> On Sat, Jul 25, 2015 at 12:42 AM, Amit Kapila
>> wrote:
>> > I thought that internal API will automatically take care of it,
>> > example for msvc it uses _InterlockedCompareExchange64
>
On Mon, Jul 27, 2015 at 8:47 PM, Robert Haas wrote:
>
> On Sat, Jul 25, 2015 at 12:42 AM, Amit Kapila
wrote:
> > I thought that internal API will automatically take care of it,
> > example for msvc it uses _InterlockedCompareExchange64
> > which if doesn't work on 32-bit systems or is not defined
On Sat, Jul 25, 2015 at 10:12 AM, Amit Kapila
wrote:
>
> >>
> >
> > Numbers look impressive and definitely shows that the idea is worth
> pursuing. I tried patch on my laptop. Unfortunately, at least for 4 and 8
> clients, I did not see any improvement.
> >
>
> I can't help in this because I thin
On Sat, Jul 25, 2015 at 12:42 AM, Amit Kapila wrote:
> I thought that internal API will automatically take care of it,
> example for msvc it uses _InterlockedCompareExchange64
> which if doesn't work on 32-bit systems or is not defined, then
> we have to use 32-bit version, but I am not certain ab
On Fri, Jul 24, 2015 at 4:26 PM, Pavan Deolasee
wrote:
>
>
>
> On Mon, Jun 29, 2015 at 8:57 PM, Amit Kapila
wrote:
>>
>>
>>
>> pgbench setup
>>
>> scale factor - 300
>> Data is on magnetic disk and WAL on ssd.
>> pgbench -M prepared tpc-b
>>
>> Head : commit 51d0fe5d
>> P
On Mon, Jun 29, 2015 at 8:57 PM, Amit Kapila
wrote:
>
>
> pgbench setup
>
> scale factor - 300
> Data is on magnetic disk and WAL on ssd.
> pgbench -M prepared tpc-b
>
> Head : commit 51d0fe5d
> Patch -1 : group_xid_clearing_at_trans_end_rel_v1
>
>
> Client Count/TPS18163
On 30 June 2015 at 07:30, Amit Kapila wrote:
> Sure and I think we might want to try something similar even
> for XLogFlush where we use LWLockAcquireOrWait for
> WALWriteLock, not sure how it will workout in that case as
> I/O is involved, but I think it is worth trying.
>
+1
--
Simon Riggs
On Tue, Jun 30, 2015 at 11:53 AM, Simon Riggs wrote:
>
> On 30 June 2015 at 04:21, Amit Kapila wrote:
>
>>
>> Now, I would like to briefly explain how allow-one-waker idea has
>> helped to improve the patch as not every body here was present
>> in that Un-conference.
>
>
> The same idea applies f
On 30 June 2015 at 04:21, Amit Kapila wrote:
> Now, I would like to briefly explain how allow-one-waker idea has
> helped to improve the patch as not every body here was present
> in that Un-conference.
>
The same idea applies for marking commits in clog, for which I have been
sitting on a patc
On Mon, Jun 29, 2015 at 11:14 PM, Simon Riggs wrote:
> What I find weird is that the discussion was so intense about
> LWLockAcquireOrWait that when someone presented a solution there were people
> that didn't notice. It makes me wonder whether large group discussions are
> worth it.
I didn't thi
On 30 June 2015 at 03:43, Robert Haas wrote:
> On Mon, Jun 29, 2015 at 1:22 PM, Simon Riggs
> wrote:
> > Yes, I know. And we all had a long conversation about how to do it
> without
> > waking up the other procs.
> >
> > Forming a list, like we use for sync rep and having just a single process
>
On Mon, Jun 29, 2015 at 10:52 PM, Simon Riggs wrote:
>
> On 29 June 2015 at 18:11, Andres Freund wrote:
>>
>> On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs
wrote:
>> >On 29 June 2015 at 16:27, Amit Kapila wrote:
>> >
>> >
>> >> Thanks to Robert Haas for having discussion (offlist) about th
On Mon, Jun 29, 2015 at 1:22 PM, Simon Riggs wrote:
> Yes, I know. And we all had a long conversation about how to do it without
> waking up the other procs.
>
> Forming a list, like we use for sync rep and having just a single process
> walk the queue was the way I suggested then and previously.
On 29 June 2015 at 18:11, Andres Freund wrote:
> On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs
> wrote:
> >On 29 June 2015 at 16:27, Amit Kapila wrote:
> >
> >
> >> Thanks to Robert Haas for having discussion (offlist) about the idea
> >> and suggestions to improve it and also Andres Freun
On June 29, 2015 7:02:10 PM GMT+02:00, Simon Riggs
wrote:
>On 29 June 2015 at 16:27, Amit Kapila wrote:
>
>
>> Thanks to Robert Haas for having discussion (offlist) about the idea
>> and suggestions to improve it and also Andres Freund for having
>> discussion and sharing thoughts about this ide
On 29 June 2015 at 16:27, Amit Kapila wrote:
> Thanks to Robert Haas for having discussion (offlist) about the idea
> and suggestions to improve it and also Andres Freund for having
> discussion and sharing thoughts about this idea at PGCon.
>
Weird. This patch is implemented exactly the way I
I have been working on to analyze different ways to reduce
the contention around ProcArrayLock. I have evaluated mainly
2 ideas, first one is to partition the ProcArrayLock (the basic idea
is to allow multiple clients (equal to number of ProcArrayLock partitions)
to perform ProcArrayEndTransaction
56 matches
Mail list logo
