On 17.07.2013 15:24, Andres Freund wrote:
On 2013-07-17 15:46:00 +0530, Amit Kapila wrote:
Few doubts while reading the code:
1. Why in function WALInsertSlotAcquireOne(int slotno), it does
START_CRIT_SECTION() to
Lock out cancel/die interrupts, whereas other places call
HOLD_INTERRUPTS()
On 17.07.2013 02:18, Michael Paquier wrote:
On Tue, Jul 16, 2013 at 2:24 AM, Fujii Masao wrote:
On Mon, Jul 8, 2013 at 6:16 PM, Heikki Linnakangas
wrote:
Ok, I've committed this patch now. Finally, phew!
I found that this patch causes the assertion failure. When I set up simple
replication
On 2013-07-17 15:46:00 +0530, Amit Kapila wrote:
> On Monday, July 08, 2013 2:47 PM Heikki Linnakangas wrote:
> > Ok, I've committed this patch now. Finally, phew!
>
> Few doubts while reading the code:
>
> 1. Why in function WALInsertSlotAcquireOne(int slotno), it does
> START_CRIT_SECTION() t
On Monday, July 08, 2013 2:47 PM Heikki Linnakangas wrote:
> Ok, I've committed this patch now. Finally, phew!
Few doubts while reading the code:
1. Why in function WALInsertSlotAcquireOne(int slotno), it does
START_CRIT_SECTION() to
Lock out cancel/die interrupts, whereas other places call
On Tue, Jul 16, 2013 at 2:24 AM, Fujii Masao wrote:
> On Mon, Jul 8, 2013 at 6:16 PM, Heikki Linnakangas
> wrote:
>> Ok, I've committed this patch now. Finally, phew!
>
> I found that this patch causes the assertion failure. When I set up simple
> replication environment and promoted the standby
On Mon, Jul 8, 2013 at 6:16 PM, Heikki Linnakangas
wrote:
> Ok, I've committed this patch now. Finally, phew!
I found that this patch causes the assertion failure. When I set up simple
replication environment and promoted the standby before executing any
transaction on the master, I got the follo
At 2013-07-08 12:16:34 +0300, hlinnakan...@vmware.com wrote:
>
> Ok, I've committed this patch now. Finally, phew!
Good.
I'd signed up to review this patch, and did spend some considerable time
on it, but although I managed to understand what was going on (which was
my objective), I didn't find a
On 2013-07-09 08:00:52 +0900, Michael Paquier wrote:
> On Mon, Jul 8, 2013 at 6:16 PM, Heikki Linnakangas
> wrote:
> > Ok, I've committed this patch now. Finally, phew!
> +1. Great patch!
And one more: +1
There seem to be quite some lowhanging fruits to make stuff faster after
this bottleneck is
On Mon, Jul 8, 2013 at 6:16 PM, Heikki Linnakangas
wrote:
> Ok, I've committed this patch now. Finally, phew!
+1. Great patch!
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Jul 8, 2013, at 4:16 AM, Heikki Linnakangas wrote:
> Ok, I've committed this patch now. Finally, phew!
Woohoo!
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Jul 8, 2013 at 1:38 PM, Heikki Linnakangas
wrote:
> GetXLogBuffer() doesn't read the content of the page - it writes to it (or
> rather, the caller of GetXLogBarrier() does). The barrier is needed between
> reading xlblocks (to check that the buffer contains the right page), and
> writing
On 2013-07-08 10:45:41 +0300, Heikki Linnakangas wrote:
> On 01.07.2013 16:40, Andres Freund wrote:
> >On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
> >>>* Could you document the way slots prevent checkpoints from occurring
> >>> when XLogInsert rechecks for full page writes? I think it
On 08.07.2013 13:21, Ants Aasma wrote:
On Mon, Jul 8, 2013 at 12:16 PM, Heikki Linnakangas
wrote:
Ok, I've committed this patch now. Finally, phew!
Fantastic work!
I simplified the handling of xlogInsertingAt per discussion, and added the
memory barrier to GetXLogBuffer(). I ran again the
On Mon, Jul 8, 2013 at 12:16 PM, Heikki Linnakangas
wrote:
> Ok, I've committed this patch now. Finally, phew!
Fantastic work!
> I simplified the handling of xlogInsertingAt per discussion, and added the
> memory barrier to GetXLogBuffer(). I ran again the pgbench tests I did
> earlier with the
On 27.06.2013 20:36, Andres Freund wrote:
On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
There's this in the comment near the top of the file:
Btw, I find the 'you' used in the comment somewhat irritating. Not badly
so, but reading something like:
* When you are about to write
*
On 08.07.2013 12:16, Heikki Linnakangas wrote:
I just remembered one detail that I'm not sure has been mentioned on the
mailing list yet. Per the commit message:
This has one user-visible change: switching to a new WAL segment with
pg_switch_xlog() now fills the remaining unused portion of the
Ok, I've committed this patch now. Finally, phew!
I think I've addressed all your comments about the comments. I moved
some of the comments around: I split up the large one near the top of
the file, moving its paragraphs closer to the code where they apply.
Regarding your performance-related
On 01.07.2013 16:40, Andres Freund wrote:
On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
* Could you document the way slots prevent checkpoints from occurring
when XLogInsert rechecks for full page writes? I think it's correct -
but not very obvious on a glance.
There's this in
On 2013-07-02 19:48:40 +0300, Heikki Linnakangas wrote:
> >If so, why isn't it sufficient to
> >initialize it in ReserveXLogInsertLocation?
>
> It would be, but then ReserveXLogInsertLocation would need to hold the
> slot's spinlock at the same time with insertpos_lck, so that it could
> atomicall
On 27.06.2013 15:27, Andres Freund wrote:
Hi,
On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
* Could you document the way slots prevent checkpoints from occurring
when XLogInsert rechecks for full page writes? I think it's correct -
but not very obvious on a glance.
There's thi
On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
> >* Could you document the way slots prevent checkpoints from occurring
> > when XLogInsert rechecks for full page writes? I think it's correct -
> > but not very obvious on a glance.
>
> There's this in the comment near the top of the fi
On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
> There's this in the comment near the top of the file:
Btw, I find the 'you' used in the comment somewhat irritating. Not badly
so, but reading something like:
* When you are about to write
* out WAL, it is important to call WaitXLogInser
Hi,
On 2013-06-26 18:52:30 +0300, Heikki Linnakangas wrote:
> >* Could you document the way slots prevent checkpoints from occurring
> > when XLogInsert rechecks for full page writes? I think it's correct -
> > but not very obvious on a glance.
>
> There's this in the comment near the top of t
On 24.06.2013 21:01, Andres Freund wrote:
Ok, I started to look at this:
Thanks!
* Could you document the way slots prevent checkpoints from occurring
when XLogInsert rechecks for full page writes? I think it's correct -
but not very obvious on a glance.
There's this in the comment ne
On Sat, Jun 22, 2013 at 4:32 AM, Heikki Linnakangas wrote:
> On 21.06.2013 21:55, Jeff Janes wrote:
>
>>
>> Hmm, it looks like the xlog-switch is trying to wait for itself to
> finish. The concurrent TRUNCATE is just being blocked behind the
> xlog-switch, which is stuck on itself.
>
> I wasn't a
On 2013-06-22 14:32:46 +0300, Heikki Linnakangas wrote:
> Attached is a new version that fixes at least the problem I saw. Not sure if
> it fixes what you saw, but it's worth a try. How easily can you reproduce
> that?
Ok, I started to look at this:
* Could you document the way slots prevent chec
On 21.06.2013 21:55, Jeff Janes wrote:
I think I'm getting an undetected deadlock between the checkpointer and a
user process running a TRUNCATE command.
This is the checkpointer:
#0 0x003a73eeaf37 in semop () from /lib64/libc.so.6
#1 0x005ff847 in PGSemaphoreLock (sema=0x7f8c0a4e
On Tue, Jun 18, 2013 at 11:28 PM, Heikki Linnakangas <
hlinnakan...@vmware.com> wrote:
> On 18.06.2013 21:17, Jeff Janes wrote:
>
>> Hi Heikki,
>>
>> I am getting conflicts applying version 22 of this patch to 9.4dev. Could
>> you rebase?
>>
>
> Here you go.
I think I'm getting an undetected de
On 18.06.2013 21:17, Jeff Janes wrote:
Hi Heikki,
I am getting conflicts applying version 22 of this patch to 9.4dev. Could
you rebase?
Here you go.
Does anyone know of an easy way to apply an external patch through git, so
I can get git-style merge conflict markers, rather than getting pat
Hi Heikki,
I am getting conflicts applying version 22 of this patch to 9.4dev. Could
you rebase?
Does anyone know of an easy way to apply an external patch through git, so
I can get git-style merge conflict markers, rather than getting patch's
reject file?
Cheers,
Jeff
On Wed, May 29, 2013 at 8:40 PM, Heikki Linnakangas
wrote:
> Thanks for asking :-). I've been fixing bitrot throughout the winter. I just
> started cleaning it up again last week, and also continued with performance
> testing.
>
> Unfortunately I lost the 8-core box I used earlier to test this, to
Heikki Linnakangas wrote:
> I've been slowly continuing to work that I started last winder to
> make XLogInsert scale better. I have tried quite a few different
> approaches since then, and have settled on the attached. This is
> similar but not exactly the same as what I did in the patches I
> pos
On Fri, Sep 28, 2012 at 12:58 AM, Heikki Linnakangas
wrote:
> Hmm, I cannot reproduce this on my Linux laptop. However, I think I see what
> the problem is: the assertion should assert that (*CurrPos* % XLOG_BLCKZ >=
> SizeOfXLogShortPHD), not currpos. The former is an XLogRecPtr, the latter is
>
On 24.09.2012 21:06, Fujii Masao wrote:
The patch could be applied cleanly and the compile could be successfully done.
Thanks for the testing!
But when I ran initdb, I got the following assertion error:
--
$ initdb -D data --locale=C --encoding=UTF-8
T
On Fri, Sep 21, 2012 at 12:29 AM, Heikki Linnakangas
wrote:
> I've been slowly continuing to work that I started last winder to make
> XLogInsert scale better. I have tried quite a few different approaches since
> then, and have settled on the attached. This is similar but not exactly the
> same a
On 20 September 2012 16:29, Heikki Linnakangas wrote:
> 1. Insert 1000 rows in each transaction. This is exactly the sort of
> workload where WALInsertLock currently becomes a bottleneck. Without the the
> patch, the test scales very badly, with about 420 TPS with a single client,
> peaking only
Heikki Linnakangas writes:
> I've been slowly continuing to work that I started last winder to make
> XLogInsert scale better. I have tried quite a few different approaches
> since then, and have settled on the attached. This is similar but not
> exactly the same as what I did in the patches I
On 20.09.2012 18:37, Tom Lane wrote:
I suspect that it's not a coincidence that you're seeing
a sweet spot for #slots ~= #CPUs.
Yeah, quite possible. I did not test with any different number of slots,
so I don't know if that's the sweet spot, but I wouldn't be surprised if
it is.
If that i
On Thursday, September 20, 2012 05:37:42 PM Tom Lane wrote:
> Heikki Linnakangas writes:
> > I've been slowly continuing to work that I started last winder to make
> > XLogInsert scale better. I have tried quite a few different approaches
> > since then, and have settled on the attached. This is s
I've been slowly continuing to work that I started last winder to make
XLogInsert scale better. I have tried quite a few different approaches
since then, and have settled on the attached. This is similar but not
exactly the same as what I did in the patches I posted earlier.
The basic idea, li
On Wednesday 16 December 2009 20:07:07 Gurjeet Singh wrote:
> 2009/12/15 Greg Smith
>
> > Jaime Casanova wrote:
> >> So in this extreme case avg tps is just 6 transactions better
> >
> > Great job trying to find the spot where the code worked better. I'm not
> > so sure I trust pgbench results w
2009/12/15 Greg Smith
> Jaime Casanova wrote:
>
>> So in this extreme case avg tps is just 6 transactions better
>>
>>
> Great job trying to find the spot where the code worked better. I'm not so
> sure I trust pgbench results where the TPS was so low though. Which leads
> us right back to exac
Jaime Casanova wrote:
So in this extreme case avg tps is just 6 transactions better
Great job trying to find the spot where the code worked better. I'm not
so sure I trust pgbench results where the TPS was so low though. Which
leads us right back to exactly how Jeff measured his original r
On Thu, Dec 10, 2009 at 3:58 PM, Alvaro Herrera
wrote:
> Jaime Casanova escribió:
>> On Wed, Dec 9, 2009 at 9:39 AM, Tom Lane wrote:
>> > so I'd like some independent confirmation that it does.
>> >
>>
>> what kind of tests could show that? or simply running pgbench several
>> times for 15 minute
Jaime Casanova escribió:
> On Wed, Dec 9, 2009 at 9:39 AM, Tom Lane wrote:
> > so I'd like some independent confirmation that it does.
> >
>
> what kind of tests could show that? or simply running pgbench several
> times for 15 minutes each run could show any benefit?
Isn't the supposed performa
On Wed, Dec 9, 2009 at 9:39 AM, Tom Lane wrote:
> so I'd like some independent confirmation that it does.
>
what kind of tests could show that? or simply running pgbench several
times for 15 minutes each run could show any benefit?
--
Atentamente,
Jaime Casanova
Soporte y capacitación de Postgr
Tom Lane wrote:
Jaime Casanova writes:
i haven't made any performance tests but it should gain something :),
maybe someone can make those tests?
The argument for changing this at all is only that it will improve
performance, so I'd like some independent confirmation that it does.
I'v
Jaime Casanova writes:
> i haven't made any performance tests but it should gain something :),
> maybe someone can make those tests?
The argument for changing this at all is only that it will improve
performance, so I'd like some independent confirmation that it does.
reg
On Sun, Sep 13, 2009 at 10:42 PM, Jeff Janes wrote:
>
> I've attached a patch which removes the iteration over the blocks to be
> backed-up from the critical section of XLogInsert. Now those blocks are
> only looped over in one piece of code which both computes the CRC and builds
> the linked lis
On Wed, Aug 19, 2009 at 9:49 AM, Tom Lane wrote:
> Jeff Janes writes:
> > If I read the code correctly, the only thing that is irrevocable is
> > that it writes into
> > rdt->next, and if it saved an old copy of rdt first, then it could
> > revoke the changes just
> > by doing rdt_old->next=NULL
Jeff Janes writes:
> If I read the code correctly, the only thing that is irrevocable is
> that it writes into
> rdt->next, and if it saved an old copy of rdt first, then it could
> revoke the changes just
> by doing rdt_old->next=NULL. If that were done, then I think this
> code could be
> moved
In XLogInsert (src/backend/access/transam/xlog.c), the part that adds back-up
blocks into the rdata chain is described:
/*
* Make additional rdata chain entries for the backup blocks, so that we
* don't need to special-case them in the write loop. Note that we have
52 matches
Mail list logo
