On Tue, 2008-09-09 at 09:11 +0100, Simon Riggs wrote:
> This gives us the Group Commit feature also, even if we are not using
> replication. So we can drop the commit_delay stuff.
>
> XLogBackgroundFlush() processes data page at a time if it can. That may
> not be the correct batch size for XLog
Simon Riggs wrote:
> Why not:
> 1. Same procedure as Warm Standby now
> a) WAL archiving to standby starts
> b) base backup
>
> 2. Startup standby, with additional option to stream WAL. WALReceiver
> starts, connects to Primary. Primary issues log switch. Archiver turns
> itself off after sending
Fujii Masao wrote:
On Fri, Sep 12, 2008 at 12:17 AM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
Hmm. There's more problems than the TLI with that. For the original master
to catch up by replaying WAL from the new slave, without restoring from a
full backup, the original master must not write
Alvaro Herrera wrote:
Simon Riggs escribió:
On Fri, 2008-09-12 at 17:08 +0300, Heikki Linnakangas wrote:
It should be clear that to make this work you must run with a base
backup that was derived correctly on the current master. You can do that
by re-copying everything, or you can do that by ju
Simon Riggs escribió:
> On Fri, 2008-09-12 at 17:08 +0300, Heikki Linnakangas wrote:
> > > It should be clear that to make this work you must run with a base
> > > backup that was derived correctly on the current master. You can do that
> > > by re-copying everything, or you can do that by just s
On Fri, Sep 12, 2008 at 12:17 AM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
> Fujii Masao wrote:
>>
>> I think that this case would often happen. So, we should establish a
>> certain
>> solution or procedure to the case where TLI of the master doesn't match
>> TLI of the slave. If we only allow
On Fri, Sep 12, 2008 at 7:41 PM, Simon Riggs <[EMAIL PROTECTED]> wrote:
>
> On Thu, 2008-09-11 at 18:17 +0300, Heikki Linnakangas wrote:
>> Fujii Masao wrote:
>> > I think that this case would often happen. So, we should establish a
>> > certain
>> > solution or procedure to the case where TLI of
Hannu Krosing escribió:
> On Fri, 2008-09-12 at 17:45 +0100, Simon Riggs wrote:
> > On Fri, 2008-09-12 at 17:11 +0200, Csaba Nagy wrote:
> >
> > > Why not have a design where the slave is in control for it's own data ?
> > > I mean the slave...
> >
> > The slave only exists because it is a copy o
On Fri, 2008-09-12 at 17:45 +0100, Simon Riggs wrote:
> On Fri, 2008-09-12 at 17:11 +0200, Csaba Nagy wrote:
>
> > Why not have a design where the slave is in control for it's own data ?
> > I mean the slave...
>
> The slave only exists because it is a copy of the master. If you try to
> "startup
On Fri, 2008-09-12 at 17:11 +0200, Csaba Nagy wrote:
> Why not have a design where the slave is in control for it's own data ?
> I mean the slave...
The slave only exists because it is a copy of the master. If you try to
"startup" a slave without first having taken a copy, how would you
bootstra
Hi,
Andrew Dunstan wrote:
If this were a nice pluggable library I'd agree, but AFAIK it's not, and
I don't see great value in reinventing the wheel.
I certainly agree.
However, I thought of it more like the archive_command, as proposed by
Hannu. That way we don't need to reinvent any wheel a
Csaba Nagy wrote:
Why not have a design where the slave is in control for it's own data ?
I mean the slave could ask for the base files (possibly through a
special function deployed on the master), then ask for the WAL stream
and so on. That would easily let a slave cascade too, as it could relay
Heikki Linnakangas wrote:
Simon Riggs wrote:
Built-in? Why? I mean make base backup using rsync. That way only
changed data blocks need be migrated, so much faster.
Yes, what I meant is that it would be cool to have that functionality
built-in, so that you wouldn't need to configure extra r
Simon Riggs wrote:
Built-in? Why? I mean make base backup using rsync. That way only
changed data blocks need be migrated, so much faster.
Yes, what I meant is that it would be cool to have that functionality
built-in, so that you wouldn't need to configure extra rsync scripts and
authenticat
On Fri, 2008-09-12 at 17:24 +0300, Hannu Krosing wrote:
> On Fri, 2008-09-12 at 17:08 +0300, Heikki Linnakangas wrote:
> > Hmm, built-in rsync capability would be cool. Probably not in the first
> > phase, though..
>
> We have it for WAL shipping, in form of GUC "archive_command" :)
>
> Why not
On Fri, 2008-09-12 at 17:08 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> I think we'll need an option to specify a maximum for the number of WAL
> files to keep around. The DBA should set that to the size of the WAL
> drive, minus some safety factor.
>
> > It should be clear that to
On Fri, 2008-09-12 at 17:08 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
>
> > It should be clear that to make this work you must run with a base
> > backup that was derived correctly on the current master. You can do that
> > by re-copying everything, or you can do that by just shipping
Simon Riggs wrote:
If we were going to recover from failed-over standby back to original
master just via WAL logs we would need all of the WAL files from the
point of failover. So you'd need to be storing all WAL file just in case
the old master recovers. I can't believe doing that would be the c
On Fri, 2008-09-12 at 00:03 +0900, Fujii Masao wrote:
> In my procedure, old WAL files are copyed by admin using scp, rsync
> or other external tool. So, I don't think that my procedure makes a
> problem more difficult. Since there are many setup cases, we should
> not leave all procedures to pos
On Thu, 2008-09-11 at 18:17 +0300, Heikki Linnakangas wrote:
> Fujii Masao wrote:
> > I think that this case would often happen. So, we should establish a certain
> > solution or procedure to the case where TLI of the master doesn't match
> > TLI of the slave. If we only allow the case where TLI o
Gregory Stark wrote:
Tom Lane <[EMAIL PROTECTED]> writes:
I'm not sure what to do if we need signals sent from processes that
aren't connected to shared memory; but maybe we need not cross that
bridge here.
Such as signals coming from the postmaster? Isn't that where most of them come
from th
Tom Lane <[EMAIL PROTECTED]> writes:
> I'm not sure what to do if we need signals sent from processes that
> aren't connected to shared memory; but maybe we need not cross that
> bridge here.
Such as signals coming from the postmaster? Isn't that where most of them come
from though?
--
Gregor
Hi,
Tom Lane wrote:
No, that's not what I had in mind at all, just the ability to deliver
one of a specified set of event notifications --- ie, get around the
fact that Unix only gives us two user-definable signal types.
Ah, okay. And I already thought you'd like imessages :-(
For signals se
Markus Wanner <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> Sooner or later we shall have to bite the bullet and set up a
>> multiplexing system to transmit multiple event types to backends with
>> just one signal. We already did it for signals to the postmaster.
> Agreed. However, it's non-tr
Fujii Masao wrote:
I think that this case would often happen. So, we should establish a certain
solution or procedure to the case where TLI of the master doesn't match
TLI of the slave. If we only allow the case where TLI of both servers is the
same, the configuration after failover always needs
On Thu, Sep 11, 2008 at 3:17 AM, Simon Riggs <[EMAIL PROTECTED]> wrote:
>
> On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
>
>> My sequence covers several cases :
>>
>> * There is no missing WAL file.
>> * There is a lot of missing WAL file.
>
> This is the likely case for any medium+ sized
Hi,
Tom Lane wrote:
Sooner or later we shall have to bite the bullet and set up a
multiplexing system to transmit multiple event types to backends with
just one signal. We already did it for signals to the postmaster.
Agreed. However, it's non-trivial if you want reliable queues (i.e. no
mes
Hi,
Fujii Masao wrote:
Umm... backends have already used SIGUSR1. PostgresMain() sets up a signal
handler for SIGUSR1 as follows.
Uh.. right. Thanks for pointing that out. Maybe just use SIGPIPE for now?
Yes, since WAL sender waits on select(), it's convenient to use signal
for the notificat
"Fujii Masao" <[EMAIL PROTECTED]> writes:
> Which signal should WAL sender send to backends?
Sooner or later we shall have to bite the bullet and set up a
multiplexing system to transmit multiple event types to backends with
just one signal. We already did it for signals to the postmaster.
On Wed, Sep 10, 2008 at 11:13 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
>> Which signal should we use for the notification to the backend from
>> WAL sender? The notable signals are already used.
>
> I'm using SIGUSR1, see src/backend/storage/ipc/imsg.c from Postgres-R, line
> 232. That isn't is
On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
> My sequence covers several cases :
>
> * There is no missing WAL file.
> * There is a lot of missing WAL file.
This is the likely case for any medium+ sized database.
> * There are missing history files. Failover always generates the gap
Hi,
Fujii Masao wrote:
On Tue, Sep 9, 2008 at 10:55 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
Hi,
ITAGAKI Takahiro wrote:
Signals and locking, borrewed from Postgres-R, are now studied
for the purpose in the log shipping,
Cool. Let me know if you have any questions WRT this imessages stuf
On Tue, Sep 9, 2008 at 10:55 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
> Hi,
>
> ITAGAKI Takahiro wrote:
>>
>> Signals and locking, borrewed from Postgres-R, are now studied
>> for the purpose in the log shipping,
>
> Cool. Let me know if you have any questions WRT this imessages stuff.
If you'
On Wed, 2008-09-10 at 09:36 -0400, Aidan Van Dyk wrote:
> * Simon Riggs <[EMAIL PROTECTED]> [080910 06:18]:
>
> > We have a number of choices, at the point of failure:
> > * Does the whole primary server stay up (probably)?
>
> The only sane choice is the one the admin makes. Any "predetermined
* Simon Riggs <[EMAIL PROTECTED]> [080910 06:18]:
> We have a number of choices, at the point of failure:
> * Does the whole primary server stay up (probably)?
The only sane choice is the one the admin makes. Any "predetermined" choice
PG makes can (and will) be wrong in some situations.
> * Do
On Wed, 2008-09-10 at 09:35 +0300, Hannu Krosing wrote:
> On Wed, 2008-09-10 at 15:15 +0900, Fujii Masao wrote:
> > On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas
> > <[EMAIL PROTECTED]> wrote:
> > > If a slave falls behind, how does it catch up? I guess you're saying that
> > > it
> > > ca
On Wed, 2008-09-10 at 11:07 +0200, Dimitri Fontaine wrote:
> Hi,
>
> Le mercredi 10 septembre 2008, Heikki Linnakangas a écrit :
> > Sure. That's the fundamental problem with synchronous replication.
> > That's why many people choose asynchronous replication instead. Clearly
> > at some point you
Hi,
Simon Riggs wrote:
The standby server won't come up until you have:
* copied the base backup
* sent it to standby server
* bring up standby, have it realise it is a replication partner and
begin requesting WAL from primary (in some way)
Right. That was your assumption as well. Required bef
On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
> Umm.. I disagree with you ;)
That's no problem and I respect your knowledge.
If we disagree, it is very likely because we have misunderstood each
other. Much has been written, so I will wait for it to all be read and
understood by you and
Hi,
Le mercredi 10 septembre 2008, Heikki Linnakangas a écrit :
> Sure. That's the fundamental problem with synchronous replication.
> That's why many people choose asynchronous replication instead. Clearly
> at some point you'll want to give up and continue without the slave, or
> kill the master
On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
> I cannot accept that WAL sender is blocked for initial setup.
Yes, very important point. We definitely agree on that. The primary must
be able to continue working while all this setup is happening. No tables
are locked, all commits are
On Wed, 2008-09-10 at 10:06 +0200, Markus Wanner wrote:
> Hi,
>
> Simon Riggs wrote:
> > 1. Standby contacts primary and says it would like to catch up, but is
> > currently at point X (which is a point at, or after the first consistent
> > stopping point in WAL after standby has performed its ow
On Wed, Sep 10, 2008 at 4:15 PM, Simon Riggs <[EMAIL PROTECTED]> wrote:
>
> On Wed, 2008-09-10 at 09:35 +0300, Hannu Krosing wrote:
>> On Wed, 2008-09-10 at 15:15 +0900, Fujii Masao wrote:
>> > On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas
>> > <[EMAIL PROTECTED]> wrote:
>> > > If a slave fa
On Wed, 2008-09-10 at 08:15 +0100, Simon Riggs wrote:
> Any working solution needs to work for all required phases. If you did
> it this way, you'd never catch up at all.
>
> When you first make the copy, it will be made at time X. The point of
> consistency will be sometime later and requires WA
On Wed, 2008-09-10 at 10:06 +0200, Markus Wanner wrote:
> Hi,
>
> Simon Riggs wrote:
> > 1. Standby contacts primary and says it would like to catch up, but is
> > currently at point X (which is a point at, or after the first consistent
> > stopping point in WAL after standby has performed its own
On Tue, 2008-09-09 at 20:59 +0200, Zeugswetter Andreas OSB sIT wrote:
> All in all a useful streamer seems like a lot of work.
I mentioned some time ago an alternative idea of having the slave
connect through a normal SQL connection and call a function which
streams the WAL file from the point req
Simon Riggs wrote:
Saying "I want to wait for a synchronous commit and I am willing to wait
for ever to ensure it" leads to long hangs in some cases.
Sure. That's the fundamental problem with synchronous replication.
That's why many people choose asynchronous replication instead. Clearly
at s
On Wed, 2008-09-10 at 11:10 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > On Wed, 2008-09-10 at 13:28 +0900, Fujii Masao wrote:
> >> On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
> >> <[EMAIL PROTECTED]> wrote:
> >>> There's one thing I haven't figured out in this discussion. Does
On Wed, Sep 10, 2008 at 1:40 PM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
>
>
> The thing that bothers me is the behavior when the synchronous slave doesn't
> respond. A timeout has been discussed, after which the master just gives up
> on sending, and starts acting as if there's no slave. How
Simon Riggs wrote:
On Wed, 2008-09-10 at 13:28 +0900, Fujii Masao wrote:
On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
There's one thing I haven't figured out in this discussion. Does the write
to the disk happen before or after the write to the slave? Can you gu
Hi,
Simon Riggs wrote:
1. Standby contacts primary and says it would like to catch up, but is
currently at point X (which is a point at, or after the first consistent
stopping point in WAL after standby has performed its own crash
recovery, if any was required).
2. primary initiates data transfe
On Wed, 2008-09-10 at 12:24 +0530, Pavan Deolasee wrote:
> Also we may not want the master to be stuck while slave is doing the catchup.
No, since it may take hours, not seconds.
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support
--
Sent via pgsql-hacker
On Wed, 2008-09-10 at 13:28 +0900, Fujii Masao wrote:
> On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
> <[EMAIL PROTECTED]> wrote:
> > There's one thing I haven't figured out in this discussion. Does the write
> > to the disk happen before or after the write to the slave? Can you guarantee
>
On Wed, Sep 10, 2008 at 12:05 PM, Hannu Krosing <[EMAIL PROTECTED]> wrote:
>
>
>> Because, at the time, current WAL position might be in the middle of
>> WAL file. Even if the master sends only current WAL data, the slave
>> which don't have the corresponding WAL file can not handle it.
>
> I agree
On Wed, 2008-09-10 at 15:15 +0900, Fujii Masao wrote:
> On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas
> <[EMAIL PROTECTED]> wrote:
> > If a slave falls behind, how does it catch up? I guess you're saying that it
> > can't fall behind, because the master will block before that happens. Also
>
On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
> If a slave falls behind, how does it catch up? I guess you're saying that it
> can't fall behind, because the master will block before that happens. Also
> in asynchronous replication? And what about when the slave is
On Tue, Sep 9, 2008 at 8:42 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
>> In the viewpoint of detection of a network failure, this feature is
>> necessary.
>> When the network goes down, WAL sender can be blocked until it detects
>> the network failure, i.e. WAL sender keeps waiting for the respo
On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
> There's one thing I haven't figured out in this discussion. Does the write
> to the disk happen before or after the write to the slave? Can you guarantee
> that if a transaction is committed in the master, it's also com
> > Don't understand. I am referring to the logic at the top of
> > AdvanceXLInsertBuffer(). We would need to wait for all people reading
> > the contents of wal_buffers.
>
> Oh, I see.
>
> If a slave falls behind, how does it catch up? I guess you're saying
> that it can't fall behind, because th
On Tue, 2008-09-09 at 18:26 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > Don't understand. I am referring to the logic at the top of
> > AdvanceXLInsertBuffer(). We would need to wait for all people reading
> > the contents of wal_buffers.
>
> Oh, I see.
>
> If a slave falls behind
Simon Riggs wrote:
Don't understand. I am referring to the logic at the top of
AdvanceXLInsertBuffer(). We would need to wait for all people reading
the contents of wal_buffers.
Oh, I see.
If a slave falls behind, how does it catch up? I guess you're saying
that it can't fall behind, because
On Tue, 2008-09-09 at 17:17 +0300, Heikki Linnakangas wrote:
> Tom Lane wrote:
> > Simon Riggs <[EMAIL PROTECTED]> writes:
> >> On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote:
> >>> "Agreed"? That last restriction is a deal-breaker.
> >
> >> OK, I should have said *if wal_buffers are full* XL
Tom Lane wrote:
Simon Riggs <[EMAIL PROTECTED]> writes:
On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote:
"Agreed"? That last restriction is a deal-breaker.
OK, I should have said *if wal_buffers are full* XLogInsert() cannot
advance to a new page while we are waiting to send or write. So
On Tue, 2008-09-09 at 16:05 +0200, Dimitri Fontaine wrote:
> Le mardi 09 septembre 2008, Simon Riggs a écrit :
> > If the WALWriter|Sender is available, it can begin the task immediately.
> > There is no need for it to wait if you want synchronous behaviour.
>
> Ok. Now I'm as lost as anyone with
Le mardi 09 septembre 2008, Simon Riggs a écrit :
> If the WALWriter|Sender is available, it can begin the task immediately.
> There is no need for it to wait if you want synchronous behaviour.
Ok. Now I'm as lost as anyone with respect to how you get Group Commit :)
--
dim
signature.asc
Descri
Hi,
Dimitri Fontaine wrote:
Exactly the point. The process is now already waiting in all cases, so maybe
we could just force waiting some WALSender signal before sending the fsync()
order, so we now have Group Commit.
A single process can only wait on either fsync() or on select(), but not
o
Hi,
ITAGAKI Takahiro wrote:
Signals and locking, borrewed from Postgres-R, are now studied
for the purpose in the log shipping,
Cool. Let me know if you have any questions WRT this imessages stuff.
Regards
Markus Wanner
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
On Tue, 2008-09-09 at 15:32 +0200, Dimitri Fontaine wrote:
> The process is now already waiting in all cases
If the WALWriter|Sender is available, it can begin the task immediately.
There is no need for it to wait if you want synchronous behaviour.
--
Simon Riggs www.2ndQuadrant.com
Le mardi 09 septembre 2008, Markus Wanner a écrit :
> ..and it will still has to wait until WAL is written to disk on the
> local node, as we do now. These are two different things to wait for.
> One is a network socket operation, the other is an fsync(). As these
> don't work together too well (bl
Hi,
Dimitri Fontaine wrote:
It might just be I'm not understanding what it's all about, but it seems to me
with WALSender process A will wait, whatever happens, either until the WAL is
sent to slave or written to disk on the slave.
..and it will still has to wait until WAL is written to disk
Hi,
Le mardi 09 septembre 2008, Heikki Linnakangas a écrit :
> The tricky part is, how does A know if it should wait, and for how long?
> commit_delay sure isn't ideal, but AFAICS the log shipping proposal
> doesn't provide any solution to that.
It might just be I'm not understanding what it's al
Simon Riggs <[EMAIL PROTECTED]> writes:
> On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote:
>> "Agreed"? That last restriction is a deal-breaker.
> OK, I should have said *if wal_buffers are full* XLogInsert() cannot
> advance to a new page while we are waiting to send or write. So I don't
> thi
On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote:
> "Fujii Masao" <[EMAIL PROTECTED]> writes:
> > On Tue, Sep 9, 2008 at 5:11 PM, Simon Riggs <[EMAIL PROTECTED]> wrote:
> >>
> >> Yes. We should have a LogwrtRqst pointer and LogwrtResult pointer for
> >> the send operation. The Write and Send ope
"Fujii Masao" <[EMAIL PROTECTED]> writes:
> On Tue, Sep 9, 2008 at 5:11 PM, Simon Riggs <[EMAIL PROTECTED]> wrote:
>>
>> Yes. We should have a LogwrtRqst pointer and LogwrtResult pointer for
>> the send operation. The Write and Send operations can then continue
>> independently of one another. XLo
On Tue, 2008-09-09 at 13:42 +0200, Markus Wanner wrote:
> How about multiple standby servers?
There are various ways for getting things to work with multiple servers.
I hope we can make this work with just a single standby before we try to
make it work on more. There are various options for sync
On Tue, 2008-09-09 at 20:12 +0900, Fujii Masao wrote:
> I'd like to introduce new parameter "synchronous_replication" which specifies
> whether backends waits for the response from WAL sender process. By
> combining synchronous_commit and synchronous_replication, users can
> choose various option
On Tue, 2008-09-09 at 12:54 +0300, Heikki Linnakangas wrote:
> Note that we already have the logic to flush all pending commit
> records at once.
But only when you can grab WALInsertLock when flushing. If you look at
the way I suggested, it does not rely upon that lock being available.
So it is
Fujii Masao wrote:
What makes the sender process bottleneck?
The keyword here is "might". There's many possibilities, like:
- Slow network.
- Ridiculously fast disk. Like a RAM disk. If you have a synchronous
slave you can fail over to, putting WAL on a RAM disk isn't that crazy.
- slower WAL
On Tue, 2008-09-09 at 20:12 +0900, Fujii Masao wrote:
> What makes the sender process bottleneck?
In my experience, the Atlantic. But I guess the Pacific does it too. :-)
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support
--
Sent via pgsql-hackers maili
Hi,
Fujii Masao wrote:
Really? In the benchmark result of my prototype, the bottleneck is
still disk I/O.
The communication (between the master and the slave) latency is smaller than
WAL writing (fsyncing) one. Of course, I assume that we use not-poor network
like 1000BASE-T.
Sure. If you do W
Hi,
ITAGAKI Takahiro wrote:
Signals and locking, borrewed from Postgres-R, are now studied
for the purpose in the log shipping, but I'm not sure it can be
also used in the group commit.
Yeah. As Heikki points out, there is a completely orthogonal question
WRT group commit: how does transactio
On Tue, Sep 9, 2008 at 5:11 PM, Simon Riggs <[EMAIL PROTECTED]> wrote:
>
> Yes. We should have a LogwrtRqst pointer and LogwrtResult pointer for
> the send operation. The Write and Send operations can then continue
> independently of one another. XLogInsert() cannot advance to a new page
> while we
Heikki Linnakangas <[EMAIL PROTECTED]> wrote:
> The tricky part is, how does A know if it should wait, and for how long?
> commit_delay sure isn't ideal, but AFAICS the log shipping proposal
> doesn't provide any solution to that.
They have no relation each other directly,
but they need simila
Simon Riggs wrote:
Multiple backends waiting while we perform a write. Commits then happen
as a group (to WAL at least), hence Group Commit.
The problem with our current commit protocol is this:
1. Backend A inserts commit record A
2. Backend A starts to flush commit record A
3. Backend B inse
On Tue, 2008-09-09 at 12:24 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > This gives us the Group Commit feature also, even if we are not using
> > replication. So we can drop the commit_delay stuff.
>
> Huh? How does that give us group commit?
Multiple backends waiting while we perf
Simon Riggs wrote:
This gives us the Group Commit feature also, even if we are not using
replication. So we can drop the commit_delay stuff.
Huh? How does that give us group commit?
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pg
On Mon, 2008-09-08 at 17:40 -0400, Bruce Momjian wrote:
> Fujii Masao wrote:
> > On Mon, Sep 8, 2008 at 8:44 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
> > >>Merge into WAL writer?
> > >
> > > Uh.. that would mean you'd loose parallelism between WAL writing to disk
> > > and
> > > WAL s
"Fujii Masao" <[EMAIL PROTECTED]> wrote:
> 3) Patch of introducing new background process which I've called
> WALSender. It takes charge of sending WAL to the slave.
>
> Now, I assume that WALSender also listens the connection from
> the slave, i.e. only one sender process manages
Hi,
I looked some comment for the synchronous replication and understood
the consensus of the community was that the sync replication should be
added using not hooks and plug-ins but core-patches. If my understanding
is right, I will change my development plan so that the sync replication
may be p
Markus Wanner wrote:
> Hi,
>
> Bruce Momjian wrote:
> > Backends would
> > update a shared memory variable specifying how far they want the wal
> > streamer to advance and send a signal to the wal streamer if necessary.
> > Backends would monitor another shared memory variable that specifies how
Hi,
Bruce Momjian wrote:
Backends would
update a shared memory variable specifying how far they want the wal
streamer to advance and send a signal to the wal streamer if necessary.
Backends would monitor another shared memory variable that specifies how
far the wal streamer has advanced.
Tha
Fujii Masao wrote:
> On Mon, Sep 8, 2008 at 8:44 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
> >>Merge into WAL writer?
> >
> > Uh.. that would mean you'd loose parallelism between WAL writing to disk and
> > WAL shipping via network. That does not sound appealing to me.
>
> That depends
On Mon, 2008-09-08 at 19:19 +0900, ITAGAKI Takahiro wrote:
> Bruce Momjian <[EMAIL PROTECTED]> wrote:
>
> > > > b) Use new background process as WALSender
> > > >
> > > >This idea needs background-process hook which enables users
> > > >to define new background processes
>
> > I think s
Hi,
Fujii Masao wrote:
1. A backend writes WAL to disk.
2. The backend wakes up WAL sender process and sleeps.
3. WAL sender process does WAL shipping and wakes up the backend.
4. The backend issues sync command.
Right, that would work. But still, the WAL writer process would block
during wri
On Mon, Sep 8, 2008 at 8:44 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
>>Merge into WAL writer?
>
> Uh.. that would mean you'd loose parallelism between WAL writing to disk and
> WAL shipping via network. That does not sound appealing to me.
That depends on the order of WAL writing and W
Hi,
ITAGAKI Takahiro wrote:
1. Is process-switching approach the best way to share one socket?
Both Postgres-R and the log-shipping prototype use the approach now.
Can I think there is no objection here?
I don't see any appealing alternative. The postmaster certainly
shouldn
Markus Wanner <[EMAIL PROTECTED]> wrote:
> ITAGAKI Takahiro wrote:
> > Are there any better idea to share one socket connection between
> > backends (and bgwriter)?
>
> I fear I'm repeating myself, but I've had the same problem for
> Postgres-R and solved it with an internal message passing infr
Hi,
ITAGAKI Takahiro wrote:
Are there any better idea to share one socket connection between
backends (and bgwriter)? The connections could be established after
fork() from postmaster, and number of them could be two or more.
This is one of the most complicated part of synchronous log shipping.
Bruce Momjian <[EMAIL PROTECTED]> wrote:
> > > b) Use new background process as WALSender
> > >
> > >This idea needs background-process hook which enables users
> > >to define new background processes
> I think starting/stopping a process for each WAL send is too much
> overhead.
Yes,
On Sat, 2008-09-06 at 22:09 -0400, Bruce Momjian wrote:
> > I'm unclear on what you want hooks for. If additional processes get
> > integrated into Postgres, those certainly need to get integrated very
> > much like we integrated other auxiliary processes. I wouldn't call that
> > 'hooking', b
1 - 100 of 106 matches
Mail list logo
