Simon Riggs wrote:
On Tue, 2010-03-02 at 13:13 +, Greg Stark wrote:
Why do we disallow turning off archive_mode anyways?
Because it is needed for safety and nobody has got around to coding the
idea of turning it on/off during normal running, which is possible, with
appropriate care
On Tue, 2010-03-02 at 13:13 +, Greg Stark wrote:
> On Tue, Mar 2, 2010 at 9:48 AM, Simon Riggs wrote:
> >> > Setting archive_mode to a command that does nothing but return true,
> >> > e.g. /bin/true,
> >>
> >> "return true" seems ambiguous for me. How about writing clearly
> >> "return a zer
On Tue, Mar 2, 2010 at 9:48 AM, Simon Riggs wrote:
>> > Setting archive_mode to a command that does nothing but return true, e.g.
>> > /bin/true,
>>
>> "return true" seems ambiguous for me. How about writing clearly
>> "return a zero exit status" instead?
>
> Docs are already quite clear on that
On Tue, 2010-03-02 at 15:20 +0900, Fujii Masao wrote:
> On Fri, Feb 26, 2010 at 2:47 AM, Bruce Momjian wrote:
> > Postgres 9.0 will be the first release to mention /bin/true as a way of
> > turning off archiving in extraordinary circumstances:
> >
> >
> > http://developer.postgresql.org/pg
On Tue, 2 Mar 2010 15:20:36 +0900, Fujii Masao
wrote:
>> Setting archive_mode to a command that does nothing but return true,
>> e.g. /bin/true,
>
> "return true" seems ambiguous for me. How about writing clearly
> "return a zero exit status" instead?
For the record. I hate the fact that I ever
Greg Smith wrote:
> Fujii Masao wrote:
>> We would be easily able to calculate the last archived log file from
>> the existence of archive status files.
>
> Right, but you have to actually scan the whole archive directory to
> figure that out, and I'd rather not see that code get duplicated
> some
Fujii Masao wrote:
We would be easily able to calculate the last archived log file from
the existence of archive status files.
Right, but you have to actually scan the whole archive directory to
figure that out, and I'd rather not see that code get duplicated
somewhere else when it's alrea
Fujii Masao wrote:
On Fri, Feb 26, 2010 at 2:47 AM, Bruce Momjian wrote:
Postgres 9.0 will be the first release to mention /bin/true as a way of
turning off archiving in extraordinary circumstances:
http://developer.postgresql.org/pgdocs/postgres/runtime-config-wal.html
Setting
On Fri, Feb 26, 2010 at 10:00 PM, Greg Stark wrote:
> Secondarily, the message printed at this time and when the process is
> finished doesn't actually give the user any information on how much
> longer to expect the process to take.
>
> It would be nice to say what the target archive log we're wa
On Fri, Feb 26, 2010 at 2:47 AM, Bruce Momjian wrote:
> Postgres 9.0 will be the first release to mention /bin/true as a way of
> turning off archiving in extraordinary circumstances:
>
> http://developer.postgresql.org/pgdocs/postgres/runtime-config-wal.html
> Setting archive_mode to a c
On Fri, Feb 26, 2010 at 9:41 AM, Bernd Helmle wrote:
>
>
> --On 24. Februar 2010 16:01:02 -0500 Tom Lane wrote:
>
>> One objection to this is that it's not very clear to the user when
>> pg_stop_backup has finished with actual work and is just waiting for the
>> archiver, ie when is it safe to hi
--On 24. Februar 2010 16:01:02 -0500 Tom Lane wrote:
One objection to this is that it's not very clear to the user when
pg_stop_backup has finished with actual work and is just waiting for the
archiver, ie when is it safe to hit control-C? Maybe we should emit a
"backup done, waiting for arc
Looks like we arrived at the best solution here. I don't think it was
clear to users that pg_stop_backup() was issuing an archive_command and
hence they wouldn't be likely to understand the delay or correct a
problem. This gives them the information they need at the time they
need it.
-
Joshua D. Drake wrote:
> On Wed, 2010-02-24 at 12:32 -0800, Josh Berkus wrote:
> > > pg_stop_backup() doesn't complete until all the WAL segments needed to
> > > restore from the backup are archived. If archive_command is failing,
> > > that never happens.
> >
> > OK, so we need a way out of that
Greg Stark wrote:
In an ideal world it would be best if pg_stop_backup could actually
print the error status of the archiving command. Is there any way for
it to get ahold of the fact that the archiving is failing?
This is in the area I mentioned I'd proposed a patch to improve not too
long
Heikki Linnakangas wrote:
> Josh Berkus wrote:
> > OK, can you go through the reasons why pg_stop_backup would not
> > complete?
>
> pg_stop_backup() doesn't complete until all the WAL segments needed to
> restore from the backup are archived. If archive_command is failing,
> that never happens.
On Wed, Feb 24, 2010 at 11:14 PM, Josh Berkus wrote:
>
> Right. I'm pointing out that production and "trying out 9.0 for the
> first time" are actually different circumstances, and we need to be able
> to handle both gracefully. Since, if people have a bad experience
> trying it out for the firs
Greg Smith writes:
> Tom Lane wrote:
>> The value of the HINT I think would be to make them (a) not afraid to
>> hit control-C and (b) aware of the fact that their archiver has got
>> a problem.
>>
> Agreed on both points. Patch attached that implements something similar
> to Josh's wording, tw
> I don't want that immediate NOTICE message, which sounds like a noise.
> Delaying it or changing the log level to DEBUG work for me.
Problem is that a new user won't be seeing DEBUG messages by default.
This issue is all about new user experience.
Alternatively, we could move the time of the f
On Thu, Feb 25, 2010 at 10:52 AM, Tom Lane wrote:
> The one thing I'm undecided about is whether we want the immediate
> NOTICE, as opposed to dialing down the time till the first WARNING
> to something like 5 or 10 seconds. I think the main argument for the
> latter approach would be to avoid lo
On 2/24/10 5:58 PM, Greg Smith wrote:
>
> I though about that for a minute, but didn't think pg_stop_backup is a
> common enough operation that anyone will complain that it's a little
> more verbose in its logging now. I know when I was new to this, I used
> to wonder just what it was busy doing
Tom Lane wrote:
The one thing I'm undecided about is whether we want the immediate
NOTICE, as opposed to dialing down the time till the first WARNING
to something like 5 or 10 seconds. I think the main argument for the
latter approach would be to avoid log-spam in normal operation
I though abo
On Wed, Feb 24, 2010 at 08:52:28PM -0500, Tom Lane wrote:
> Josh Berkus writes:
> > On 2/24/10 5:36 PM, Greg Smith wrote:
> >> gsmith=# select pg_stop_backup();
> >> NOTICE: pg_stop_backup cleanup done, waiting for required segments to
> >> archive
> >> WARNING: pg_stop_backup still waiting for
Josh Berkus writes:
> On 2/24/10 5:36 PM, Greg Smith wrote:
>> gsmith=# select pg_stop_backup();
>> NOTICE: pg_stop_backup cleanup done, waiting for required segments to
>> archive
>> WARNING: pg_stop_backup still waiting for all required segments to
>> archive (60 seconds elapsed)
>> HINT: Con
On 2/24/10 5:36 PM, Greg Smith wrote:
> gsmith=# select pg_stop_backup();
> NOTICE: pg_stop_backup cleanup done, waiting for required segments to
> archive
> WARNING: pg_stop_backup still waiting for all required segments to
> archive (60 seconds elapsed)
> HINT: Confirm your archive_command is
Tom Lane wrote:
The value of the HINT I think would be to make them (a) not afraid to
hit control-C and (b) aware of the fact that their archiver has got
a problem.
Agreed on both points. Patch attached that implements something similar
to Josh's wording, tweaking the original warning too.
On Wed, 2010-02-24 at 19:08 -0500, Tom Lane wrote:
> Simon Riggs writes:
> > On Wed, 2010-02-24 at 16:52 -0500, Tom Lane wrote:
> >> * emit a NOTICE as soon as pg_stop_backup's actual work is done and
> >> it's starting to wait for the archiver (or maybe after it's waited
> >> for a few seconds, b
On Wed, 2010-02-24 at 23:57 +, Simon Riggs wrote:
>
> > * emit a NOTICE as soon as pg_stop_backup's actual work is done and
> > it's starting to wait for the archiver (or maybe after it's waited
> > for a few seconds, but much less than the present 60).
>
> Pointless really. Nobody runs back
Simon Riggs writes:
> On Wed, 2010-02-24 at 16:52 -0500, Tom Lane wrote:
>> * emit a NOTICE as soon as pg_stop_backup's actual work is done and
>> it's starting to wait for the archiver (or maybe after it's waited
>> for a few seconds, but much less than the present 60).
> Pointless really. Nobod
On Wed, 2010-02-24 at 14:20 -0800, Josh Berkus wrote:
> Since Kevin suggested this in his first post and I agreed with that in
> > the first paragraph of my first post, I think you've wasted a lot of
> > time here going in circles. 42 posts, more than a dozen people. I
> think
>
> Please tone down
On Wed, 2010-02-24 at 16:52 -0500, Tom Lane wrote:
> Before you could enter pg_abort_backup you'd have to control-C out of
> the pg_stop_backup call, and that action already accomplishes the only
> thing pg_abort_backup could do for you.
Agreed. I was responding to perceived user need.
> So what
> If you like the concept, then the next question is exactly how to phrase
> the messages. All we have at the moment is the inside-the-delay-loop
> warning:
>
> ereport(WARNING,
> (errmsg("pg_stop_backup still waiting for archive to complete (%d
> seconds elapsed)",
>
On Feb 24, 2010, at 3:24 PM, Tom Lane wrote:
> If you like the concept, then the next question is exactly how to phrase
> the messages. All we have at the moment is the inside-the-delay-loop
> warning:
>
>ereport(WARNING,
>(errmsg("pg_stop_backup still waiting for archive to com
Josh Berkus writes:
> Tom, Simon,
>> * emit a NOTICE as soon as pg_stop_backup's actual work is done and
>> it's starting to wait for the archiver (or maybe after it's waited
>> for a few seconds, but much less than the present 60).
>>
>> * extend the existing WARNING (and the NOTICE too if we el
> That's not what I said to do first. If it's possible to fix your
> archive_command, and it never returned bad "I'm saying success but I
> didn't really do the right thing" information to the server--it just
> failed--this situation is completely recoverable with no damage to the
> backup. Just
Josh Berkus wrote:
Thing is, if archive_command is failing, then the backup is useless
regardless until it's fixed. And sending the archives to /dev/null (the
fix you're essentially recommending above) doesn't make the backup any
more useful.
That's not what I said to do first. If it's possib
Tom, Simon,
> * emit a NOTICE as soon as pg_stop_backup's actual work is done and
> it's starting to wait for the archiver (or maybe after it's waited
> for a few seconds, but much less than the present 60).
>
> * extend the existing WARNING (and the NOTICE too if we elect to have
> one) with a H
Simon Riggs writes:
> Please test the following patch to see if it meets your needs and check
> the wordings used in the docs.
What exactly will that function accomplish, given the assumption that
the user already tried pg_stop_backup?
regards, tom lane
--
Sent via pgsq
On Wed, 2010-02-24 at 13:30 -0800, Josh Berkus wrote:
> So I'm seeing pg_abort_backup(), which also produces a
> markers which prevent the backup from loading, as an improvement on
> current UI.
Since Kevin suggested this in his first post and I agreed with that in
the first paragraph of my first
Josh Berkus writes:
> Thing is, if archive_command is failing, then the backup is useless
> regardless until it's fixed. And sending the archives to /dev/null (the
> fix you're essentially recommending above) doesn't make the backup any
> more useful. So I'm seeing pg_abort_backup(), which also
Josh Berkus wrote:
> So I'm seeing pg_abort_backup(), which also produces a
> markers which prevent the backup from loading, as an improvement on
> current UI.
Starting with 9.0, if recovery doesn't see a end-of-backup record, it
will refuse to start up. In earlier versions we had a similar mechan
Greg,
> I expect for your particular bad situation, you can replace the
> archive_command with a corrected one, use "pg_ctl reload" to send a
> SIGHUP to make that fix active, and escape from this. That's the only
> right way out of this situation. You can't just abort a backup someone
> has ask
Heikki Linnakangas writes:
> Josh Berkus wrote:
>> OK, so we need a way out of that cycle if the user is issuing
>> pg_stop_backup because they *already know* that archive_command is
>> failing. Right now, there's no way out other than a fast shutdown,
> Sure there is. Just kill the session, Ctr
Josh Berkus wrote:
pg_stop_backup() doesn't complete until all the WAL segments needed to
restore from the backup are archived. If archive_command is failing,
that never happens.
OK, so we need a way out of that cycle if the user is issuing
pg_stop_backup because they *already know* that a
On Feb 24, 2010, at 12:47 PM, Tom Lane wrote:
>> OK, so we need a way out of that cycle if the user is issuing
>> pg_stop_backup because they *already know* that archive_command is
>> failing. Right now, there's no way out other than a fast shutdown,
>> which is a bit user-hostile.
>
> The pg_ab
Josh Berkus writes:
>> pg_stop_backup() doesn't complete until all the WAL segments needed to
>> restore from the backup are archived. If archive_command is failing,
>> that never happens.
> OK, so we need a way out of that cycle if the user is issuing
> pg_stop_backup because they *already know*
On Wed, 2010-02-24 at 11:55 -0800, Joshua D. Drake wrote:
> On Wed, 2010-02-24 at 19:02 +, Simon Riggs wrote:
> > On Wed, 2010-02-24 at 10:17 -0800, Joshua D. Drake wrote:
>
> > You make the mistake of assuming that someone that can develop has no
> > solution experience. That is exactly how I
Josh Berkus wrote:
>> pg_stop_backup() doesn't complete until all the WAL segments
>> needed to restore from the backup are archived. If
>> archive_command is failing, that never happens.
>
> OK, so we need a way out of that cycle if the user is issuing
> pg_stop_backup because they *already kn
Josh Berkus wrote:
>> pg_stop_backup() doesn't complete until all the WAL segments needed to
>> restore from the backup are archived. If archive_command is failing,
>> that never happens.
>
> OK, so we need a way out of that cycle if the user is issuing
> pg_stop_backup because they *already know*
On Wed, 2010-02-24 at 12:32 -0800, Josh Berkus wrote:
> > pg_stop_backup() doesn't complete until all the WAL segments needed to
> > restore from the backup are archived. If archive_command is failing,
> > that never happens.
>
> OK, so we need a way out of that cycle if the user is issuing
> pg_s
> pg_stop_backup() doesn't complete until all the WAL segments needed to
> restore from the backup are archived. If archive_command is failing,
> that never happens.
OK, so we need a way out of that cycle if the user is issuing
pg_stop_backup because they *already know* that archive_command is
fa
Josh Berkus wrote:
> OK, can you go through the reasons why pg_stop_backup would not
> complete?
pg_stop_backup() doesn't complete until all the WAL segments needed to
restore from the backup are archived. If archive_command is failing,
that never happens.
> And why it's a problem to have it co
On 2/24/10 11:55 AM, Joshua D. Drake wrote:
> On Wed, 2010-02-24 at 19:02 +, Simon Riggs wrote:
>> On Wed, 2010-02-24 at 10:17 -0800, Joshua D. Drake wrote:
>
>> You make the mistake of assuming that someone that can develop has no
>> solution experience. That is exactly how I fund further dev
Simon,
> If you have a concrete proposal, get off your soapbox and make one,
> based upon the technical information you've received. There are clear
> reasons why things are the way they are and those reasons will not be
> ignored, by me.
OK, can you go through the reasons why pg_stop_backup woul
On Wed, 2010-02-24 at 11:31 -0800, Josh Berkus wrote:
> > This is about what happens in production, not your laptop. The required
> > behaviour in-production is to assume that the sysadmin has configured it
> > correctly and we wait for them to fix the problem.
>
> 90% of our user base does not
On Wed, 2010-02-24 at 19:02 +, Simon Riggs wrote:
> On Wed, 2010-02-24 at 10:17 -0800, Joshua D. Drake wrote:
> You make the mistake of assuming that someone that can develop has no
> solution experience. That is exactly how I fund further development, so
> you are off base by a long way.
I n
> You haven't posted a reproduceable bug, nor is this new to 9.0.
Yes, I have.
1) set up a failing archive_command on an idle database
2) do pg_start_backup
3) do pg_stop_backup
4) pg_stop_backup waits forever (or at least 5 minutes, which as long as
I've given it so far).
> This is about wh
On Wed, 2010-02-24 at 11:07 -0800, Josh Berkus wrote:
> > I haven't ignored the issue. The behaviour you are complaining about was
> > put there following complaints that it didn't wait. You're ignoring the
> > point that there hasn't been any change in this release and so your
> > comments are unf
> I haven't ignored the issue. The behaviour you are complaining about was
> put there following complaints that it didn't wait. You're ignoring the
> point that there hasn't been any change in this release and so your
> comments are unfounded in reality.
I've posted a reproduceable bug (pg_stop_
On 2/24/10 10:40 AM, Heikki Linnakangas wrote:
> Josh Berkus wrote:
>> And, while it makes sense for smart shutdown to wait for
>> pg_stop_backup(), it does not make sense for fast shutdown to wait.
>
> Hang on, fast shutdown does *not* wait for backup to finish.
It did when I tried it. I'll tes
On Wed, 2010-02-24 at 10:17 -0800, Joshua D. Drake wrote:
> Basically the reports boil down to people who are actually going to be
> dealing with this in the field. Simon with respect you have been 6 feet
> deep in code for too long on this. You need to step back and take some
> constructive feedb
On Wed, 2010-02-24 at 10:07 -0800, Josh Berkus wrote:
> >
> > The behaviour to wait for pg_stop_backup() was added by user request.
> > The behaviour for shutdown to wait for pg_stop_backup() was also added
> > by user request.
>
> Your two statements above contradict each other.
No they don't.
Josh Berkus wrote:
> And, while it makes sense for smart shutdown to wait for
> pg_stop_backup(), it does not make sense for fast shutdown to wait.
Hang on, fast shutdown does *not* wait for backup to finish.
> Aside from that, the main issue is not having shutdown wait for
> pg_stop_backup; it's
> Your OP was not too clear about whether you tried a smart shutdown or
> a fast shutdown, but if you meant a fast shutdown, this is apparently
> (he says without testing) a regression.
Ah, sorry. Yes, I attempted a fast shutdown.
--Josh Berkus
--
Sent via pgsql-hackers mailing list (pgsql-ha
On Wed, Feb 24, 2010 at 1:07 PM, Josh Berkus wrote:
> And, while it makes sense for smart shutdown to wait for
> pg_stop_backup(), it does not make sense for fast shutdown to wait.
TFM in fact says:
http://www.postgresql.org/docs/8.4/static/app-pg-ctl.html#APP-PG-CTL-DESCRIPTION
In stop mode, t
On Wed, 2010-02-24 at 10:07 -0800, Josh Berkus wrote:
> Simon,
> > Your mistake was not typoing an archive_command, it was not correctly
> > testing that what you had done was actually working. The fix is to read
> > the manual and correct the typo. Shutting down the server after failing
> > to co
Simon,
> It's not a common setup mistake. Nothing changed in this release and
> this has never been reported before.
>
> The behaviour to wait for pg_stop_backup() was added by user request.
> The behaviour for shutdown to wait for pg_stop_backup() was also added
> by user request.
Your two stat
On Tue, 2010-02-23 at 17:46 -0800, Josh Berkus wrote:
> On 2/23/10 10:58 AM, Simon Riggs wrote:
> > So I don't see this as something that needs fixing for 9.0. There is
> > already too much non-essential code there, all of which needs to be
> > tested. I don't think adding in new corner cases to "h
On 2/23/10 10:58 AM, Simon Riggs wrote:
> So I don't see this as something that needs fixing for 9.0. There is
> already too much non-essential code there, all of which needs to be
> tested. I don't think adding in new corner cases to "help" people makes
> any sense until we have automated testing
On Wed, Feb 24, 2010 at 4:49 AM, Robert Haas wrote:
> Maybe. But for sure, if it doesn't, and instead tells the user to
> issue pg_stop_backup(), then pg_stop_backup() had better WORK when the
> user tries to execute it. I gather that the problem is that it has to
> finish all that outstanding a
On Tue, Feb 23, 2010 at 3:09 PM, Joshua D. Drake wrote:
> On Tue, 2010-02-23 at 14:49 -0500, Robert Haas wrote:
>
>> > If I issue a shutdown, PostgreSQL should do whatever it needs to do to
>> > shutdown; including issuing a pg_stop_backup.
>>
>> Maybe. But for sure, if it doesn't, and instead te
On Tue, 2010-02-23 at 14:49 -0500, Robert Haas wrote:
> > If I issue a shutdown, PostgreSQL should do whatever it needs to do to
> > shutdown; including issuing a pg_stop_backup.
>
> Maybe. But for sure, if it doesn't, and instead tells the user to
> issue pg_stop_backup(), then pg_stop_backup()
On Tue, Feb 23, 2010 at 12:52 PM, Joshua D. Drake
wrote:
> On Tue, 2010-02-23 at 09:45 -0800, Josh Berkus wrote:
>> Simon, Fujii, All:
>>
>> While demoing HS/SR at SCALE, I ran into a problem which is likely to be
>> a commonly encountered bug when people first setup HS/SR. Here's the
>> sequenc
On Tue, 2010-02-23 at 11:24 -0800, Joshua D. Drake wrote:
> This will bite us if we release like this.
No it won't. The current behaviour was put there by user request a few
releases back. This isn't a 9.0 issue, and as I've said its addressing
something that we now longer see as mainstream going
On Tue, Feb 23, 2010 at 06:58:22PM +, Simon Riggs wrote:
> On Tue, 2010-02-23 at 09:45 -0800, Josh Berkus wrote:
>
> > 1) Set up a brand new master with an archive-commmand and
> > archive=on.
> >
> > 2) Start the master
> >
> > 3) Do a pg_start_backup()
> >
> > 4) Realize, based on log err
Simon Riggs wrote:
> The correct resolution is to put in an archive_command that works.
One really should ensure that WAL files (or should I now say data?
;-) are flowing before issuing running the pg_start_backup()
function. The documentation has always been pretty explicit about
that:
htt
On Tue, 2010-02-23 at 18:58 +, Simon Riggs wrote:
> On Tue, 2010-02-23 at 09:45 -0800, Josh Berkus wrote:
> > This issue is 100% reproduceable.
>
> IMHO there in no problem in that behaviour. If somebody requests a
> backup then we should wait for it to complete. Kevin's suggestion of
> pg_fa
On Tue, 2010-02-23 at 09:45 -0800, Josh Berkus wrote:
> 1) Set up a brand new master with an archive-commmand and archive=on.
>
> 2) Start the master
>
> 3) Do a pg_start_backup()
>
> 4) Realize, based on log error messages, that I've misconfigured the
> archive_command.
> 5) Attempt to shut d
"Joshua D. Drake" wrote:
> If I issue a shutdown, PostgreSQL should do whatever it needs to
> do to shutdown; including issuing a pg_stop_backup.
Should we have a pg_fail_backup function, so that it doesn't put out
a file which suggests that we have a complete backup?
-Kevin
--
Sent via p
On Tue, 2010-02-23 at 09:45 -0800, Josh Berkus wrote:
> Simon, Fujii, All:
>
> While demoing HS/SR at SCALE, I ran into a problem which is likely to be
> a commonly encountered bug when people first setup HS/SR. Here's the
> sequence:
>
> 1) Set up a brand new master with an archive-commmand and
> This issue is 100% reproduceable.
Oh, btw, this is on Alpha4.
--Josh Berkus
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Simon, Fujii, All:
While demoing HS/SR at SCALE, I ran into a problem which is likely to be
a commonly encountered bug when people first setup HS/SR. Here's the
sequence:
1) Set up a brand new master with an archive-commmand and archive=on.
2) Start the master
3) Do a pg_start_backup()
4) Rea
82 matches
Mail list logo
