"Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> If fsync may return before data *actually* flushed then you may have
> unlogged data page changes which breakes WAL rule and means corrupted
> (inconsistent) database without ANY ABILITY TO RECOVERY TO CONSISTENT
> STATE. Now please explain me how sa
Was the following bug already fixed ?
Regards,
Hiroshi Inoue
Tom Lane wrote:
>
> With current sources:
>
> DEBUG: copy: line 629980, XLogWrite: new log file created - try to increase
>WAL_FILES
> DEBUG: copy: line 694890, XLogWrite: new log file created - try to increase
>WAL_FILES
> FATAL
> I wouldn't be at all surprised if you found a better approach - my
> configuration above, to my mind at least, is not pretty. I hope you do find
> a better approach - I know I'll be peeking at your code to see.
Yes, I have an idea and hope it works.
--
Bruce Momjian|
> 1) WAL
> We have buffer manager, ok. So why not to use WAL as part of
> it and don't log INSERT/UPDATE/DELETE xlog records but directly
> changes into buffer pages ? When someone dirties page it has to
> inform bmgr about dirty region and bmgr would formulate xlog record.
> The record could be
> > If there is no pg_control or it's corrupted or points to
> > unexistent/corrupted checkpoint record then scan logs from
> > newest to oldest one till we find last valid checkpoint record
> > or oldest valid log record and than redo from there.
>
> And how well will that approach work if the l
Yes, seems that is best. I will probably hack something up here so I
can do some testing of the app itself.
> Mike Mascari's idea (er... his assembling of the other ideas) still
> sounds like the Best Solution though.
>
> :-)
>
> + Justin
>
> +++
>
> I like the idea of updating shared memory
Mike Mascari's idea (er... his assembling of the other ideas) still
sounds like the Best Solution though.
:-)
+ Justin
+++
I like the idea of updating shared memory with the performance
statistics,
current query execution information, etc., providing a function to fetch
those statistics, and
> At 18:05 7/03/01 -0500, Bruce Momjian wrote:
> >> All in all, I do not see this as an easy task that you can whip out and
> >> then release as a 7.1 patch without extensive testing. And given that,
> >> I'd rather see it done with what I consider the right long-term approach,
> >> rather than a
> I think Bruce wants per-backend data, and this approach would seem to only
> get the data for the current backend.
>
> Also, I really don't like the proposal to write files to /tmp. If we want a
> perf tool, then we need to have something like 'top', which will
> continuously update. With 40 b
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> "two checkpoints" approach is not the best way to follow.
> From my POV, scanning logs is much better - it doesn't require
> doubling size of on-line logs and allows to *restart* if pg_control
> was lost/corrupted:
> If there is no pg_control or it'
> >> * Store two past checkpoint locations, not just one, in pg_control.
> >> On startup, we fall back to the older checkpoint if the newer one
> >> is unreadable. Also, a physical copy of the newest
> >> checkpoint record
>
> > And what to do if older one is unreadable too?
> > (Isn't it like
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> If fsync may return before data *actually* flushed then you may have
> unlogged data page changes which breakes WAL rule and means corrupted
> (inconsistent) database without ANY ABILITY TO RECOVERY TO CONSISTENT
> STATE. Now please explain me how sav
> > But what can be done if fsync returns before pages flushed?
>
> When you write out critical information, you keep earlier versions of
> it. On startup, if the critical information is corrupt, you use the
> earlier versions of it. This helps protect against the scenario I
> mentioned: a few
Tom Lane wrote:
>
> Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> > Tom Lane wrote:
> >> Why not? How is this a critical parameter (more critical than, say,
> >> fsync enable)?
>
> > Does it have any meaning other than testing ? IMHO recovery system
> > doesn't allow any optimism and archdir is al
At 19:59 7/03/01 -0500, Mike Mascari wrote:
>I like the idea of updating shared memory with the performance statistics,
>current query execution information, etc., providing a function to fetch
>those statistics, and perhaps providing a system view (i.e. pg_performance)
>based upon such functio
I like the idea of updating shared memory with the performance statistics,
current query execution information, etc., providing a function to fetch
those statistics, and perhaps providing a system view (i.e. pg_performance)
based upon such functions which can be queried by the administrator.
F
Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> Why not? How is this a critical parameter (more critical than, say,
>> fsync enable)?
> I don't think 'fsync enable' is a critical parameter.
> It's a dangerous parameter and it's not appropriate
> as a GUC paramter either.
That's
Hi:
When trying to pg_dump a database as the postgres user or as any other user I got this
error:
dumpProcLangs(): handler procedure for language plpgsql not found
what's going on ?
the command to dump this is: pg_dump -x alyma > alyma.dat.sql
Thank you in advanced.
--
Ing. Luis Magaña.
G
Tom Lane wrote:
>
> Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> >> But what possible reason is there for keeping it in pg_control?
> >> AFAICS that would just mean that we'd need special code for setting it,
> >> instead of making use of all of Peter's hard work on GUC.
>
> > I don't think it's a
At 11:33 8/03/01 +1100, Justin Clift wrote:
>Hi all,
>
>Wouldn't another approach be to write a C function that does the
>necessary work, then just call it like any other C function?
>
>i.e. Connect to the database and issue a "select
>perf_stats('/tmp/stats-2001-03-08-01.txt')" ?
>
I think Bruc
Hi all,
Wouldn't another approach be to write a C function that does the
necessary work, then just call it like any other C function?
i.e. Connect to the database and issue a "select
perf_stats('/tmp/stats-2001-03-08-01.txt')" ?
Or similar?
Sure, that means another database connection which w
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> > > I feel that the fact that
> > >
> > > WAL can't help in the event of disk errors
> > >
> > > is often overlooked.
> >
> > This is true in general. But, nevertheless, WAL can be written to
> > protect against predictable disk errors, when poss
Hiroshi Inoue <[EMAIL PROTECTED]> writes:
>> But what possible reason is there for keeping it in pg_control?
>> AFAICS that would just mean that we'd need special code for setting it,
>> instead of making use of all of Peter's hard work on GUC.
> I don't think it's appropriate to edit archdir by
At 18:05 7/03/01 -0500, Bruce Momjian wrote:
>> All in all, I do not see this as an easy task that you can whip out and
>> then release as a 7.1 patch without extensive testing. And given that,
>> I'd rather see it done with what I consider the right long-term approach,
>> rather than a dead-end
Philip Warner wrote:
>At 20:48 7/03/01 +, Oliver Elphick wrote:
>>kovacsz wrote:
>> >
>> >You answered that noone experienced anything like this. Here I get this
>> >behaviour with the most simple table as well.
>>
>
>Is there a problem with the lists? I reveived Zoltan's me
Tom Lane wrote:
>
> "Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> > So, it's better to leave archdir in pg_control now - if we'll
> > decide that GUC is better place then we'll just ignore archdir
> > in pg_control. But if it will be better to have it in pg_control
> > then we'll not be able to a
> All in all, I do not see this as an easy task that you can whip out and
> then release as a 7.1 patch without extensive testing. And given that,
> I'd rather see it done with what I consider the right long-term approach,
> rather than a dead-end hack. I think doing it in a signal handler is
>
At 20:48 7/03/01 +, Oliver Elphick wrote:
>kovacsz wrote:
> >
> >You answered that noone experienced anything like this. Here I get this
> >behaviour with the most simple table as well.
>
Is there a problem with the lists? I reveived Zoltan's message twice, and
now this one that seems to i
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > How do people feel about adding a single handler to 7.1? Is it
> > something I can slip into the current CVS, or will it have to exist as a
> > patch to 7.1. Seems it would be pretty isolated unless someone sends
> > the signal, but it is clearly a
Bruce Momjian <[EMAIL PROTECTED]> writes:
> How do people feel about adding a single handler to 7.1? Is it
> something I can slip into the current CVS, or will it have to exist as a
> patch to 7.1. Seems it would be pretty isolated unless someone sends
> the signal, but it is clearly a feature a
"Mikheev, Vadim" <[EMAIL PROTECTED]> writes:
> So, it's better to leave archdir in pg_control now - if we'll
> decide that GUC is better place then we'll just ignore archdir
> in pg_control. But if it will be better to have it in pg_control
> then we'll not be able to add it there.
But what possi
On Wed, Mar 07, 2001 at 12:03:41PM -0800, Mikheev, Vadim wrote:
> Ian wrote:
> > > I feel that the fact that
> > >
> > > WAL can't help in the event of disk errors
> > >
> > > is often overlooked.
> >
> > This is true in general. But, nevertheless, WAL can be written to
> > protect against pre
The Hermit Hacker <[EMAIL PROTECTED]> writes:
>> How do people feel about adding a single handler to 7.1?
> Totally dead set against it ...
Ditto. Particularly a signal handler that performs I/O. That's going
to create all sorts of re-entrancy problems.
regards, tom la
> > How do people feel about adding a single handler to 7.1? Is it
> > something I can slip into the current CVS, or will it have to exist as a
> > patch to 7.1. Seems it would be pretty isolated unless someone sends
> > the signal, but it is clearly a feature addition.
>
> Totally dead set aga
> It is possible to build a logging system so that you mostly don't care
> when the data blocks get written; a particular data block on disk is
> considered garbage until the next checkpoint, so that you
How to know if a particular data page was modified if there is no
log record for that modif
On Wed, Mar 07, 2001 at 11:21:37AM -0500, Tom Lane wrote:
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > The only problem is that we would no longer have control over which
> > pages made it to disk. The OS would perhaps write pages as we modified
> > them. Not sure how important that is.
>
>
On Wed, Mar 07, 2001 at 11:09:25AM -0500, Tom Lane wrote:
> "Vadim Mikheev" <[EMAIL PROTECTED]> writes:
> >> * Store two past checkpoint locations, not just one, in pg_control.
> >> On startup, we fall back to the older checkpoint if the newer one
> >> is unreadable. Also, a physical copy of the
On Wed, 7 Mar 2001, Mikheev, Vadim wrote:
> But what can be done if fsync returns before pages flushed?
No, it won't. When fsync returns, data is promised by the OS to be on
disk.
-alex
---(end of broadcast)---
TIP 6: Have you searched our list a
> > But what can be done if fsync returns before pages flushed?
> No, it won't. When fsync returns, data is promised by the OS to be on
> disk.
Seems you didn't follow discussions about this issue.
Vadim
---(end of broadcast)---
TIP 3: if posting/
kovacsz wrote:
>I reported this problem about 3 weeks ago or even more. The problem hasn't
>disappeared yet. In 7.1beta4 if I use pg_dump with -a switch together, I
>get each CREATE SEQUENCE twice. I suspected if this is an installation
>problem at my place but now I think it maybe isn't.
> > > I have just sent to the pgsql-patches list a rather large set of
> >
> > Please send it to me directly - pgsql-patches' archieve is
> dated by Feb -:(
>
> Huh?
>
> http://www.postgresql.org/mhonarc/pgsql-patches/2001-03/index.html
But there was nothing there yesterday -:)
(Seems archive
> > I feel that the fact that
> >
> > WAL can't help in the event of disk errors
> >
> > is often overlooked.
>
> This is true in general. But, nevertheless, WAL can be written to
> protect against predictable disk errors, when possible. Failing to
> write a couple of disk blocks when the sys
> > > What I've thought is to implement a new command to
> > > change archdir under WAL's control.
> > > If it's different from Vadim's plan I don't object.
> >
> > Actually, I have no concrete plans for archdir yet - this
> > one is for WAL based BAR we should discuss in future.
> > So, I don't
On Wed, 7 Mar 2001, Bruce Momjian wrote:
> I have started coding a PostgreSQL performance monitor. It will be like
> top, but allow you to click on a backend to see additional information.
>
> It will be written in Tcl/Tk. I may ask to add something to 7.1 so when
> a backend receives a special
I have started coding a PostgreSQL performance monitor. It will be like
top, but allow you to click on a backend to see additional information.
It will be written in Tcl/Tk. I may ask to add something to 7.1 so when
a backend receives a special signal, it dumps a file in /tmp with some
backend
I reported this problem about 3 weeks ago or even more. The problem hasn't
disappeared yet. In 7.1beta4 if I use pg_dump with -a switch together, I
get each CREATE SEQUENCE twice. I suspected if this is an installation
problem at my place but now I think it maybe isn't.
You answered that noone ex
"Vadim Mikheev" <[EMAIL PROTECTED]> writes:
> I feel that the fact that
>
> WAL can't help in the event of disk errors
>
> is often overlooked.
This is true in general. But, nevertheless, WAL can be written to
protect against predictable disk errors, when possible. Failing to
write a couple
On Wed, 7 Mar 2001, Vadim Mikheev wrote:
> > I have just sent to the pgsql-patches list a rather large set of
>
> Please send it to me directly - pgsql-patches' archieve is dated by Feb -:(
Huh?
http://www.postgresql.org/mhonarc/pgsql-patches/2001-03/index.html
---(en
Tom Lane writes:
> Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> > Do I have to send SIGHUP after changing postgresql.conf ?
>
> In general, yes. I think that for the (still vaporware) archdir option,
> you might not need to: archdir will only be looked at by the checkpoint
> subprocess, and I thi
> Bruce Momjian <[EMAIL PROTECTED]> writes:
> > The only problem is that we would no longer have control over which
> > pages made it to disk. The OS would perhaps write pages as we modified
> > them. Not sure how important that is.
>
> Unfortunately, this alone is a *fatal* objection. See nea
Bruce Momjian <[EMAIL PROTECTED]> writes:
> The only problem is that we would no longer have control over which
> pages made it to disk. The OS would perhaps write pages as we modified
> them. Not sure how important that is.
Unfortunately, this alone is a *fatal* objection. See nearby
discussi
Peter Eisentraut writes:
> Michal Maru¹ka writes:
>
> > What about (optionally) printing the type of the column data?
>
> >io | tu | tipo |data
> >int | int | int2 |date
> > +---+--+
> > 102242 | 26404 | 1203 | 2000-11-22
> > (1 ro
"Vadim Mikheev" <[EMAIL PROTECTED]> writes:
>> I have just sent to the pgsql-patches list a rather large set of
> Please send it to me directly - pgsql-patches' archieve is dated by Feb -:(
Done under separate cover.
>> proposed diffs for the WAL code. These changes:
>>
>> * Store two past che
"Vadim Mikheev" <[EMAIL PROTECTED]> writes:
> So, I don't see why to remove archdir from pg_control now.
I didn't like the space consumption. I think it's important that the
pg_control data fit in less than 512 bytes so that it doesn't cross
physical sectors on the disk. This reduces the odds o
Zeugswetter Andreas SB <[EMAIL PROTECTED]> writes:
> I really think that xlog entries should be done by a layer below the
> userland functions.
That seems somewhere between impractical and impossible: how will you
tie the functional xlog entries ("insert foo into index bar") to the
resulting pag
"Vadim Mikheev" <[EMAIL PROTECTED]> writes:
> I've asked if anyone can send crc64 impl to me and got only one from
> Nathan Myers. Unfortunately, SWISS-PROT impl assumes that long long
> is 8 bytes - is it portable?
No, it's not. I have written an implementation that uses uint64 if
available (pe
> This was brought up a week ago, and I consider it an interesting idea.
> The only problem is that we would no longer have control over which
> pages made it to disk. The OS would perhaps write pages as we modified
> them. Not sure how important that is.
Yes. As I work on linux kernel I know
On Wed, Mar 07, 2001 at 04:17:09PM +0100, Olivier PRENANT wrote:
> Hi Matthew,
>
> I would love to get this stuff...
>
> Could you send it to me or tell me where it is ?
>
> TIA
> On Tue, 6 Mar 2001, Matthew Hagerty wrote:
>
> > Greetings,
> >
> > I wrote a few simple programs to log Apache a
Hiroshi Inoue <[EMAIL PROTECTED]> writes:
> Do I have to send SIGHUP after changing postgresql.conf ?
In general, yes. I think that for the (still vaporware) archdir option,
you might not need to: archdir will only be looked at by the checkpoint
subprocess, and I think that a newly spawned backe
Michal Maru¹ka writes:
> What about (optionally) printing the type of the column data?
>io | tu | tipo |data
>int | int | int2 |date
> +---+--+
> 102242 | 26404 | 1203 | 2000-11-22
> (1 row)
I've been meaning to implement this for a while. N
Hi Matthew,
I would love to get this stuff...
Could you send it to me or tell me where it is ?
TIA
On Tue, 6 Mar 2001, Matthew Hagerty wrote:
> Greetings,
>
> I wrote a few simple programs to log Apache access_log entries to pg. If
> this is something anyone would be interested in or if the
Tom Lane wrote:
> Christof Petig <[EMAIL PROTECTED]> writes:
> > We noticed that after upgrading to 7.1beta[245] the execution time for
> > some often used queries went up by a factor of 2 or more.
>
> I get the desired plan after doing VACUUM ANALYZE ...
>
> regards, tom
> 2) SHM vs. MMAP
> Why don't use mmap to share pages (instead of shm) ? There would be no
> problem with tuning pg's buffer cache size - it is balanced by OS.
> When using SHM there are often two copies of page: one in OS' page cache
> and one in SHM (vaste of memory).
> When using mmap the data
> > > I thought the intended way to change a GUC parameter permanently was to
> > > edit data/postgresql.conf . No ?
> > >
> >
> > What I've thought is to implement a new command to
> > change archdir under WAL's control.
> > If it's different from Vadim's plan I don't object.
>
> Actually, I h
> > > Hm, wasn't it handling non-atomic disk writes, Andreas?
> >
> > Yes, but for me, that was only one (for me rather minor) issue.
> > I still think that the layout of PostgreSQL pages was designed to
> > reduce the risc of a (heap) page beeing inconsistent because it is
> > only partly writ
Hello,
maybe I missed something, but in last days I was thinking how would I
write my own sql server. I got several ideas and because these are not
used in PG they are probably bad - but I can't figure why.
1) WAL
We have buffer manager, ok. So why not to use WAL as part of it and don't
log INSE
> > Hm, wasn't it handling non-atomic disk writes, Andreas?
>
> Yes, but for me, that was only one (for me rather minor) issue.
> I still think that the layout of PostgreSQL pages was designed to
> reduce the risc of a (heap) page beeing inconsistent because it is
> only partly written to an acc
Is anyone on this list in Hannover for CeBit? Maybe we could arrange a
meeting.
Michael
--
Michael Meskes
[EMAIL PROTECTED]
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!
---(end of broadcast)---
TIP 1: subscribe and unsubscribe
> > In short I do not think that the current implementation of
> > "physical log" does what it was intended to do :-(
>
> Hm, wasn't it handling non-atomic disk writes, Andreas?
Yes, but for me, that was only one (for me rather minor) issue.
I still think that the layout of PostgreSQL pages was
> On third thought --- we could still log the original page contents and
> the modification log record atomically, if what were logged in the xlog
> record were (essentially) the parameters to the operation being logged,
> not its results. That is, make the log entry before you start doing the
> m
Zeugswetter Andreas SB <[EMAIL PROTECTED]> wrote:
>
> In short I do not think that the current implementation of
> "physical log" does what it was intended to do :-(
Hm, wasn't it handling non-atomic disk writes, Andreas?
And for what else "physical log" could be used?
The point was - copy entir
> The point is to make the allocation of XIDs and OIDs work the same way.
> In particular, if we are forced to reset the XLOG using what's stored in
> pg_control, it would be good if what's stored in pg_control is a value
> beyond the last-used XID/OID, not a value less than the last-used ones.
I
Vadim Mikheev wrote:
>
> Nevertheless, subj is rised. BTW, does anybody know results of kill -9
> in Oracle/Informix/etc? Just curious -:)
Progress has no problem with it that I have ever seen.
Regards,
Andrew.
--
> Consider the following scenario:
>
> 1. A new transaction inserts a tuple. The tuple is entered into its
> heap file with the new transaction's XID, and an associated WAL log
> entry is made. Neither one of these are on disk yet --- the heap tuple
> is in a shmem disk buffer, and the WAL entr
> Nevertheless, subj is rised. BTW, does anybody know results of kill -9
> in Oracle/Informix/etc? Just curious -:)
Informix has no problem with it. Oracle dba's fear it, to say the least.
Andreas
---(end of broadcast)---
TIP 6: Have you search
> > > Before commit or rollback the xlog is not flushed to disk, thus you can loose
> > > those xlog entries, but the index page might already be on disk because of
> > > LRU buffer reuse, no ?
> >
> > No. Buffer page is written to disk *only after corresponding records are flushed
> > to log* (W
> I have spent several days now puzzling over the corrupted WAL logfile
> that Scott Parish was kind enough to send me from a 7.1beta4 crash.
> It looks a lot like two different series of transactions were getting
> written into the same logfile. I'd been digging like mad in the WAL
> code to try
> I have just sent to the pgsql-patches list a rather large set of
Please send it to me directly - pgsql-patches' archieve is dated by Feb -:(
> proposed diffs for the WAL code. These changes:
>
> * Store two past checkpoint locations, not just one, in pg_control.
> On startup, we fall back
> > Before commit or rollback the xlog is not flushed to disk, thus you can loose
> > those xlog entries, but the index page might already be on disk because of
> > LRU buffer reuse, no ?
>
> No. Buffer page is written to disk *only after corresponding records are flushed
> to log* (WAL means Wr
> Before commit or rollback the xlog is not flushed to disk, thus you can loose
> those xlog entries, but the index page might already be on disk because of
> LRU buffer reuse, no ?
No. Buffer page is written to disk *only after corresponding records are flushed
to log* (WAL means Write-Ahead-Log
> > I do not however see how the current solution fixes the original problem,
> > that we don't have a rollback for index modifications.
> > The index would potentially point to an empty heaptuple slot.
>
> How? There will be an XLOG entry inserting the heap tuple before the
> XLOG entry that u
> > I thought the intended way to change a GUC parameter permanently was to
> > edit data/postgresql.conf . No ?
> >
>
> What I've thought is to implement a new command to
> change archdir under WAL's control.
> If it's different from Vadim's plan I don't object.
Actually, I have no concrete pla
82 matches
Mail list logo
