Simon Riggs wrote:
Taking snapshots from primary has a few disadvantages
...
* snapshots on primary prevent row removal (but this was also an
advantage of this technique!)
That makes it an awful solution for high availability. A backend hung in
transaction-in-progress state in
Hackers,
At this point, almost all patches have been assigned to reviewers. If
you submitted a patch and don't get feedback by Saturday, take a look at
who's in the "reviewers" column and send them a query. Since I have no
way to track when patches are assigned to reviewers, I have no idea i
On Wed, Sep 10, 2008 at 9:22 PM, Tom Lane <[EMAIL PROTECTED]> wrote:
>
> On reflection I'm not even sure that this is strictly an autovacuum
> bug. It can be cast more generically as "RecentGlobalXmin getting
> used without ever having been set", and it sure looks to me like the
> HOT patch may h
(1) In the spec, an interval value or literal must be either
year-month or day-time. (I guess they didn't want to try to deal with
the sticky issues of what it means to have an interval of, for
example, seven months and three days -- since an interval has no sense
of which seven months.)
Not
Tom Lane wrote:
> BTW, I did a quick look at all the uses of RecentGlobalXmin in the back
> branches, and I think we might be all right before 8.2. The older
> branches do in fact init RecentGlobalXmin to InvalidTransactionId,
> and the only thing they use it for is "is this tuple dead to everyon
"Kevin Grittner" <[EMAIL PROTECTED]> writes:
> Tom Lane <[EMAIL PROTECTED]> wrote:
>> I am not sure about some of the corner cases --- anyone want to see if
>> their understanding of the spec for is different?
> The patch seems to support extensions to the standard.
Right. All of these were e
>>> "Kevin Grittner" <[EMAIL PROTECTED]> wrote:
> (4) I'm not 100% sure on this one, but it seemed to me that they
were
> requiring year to be four digits and other components (except for
> fractional seconds) to be two digits.
That can't be right. Maybe I saw that in datetime literal specs.
Transaction snapshots is probably the most difficult problem for Hot
Standby to resolve. We *must* have a transaction snapshot to access
table data in a consistent way, or we must accept some inconsistency,
though that is not simple either.
We can derive transaction snapshots
* remotely fro
>>> Tom Lane <[EMAIL PROTECTED]> wrote:
> The patch includes regression test changes that illustrate what it
does.
> I am not sure about some of the corner cases --- anyone want to see
if
> their understanding of the spec for is different?
The patch seems to support extensions to the standard
I wrote:
> Yeah, this looks like exactly what I had in mind for HEAD. I'm not too
> sure about the back branches though. I think we could apply all of it
> to 8.3, but further back is going to require a separate investigation
> for each branch. Will you take that on?
BTW, I did a quick look at
Tom Lane wrote:
> Alvaro Herrera <[EMAIL PROTECTED]> writes:
> > Tom Lane wrote:
> >> Anyway I think we are on the same page about the rest of the issues.
> >> Did you want to work on fixing them, or shall I?
>
> > Is this more or less what you had in mind?
>
> Yeah, this looks like exactly what
Alvaro Herrera <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> Anyway I think we are on the same page about the rest of the issues.
>> Did you want to work on fixing them, or shall I?
> Is this more or less what you had in mind?
Yeah, this looks like exactly what I had in mind for HEAD. I'm not
On Wed, 2008-09-10 at 16:11 -0400, Tom Lane wrote:
> If there were a serious performance argument against that, then yeah,
> but I don't see one.
Maybe! Just finishing other patch then I'll be starting Hot Standby
discussions, so we'll see.
--
Simon Riggs www.2ndQuadrant.com
Postgre
On Tue, 2008-09-09 at 11:21 -0700, Lawrence, Ramon wrote:
> Our research group has been using the PostgreSQL code base to test new
> join algorithms.
Sounds cool. I'm sure you'll come up with some good things.
You might be interested in this also
http://archives.postgresql.org/pgsql-hackers/200
Tom Lane wrote:
> Sorry, I got a bit confused there. The vacuum's intentional pruning
> will use its own OldestXmin variable, which is known current at the
> start of the vacuum (and I think there were even proposals to advance
> it more frequently than that). However, a vacuum could require som
Tom Lane wrote:
Tom Raney <[EMAIL PROTECTED]> writes:
Why does the index scan for tenk1 include a path key from
onek.unique2? Is it implying an equivalence there?
bench=# explain select * from tenk1 JOIN onek ON
tenk1.unique2=onek.unique2;
Yes, for an example like that the planner knows th
Simon Riggs <[EMAIL PROTECTED]> writes:
> On Wed, 2008-09-10 at 11:52 -0400, Tom Lane wrote:
>> I'm thinking that maybe an appropriate fix is to insert a
>> GetTransactionSnapshot call at the beginning of InitPostgres'
>> transaction, thus ensuring that every backend has some vaguely sane
>> value
On Wed, 2008-09-10 at 11:52 -0400, Tom Lane wrote:
> I'm thinking that maybe an appropriate fix is to insert a
> GetTransactionSnapshot call at the beginning of InitPostgres'
> transaction, thus ensuring that every backend has some vaguely sane
> value for RecentGlobalXmin before it tries to do a
Heikki Linnakangas napsal(a):
Zdenek Kotala wrote:
Do you still have the iGen setup available? Want to give it a shot?
Not sure if I have it configured, need to check. But I'll try it or I'll ask
Jignesh or Paul if they have a free time. They are real benchmark gurus.
Zden
Tom Raney <[EMAIL PROTECTED]> writes:
> Why does the index scan for tenk1 include a path key from
> onek.unique2? Is it implying an equivalence there?
> bench=# explain select * from tenk1 JOIN onek ON
> tenk1.unique2=onek.unique2;
Yes, for an example like that the planner knows that tenk1.uni
> I meant that -- hypothetically if we did accept the feature as-is -- the
> number of evaluations would be documented to be undefined, not N. That
> would avoid the backwards-compatibility problem.
>
> This one point is probably not worth discussing now, because argument
> #1 and #2 stand on their
Tom Lane wrote:
Tom Raney <[EMAIL PROTECTED]> writes:
Why does the planner consider both input variations of each symmetric merge join? The
README says "there is not a lot of difference" between the two options. When
are there any differences?
The righthand side needs to support mark/resto
On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
> My sequence covers several cases :
>
> * There is no missing WAL file.
> * There is a lot of missing WAL file.
This is the likely case for any medium+ sized database.
> * There are missing history files. Failover always generates the gap
Alvaro Herrera wrote:
I didn't know we were doing HOT pruning during vacuum; does
it make sense?
Removing HOT-updated, dead tuples is a bit complicated. It needs to be
done one HOT-chain at a time, and the root line pointer needs to be
updated to the next live tuple in the chain. lazy_scan_he
Alvaro Herrera <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> I'm worried about keeping RecentGlobalXmin up to date during the
>> vacuums, not only at the end, because it will be used for HOT page
>> pruning during the vacuums.
> Oh, I see. I didn't know we were doing HOT pruning during vacuum;
Tom Lane wrote:
> Alvaro Herrera <[EMAIL PROTECTED]> writes:
> > Tom Lane wrote:
> >> Lastly, now that we have the PROC_IN_VACUUM test in GetSnapshotData,
> >> is it actually necessary for lazy vacuum to avoid setting a snapshot?
> >> It seems like it might be a good idea for it to do so in order
Alvaro Herrera <[EMAIL PROTECTED]> writes:
> Tom Lane wrote:
>> I observed a curious bug in autovac just now. ...
> Maybe we should boot RecentGlobalXmin with InvalidOid, and ensure where
> it's going to be used that it's not that.
Good idea --- an Assert at the use sites should be sufficient.
Tom Lane wrote:
> I observed a curious bug in autovac just now. Since plain vacuum avoids
> calling GetTransactionSnapshot, an autovac worker that happens not to
> analyze any tables will never call GetTransactionSnapshot at all.
> This means it will arrive at vac_update_datfrozenxid with
> Recent
On Tue, 2008-09-09 at 09:47 -0400, Robert Haas wrote:
> > 3. "I think this is a "must fix" because of the point about volatile
> > functions --- changing it later will result in user-visible semantics
> > changes, so we have to get it right the first time."
> >
> > I don't entirely agree with #3. I
Zdenek Kotala <[EMAIL PROTECTED]> writes:
> I think it depends of ration of unique integer number in a table and
> numbers of requested interval, number distribution and total number of rows.
> For example if you have 10 distinct number and each has 100 occurrence
> then full scan is better (for
On Tue, Sep 09, 2008 at 10:50:57PM -0400, Robert Haas wrote:
> > When people aren't keeping track of their DDL, that is very
> > strictly a process problem on their end. When people are shooting
> > themselves in the foot, it's a great disservice to market Kevlar
> > shoes to them.
>
> I can't be
I observed a curious bug in autovac just now. Since plain vacuum avoids
calling GetTransactionSnapshot, an autovac worker that happens not to
analyze any tables will never call GetTransactionSnapshot at all.
This means it will arrive at vac_update_datfrozenxid with
RecentGlobalXmin never having be
> a) Verbosely spelling out the units in the default config file
> temp_buffers = 16 megabytes
> or
> temp_buffers = 16 milliblocks :-)
> Naive users who favor cut&paste will use the verbose words
> that should leave little room for confusion. Power-users
> who know the short forms fro
Zdenek Kotala wrote:
Yesterday, I started to reviewing your patch.
Thanks!
1) If I understand correctly the main goal is to improve FSM to cover
all pages in file which is useful for huge database.
That's not a goal per se, though it's true that the new FSM does cover
all pages. The goals
On Wed, Sep 10, 2008 at 10:44:00AM -0400, Mark Mielke wrote:
> >There are 2 killer problems:
> >
> >- decode does not seem to handle architectures that segfault
> > on unaligned int32 accesses.
>
> Out of curiosity - does this problem exist on any platform for which
> PostgreSQL is currently por
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Wed, Sep 10, 2008 at 10:44:00AM -0400, Mark Mielke wrote:
> Marko Kreen wrote:
[...]
>> - decode does not seem to handle architectures that segfault
>> on unaligned int32 accesses.
>
> Out of curiosity - does this problem exist on any platform for
Heikki Linnakangas napsal(a):
Here's an updated FSM patch. Changes since last patch:
Yesterday, I started to reviewing your patch. At the beginning I have
general questions:
1) If I understand correctly the main goal is to improve FSM to cover
all pages in file which is useful for huge dat
Marko Kreen wrote:
(Note: the b64encode there reads 3 chars at a time, b64decode int32
at a time.)
There are 2 killer problems:
- decode does not seem to handle architectures that segfault
on unaligned int32 accesses.
Out of curiosity - does this problem exist on any platform for which
Pos
Robert Haas wrote:
bits...bytes...blocks...m...M
I can't imagine that taking away the "B" is somehow going to
be more clear.
If clarity is the goal, I'd want the following:
a) Verbosely spelling out the units in the default config file
temp_buffers = 16 megabytes
or
temp_buffers = 1
On 9/10/08, Gaetano Mendola <[EMAIL PROTECTED]> wrote:
> I have been experimenting with some base64 encoding/decoding implementation.
>
> I find out that the one at http://code.google.com/p/stringencoders is the
> best
> obtaining a 1.3 speedup vs the postgres one.
>
> Do you think is worth to
Hannu Krosing napsal(a):
On Wed, 2008-09-10 at 07:13 -0400, Robert Haas wrote:
I'm not planner guru but it seems to me that BETWEEN clause could be
rewritten as a IN clause for integer data types and small interval.
Where should the line be drawn.
Define small :)
When the estimated cost is low
Hi,
Fujii Masao wrote:
On Tue, Sep 9, 2008 at 10:55 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
Hi,
ITAGAKI Takahiro wrote:
Signals and locking, borrewed from Postgres-R, are now studied
for the purpose in the log shipping,
Cool. Let me know if you have any questions WRT this imessages stuf
On Tue, Sep 9, 2008 at 10:55 PM, Markus Wanner <[EMAIL PROTECTED]> wrote:
> Hi,
>
> ITAGAKI Takahiro wrote:
>>
>> Signals and locking, borrewed from Postgres-R, are now studied
>> for the purpose in the log shipping,
>
> Cool. Let me know if you have any questions WRT this imessages stuff.
If you'
Hi,
On Tue, 2008-09-09 at 23:14 -0400, Andrew Dunstan wrote:
> Nothing is cost free. Every feature adds to code complexity, and has
> to be maintained.
With full respect to you: I'm only talking about creation time. How much
overhead and complexity are you expecting?
Cheers,
--
Devrim GÜNDÜZ, R
On Wed, 2008-09-10 at 09:36 -0400, Aidan Van Dyk wrote:
> * Simon Riggs <[EMAIL PROTECTED]> [080910 06:18]:
>
> > We have a number of choices, at the point of failure:
> > * Does the whole primary server stay up (probably)?
>
> The only sane choice is the one the admin makes. Any "predetermined
Hannu Krosing wrote:
> Timestamps should rather be considered a forensic tool.
>
> You may have the best VCS system, but if somebody bypasses it, you may
> still need to find out, when it was done.
So you're arguing for modification time, which is not was Devrim is
proposing -- he's proposing cr
* Simon Riggs <[EMAIL PROTECTED]> [080910 06:18]:
> We have a number of choices, at the point of failure:
> * Does the whole primary server stay up (probably)?
The only sane choice is the one the admin makes. Any "predetermined" choice
PG makes can (and will) be wrong in some situations.
> * Do
On Wed, 2008-09-10 at 07:13 -0400, Robert Haas wrote:
> >> I'm not planner guru but it seems to me that BETWEEN clause could be
> >> rewritten as a IN clause for integer data types and small interval.
> >
> > Where should the line be drawn.
> > Define small :)
>
> When the estimated cost is lower?
Hi,
I have been experimenting with some base64 encoding/decoding implementation.
I find out that the one at http://code.google.com/p/stringencoders is the best
obtaining a 1.3 speedup vs the postgres one.
Do you think is worth to submit a patch that replaces the postgres base64
implementation
wi
>> I'm not planner guru but it seems to me that BETWEEN clause could be
>> rewritten as a IN clause for integer data types and small interval.
>
> Where should the line be drawn.
> Define small :)
When the estimated cost is lower?
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@
On Wed, Sep 10, 2008 at 1:39 PM, Zdenek Kotala <[EMAIL PROTECTED]>wrote:
> I has played with new hash index implementation and I tried following
> command:
>
> postgres=# select * from test where id between 1 and 5;
> Time: 9651,033 ms
> postgres=# explain select * from test where id between 1 and
I has played with new hash index implementation and I tried following
command:
postgres=# select * from test where id between 1 and 5;
Time: 9651,033 ms
postgres=# explain select * from test where id between 1 and 5;
QUERY PLAN
-
On Wed, Sep 10, 2008 at 12:51:02PM +0300, Heikki Linnakangas wrote:
> >Since the set of collations isn't exactly denumerable, we need some way
> >to allow the user to specify the collation they want. The only
> >collation PostgreSQL knows about is the C collation. Anything else is
> >user-defined.
On Wed, 2008-09-10 at 09:35 +0300, Hannu Krosing wrote:
> On Wed, 2008-09-10 at 15:15 +0900, Fujii Masao wrote:
> > On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas
> > <[EMAIL PROTECTED]> wrote:
> > > If a slave falls behind, how does it catch up? I guess you're saying that
> > > it
> > > ca
On Wed, 2008-09-10 at 11:07 +0200, Dimitri Fontaine wrote:
> Hi,
>
> Le mercredi 10 septembre 2008, Heikki Linnakangas a écrit :
> > Sure. That's the fundamental problem with synchronous replication.
> > That's why many people choose asynchronous replication instead. Clearly
> > at some point you
Heikki Linnakangas napsal(a):
Design & functionality changes left:
- move retrieveing collation from pg_database to pg_type
The problem there is that pg_collation is local catalog, but pg_database
is global catalog. IIRC, It was discussed during last commitfest. I
think it is bad idea to m
Martijn van Oosterhout wrote:
On Wed, Sep 10, 2008 at 11:29:14AM +0300, Heikki Linnakangas wrote:
Radek Strnad wrote:
- because of pg_collation and pg_charset are catalogs individual for each
database, if you want to create a database with collation other than
specified, create it in template1
Hi,
Simon Riggs wrote:
The standby server won't come up until you have:
* copied the base backup
* sent it to standby server
* bring up standby, have it realise it is a replication partner and
begin requesting WAL from primary (in some way)
Right. That was your assumption as well. Required bef
On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
> Umm.. I disagree with you ;)
That's no problem and I respect your knowledge.
If we disagree, it is very likely because we have misunderstood each
other. Much has been written, so I will wait for it to all be read and
understood by you and
Hi,
Le mercredi 10 septembre 2008, Heikki Linnakangas a écrit :
> Sure. That's the fundamental problem with synchronous replication.
> That's why many people choose asynchronous replication instead. Clearly
> at some point you'll want to give up and continue without the slave, or
> kill the master
On Wed, 2008-09-10 at 17:57 +0900, Fujii Masao wrote:
> I cannot accept that WAL sender is blocked for initial setup.
Yes, very important point. We definitely agree on that. The primary must
be able to continue working while all this setup is happening. No tables
are locked, all commits are
On Wed, 2008-09-10 at 10:06 +0200, Markus Wanner wrote:
> Hi,
>
> Simon Riggs wrote:
> > 1. Standby contacts primary and says it would like to catch up, but is
> > currently at point X (which is a point at, or after the first consistent
> > stopping point in WAL after standby has performed its ow
On Wed, Sep 10, 2008 at 4:15 PM, Simon Riggs <[EMAIL PROTECTED]> wrote:
>
> On Wed, 2008-09-10 at 09:35 +0300, Hannu Krosing wrote:
>> On Wed, 2008-09-10 at 15:15 +0900, Fujii Masao wrote:
>> > On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas
>> > <[EMAIL PROTECTED]> wrote:
>> > > If a slave fa
On Wed, 2008-09-10 at 08:15 +0100, Simon Riggs wrote:
> Any working solution needs to work for all required phases. If you did
> it this way, you'd never catch up at all.
>
> When you first make the copy, it will be made at time X. The point of
> consistency will be sometime later and requires WA
On Wed, 2008-09-10 at 10:06 +0200, Markus Wanner wrote:
> Hi,
>
> Simon Riggs wrote:
> > 1. Standby contacts primary and says it would like to catch up, but is
> > currently at point X (which is a point at, or after the first consistent
> > stopping point in WAL after standby has performed its own
On Wed, 2008-09-10 at 09:27 +0300, Volkan YAZICI wrote:
> On Tue, 9 Sep 2008, David Fetter <[EMAIL PROTECTED]> writes:
> >> AFAICS, PostgreSQL is not keeping info about when a table, database,
> >> sequence, etc was created. We cannot get that info even from OS,
> >> since CLUSTER or VACUUM FULL m
On Wed, Sep 10, 2008 at 11:29:14AM +0300, Heikki Linnakangas wrote:
> Radek Strnad wrote:
> >- because of pg_collation and pg_charset are catalogs individual for each
> >database, if you want to create a database with collation other than
> >specified, create it in template1 and then create databas
On Tue, 2008-09-09 at 20:59 +0200, Zeugswetter Andreas OSB sIT wrote:
> All in all a useful streamer seems like a lot of work.
I mentioned some time ago an alternative idea of having the slave
connect through a normal SQL connection and call a function which
streams the WAL file from the point req
Simon Riggs wrote:
Saying "I want to wait for a synchronous commit and I am willing to wait
for ever to ensure it" leads to long hangs in some cases.
Sure. That's the fundamental problem with synchronous replication.
That's why many people choose asynchronous replication instead. Clearly
at s
Radek Strnad wrote:
Progress so far:
- created catalogs pg_collation a pg_charset which are filled with three
standard collations
- initdb changes rows called "DEFAULT" in both catalogs during the bki
bootstrap phase with current system LC_COLLATE and LC_CTYPE or those set by
command line.
- new
On Wed, 2008-09-10 at 11:10 +0300, Heikki Linnakangas wrote:
> Simon Riggs wrote:
> > On Wed, 2008-09-10 at 13:28 +0900, Fujii Masao wrote:
> >> On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
> >> <[EMAIL PROTECTED]> wrote:
> >>> There's one thing I haven't figured out in this discussion. Does
On Wed, Sep 10, 2008 at 1:40 PM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
>
>
> The thing that bothers me is the behavior when the synchronous slave doesn't
> respond. A timeout has been discussed, after which the master just gives up
> on sending, and starts acting as if there's no slave. How
Simon Riggs wrote:
On Wed, 2008-09-10 at 13:28 +0900, Fujii Masao wrote:
On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
<[EMAIL PROTECTED]> wrote:
There's one thing I haven't figured out in this discussion. Does the write
to the disk happen before or after the write to the slave? Can you gu
Hi,
Simon Riggs wrote:
1. Standby contacts primary and says it would like to catch up, but is
currently at point X (which is a point at, or after the first consistent
stopping point in WAL after standby has performed its own crash
recovery, if any was required).
2. primary initiates data transfe
On Wed, 2008-09-10 at 12:24 +0530, Pavan Deolasee wrote:
> Also we may not want the master to be stuck while slave is doing the catchup.
No, since it may take hours, not seconds.
--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support
--
Sent via pgsql-hacker
Hi,
Devrim GÜNDÜZ wrote:
On Tue, 2008-09-09 at 15:36 -0400, Tom Lane wrote:
why would creation time (as opposed to any other time, eg last schema
modification, last data modification, yadda yadda) be especially
significant?
Hmm, those would be cool, too.
maybe except last data modification.
On Wed, 2008-09-10 at 13:28 +0900, Fujii Masao wrote:
> On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas
> <[EMAIL PROTECTED]> wrote:
> > There's one thing I haven't figured out in this discussion. Does the write
> > to the disk happen before or after the write to the slave? Can you guarantee
>
77 matches
Mail list logo
