[HACKERS] hot standby on mingw

2008-12-24 Thread Jaime Casanova
Hi, i'm getting this error when triyng to build the hot standby (v5 from 20081218 obtained in the wiki) on mingw make[4]: Entering directory `/home/Administrador/pgsql/src/backend/access/heap' gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-str

Re: [HACKERS] Window-functions patch handling of aggregates

2008-12-24 Thread Hitoshi Harada
2008/12/25 Tom Lane : > Gregory Stark writes: >> Tom Lane writes: >>> Unless we want to move the goalposts on what an aggregate is allowed >>> to do internally, we're going to have to change this to re-aggregate >>> repeatedly. Neither prospect is appetizing in the least. > >> Does it currently

Re: [HACKERS] [idea] a copied relkind in pg_attribute

2008-12-24 Thread KaiGai Kohei
Jaime Casanova wrote: On Wed, Dec 24, 2008 at 6:50 AM, KaiGai Kohei wrote: The current implementation need to lookup RELOID system cache to identify the relkind of the relation, because pg_attribtue does not have any information about "relkind". However, I also think it is not an ideal implemen

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Robert Haas
On Dec 24, 2008, at 6:46 PM, Ron Mayer wrote: Robert Haas wrote: ... serializable transaction ... If we were to construct a database that had one giant lock for the entire database so that only a single query could execute at one time, transactions would be serializable (because they'd i

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Ron Mayer
Robert Haas wrote: ... serializable transaction ... If we were to construct a database that had one giant lock for the entire database so that only a single query could execute at one time, transactions would be serializable (because they'd in fact be serialized). However, performance would suc

Re: [HACKERS] parallel restore vs. windows

2008-12-24 Thread Andrew Dunstan
Jaime Casanova wrote: On Sun, Dec 14, 2008 at 12:13 PM, Andrew Dunstan wrote: This version completes properly on Windows with the regression database. actually, this one doesn't apply cleanly on head I will have a new patch a day or two after Christmas, which I hope will be v

Re: [HACKERS] parallel restore vs. windows

2008-12-24 Thread Jaime Casanova
On Sun, Dec 14, 2008 at 12:13 PM, Andrew Dunstan wrote: > > This version completes properly on Windows with the regression database. > actually, this one doesn't apply cleanly on head -- Atentamente, Jaime Casanova Soporte y capacitación de PostgreSQL Asesoría y desarrollo de sistemas Guayaquil

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Emmanuel Cecchet
Hi Kevin, The link didn't seem to work for me, but I think I found the article you meant: "Serializable Isolation for Snapshot Databases" by Michael J. Cahill, et al An interesting work. If nothing else, it will help flesh out the documentation of anomalies. If the PostgreSQL community ever

Re: [HACKERS] DTrace probes patch

2008-12-24 Thread Bruce Momjian
Thanks, applied. --- Robert Lor wrote: > Tom Lane wrote: > > Robert Lor writes: > > > >> Tom Lane wrote: > >> > >>> I agree. If the probe is meant to track only *some* WAL writes > >>> then it needs to be named som

[HACKERS] merry christmas folks

2008-12-24 Thread Grzegorz Jaskiewicz
WITH RECURSIVE tree(b, l, lv) AS ( ( WITH RECURSIVE t(b, l) AS ( select b/11.6, 0 AS l from (select generate_series(0,30)::float8 b union all select generate_series(30,0, -1)::float8 b) ziew UNION ALL select (b*1.06), l+1 FROM t WHERE l < 3 ) select b, l,

Re: [HACKERS] incoherent view of serializable transactions

2008-12-24 Thread Kevin Grittner
>>> Emmanuel Cecchet 12/22/08 11:42 PM >>> > If you want to know how to build SERIALIZABLE with a database that > provides SI (Snapshot Isolation), read > http://portal.acm.org/citation.cfm?doid=1376616.137669 The link didn't seem to work for me, but I think I found the article you meant: "Se

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Pavan Deolasee
On Wed, Dec 24, 2008 at 7:18 PM, Simon Riggs wrote: > > > > With respect, I was hoping you might look in the patch and see if you > agree with the way it is handled. No need to remember. The whole > latestRemovedXid concept is designed to do help. > Well, that's common for all cleanup record incl

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-24 Thread Fujii Masao
Hi, I fixed some bugs. On Thu, Dec 25, 2008 at 12:31 AM, Simon Riggs wrote: > > Can we change to IMMEDIATE when it we need the checkpoint? Perhaps yes, though current patch doesn't care about it. I'm not sure if we really need the feature. Yes, as you say, I'd like to also listen to everybody e

Re: [HACKERS] uuids on freebsd

2008-12-24 Thread David Lee Lambert
On Dec 17, 2:30 pm, Andrew Gierth wrote: > Has anyone ever managed to get uuid generation working on FreeBSD? [...] > > ([...] The only solution I could come up with was to knock > off a quick uuid-freebsd module that uses the base system uuid > functions rather than the ossp ones. I could put thi

Re: [HACKERS] Window-functions patch handling of aggregates

2008-12-24 Thread Tom Lane
Gregory Stark writes: > Tom Lane writes: >> Unless we want to move the goalposts on what an aggregate is allowed >> to do internally, we're going to have to change this to re-aggregate >> repeatedly. Neither prospect is appetizing in the least. > Does it currently copy the state datum before ca

Re: [HACKERS] generic reloptions improvement

2008-12-24 Thread Tom Lane
Alvaro Herrera writes: > Peter Eisentraut wrote: >> I'm not sure how important this is, but if you are enumerating the access >> methods (RELOPT_KIND_BTREE, etc.), how will this work with user-defined >> access methods? > It is important. > I'm intending to have a new routine which would reser

Re: [HACKERS] [idea] a copied relkind in pg_attribute

2008-12-24 Thread Jaime Casanova
On Wed, Dec 24, 2008 at 6:50 AM, KaiGai Kohei wrote: > > The current implementation need to lookup RELOID system cache to > identify the relkind of the relation, because pg_attribtue does > not have any information about "relkind". However, I also think > it is not an ideal implementation, even if

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 09:59 -0500, Robert Treat wrote: > I think the uncertainty comes from peoples experience with typical > replication > use cases vs a lack of experience with this current implementation. Quite possibly. Publishing user feedback on this will be very important in making t

Re: [Fwd: Re: [HACKERS] Transactions and temp tables]

2008-12-24 Thread Emmanuel Cecchet
Hi Heikki, The point of using temp tables was performance. Using regular tables in our case would hurt performance too much. Well if we cannot get a temporary fix in 8.4, we will maintain a separate patch to get that functionality just for temp tables that are created and dropped in the same

Re: [HACKERS] Sync Rep: Second thoughts

2008-12-24 Thread Emmanuel Cecchet
Hi Markus, I didn't have much reliability issues with ensemble, appia or spread, so far. Although, I admit I didn't ever run any of these in production. Performance is certainly an issue, yes. I may suggest another reading even though a bit dates, most of the results still apply: http://jmo

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-24 Thread Simon Riggs
On Thu, 2008-12-25 at 00:10 +0900, Fujii Masao wrote: > Hi, > > On Wed, Dec 24, 2008 at 7:58 PM, Fujii Masao wrote: > > Hi, > > > > On Wed, Dec 24, 2008 at 6:57 PM, Simon Riggs wrote: > >> Yes, OK. So I think it would only work when full_page_writes = on, and > >> has been on since last checkpo

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Mark Mielke
Fujii Masao wrote: - WALSender reads from WAL buffers and/or WAL files and sends the buffers to WALReceiver. In phase one, we may assume that WALSender can only read from WAL buffers and WAL files in pg_xlog directory. Later on, this can be improved so that WALSender can temporarily restore archi

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-24 Thread Fujii Masao
Hi, On Wed, Dec 24, 2008 at 7:58 PM, Fujii Masao wrote: > Hi, > > On Wed, Dec 24, 2008 at 6:57 PM, Simon Riggs wrote: >> Yes, OK. So I think it would only work when full_page_writes = on, and >> has been on since last checkpoint. So two changes: >> >> * We just need a boolean that starts at true

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Robert Treat
On Wednesday 24 December 2008 08:48:04 Simon Riggs wrote: > On Wed, 2008-12-24 at 17:56 +0530, Pavan Deolasee wrote: > > Again, I haven't seen how frequently queries may get canceled. Or if > > the delay is set to a large value, how far behind standby may get > > during replication, so I can't real

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 17:56 +0530, Pavan Deolasee wrote: > On Wed, Dec 24, 2008 at 5:26 PM, Simon Riggs wrote: > > > > > > > The patch does go to some trouble to handle that case, as I'm sure > > you've seen. Are you saying that part of the patch is ineffective and > > should be removed, or? > >

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Pavan Deolasee
On Wed, Dec 24, 2008 at 5:26 PM, Simon Riggs wrote: > > > The patch does go to some trouble to handle that case, as I'm sure > you've seen. Are you saying that part of the patch is ineffective and > should be removed, or? > Umm.. are you talking about the "wait" mechanism ? That's the only thing

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 16:48 +0530, Pavan Deolasee wrote: > On Wed, Dec 24, 2008 at 4:41 PM, Simon Riggs wrote: > > > > > > Greg and Heikki have highlighted in this thread some aspects of btree > > garbage collection that will increase the chance of queries being > > cancelled in various circumsta

[HACKERS] [idea] a copied relkind in pg_attribute

2008-12-24 Thread KaiGai Kohei
It is an idea to improve the implementation of SE-PostgreSQL. I need a copied relkind in pg_attribute, to help its decision making. When we access on pg_attribute via ALTER TABLE or DML statement directly, SE-PostgreSQL checks privilleges for the fetched tuples. If the relation pointed by its attr

Re: [HACKERS] Archiving control (a part of synch rep patches)

2008-12-24 Thread Fujii Masao
Hi, On Wed, Dec 24, 2008 at 7:26 PM, Simon Riggs wrote: > > On Wed, 2008-12-24 at 19:04 +0900, Fujii Masao wrote: >> Hi, >> >> On Wed, Dec 24, 2008 at 4:39 PM, Fujii Masao wrote: >> > (3) >> > Revert the arhive status file of the specified file from .done to .ready. >> > This would try to archiv

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Pavan Deolasee
On Wed, Dec 24, 2008 at 4:41 PM, Simon Riggs wrote: > > > Greg and Heikki have highlighted in this thread some aspects of btree > garbage collection that will increase the chance of queries being > cancelled in various circumstances Even HOT-prune may lead to frequent query cancellations and unli

Re: [HACKERS] Hot standby and b-tree killed items

2008-12-24 Thread Simon Riggs
On Tue, 2008-12-23 at 23:59 -0500, Robert Treat wrote: > On Friday 19 December 2008 19:36:42 Simon Riggs wrote: > > Perhaps we should listen to the people that have said they don't want > > queries cancelled, even if the alternative is inconsistent answers. That > > is easily possible yet is not c

Re: [HACKERS] generic reloptions improvement

2008-12-24 Thread Alvaro Herrera
Peter Eisentraut wrote: > I'm not sure how important this is, but if you are enumerating the access > methods (RELOPT_KIND_BTREE, etc.), how will this work with user-defined > access methods? It is important. I'm intending to have a new routine which would reserve a value at runtime. This val

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-24 Thread Fujii Masao
Hi, On Wed, Dec 24, 2008 at 6:57 PM, Simon Riggs wrote: > Yes, OK. So I think it would only work when full_page_writes = on, and > has been on since last checkpoint. So two changes: > > * We just need a boolean that starts at true every checkpoint and gets > set to false anytime someone resets fu

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 15:51 +0530, Pavan Deolasee wrote: > On Wed, Dec 24, 2008 at 3:40 PM, Simon Riggs wrote: > > > > > > If we want to speed up recovery more, I think we'll see the need for an > > additional process to do WAL CRC checks. > > > > Yeah, any such helper process along with other o

Re: [HACKERS] generic reloptions improvement

2008-12-24 Thread Peter Eisentraut
On Monday 22 December 2008 18:24:53 Alvaro Herrera wrote: > > Alvaro Herrera wrote: > > > Here's a patch for improving the general reloptions mechanism. What > > > this patch does is add a table-based option parser. This allows adding > > > new options very easily, and stops the business of havi

Re: [HACKERS] Archiving control (a part of synch rep patches)

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 19:04 +0900, Fujii Masao wrote: > Hi, > > On Wed, Dec 24, 2008 at 4:39 PM, Fujii Masao wrote: > > (3) > > Revert the arhive status file of the specified file from .done to .ready. > > This would try to archive the corresponding xlog file again. The archive > > status of onl

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Pavan Deolasee
On Wed, Dec 24, 2008 at 3:40 PM, Simon Riggs wrote: > > > If we want to speed up recovery more, I think we'll see the need for an > additional process to do WAL CRC checks. > Yeah, any such helper process along with other optimizations would certainly help. But I can't still believe that on a hig

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Pavan Deolasee
On Wed, Dec 24, 2008 at 3:01 PM, Fujii Masao wrote: > > > Good question. If streaming and copying are performed parallelly, > such situation doesn't happen because the speed to generate xlog > also depends on streaming. This is a price to pay. I think that the > serial operations would need a "pac

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 18:31 +0900, Fujii Masao wrote: > > Frankly, I myself have every doubt > > that on a relatively high load setup, the standby will not be able > > keep pace with the primary for two reasons: > > > > - Lack of read ahead of data blocks (Suzuki-san's work may help this) > > - S

Re: [HACKERS] Archiving control (a part of synch rep patches)

2008-12-24 Thread Fujii Masao
Hi, On Wed, Dec 24, 2008 at 4:39 PM, Fujii Masao wrote: > (3) > Revert the arhive status file of the specified file from .done to .ready. > This would try to archive the corresponding xlog file again. The archive > status of only skipped xlog file can be reverted. > > We resume xlog archiving by

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-24 Thread Simon Riggs
On Wed, 2008-12-24 at 11:39 +0900, Fujii Masao wrote: > > We might ask why pg_start_backup() needs to perform checkpoint though, > > since you have remarked that is a problem also. > > > > The answer is that it doesn't really need to, we just need to be certain > > that archiving has been running

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Fujii Masao
Hi, On Wed, Dec 24, 2008 at 5:48 PM, Pavan Deolasee wrote: > On Wed, Dec 24, 2008 at 1:50 PM, Fujii Masao wrote: >> >> >> And, I'm worried about the situation that the speed to generate xlog >> on the primary is higher than that to copy them to the standby. We >> might not be able to start xlog

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Pavan Deolasee
On Wed, Dec 24, 2008 at 1:50 PM, Fujii Masao wrote: > > > And, I'm worried about the situation that the speed to generate xlog > on the primary is higher than that to copy them to the standby. We > might not be able to start xlog streaming forever. > If that's the case, how do you expect the stan

Re: [HACKERS] Synchronous replication, reading WAL for sending

2008-12-24 Thread Fujii Masao
Hi, On Wed, Dec 24, 2008 at 2:34 PM, Pavan Deolasee wrote: > On Tue, Dec 23, 2008 at 9:12 PM, Heikki Linnakangas > wrote: >> As the patch stands, whenever XLOG segment is switched in XLogInsert, we >> wait for the segment to be sent to the standby server. That's not good. >> Particularly in asyn