Re: [HACKERS] directory archive format for pg_dump

2010-12-01 Thread Heikki Linnakangas
On 02.12.2010 04:35, Joachim Wieland wrote: There is one thing however that I am not in favor of, which is the removal of the "sizeHint" parameter for the read functions. The reason for this parameter is not very clear now without LZF but I have tried to put in a few comments to explain the situa

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread ghatpande
I know world's population. Non of the person thinks alike and still many peoples goal can be the same. Nothing is virgin in this world. If someone thinks like that then it is a mistake. My aim is to prove that Postgresql can be the great leader if we put natural intelligence in database which

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread ghatpande
If possible can you provide glimpses of History. Corrected History will always help for future. Success rate also increases if we could avoid mistakes we made in history. regards, Vijay. Experience the Excellence.. - Original Message - From: Andrew Dunstan Date: Wednesday, Decembe

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Csaba Nagy
Hi all, On Tue, 2010-11-30 at 12:05 -0800, Josh Berkus wrote: > Can you explain, for our benefit, the use case for this? Specifically, > what can be done with synonyms which can't be done with search_path and > VIEWs? I had a few cases where synonyms for user/data base names would have helped me

Re: [HACKERS] is cachedFetchXid ever invalidated?

2010-12-01 Thread Jeff Davis
On Wed, 2010-12-01 at 23:34 -0500, Tom Lane wrote: > Jeff Davis writes: > > I can't see any place that "cachedFetchXid" is ever invalidated. > > Shouldn't it be invalidated before transaction ID wraparound? > > The assumption is that the value won't sit there (in a particular > session), without

Re: [HACKERS] Spread checkpoint sync

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 23:30, Greg Smith wrote: Heikki Linnakangas wrote: Do you have any idea how to autotune the delay between fsyncs? I'm thinking to start by counting the number of relations that need them at the beginning of the checkpoint. Then use the same basic math that drives the spread write

Re: [HACKERS] build problem

2010-12-01 Thread Jeff Davis
On Wed, 2010-12-01 at 22:16 -0500, Tom Lane wrote: > Jeff Davis writes: > > When I build HEAD using just "make -s install", everything works fine. > > But when I add "-j12", it appears to cause problems. This problem > > appeared very recently. Output below. > > Platform? Oops, sorry: $ uname -

Re: [HACKERS] Proposal: First step towards Intelligent, integrated database

2010-12-01 Thread ghatpande
The aim of HTSQL is only to avoid joins. Need to setup HTSQL server. On startup introspects table relationships. Relationships are edges in a graph model. Processor translates graph requests into SQL. Some limitations: No custom commands, primitive formatters. Query results are not streaming.

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 6:41 PM, Jim Nasby wrote: > On Dec 1, 2010, at 2:59 PM, Robert Haas wrote: >> 2. Hint bits are necessary because an old XID can't be viewed as >> guaranteed committed. > > Hmm... I thought hint bits were necessary because it's too expensive to query > CLOG for every tuple.

Re: [HACKERS] is cachedFetchXid ever invalidated?

2010-12-01 Thread Tom Lane
Jeff Davis writes: > I can't see any place that "cachedFetchXid" is ever invalidated. > Shouldn't it be invalidated before transaction ID wraparound? The assumption is that the value won't sit there (in a particular session), without ever being replaced, while more than 2G transactions elapse. A

Re: [HACKERS] unlogged tables

2010-12-01 Thread Andy Colson
On 11/30/2010 10:27 PM, Robert Haas wrote: This appears as though you've somehow gotten a normal table connected to an unlogged index. That certainly sounds like a bug, but there's not enough details here to figure out what series of steps I should perform to recreate the problem. There is \h

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 5:24 PM, Jeff Davis wrote: > On Wed, 2010-12-01 at 15:59 -0500, Robert Haas wrote: >> As for CRCs, there's a pretty direct chain of inference here: >> >> 1. CRCs are hard (really impossible) because we have hint bits. > > I would disagree with "impossible". If we don't set h

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Dec2, 2010, at 00:59 , Jim Nasby wrote: > On Dec 1, 2010, at 11:09 AM, Florian Pflug wrote: >> An UPDATE on such a SHARE locked row would be allowed despite the lock if it >> only changed columns not mentioned by any unique index. > > On a side-note, by "changed columns" do you mean the column

Re: [HACKERS] pg_execute_from_file review

2010-12-01 Thread Itagaki Takahiro
On Thu, Dec 2, 2010 at 07:00, Dimitri Fontaine wrote: > Here's the result: > > dim=# \df pg_exe*|replace_*|*binary* >                                     List of functions >   Schema   |         Name          | Result data type |    Argument data > types    |  Type > +---

Re: [HACKERS] build problem

2010-12-01 Thread Tom Lane
Jeff Davis writes: > When I build HEAD using just "make -s install", everything works fine. > But when I add "-j12", it appears to cause problems. This problem > appeared very recently. Output below. Platform? regards, tom lane -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Josh Berkus
> Something that did what Pavel mentioned: > > SELECT name, parent->name FROM children; > > would be very useful. It means you no longer have to write explicit joins (or > perhaps more accurately, you no longer have to specify exactly how to join > the two tables). Already exists: http://htsq

Re: [HACKERS] directory archive format for pg_dump

2010-12-01 Thread Joachim Wieland
On Wed, Dec 1, 2010 at 9:05 AM, Heikki Linnakangas wrote: > Forgot attachment. This is also available in the above git repo. I have quickly checked your modifications, on the one hand I like the reduction of functions, I would have said that we have AH around all the time and so we could just all

[HACKERS] build problem

2010-12-01 Thread Jeff Davis
When I build HEAD using just "make -s install", everything works fine. But when I add "-j12", it appears to cause problems. This problem appeared very recently. Output below. I can get it to work simply by running "make -s -j12 install" over and over, and eventually it succeeds. Regards,

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread ghatpande
No negativity. This world is beautiful, I want to make it more beautiful. Together we can make it, I am sure. Cheers, Vijay. Experience the excellence - Original Message - From: David Fetter Date: Wednesday, December 1, 2010 9:30 pm Subject: Re: [HACKERS] Proposal: First step t

[HACKERS] is cachedFetchXid ever invalidated?

2010-12-01 Thread Jeff Davis
I can't see any place that "cachedFetchXid" is ever invalidated. Shouldn't it be invalidated before transaction ID wraparound? It's difficult to construct a test case to show whether this is a problem or not, but I couldn't find how this is solved in the code. My understanding is that, before trun

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 8:59 AM, Andrew Dunstan wrote: > On 12/01/2010 09:41 AM, Tom Lane wrote: >> ghatpa...@vsnl.net writes: >>> Create domain is only useful for abstracting common constraints on fields >>> into single location for maintenance. It may not be useful to link tables. >> It's still uncle

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 8:07 AM, Yeb Havinga wrote: > FK's cannot refer to rows in inheritance childs. We have partially solved this issue at work. In our scenario, we're not using inheritance for partitioning, we're using it for, well, inheriting. As part of that, we have a field in the parent table

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 11:09 AM, Florian Pflug wrote: > An UPDATE on such a SHARE locked row would be allowed despite the lock if it > only changed columns not mentioned by any unique index. On a side-note, by "changed columns" do you mean the column appeared in the UPDATE statement, or the data act

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Jim Nasby
On Dec 1, 2010, at 2:59 PM, Robert Haas wrote: > 2. Hint bits are necessary because an old XID can't be viewed as > guaranteed committed. Hmm... I thought hint bits were necessary because it's too expensive to query CLOG for every tuple. If my understanding is correct then if we fix the CLOG per

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Greg Smith
Tom Lane wrote: I think the best answer is to get out of the business of using O_DIRECT by default, especially seeing that available evidence suggests it might not be a performance win anyway. I was concerned that open_datasync might be doing a better job of forcing data out of drive write

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Jeff Davis
On Wed, 2010-12-01 at 15:59 -0500, Robert Haas wrote: > As for CRCs, there's a pretty direct chain of inference here: > > 1. CRCs are hard (really impossible) because we have hint bits. I would disagree with "impossible". If we don't set hint bits during reading; and when we do set them, we log t

Re: [HACKERS] pg_execute_from_file review

2010-12-01 Thread Dimitri Fontaine
Dimitri Fontaine writes: > Itagaki Takahiro writes: >> My suggestion is to introduce pg_read_binary_file() function that can >> read any files in the server, and make CREATE EXTENSION to use the >> function. Of course, pg_execute_[sql|from]_file() can simplify queries > > It seems like all you're

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 18:44 , Tom Lane wrote: > Florian Pflug writes: >> On Dec1, 2010, at 17:17 , Tom Lane wrote: >>> There's not enough space in the infomask to record which columns (or >>> which unique index) are involved. And if you're talking about data that >>> could remain on disk long after t

Re: [HACKERS] Spread checkpoint sync

2010-12-01 Thread Greg Smith
Heikki Linnakangas wrote: Do you have any idea how to autotune the delay between fsyncs? I'm thinking to start by counting the number of relations that need them at the beginning of the checkpoint. Then use the same basic math that drives the spread writes, where you assess whether you're on

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Robert Haas writes: > If we switched from per-tuple MVCC based on XIDs to per-page MVCC > based on LSNs and a rollback segment, all of this stuff would go out > the window. Hint bits, gone. Anti-wraparound VACUUM, gone. CRCs, > feasible. Visibility map... we might still need that, but the > p

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 3:31 PM, Jeff Davis wrote: > On Wed, 2010-12-01 at 11:25 -0500, Robert Haas wrote: >> 1. Every time we observe a page as all-visible, (a) set the >> PD_ALL_VISIBLE bit on the page, without bumping the LSN; > > ... > >> 2. Every time we observe a page as all-visible, (a) set

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of mié dic 01 17:13:35 -0300 2010: > > > Well, porting applications from other database systems that support synonyms > > (i.e. Oracle, DB2, SQL Server). > > SQL Server supports synonyms? If it's not Oracle-only, it's a more > powerful argument to have the fea

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Joshua D. Drake
On Wed, 2010-12-01 at 12:46 -0800, Josh Berkus wrote: > >> I'd love to hear from someone at EDB: how are you dealing with synonym > >> name collisions right now? > > > > I think the way we deal with that is the way PostgreSQL deals with it. > > Unique names per search path. > > Have you had an em

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Dave Page
On Wed, Dec 1, 2010 at 8:46 PM, Josh Berkus wrote: > >>> I'd love to hear from someone at EDB: how are you dealing with synonym >>> name collisions right now? >> >> I think the way we deal with that is the way PostgreSQL deals with it. >> Unique names per search path. > > Have you had an employmen

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Josh Berkus
>> I'd love to hear from someone at EDB: how are you dealing with synonym >> name collisions right now? > > I think the way we deal with that is the way PostgreSQL deals with it. > Unique names per search path. Have you had an employment change I didn't know about, JD? ;-) --

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Joshua D. Drake
On Wed, 2010-12-01 at 12:13 -0800, Josh Berkus wrote: > > Well, porting applications from other database systems that support synonyms > > (i.e. Oracle, DB2, SQL Server). > > SQL Server supports synonyms? If it's not Oracle-only, it's a more > powerful argument to have the feature. Oracle, DB2 a

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Jeff Davis
On Wed, 2010-12-01 at 11:25 -0500, Robert Haas wrote: > 1. Every time we observe a page as all-visible, (a) set the > PD_ALL_VISIBLE bit on the page, without bumping the LSN; ... > 2. Every time we observe a page as all-visible, (a) set the > PD_ALL_VISIBLE bit on the page, without bumping the LS

Re: [HACKERS] Idle git question: how come so many "objects"?

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 2:08 AM, Martijn van Oosterhout wrote: > On Wed, Dec 01, 2010 at 01:03:26AM -0500, Tom Lane wrote: >> So I just made a commit that touched four files in all six active >> branches, and I see: >> >> $ git push >> Counting objects: 172, done. >> Compressing objects: 100% (89/8

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Josh Berkus
> Well, porting applications from other database systems that support synonyms > (i.e. Oracle, DB2, SQL Server). SQL Server supports synonyms? If it's not Oracle-only, it's a more powerful argument to have the feature. (IMHO, the main reason why Oracle has synonyms is that their implementation

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Alexey Klyukin
On Nov 30, 2010, at 10:05 PM, Josh Berkus wrote: > Alexey, > >> Here is the proposal to add synonyms to PostgreSQL. Initial goal is to add >> synonyms >> for relations (tables, views, sequences) and an infrastructure to allow >> synonyms >> for other database objects in the future. > > Can y

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Peter Eisentraut
On tis, 2010-11-30 at 14:20 -0500, Andrew Dunstan wrote: > I agree, that argument is completely misconceived. If the DBA is > paying enough attention to use LIMIT, s/he should be paying enough > attention not to do damage in the first place. If that were the only > argument in its favor I'd be comp

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Andrew Dunstan
On 12/01/2010 01:41 PM, Andres Freund wrote: On Wednesday 01 December 2010 19:09:05 Tom Lane wrote: Josh Berkus writes: It's a bug and it's our bug. No, it's a filesystem bug that this particular filesystem doesn't support a perfectly reasonable combination of options, and doesn't even fail

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 19:23, aaliya zarrin wrote: Thanks for quick response.. Can I change this 5 second time? I have seen the postgres code as well. You can, if you don't mind changing the sources. What is the functionality of WaitLatch() function. I could not understand completely. The recoveryWa

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Josh Berkus
> However, this doesn't really address the question of what a sensible > choice of default is. If there's little evidence about whether the > current flavor of open_datasync is really the fastest way, there's > none whatsoever that establishes open_datasync_without_o_direct > being a sane choice

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Tom Lane
David Fetter writes: > On Wed, Dec 01, 2010 at 10:15:38AM -0600, Kevin Grittner wrote: >> If we move to git, don't forget that there is not one repository >> which has the entire history for PostgreSQL JDBC -- the current >> repository is missing some work, including releases, from one stable >> b

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Tom Lane
Josh Berkus writes: > It might be nice to add new sync_method options, "osync_odirect" and > "odatasync_odirect" for DBAs who think they know enough to tune with > non-defaults. That would have the benefit that we'd not have to argue with people who liked the current behavior (assuming there are

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-01 Thread Heikki Linnakangas
On 24.11.2010 12:48, Heikki Linnakangas wrote: When recovery starts, we fetch the oldestActiveXid from the checkpoint record. Let's say that it's 100. We then start replaying WAL records from the Redo pointer, and the first record (heap insert in your case) contains an Xid that's much larger than

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Andres Freund
On Wednesday 01 December 2010 19:09:05 Tom Lane wrote: > Josh Berkus writes: > > It's a bug and it's our bug. > > No, it's a filesystem bug that this particular filesystem doesn't > support a perfectly reasonable combination of options, and doesn't > even fail gracefully as it could easily do. B

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Josh Berkus
> I think the best answer is to get out of the business of using > O_DIRECT by default, especially seeing that available evidence > suggests it might not be a performance win anyway. Well, we don't have any performance evidence ... there's an issue with the fsync-test script which causes it not t

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Tom Lane
Josh Berkus writes: > It's a bug and it's our bug. No, it's a filesystem bug that this particular filesystem doesn't support a perfectly reasonable combination of options, and doesn't even fail gracefully as it could easily do. But assigning blame doesn't help much. > Back when we added O_DIREC

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Dmitriy Igrishin
Hey, I don't clearly understand why anybody should perform DELETE directly from a psql terminal on a production system. WHY ? I can't understand what problem with DELETE without WHERE clause for application developers and why DBMS should "protect" them from DELETE FROM table. PS. Anybody can perf

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Josh Berkus
> We need a convincing use case for it. So far the only one that's seemed > at all convincing to me is the one about deleting in batches. But that > might be enough. Queueing. If logless tables are in 9.1, then using PostgreSQL as the backend for a queue becomes a sensible thing to do. And wha

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread David Fetter
On Wed, Dec 01, 2010 at 10:15:38AM -0600, Kevin Grittner wrote: > David Fetter wrote: > > > Would the people now developing the JDBC driver object to > > switching to git? > > If we move to git, don't forget that there is not one repository > which has the entire history for PostgreSQL JDBC --

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-01 Thread Josh Berkus
Ana, > We would like to inform you all that our extension to PostgreSQL, that > includes hypothetical indexes (and soon index self-tuning), is available > through a sourgeforge project. > This was suggested at PgCon 2010 and we hope some of you may find it useful, > contribute and give us your

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Josh Berkus
Tom, > Well, no, actually it's the same (only) argument. We'd never consider > back-patching such a change if our hand weren't being forced by kernel > changes :-( I think we have to back-patch the change. The way it is now, a DBA who thinks they are doing normal sensible configuration can caus

[HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-01 Thread Ana Carolina Brito de Almeida
Hackers, We would like to inform you all that our extension to PostgreSQL, that includes hypothetical indexes (and soon index self-tuning), is available through a sourgeforge project. This was suggested at PgCon 2010 and we hope some of you may find it useful, contribute and give us your feedback.

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Tom Lane
Florian Pflug writes: > On Dec1, 2010, at 17:17 , Tom Lane wrote: >> There's not enough space in the infomask to record which columns (or >> which unique index) are involved. And if you're talking about data that >> could remain on disk long after the unique index is gone, that's not >> going to

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 12:22 PM, Tom Lane wrote: > Robert Haas writes: >> I think we can improve this a bit further by also introducing a >> HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with >> FrozenXID.  This allows us to freeze tuples aggressively - if we want >> - without losi

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 11:40 AM, Heikki Linnakangas wrote: > On 01.12.2010 18:25, Robert Haas wrote: >> >> I think we can improve this a bit further by also introducing a >> HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with >> FrozenXID.  This allows us to freeze tuples aggressivel

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread aaliya zarrin
Thanks for quick response.. Can I change this 5 second time? I have seen the postgres code as well. What is the functionality of WaitLatch() function. I could not understand completely. Plz help.. On Wed, Dec 1, 2010 at 5:53 PM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > O

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Robert Haas writes: > I think we can improve this a bit further by also introducing a > HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with > FrozenXID. This allows us to freeze tuples aggressively - if we want > - without losing any forensic information. So far so good ... > We c

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 17:17 , Tom Lane wrote: > Florian Pflug writes: >> The validity wouldn't change, only the kind of lock taken. If all columns to >> be locked are part of some unique index, we'd record that fact in the locked >> tuple's infomask, and thus know that only a certain subset of colum

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Heikki Linnakangas writes: > On 01.12.2010 18:40, Tom Lane wrote: >> Um, no it isn't. Suppose the heap page gets to disk but we crash before >> the WAL record does. Now we have a persistent state where the heap page >> is marked PD_ALL_VISIBLE but the corresponding VM bit is not set. The >> VM

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 18:40, Tom Lane wrote: Robert Haas writes: As far as I can tell, there are basically two viable solutions on the table here. 1. Every time we observe a page as all-visible, (a) set the PD_ALL_VISIBLE bit on the page, without bumping the LSN; (b) set the bit in the visibility ma

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Heikki Linnakangas writes: > Hmm, actually, if we're willing to believe PD_ALL_VISIBLE in the page > header over the xmin/xmax on the tuples, we could simply not bother > doing anti-wraparound vacuums for pages that have the flag set. I'm not > sure what changes that would require outside heapa

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Robert Haas writes: > As far as I can tell, there are basically two viable solutions on the > table here. > 1. Every time we observe a page as all-visible, (a) set the > PD_ALL_VISIBLE bit on the page, without bumping the LSN; (b) set the > bit in the visibility map page, bumping the LSN as usual

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 18:25, Robert Haas wrote: I think we can improve this a bit further by also introducing a HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with FrozenXID. This allows us to freeze tuples aggressively - if we want - without losing any forensic information. We can then m

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 17:11 , Kevin Grittner wrote: > Florian Pflug wrote: > >> BTW, my "serializable_lock_consisteny" patch would allow you to do >> this purely within pl/pgsql in a race-condition free way. So if >> that patch should get applied you might want to consider this as a >> workaround. Wh

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 10:36 AM, Bruce Momjian wrote: > Oh, we don't update the LSN when we set the PD_ALL_VISIBLE flag?  OK, > please let me think some more.  Thanks. As far as I can tell, there are basically two viable solutions on the table here. 1. Every time we observe a page as all-visible

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Tom Lane
Florian Pflug writes: > The validity wouldn't change, only the kind of lock taken. If all columns to > be locked are part of some unique index, we'd record that fact in the locked > tuple's infomask, and thus know that only a certain subset of columns are to > be prevented from being updated.

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Simon Riggs
On Wed, 2010-12-01 at 15:07 +0100, Yeb Havinga wrote: > FK's cannot refer to rows in inheritance childs. With some changes in > LockRows, together with removing the ONLY keyword in ri_trigger.c, it > was possible to refer to the rows in child relations. (WIP patch attached) This has a userspace

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Kevin Grittner
David Fetter wrote: > Would the people now developing the JDBC driver object to > switching to git? If we move to git, don't forget that there is not one repository which has the entire history for PostgreSQL JDBC -- the current repository is missing some work, including releases, from one sta

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Kevin Grittner
Florian Pflug wrote: > BTW, my "serializable_lock_consisteny" patch would allow you to do > this purely within pl/pgsql in a race-condition free way. So if > that patch should get applied you might want to consider this as a > workaround. Whether it will get applied is yet to be seen, though > -

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Nov29, 2010, at 22:33 , Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Alvaro Herrera's message of lun nov 29 18:00:55 -0300 2010: >>> Additionally, we'd have to expend some more cycles at the parse analysis >>> phase (of the "FOR SHARE OF x.col1, x.col2" query) to verify that those

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread David Fetter
On Wed, Dec 01, 2010 at 12:47:13PM +0100, Magnus Hagander wrote: > On Tue, Nov 30, 2010 at 19:49, Radosław Smogura wrote: > > Hello, > > > > Maybe you are interested about this what I done with JDBC > > > > > > Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is > > cu

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread David Fetter
My point exactly. You started off with high negativity, and you should not expect good results from same. Cheers, David. On Wed, Dec 01, 2010 at 08:15:25PM +0500, ghatpa...@vsnl.net wrote: > Be positive ... Negative thoughts are not good... > > - Original Message - > From: David Fetter

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 15:27 , Tom Lane wrote: > Yeb Havinga writes: >> FK's cannot refer to rows in inheritance childs. With some changes in >> LockRows, together with removing the ONLY keyword in ri_trigger.c, it >> was possible to refer to the rows in child relations. (WIP patch attached) > >> Th

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 9:57 AM, Kevin Grittner wrote: > Heikki Linnakangas wrote: > >> it would be annoying to have to checkpoint after a data load > > Heck, in my world it's currently pretty much a necessity to run > VACUUM FREEZE ANALYZE on a table after a data load before it's > reasonable to

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Bruce Momjian
Heikki Linnakangas wrote: > On 01.12.2010 15:39, Bruce Momjian wrote: > > Heikki Linnakangas wrote: > >> On 01.12.2010 03:35, Bruce Momjian wrote: > >>> Heikki Linnakangas wrote: > Let's recap what happens when a VM bit is set: You set the > PD_ALL_VISIBLE flag on the heap page (assuming

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> Right, AFAIK there is nothing in KNNGIST that would involve an on-disk >> data change. > But any external module relying on GiST will have to provide for the new > function you're thinking about, right? Updating was already needed to > cope with the

Re: [HACKERS] profiling connection overhead

2010-12-01 Thread Kevin Grittner
Robert Haas wrote: > Jeff Janes wrote: >> Oracle's backend start up time seems to be way higher than PG's. > Interesting. How about MySQL and SQL Server? My recollection of Sybase ASE is that establishing a connection doesn't start a backend or even a thread. It establishes a network conn

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Dimitri Fontaine
Tom Lane writes: > Right, AFAIK there is nothing in KNNGIST that would involve an on-disk > data change. Nice, that matches my Royal Oak memories. But any external module relying on GiST will have to provide for the new function you're thinking about, right? Updating was already needed to cope w

Re: [HACKERS] pg_execute_from_file review

2010-12-01 Thread Dimitri Fontaine
Itagaki Takahiro writes: > My suggestion is to introduce pg_read_binary_file() function that can > read any files in the server, and make CREATE EXTENSION to use the > function. Of course, pg_execute_[sql|from]_file() can simplify queries It seems like all you're missing in the current patch is t

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread ghatpande
Be positive ... Negative thoughts are not good... - Original Message - From: David Fetter Date: Wednesday, December 1, 2010 8:42 pm Subject: Re: [HACKERS] Proposal: First step towards Intelligent,integrateddatabase To: ghatpa...@vsnl.net Cc: pgsql hackers > On Wed, Dec 01, 2010 at 03:1

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread Euler Taveira de Oliveira
Heikki Linnakangas escreveu: > On 01.12.2010 13:27, aaliya zarrin wrote: >> I want to know how frequently postgres search for trigger file to switch >> over. > > In 9.0, every 100ms while streaming replication is active and connected. > 5 seconds otherwise. In current git master branch, it's alway

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Kevin Grittner
Mario Weilguni wrote: > Is it really up to the database to decide what queries are ok? > It's the task of the developers to test their applikations. We're talking about ad hoc queries here, entered directly through psql or similar. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Proposal: First step towards Intelligent,integrated database

2010-12-01 Thread David Fetter
On Wed, Dec 01, 2010 at 03:19:32PM +0500, ghatpa...@vsnl.net wrote: > Hello, > > Here is the proposal: My 1st step towards Intelligent, Integrated database. You're implying that databases are stupid and incoherent. This is *not* a great way to start. Cheers, David. -- David Fetter http://fet

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Tom Lane
Yeb Havinga writes: > On 2010-12-01 15:27, Tom Lane wrote: >> Indeed. This isn't even worth the time to review, unless you have a >> proposal for fixing the unique-index-across-multiple-tables problem. > That was in the part that you chose to not quote. Perhaps I should have said "possibly work

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Yeb Havinga
On 2010-12-01 15:27, Tom Lane wrote: Yeb Havinga writes: FK's cannot refer to rows in inheritance childs. With some changes in LockRows, together with removing the ONLY keyword in ri_trigger.c, it was possible to refer to the rows in child relations. (WIP patch attached) Though it passes simple

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Andrew Dunstan
On 12/01/2010 09:41 AM, Tom Lane wrote: ghatpa...@vsnl.net writes: Create domain is only useful for abstracting common constraints on fields into single location for maintenance. It may not be useful to link tables. It's still unclear what this does that you don't get from inheritance, typed

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Mario Weilguni
Am 01.12.2010 15:37, schrieb Rob Wultsch: "For beginners, a useful startup option is --safe-updates (or --i-am-a-dummy, which has the same effect). This option was introduced in MySQL 3.23.11. It is helpful for cases when you might have issued a DELETE FROM tbl_name statement but forgotten the WH

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Kevin Grittner
Heikki Linnakangas wrote: > it would be annoying to have to checkpoint after a data load Heck, in my world it's currently pretty much a necessity to run VACUUM FREEZE ANALYZE on a table after a data load before it's reasonable to expose the table to production use. It would hardly be an incon

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Tom Lane
Robert Haas writes: > On Wed, Dec 1, 2010 at 5:22 AM, Dimitri Fontaine > wrote: >> IIRC, the goal here was to be able to benefit from KNN GiST from >> existing GiST indexes as soon as you restart the server with the new >> code compiled in. I'm not sure it's that important in the context of >> p

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Tom Lane
ghatpa...@vsnl.net writes: > Create domain is only useful for abstracting common constraints on fields > into single location for maintenance. It may not be useful to link tables. It's still unclear what this does that you don't get from inheritance, typed tables, use of a table's rowtype as a fi

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Rob Wultsch
On Wed, Dec 1, 2010 at 4:01 AM, Daniel Loureiro wrote: > A) an feature MySQL-like which will DELETE/UPDATE just K tuples > B) an feature to protect the database in case the DBA forget the "WHERE" > statement > MySQL has B as well. To quote the manual: "For beginners, a useful startup option is --

Re: [HACKERS] GiST insert algorithm rewrite

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 4:00 AM, Heikki Linnakangas wrote: > On 01.12.2010 04:10, Robert Haas wrote: >> >> On Tue, Nov 30, 2010 at 10:26 AM, Heikki Linnakangas >>  wrote: Does the current code cope with the corruption? >>> >>> It's not corruption, but "intended degradation". Yes, the cur

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Tom Lane
Yeb Havinga writes: > FK's cannot refer to rows in inheritance childs. With some changes in > LockRows, together with removing the ONLY keyword in ri_trigger.c, it > was possible to refer to the rows in child relations. (WIP patch attached) > Though it passes simple tests, it is far from comple

Re: [HACKERS] profiling connection overhead

2010-12-01 Thread Andres Freund
On Wednesday 01 December 2010 15:20:32 Robert Haas wrote: > On Tue, Nov 30, 2010 at 11:32 PM, Jeff Janes wrote: > > On 11/28/10, Robert Haas wrote: > >> To some degree we're a > >> victim of our own flexible and extensible architecture here, but I > >> find it pretty unsatisfying to just say, OK,

Re: [HACKERS] profiling connection overhead

2010-12-01 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:32 PM, Jeff Janes wrote: > On 11/28/10, Robert Haas wrote: >> >> In a close race, I don't think we should get bogged down in >> micro-optimization here, both because micro-optimizations may not gain >> much and because what works well on one platform may not do much at

Re: [HACKERS] directory archive format for pg_dump

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 16:03, Heikki Linnakangas wrote: On 29.11.2010 22:21, Heikki Linnakangas wrote: I combined those, and the Free/Flush steps, and did a bunch of other editorializations and cleanups. Here's an updated patch, also available in my git repository at git://git.postgresql.org/git/users/he

  1   2   >