Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Heikki Linnakangas
Simon Riggs wrote: On Thu, 2009-01-29 at 12:18 +0900, Fujii Masao wrote: Though this is a matter of taste, I think that it's weird that bgwriter runs with ThisTimeLineID = 0 during recovery. This is because XLogCtl->ThisTimeLineID is set at the end of recovery. ISTM this will be a cause of bug i

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Simon Riggs
On Thu, 2009-01-29 at 10:36 +0900, Fujii Masao wrote: > Hi, > > On Wed, Jan 28, 2009 at 11:19 PM, Fujii Masao wrote: > >> I feel quite good about this patch now. Given the amount of code churn, it > >> requires testing, and I'll read it through one more time after sleeping > >> over > >> it. Si

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Simon Riggs
On Thu, 2009-01-29 at 12:18 +0900, Fujii Masao wrote: > Hi, > > On Wed, Jan 28, 2009 at 11:19 PM, Fujii Masao wrote: > >> I feel quite good about this patch now. Given the amount of code churn, it > >> requires testing, and I'll read it through one more time after sleeping > >> over > >> it. Si

Re: [HACKERS] V4 of PITR performance improvement for 8.4

2009-01-28 Thread Koichi Suzuki
Hi, This is also a reply to your latest post.I'm replying to the older one because I need original text. 2009/1/26 Koichi Suzuki : > Hi, > > It's simply because we should not refer to system catalog during the recovery. This is the reason why I didn't use PrefetchBuffer(). Prefetch buffer

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Treat
On Wednesday 28 January 2009 23:42:11 Robert Haas wrote: > > Our usual process *is* to try and circumvent our usual process. And I > > believe it will continue to be that way until we lower the incentive to > > lobby for circumvention. > > I think Tom and Bruce have both pretty much stated that the

[HACKERS] possible bug in cover density ranking?

2009-01-28 Thread Sushant Sinha
I am running postgres 8.3.1. In tsrank.c I am looking at the cover density function used for ranking while doing text search: float4 calc_rank_cd(float4 *arrdata, TSVector txt, TSQuery query, int method) Here is the excerpt of code that I think may possibly have bug when document is big enough to

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Jonah H. Harris
On Wed, Jan 28, 2009 at 9:49 PM, KaiGai Kohei wrote: > IIRC, 0racle or M$ has a patent to rewrite WHERE clause for security > purpose, so Tom suggested it should be implemented using a hook > deployed within executor. Yes, it was Oracle. There are a couple newer revisions, but they're all base

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Bruce Momjian wrote: Robert Haas wrote: On Wed, Jan 28, 2009 at 6:57 PM, Tom Lane wrote: Hmm. If that's the expected application environment then the patch as proposed has fatal performance problems anyway, for lack of a way to get rid of no-longer-referenced pg_security rows. We had been le

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Bruce Momjian
FYI, I have created a wiki page that summerizes the SE-PostgreSQL information we have gathered so far: http://wiki.postgresql.org/wiki/SEPostgreSQL-patch -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + If your life is

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
KaiGai Kohei wrote: > Bruce Momjian wrote: > > OK, time for me to chime in. > > > > I think the outstanding commit-fest items can be broken down into four > > sections: > > > > o Log streaming > > o Hot standby > > o SE-PostgreSQL > > o Others > > - snip - > > > SE-Postgre

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Bruce Momjian wrote: Tom Lane wrote: Gregory Stark writes: I don't think partitioning is really the same thing as row-level security. Of course not, but it seems to me that it can be used to accomplish most of the same practical use-cases. The main gripe about doing it via partitioning is th

Re: Commitfest infrastructure (was Re: [HACKERS] 8.4 release =?iso-8859-1?q?=09planning?=)

2009-01-28 Thread Bruce Momjian
Marc G. Fournier wrote: > On Wed, 28 Jan 2009, Bruce Momjian wrote: > > >> Details? I find no public record of this. > > > > I think it was Keystone; Marc set it up. > > If that was it, god, that was what, 10 years ago when we tried that? And > yes, it was atrocious ... Yep, that matches my

Re: Commitfest infrastructure (was Re: [HACKERS] 8.4 release =?iso-8859-1?q?=09planning?=)

2009-01-28 Thread Marc G. Fournier
On Wed, 28 Jan 2009, Bruce Momjian wrote: Details? I find no public record of this. I think it was Keystone; Marc set it up. If that was it, god, that was what, 10 years ago when we tried that? And yes, it was atrocious ... Marc G. Fournier Hub.Org Networking Services (h

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Bruce Momjian
Robert Haas wrote: > On Wed, Jan 28, 2009 at 6:57 PM, Tom Lane wrote: > > Hmm. If that's the expected application environment then the patch as > > proposed has fatal performance problems anyway, for lack of a way to > > get rid of no-longer-referenced pg_security rows. We had been led to > > un

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Bruce Momjian
Tom Lane wrote: > Gregory Stark writes: > > I don't think partitioning is really the same thing as row-level > > security. > > Of course not, but it seems to me that it can be used to accomplish most > of the same practical use-cases. The main gripe about doing it via > partitioning is that the

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Haas
> Our usual process *is* to try and circumvent our usual process. And I believe > it will continue to be that way until we lower the incentive to lobby for > circumvention. I think Tom and Bruce have both pretty much stated that they're not keen on a shorter release cycle, and they're the ones who

Re: Commitfest infrastructure (was Re: [HACKERS] 8.4 release =?iso-8859-1?q?=09planning?=)

2009-01-28 Thread Bruce Momjian
Peter Eisentraut wrote: > On Tuesday 27 January 2009 23:59:46 Magnus Hagander wrote: > > Marko Kreen wrote: > > > On 1/27/09, Peter Eisentraut wrote: > > >> On Tuesday 27 January 2009 15:51:02 Marko Kreen wrote: > > >> > Such app already exists: > > >> > > > >> > http://ozlabs.org/~jk/project

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Robert Haas wrote: My concern is that superuser is allowed to modify system catalog by hand, like: UPDATE pg_proc SET probin = '/tmp/malicious_library.so' WHERE oid = ...; It is logically same as ALTER FUNCTION. Even if I remove a hook from simple_heap_(), it is necessary to check que

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Bruce Momjian
Tom Lane wrote: > The appeal of the pg_dump approach is that it will automatically handle > everything that there exists a plain-SQL representation for, which is to > say darn near everything. We will need special purpose code to deal > with the dropped-column and TOAST-oid issues, but that can pr

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Robert Haas wrote: On Wed, Jan 28, 2009 at 10:15 PM, KaiGai Kohei wrote: It seems to me reference-counter is more preferable than boolean, at least. But it makes performance pain on writer access when it is expanded to row-level security. A reference counter will never work. You could easily

Re: [HACKERS] Column-Level Privileges

2009-01-28 Thread Stephen Frost
Tom, et al, * Tom Lane (t...@sss.pgh.pa.us) wrote: > There are still some significant loose ends though: Apologies for not having this finished already, been kind of caught up in some discussions. :) > * Some of the information_schema views are specified to respond to > per-column privileges; th

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 10:15 PM, KaiGai Kohei wrote: > It seems to me reference-counter is more preferable than boolean, > at least. But it makes performance pain on writer access when it > is expanded to row-level security. A reference counter will never work. You could easily end up serializin

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Robert Haas wrote: On Wed, Jan 28, 2009 at 9:27 PM, Stephen Frost wrote: Robert, * Robert Haas (robertmh...@gmail.com) wrote: pg_security (which I really think out to be renamed to pg_selinux_context or something, and make a new table if we someday support Trusted Solaris or whatever). Err,

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Andrew Dunstan
Robert Treat wrote: On Wednesday 28 January 2009 20:12:40 Bruce Momjian wrote: Robert Treat wrote: The revisionism was that of "remarkable failure". That was our shortest release cycle in the modern era. And it didn't have the advantage of the commitfest process. But I think what is

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
> My concern is that superuser is allowed to modify system catalog > by hand, like: > > UPDATE pg_proc SET probin = '/tmp/malicious_library.so' > WHERE oid = ...; > > It is logically same as ALTER FUNCTION. > > Even if I remove a hook from simple_heap_(), it is necessary > to check queries

Re: [HACKERS] mingw check hung

2009-01-28 Thread Andrew Dunstan
Andrew Dunstan wrote: Tom Lane wrote: Magnus Hagander writes: Andrew Dunstan wrote: The suspect patch is quite definitely the source of the problem. I can't spot the error right off :-( Can you try to see if it's the putenv() or the unsetenv() that gets broken? Ar

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 9:27 PM, Stephen Frost wrote: > Robert, > > * Robert Haas (robertmh...@gmail.com) wrote: >> pg_security (which I really think out to be renamed to >> pg_selinux_context or something, and make a new table if we someday >> support Trusted Solaris or whatever). > > Err, this d

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Treat
On Wednesday 28 January 2009 20:12:40 Bruce Momjian wrote: > Robert Treat wrote: > > The revisionism was that of "remarkable failure". That was our shortest > > release cycle in the modern era. And it didn't have the advantage of the > > commitfest process. > > > > But I think what is important he

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Fujii Masao
Hi, On Wed, Jan 28, 2009 at 11:19 PM, Fujii Masao wrote: >> I feel quite good about this patch now. Given the amount of code churn, it >> requires testing, and I'll read it through one more time after sleeping over >> it. Simon, do you see anything wrong with this? > > I also read this patch and

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Robert Haas wrote: On Wed, Jan 28, 2009 at 6:57 PM, Tom Lane wrote: Hmm. If that's the expected application environment then the patch as proposed has fatal performance problems anyway, for lack of a way to get rid of no-longer-referenced pg_security rows. We had been led to understand that t

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 6:05 PM, Tom Lane wrote: > Well, what it really means is that all the special-purpose conversion > code is in SQL instead of C. Which is sort of good as long as whatever > transformation you have in mind can be done easily in SQL. (Good luck > if you need to control the O

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Stephen Frost
* KaiGai Kohei (kai...@ak.jp.nec.com) wrote: >> I agree that that's no good. As do I. > My concern is that superuser is allowed to modify system catalog > by hand, like: > > UPDATE pg_proc SET probin = '/tmp/malicious_library.so' > WHERE oid = ...; That UPDATE still goes through permissio

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Robert Haas wrote: On Wed, Jan 28, 2009 at 6:34 PM, Tom Lane wrote: As an example, the present patch imagines that it will have adequate control over inserts by putting hooks into simple_heap_insert and the (rather small number of) places that call heap_insert directly. But there are dozens of

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Joshua Brindle
Robert Haas wrote: On Wed, Jan 28, 2009 at 6:57 PM, Tom Lane wrote: Then you can write something which goes through and sets all the rows to false and then visits every row of every table in the database and forces OID lookups on the security ID of each. When you get done, any rows that still

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > pg_security (which I really think out to be renamed to > pg_selinux_context or something, and make a new table if we someday > support Trusted Solaris or whatever). Err, this doesn't really make sense if we're doing row-level security, that's

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Stephen Frost
KaiGai, * KaiGai Kohei (kai...@ak.jp.nec.com) wrote: > Obviously, we cannot make clear state of the row-level access > controls by the date of v8.4 freeze. Indeed, I have to agree that's where things are headed, which is certainly unfortunate but I think it's good that we were able to provide som

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
> I found a proverbial phrase in my dictionaly: > Between two stools you fall to the ground. LOL. I like that one - and it's very apt. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 6:57 PM, Tom Lane wrote: > Hmm. If that's the expected application environment then the patch as > proposed has fatal performance problems anyway, for lack of a way to > get rid of no-longer-referenced pg_security rows. We had been led to > understand that there wouldn't

Re: [HACKERS] mingw check hung

2009-01-28 Thread Andrew Dunstan
Tom Lane wrote: Magnus Hagander writes: Andrew Dunstan wrote: The suspect patch is quite definitely the source of the problem. I can't spot the error right off :-( Can you try to see if it's the putenv() or the unsetenv() that gets broken? Are we sure pgwin32_unse

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 6:34 PM, Tom Lane wrote: > As an example, the present patch imagines that it will have adequate > control over inserts by putting hooks into simple_heap_insert and the > (rather small number of) places that call heap_insert directly. But > there are dozens of calls of simp

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Obviously, we cannot make clear state of the row-level access controls by the date of v8.4 freeze. I agree the row-level access controls can be separated and postponed to v8.5 development cycle. So, I'll cut off a few part of previous patches for v8.4. Stephen Frost gave me a guideline about wha

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Gregory Stark wrote: > I'm a bit shocked by how long Tom expects the release cycle to take even if we > froze the code today. I guess I forget how long it takes and how many steps > there are from past releases. If it's going to be 9+ months between Nov 1st > and the first commitfest I'm worried ab

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Robert Treat wrote: > > We are going to have exactly > > no credibility if we tell Simon et al "we're pushing these patches to > > 8.5, but don't worry, it'll be a short release cycle". > > > > The other options being we stall 8.4 indefinatly waiting for HS (which, > honestly I am comfortable wi

Re: [HACKERS] Hot standby, recovery infra

2009-01-28 Thread Fujii Masao
Hi, On Wed, Jan 28, 2009 at 11:19 PM, Fujii Masao wrote: >> I feel quite good about this patch now. Given the amount of code churn, it >> requires testing, and I'll read it through one more time after sleeping over >> it. Simon, do you see anything wrong with this? > > I also read this patch and

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Robert Treat wrote: > The revisionism was that of "remarkable failure". That was our shortest > release cycle in the modern era. And it didn't have the advantage of the > commitfest process. > > But I think what is important here is to recognize why it didn't work. Once > again we ended up wi

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Andrew Dunstan wrote: Tom Lane wrote: As an example, the present patch imagines that it will have adequate control over inserts by putting hooks into simple_heap_insert and the (rather small number of) places that call heap_insert directly. But there are dozens of calls of simple_heap_insert an

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Joshua Brindle
Tom Lane wrote: Joshua Brindle writes: In reality it isn't, unless postgres won't mind thousands of partitions being made. In the military/gov implementations BLP lets you have hierarchical levels and non-hierarchical categories. Since I linked to an article about it upthread I assumed everyone

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread KaiGai Kohei
Good morning, I started to follow the discussion. (Time difference is unconfortable for me!) >> adding SELinux support for the existing levels of access control in PG > > is > > - table/column level access controls > - permission checks on database login > - permission checks on function invocat

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Joshua Brindle writes: > In reality it isn't, unless postgres won't mind thousands of > partitions being made. In the military/gov implementations BLP lets > you have hierarchical levels and non-hierarchical categories. Since I > linked to an article about it upthread I assumed everyone > particip

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Andrew Dunstan
Tom Lane wrote: As an example, the present patch imagines that it will have adequate control over inserts by putting hooks into simple_heap_insert and the (rather small number of) places that call heap_insert directly. But there are dozens of calls of simple_heap_insert and no way for the func

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Robert Haas writes: > I'm not clear that I understand why it would be necessary for > row-level security to touch every part of the code. OK, I admit it's not "every" part, just every place that fetches, inserts, updates, or deletes tuples. There are quite a few ;-) As an example, the present p

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Ron Mayer
Joshua Brindle wrote: > Nonetheless, this conversation seems moot now that Tom has walled off > and won't even discuss row-level access controls. I think that's a bit of an overstatement. He says he's against them[1] and he says that they are the sticking point on this patch[2], and that they bre

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 17:04 -0500, Tom Lane wrote: > The key committers are overstressed already. Some developers are too. I'm sure there's a way to avoid it being a zero-sum game. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-ha

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Gregory Stark
Tom Lane writes: > I don't believe I will ever think that row-level checks are a good idea; as > long as those are in the patch I will vote against applying it. I think we're conflating two behaviours here. The data hiding behaviour clearly changes the semantics of queries in ways that make a

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Tom Lane
Robert Haas writes: > I really like this idea, assuming I understand it. Basically, I think > you're proposing that we move the old system catalogs out of the way, > bootstrap a new catalog, and then using SQL (running inside a > standalone backend?) to migrate data from the old catalog to the n

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Simon Riggs wrote: > > On Tue, 2009-01-27 at 15:46 -0500, Tom Lane wrote: > > Peter Eisentraut writes: > > > Not to pick on you personally, but this is the kind of review that should > > > have > > > happened six months ago, not during a "why is our development process > > > inadequate" discus

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Peter Eisentraut wrote: > On Tuesday 27 January 2009 16:36:50 Stephen Frost wrote: > > * Peter Eisentraut (pete...@gmx.net) wrote: > > > As one of the earlier reviewers, I think the design is OK, but the way > > > the implementation is presented was not acceptable, and very little has > > > been ac

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Joshua Brindle
Ron Mayer wrote: Stephen Frost wrote: And, just to go full circle, row-level access controls are exactly what the other enterprise RDBMSs have and is what is used in these security circles today. One of the major issues, as I understand it, is to be able to use stock applications with multiple

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> As the SEPostgres patch is constructed, the planner could *never* trust >> an FK for optimization since it would have no way to know whether row >> level permissions might be present (perhaps only for some rows) at >> execution time. You could only get b

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Robert Haas wrote: > > The flaw in that argument is that as you are doing it, the > > de-optimization only happens on queries that actually need the behavior. > > As the SEPostgres patch is constructed, the planner could *never* trust > > an FK for optimization since it would have no way to know wh

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Simon Riggs wrote: > > On Tue, 2009-01-27 at 14:18 -0500, Tom Lane wrote: > > Joshua Brindle writes: > > > Tom Lane wrote: > > >> Right, which is why it's bad for something like a foreign key constraint > > >> to expose the fact that the row does exist after all. > > > > > Once again, this is no

Re: [HACKERS] Output filter for psql

2009-01-28 Thread Andrew Dunstan
D'Arcy J.M. Cain wrote: On Wed, 28 Jan 2009 14:08:54 -0500 Andrew Dunstan wrote: D'Arcy J.M. Cain wrote: I suppose we could define another line with options that we could define for meta information such as the border setting and the table name and whatever we define later. E.g: "t

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Tom Lane wrote: > Robert Haas writes: > > On Tue, Jan 27, 2009 at 12:52 PM, Tom Lane wrote: > >> Right, but you expect that to be a small and predictable cost, say in > >> the single-digits-percentage range. Plan optimizations that > >> suddenly stop happening can cost you multiple orders of mag

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Treat
On Wednesday 28 January 2009 12:35:42 Tom Lane wrote: > Robert Treat writes: > > On Wednesday 28 January 2009 08:55:56 Magnus Hagander wrote: > >> We're still going to have to pay the full cost of doing a release every > >> time. With beta/rc management, release notes, announcements, postings, > >

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Ron Mayer
Stephen Frost wrote: > And, just to go full circle, row-level access controls are exactly what > the other enterprise RDBMSs have and is what is used in these security > circles today. One of the major issues, as I understand it, is to be > able to use stock applications with multiple security lev

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Ron Mayer
Joshua Brindle wrote: > Tom Lane wrote: >> Joshua Brindle writes: >>> I'm not sure how much it would cut to remove row level access >>> controls, but I do have some points here. To me, row level access >>> controls are the most important part, this is the feature that lets us >>> put secret and to

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > For me, the row-level access controls are really the sticking point. > There is absolutely nothing you can say that will convince me that they > don't break SQL in fundamental ways, and I also don't believe that it's > going to be possible to implement them

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Robert Haas
> I considered pg_upgrade one of the "others" on the list; it is not as > complex as the previous three. LOL. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 5:00 PM, Zdenek Kotala wrote: > Tom Lane píše v st 28. 01. 2009 v 14:06 -0500: >> Trying to do catalog upgrade >> in-place is going to be a complete mess. I'd be interested to know, >> for example, how you imagine rearranging the contents of pg_class would >> work. You do

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Bruce Momjian
Zdenek Kotala wrote: > > Bruce Momjian p??e v po 26. 01. 2009 v 23:02 -0500: > > OK, time for me to chime in. > > > > I think the outstanding commit-fest items can be broken down into four > > sections: > > > > o Log streaming > > o Hot standby > > o SE-PostgreSQL > > o Other

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Robert Haas
On Wed, Jan 28, 2009 at 3:58 PM, Tom Lane wrote: > Andrew Sullivan writes: >> On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: >>> The costs are nil for people who don't want this feature. > >> That's also false, because developers who don't care about the feature >> have to contin

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Tom Lane
Dimitri Fontaine writes: > Le 28 janv. 09 à 16:22, Simon Riggs a écrit : >> The only way to keep the dev window open longer is to overlap the >> start of the next cycle with the previous one. i.e. branch new version >> before final release. > This is the second time the idea is raised and

Re: [HACKERS] pg_upgrade project status

2009-01-28 Thread Zdenek Kotala
Tom Lane píše v st 28. 01. 2009 v 14:06 -0500: > Trying to do catalog upgrade > in-place is going to be a complete mess. I'd be interested to know, > for example, how you imagine rearranging the contents of pg_class would > work. You don't get to modify pg_class if you can't even find it, which

Re: [HACKERS] Column privileges for system catalogs

2009-01-28 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I don't have any objection to changing the catalog's own permissions > that way, but the filtered view still has a usability advantage: you > can just go "select * from ...". Is it reasonable to change the catalog > permissions and keep the view too? I've

Re: [HACKERS] 8.4 release planning

2009-01-28 Thread Dimitri Fontaine
Le 28 janv. 09 à 16:22, Simon Riggs a écrit : The only way to keep the dev window open longer is to overlap the start of the next cycle with the previous one. i.e. branch new version before final release. This is the second time the idea is raised and I like it. Do we have anywhere near e

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Tom Lane
Robert Haas writes: > I vote with Simon. The thing is that if you get some queries > cancelled, you'll realize you have a problem. ... or if you don't, they couldn't have been all that critical. > Having your failover be 12 hours > behind (or 12 months behind) is something that it would be much

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Gregory Stark
Simon Riggs writes: > On Wed, 2009-01-28 at 14:56 -0500, Tom Lane wrote: > >> Well, those unexpectedly cancelled queries could have represented >> critical functionality too. I think this argument calls the entire >> approach into question. If there is no safe setting for the parameter >> then

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Dimitri Fontaine
Sorry for top-posting and avoiding to paste online doc URL. Joshua, you sound like you're missing an important point. Sorry for misunderstanding if that's not true. Partitioning is supported in a way that a query does not need to know about it, be it a SELECT, INSERT, UPDATE or DELETE. And

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Robert Haas
>> I don't. Primarily, we must support high availability. It is much better >> if we get people saying "I get my queries cancelled" and we say RTFM and >> change parameter X, than if people say "my failover was 12 hours behind >> when I needed it to be 10 seconds behind and I lost a $1 million beca

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Andrew Sullivan writes: > On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: >> The costs are nil for people who don't want this feature. > That's also false, because developers who don't care about the feature > have to continue to maintain it as part of the system. If maintenance

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Jeff Davis
On Wed, 2009-01-28 at 22:47 +0200, Heikki Linnakangas wrote: > It's not quite that simple. Setting max_standby_delay=5mins means that > you're willing to wait 5 minutes for each query to die. Which means that > in worst case you have to stop for 5 minutes at every single vacuum > record, and fal

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 22:47 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > The essential choice is "What would you like the max failover time to > > be?". Some users want one server with max 5 mins behind, some want two > > servers, one with 0 seconds behind, one with 12 hours behind >

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Heikki Linnakangas
Simon Riggs wrote: The essential choice is "What would you like the max failover time to be?". Some users want one server with max 5 mins behind, some want two servers, one with 0 seconds behind, one with 12 hours behind It's not quite that simple. Setting max_standby_delay=5mins means that yo

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 14:56 -0500, Tom Lane wrote: > Well, those unexpectedly cancelled queries could have represented > critical functionality too. I think this argument calls the entire > approach into question. If there is no safe setting for the parameter > then we need to find a way to not

Re: [HACKERS] mingw check hung

2009-01-28 Thread Magnus Hagander
Mark Cave-Ayland wrote: > Magnus Hagander wrote: > >> Per discussion I looked at just reverting that part, but that won't >> work. If we do that, the call to SetEnvironmentVariable() will not be >> run, which certainly isn't right.. >> >> The problem has to be in win32env.c. I originally thought w

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 11:33 -0800, Joshua D. Drake wrote: > On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: > > On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: > > > Agreed. As explained when I published that patch it is deliberately > > severe to allow testing of conflict resolutio

Re: [HACKERS] Output filter for psql

2009-01-28 Thread D'Arcy J.M. Cain
On Wed, 28 Jan 2009 14:08:54 -0500 Andrew Dunstan wrote: > D'Arcy J.M. Cain wrote: > > I suppose we could define another line with options that we could > > define for meta information such as the border setting and the table > > name and whatever we define later. E.g: > > > > "table:person","bor

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Heikki Linnakangas
Tom Lane wrote: "Joshua D. Drake" writes: On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: I still *strongly* feel the default has to be the non-destructive conservative -1. I don't. Primarily, we must support high availability. It

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Jeff Davis
On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: > "my failover was 12 hours behind when I needed it to be 10 seconds > behind and I lost a $1 million because of downtime of Postgres" The same could be said for warm standby right now. Or Slony-I, for that matter. I think that we can reasonab

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 21:41 +0200, Heikki Linnakangas wrote: > So, you can think of the unobserved xids array as an extension of > ProcArray. The entries are like light-weight PGPROC entries. In fact I > proposed earlier to simply create dummy PGPROC entries instead. Which we don't do because w

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Greg Stark
Put another way: your characterization is no more true than claiming there's no "safe" setting for statement_timeout since a large value means clog could overflow your disk and your tables could bloat. (I note we default statement_timeout to off though) -- Greg On 28 Jan 2009, at 19:56, To

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Greg Stark
(sorry for top posting -- blame apple) I don't see anything "dangerous" with either setting. For use cases where the backup is the primary purpose then killing queries is fine. For use cases where the maching is a reporting machine then saving large amounts of archived logs is fine. Engin

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Aidan Van Dyk
* Tom Lane [090128 15:02]: > Well, those unexpectedly cancelled queries could have represented > critical functionality too. I think this argument calls the entire > approach into question. If there is no safe setting for the parameter > then we need to find a way to not have the parameter. T

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Tom Lane
"Joshua D. Drake" writes: > On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: >> On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: >>> I still *strongly* feel the default has to be the >>> non-destructive conservative -1. >> >> I don't. Primarily, we must support high availability. It i

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Simon Riggs
On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: > I still don't understand why you need unobserved_xids. We don't need > this in normal running, an xid we don't know for certain is committed > is exactly the same as a transaction we know is currently running or > aborted. So why do you nee

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Andrew Sullivan
On Wed, Jan 28, 2009 at 01:49:21PM -0500, Joshua Brindle wrote: > use. The people that need them understand the ramifications of row > filtering and the absence of inaccessible rows won't be surprising. I wish there was even a little bit of evidence that users of a security system may be relied u

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Heikki Linnakangas
Gregory Stark wrote: 6) I still don't understand why you need unobserved_xids. We don't need this in normal running, an xid we don't know for certain is committed is exactly the same as a transaction we know is currently running or aborted. So why do you need it during HS? In normal operation,

Re: [HACKERS] How to get SE-PostgreSQL acceptable

2009-01-28 Thread Tom Lane
Gregory Stark writes: > I don't think partitioning is really the same thing as row-level > security. Of course not, but it seems to me that it can be used to accomplish most of the same practical use-cases. The main gripe about doing it via partitioning is that the user's nose gets rubbed in the

Re: [HACKERS] mingw check hung

2009-01-28 Thread Mark Cave-Ayland
Magnus Hagander wrote: Per discussion I looked at just reverting that part, but that won't work. If we do that, the call to SetEnvironmentVariable() will not be run, which certainly isn't right.. The problem has to be in win32env.c. I originally thought we accidentally called the putenv functio

Re: [HACKERS] Hot Standby (v9d)

2009-01-28 Thread Joshua D. Drake
On Wed, 2009-01-28 at 19:27 +, Simon Riggs wrote: > On Wed, 2009-01-28 at 18:55 +, Gregory Stark wrote: > Agreed. As explained when I published that patch it is deliberately > severe to allow testing of conflict resolution and feedback on it. > > > I still *strongly* feel the default has

  1   2   >