Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites

2004-11-10 Thread Mark Kirkwood
Probably for a small table, where the machinery of reading the index, followed by checking the table for non-visible tuples is more costly than just scanning the table! regards Mark John Hansen wrote: Why not just change the function all together to 'select $1 from $2 order by $1 desc limit 1;'

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites

2004-11-10 Thread Mark Kirkwood
There seems to be (as Tom indicated) a choice of approaches: i) rewrite max/min querys and then plan 'em ii) provide alternate plans based on presence of certain aggregate types in the query when I first examined this TODO item, I was really thinking about i), but I suspect that ii) is probably

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites

2004-11-10 Thread Jim C. Nasby
Certainly handling only one case is better than none. I just wanted to bring up the multiple aggregate scenario. Also, consider that SELECT min(a), max(a), min(b), max(c) FROM table could be optimized as well (into 4 index scans, assuming a, b, and c all had indexes). I don't think any other agg

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread John Hansen
Why not just change the function all together to 'select $1 from $2 order by $1 desc limit 1;' Is there ANY situation where max(col) as it is, would be faster? ... John ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > What about having a new column in pg_aggregate which would point to a > function that would try to optimize the aggregate's handling? I can't get very excited about this, because how would you make a reasonably stable/narrow API for such a thing? The f

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites

2004-11-10 Thread Bruno Wolff III
On Thu, Nov 11, 2004 at 17:57:42 +1300, Mark Kirkwood <[EMAIL PROTECTED]> wrote: > Your example and ones like : > > SELECT max(foo), count(foo) FROM bar > SELECT max(a.foo1), max(b.foo2) FROM bar1 AS a NATURAL JOIN bar2 AS b > > have made me realize that the scope of "what should be optimized"

[HACKERS] Vacuum info clarification

2004-11-10 Thread Bruce Momjian
The last two lines of VACUUM VERBOSE are: INFO: free space map: 49 relations, 32 pages stored; 784 total pages needed DETAIL: Allocated FSM size: 1000 relations + 2 pages = 182 kB shared memory. I think "total pages needed" should be "total pages used" or "total pages requested". I am

Re: [HACKERS] Increasing the length of pg_stat_activity.current_query...

2004-11-10 Thread Greg Stark
Simon Riggs <[EMAIL PROTECTED]> writes: > My intention was towards a data warehouse situation, and my comments are > only relevant in that context. Possibly 25+% of the user base use this > style of processing. In that case, I expect queries to run for minutes > or hours. I come from the opposit

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites

2004-11-10 Thread Mark Kirkwood
Your example and ones like : SELECT max(foo), count(foo) FROM bar SELECT max(a.foo1), max(b.foo2) FROM bar1 AS a NATURAL JOIN bar2 AS b have made me realize that the scope of "what should be optimized" is somewhat subtle. I am inclined to keep it simple (i.e rather limited) for a first cut, and

Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT variable category

2004-11-10 Thread Greg Stark
Stephan Szabo <[EMAIL PROTECTED]> writes: > > Would that mean I wouldn't be able to change the logging level on the fly at > > all? > > I would think you'd still be able to do it through a security definer > wrapper function owned by a superuser. Oh yeah, well that would be sufficient for my pu

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread Jim C. Nasby
On Thu, Nov 11, 2004 at 11:48:49AM +1300, Mark Kirkwood wrote: > I am looking at implementing this TODO item. e.g. (max case): > > rewrite > SELECT max(foo) FROM bar > as > SELECT foo FROM bar ORDER BY foo DESC LIMIT 1 > if there is an index on bar(foo) Out of curiosity, will you be doing this i

Re: [HACKERS] Reorganization of the translation files

2004-11-10 Thread Aleksander Kmetec
I'm not sure about other translators, but I'd like to see two files: one for the backend and one for the utilities. Translating strings can be a slow and tedious process and completing them all at the same time is not likely. With two files you can choose to translate the tools during one devel

Re: [HACKERS] Reorganization of the translation files

2004-11-10 Thread Aleksander Kmetec
Alvaro Herrera wrote: The biggest downside I see is the long execution time for msgmerge. I can live with that, especially if it will save me translating "out of memory" six or seven times. I'm using poedit which can replace empty translations with previous translated values for the same/similar s

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread Bruno Wolff III
On Wed, Nov 10, 2004 at 22:21:31 -0300, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > On Wed, Nov 10, 2004 at 07:18:59PM -0500, Tom Lane wrote: > > > A more radical way of handling it would be to detect the relevance of an > > indexscan in indxpath.c and generate a special kind of Path node; this

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites

2004-11-10 Thread Mark Kirkwood
Tom Lane wrote: A more radical way of handling it would be to detect the relevance of an indexscan in indxpath.c and generate a special kind of Path node; this would not generalize to other sorts of things as you were hoping, but I'm unconvinced that the mechanism is going to be very general-purpos

Re: [HACKERS] Reorganization of the translation files

2004-11-10 Thread Alvaro Herrera
On Tue, Nov 09, 2004 at 12:00:46PM -0500, Tom Lane wrote: > Peter Eisentraut <[EMAIL PROTECTED]> writes: > > I was thinking about organizing the translation files in a more efficient > > manner. (not for 8.0, obviously) > > [snip] > > And it would easily solve issues like translatable strings > >

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread Alvaro Herrera
On Wed, Nov 10, 2004 at 07:18:59PM -0500, Tom Lane wrote: > A more radical way of handling it would be to detect the relevance of an > indexscan in indxpath.c and generate a special kind of Path node; this > would not generalize to other sorts of things as you were hoping, but > I'm unconvinced th

Re: [HACKERS] sp-gist porting to postgreSQL

2004-11-10 Thread Ramy M. Hassan
On Wed, 10 Nov 2004, Oleg Bartunov wrote: On Wed, 10 Nov 2004, Ramy M. Hassan wrote: I believe that it is still possible to have several index access methods for the same type and the same operations. But this requires that each index access method has its own tuple in the pg_am relation and ther

Re: [HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread Tom Lane
Mark Kirkwood <[EMAIL PROTECTED]> writes: > I am looking at implementing this TODO item. e.g. (max case): > My initial thoughts revolved around extending the existing RULE system > to be able to handle more general types of rewrite - like conditionals > in SELECT rules and rewrites that change e

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > On Thu, 2004-11-11 at 04:11, Tom Lane wrote: >> You can't override a builtin C function that way because there is a >> built-in map from function OID to builtin function address, and it's >> consulted before trying to look in pg_proc. > Then shouldn't we d

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread Neil Conway
On Thu, 2004-11-11 at 04:11, Tom Lane wrote: > You can't override a builtin C function that way because there is a > built-in map from function OID to builtin function address, and it's > consulted before trying to look in pg_proc. > > This behavior is not really open to negotiation Then shouldn'

[HACKERS] multiline CSV fields

2004-11-10 Thread Andrew Dunstan
Darcy Buskermolen has drawn my attention to unfortunate behaviour of COPY CSV with fields containing embedded line end chars if the embedded sequence isn't the same as those of the file containing the CSV data. In that case we error out when reading the data in. This means there are cases where

Re: [HACKERS] Increasing the length of

2004-11-10 Thread Andrew Sullivan
On Wed, Nov 10, 2004 at 05:51:01PM -0500, Andrew Sullivan wrote: > log_statement_after_min_duration (integer) -- which did what Simon > wants. Uh, well, not what Simon wants, of course, but which gave us a useful capability anyway. I agree that the full-bore profiling for the DBA would be awful

Re: [HACKERS] Increasing the length of

2004-11-10 Thread Andrew Sullivan
On Wed, Nov 10, 2004 at 09:52:17PM +, Simon Riggs wrote: > On Wed, 2004-11-10 at 21:48, Richard Huxton wrote: > > > > Isn't that: > > log_min_duration_statement (integer) > > That gets written when a statement completes, not during execution. I've been following this thread, and I was thinki

[HACKERS] MAX/MIN optimization via rewrite (plus query rewrites generally)

2004-11-10 Thread Mark Kirkwood
I am looking at implementing this TODO item. e.g. (max case): rewrite SELECT max(foo) FROM bar as SELECT foo FROM bar ORDER BY foo DESC LIMIT 1 if there is an index on bar(foo) Suggestions about the most suitable point in the parser/planner stage to perform this sort of rewrite would be most welco

[HACKERS] Beta time early next week

2004-11-10 Thread Bruce Momjian
FYI, I am going to need a few more days to apply patches submitted in the past few days and Magnus needs a few more days to fix the windows signal problems. So, rather than planning a beta for later this week, I think we should focus on early next week. -- Bruce Momjian

Re: [HACKERS] Increasing the length of

2004-11-10 Thread Simon Riggs
Hi Richard On Wed, 2004-11-10 at 21:48, Richard Huxton wrote: > Simon Riggs wrote: > > On Mon, 2004-11-08 at 22:32, Tom Lane wrote: > > > >>Another relevant question is why you are expecting to get this > >>information through pgstats and not by looking in the postmaster log. > > > > > > This i

Re: [HACKERS] Increasing the length of

2004-11-10 Thread Richard Huxton
Simon Riggs wrote: On Mon, 2004-11-08 at 22:32, Tom Lane wrote: Another relevant question is why you are expecting to get this information through pgstats and not by looking in the postmaster log. This is only available if you log all queries, which isn't normally done while you are in production.

Re: [HACKERS] Increasing the length of

2004-11-10 Thread Simon Riggs
On Wed, 2004-11-10 at 20:25, Jan Wieck wrote: > On 11/8/2004 5:32 PM, Tom Lane wrote: > > > Another relevant question is why you are expecting to get this > > information through pgstats and not by looking in the postmaster log. > > The pgstats were originally designed to give "hints" for tuning.

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread Tom Lane
"John Hansen" <[EMAIL PROTECTED]> writes: >> You can't override a builtin C function that way because >> there is a built-in map from function OID to builtin function >> address, and it's consulted before trying to look in pg_proc. > That doesn't make sense, since if I delete the entry from pg_p

Re: [HACKERS] Increasing the length of pg_stat_activity.current_query...

2004-11-10 Thread Jan Wieck
On 11/8/2004 5:32 PM, Tom Lane wrote: Another relevant question is why you are expecting to get this information through pgstats and not by looking in the postmaster log. The pgstats were originally designed to give "hints" for tuning. That's why they cover cache hits vs. misses per table and numb

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread John Hansen
> > The builtin function is still called, not the userdefined > function for > > 'C' language functions. > > You can't override a builtin C function that way because > there is a built-in map from function OID to builtin function > address, and it's consulted before trying to look in pg_proc.

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread Tom Lane
elein <[EMAIL PROTECTED]> writes: > Isn't there a load/unload function for the .so that would work > in this case? Won't affect the fmgr_builtins table ... he wanted to replace a builtin, not a previously-dynamically-loaded function. regards, tom lane

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread elein
Isn't there a load/unload function for the .so that would work in this case? --elein On Wed, Nov 10, 2004 at 12:11:27PM -0500, Tom Lane wrote: > John Hansen <[EMAIL PROTECTED]> writes: > > When doing CREATE or REPLACE FUNCTION of a builtin function, it seems to > > have no effect if its in the 'C

Re: [HACKERS] sp-gist porting to postgreSQL

2004-11-10 Thread Tom Lane
Oleg Bartunov <[EMAIL PROTECTED]> writes: > AFAIK, posgresql doesnt' supports several indices for the same type. I think what's really being asked for is several operator classes for the same type. This most certainly *is* possible. regards, tom lane ---

Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT variable category

2004-11-10 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> I'd like to propose that we get rid of GUC's USERLIMIT category and >> convert all the variables in it to plain SUSET. > Would that mean I wouldn't be able to change the logging level on the fly at > all? No, it woul

Re: [HACKERS] CREATE or REPLACE function pg_catalog.*

2004-11-10 Thread Tom Lane
John Hansen <[EMAIL PROTECTED]> writes: > When doing CREATE or REPLACE FUNCTION of a builtin function, it seems to > have no effect if its in the 'C" language. SQL functions seem to work, > but as neilc pointed out, it may be due to the SQL function being > inlined. > The builtin function is still

Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT variable category

2004-11-10 Thread Tom Lane
Andrew McMillan <[EMAIL PROTECTED]> writes: > When tracking down gnarly problems in heavily multi-user applications > enabling higher log levels at selective points has the potential to help > _a lot_ with diagnostic detail, without smothering you in _every_ > detail. Sure. As I pointed out in th

Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT variable

2004-11-10 Thread Stephan Szabo
On Tue, 10 Nov 2004, Greg Stark wrote: > > Tom Lane <[EMAIL PROTECTED]> writes: > > > I'd like to propose that we get rid of GUC's USERLIMIT category and > > convert all the variables in it to plain SUSET. In my mind, USERLIMIT > > is a failed experiment: it's way too complicated, and it still do

Re: [HACKERS] sp-gist porting to postgreSQL

2004-11-10 Thread Oleg Bartunov
On Wed, 10 Nov 2004, Ramy M. Hassan wrote: I believe that it is still possible to have several index access methods for the same type and the same operations. But this requires that each index access method has its own tuple in the pg_am relation and therefore postgresql recognizes it by itself. Bu

Re: [HACKERS] sp-gist porting to postgreSQL

2004-11-10 Thread Ramy M. Hassan
I believe that it is still possible to have several index access methods for the same type and the same operations. But this requires that each index access method has its own tuple in the pg_am relation and therefore postgresql recognizes it by itself. But this is not the case with GiST based indi

Re: [HACKERS] [Pgsphere-dev] GIST index concurrency concern

2004-11-10 Thread Oleg Bartunov
Patrick, you didn't say us about your setup. Have you proved you've seen locking issue for reading ? Are you sure you have no any locks in your code ? Any tests demonstrated your problem would be great. Oleg On Tue, 9 Nov 2004, Patrick Clery wrote: Oleg, Daniel and I have both been collabor

Re: [HACKERS] A modest proposal: get rid of GUC's USERLIMIT

2004-11-10 Thread Andrew McMillan
On Tue, 2004-11-09 at 17:27 -0500, Tom Lane wrote: > > What the USERLIMIT code tries to do is allow non-superusers to > "increase" but not "decrease" the logging verbosity for their sessions > only. (For instance, a non-superuser could turn log_duration on, but > can't turn it off if the DBA has

Re: [HACKERS] sp-gist porting to postgreSQL

2004-11-10 Thread Oleg Bartunov
On Wed, 10 Nov 2004, Ramy M. Hassan wrote: Oleg, Thanks for your prompt reply. Actually, I am able to create a new access method for testing and add an operator class for the type "integer" using the new access method. Then created a table with two integer fields, one indexed using the new access m