Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-11-03 Thread Dimitri Fontaine
Le vendredi 31 octobre 2008, Tom Lane a écrit : > With the attached patch, SQL functions support returning the results of > INSERT/UPDATE/DELETE RETURNING clauses. Thanks for your work and for having considered user whining in-passing! :) -- dim signature.asc Description: This is a digitally si

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-31 Thread Tom Lane
With the attached patch, SQL functions support returning the results of INSERT/UPDATE/DELETE RETURNING clauses. An INSERT/UPDATE/DELETE statement is always executed to completion before returning (including firing any related triggers or rules), so we always materialize the RETURNING output. When

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-31 Thread Pavel Stehule
2008/10/31 Tom Lane <[EMAIL PROTECTED]>: > "Pavel Stehule" <[EMAIL PROTECTED]> writes: >> RETURN QUERY should be implemented for lazy execution model. And it >> should be fast and not to much dificult. > > Really? Consider what happens if it's inside a loop, or an exception > block, or any other n

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-31 Thread Tom Lane
"Pavel Stehule" <[EMAIL PROTECTED]> writes: > RETURN QUERY should be implemented for lazy execution model. And it > should be fast and not to much dificult. Really? Consider what happens if it's inside a loop, or an exception block, or any other nesting construct. regards

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-31 Thread Pavel Stehule
2008/10/30 Robert Haas <[EMAIL PROTECTED]>: >> With session variables we could implement srf function in plpgsql like >> current C srf function. Like >> >> create or replace function foo() >> returns record as $$ >> #option with_srf_context(datatype of srf context) >> begin >> return row(...)

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-30 Thread Robert Haas
> With session variables we could implement srf function in plpgsql like > current C srf function. Like > > create or replace function foo() > returns record as $$ > #option with_srf_context(datatype of srf context) > begin > return row(...); > end; > $$ language plpgsql; Oh, sure - but what

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-30 Thread Tom Lane
I wrote: > I'm currently going to have a look at just what it would take to support > both lazy and eager evaluation in functions.c (independently of what > syntax, if any, we settle on to expose the choice to the user). If it's > either really awful or really easy we should know that before argui

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Pavel Stehule
2008/10/30 Robert Haas <[EMAIL PROTECTED]>: >> All of this is pie-in-the-sky for PL functions, and I think properly so: >> the whole reason for supporting PLs is to enable doing things that SQL >> does poorly or not at all. So expecting SQL to interoperate very >> closely with them seems impossibl

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Robert Haas
> All of this is pie-in-the-sky for PL functions, and I think properly so: > the whole reason for supporting PLs is to enable doing things that SQL > does poorly or not at all. So expecting SQL to interoperate very > closely with them seems impossible, or at least unreasonably limiting. > The real

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Hannu Krosing
On Wed, 2008-10-29 at 11:58 -0400, Tom Lane wrote: > Dimitri Fontaine <[EMAIL PROTECTED]> writes: > > And I fail to see how the user would control which behavior will get > > chosen, > > Oh, I'm sorry, I didn't realize you misunderstood my syntax example. > I was suggesting that the SQL function

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Tom Lane
Dimitri Fontaine <[EMAIL PROTECTED]> writes: > Le mercredi 29 octobre 2008, Tom Lane a écrit : >> However, I see no >> solution to that problem except function inlining; and if the function >> gets inlined then all this discussion is moot anyhow. > How to inline PLs functions? All of this is p

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Dimitri Fontaine
Le mercredi 29 octobre 2008, Tom Lane a écrit : > Now of course the bigger problem with either this syntax or yours is > that attaching such a property to a function is arguably the Wrong Thing > in the first place. Which one is the best way is likely to depend on > the calling query more than it

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Tom Lane
Dimitri Fontaine <[EMAIL PROTECTED]> writes: > And I fail to see how the user would control which behavior will get chosen, Oh, I'm sorry, I didn't realize you misunderstood my syntax example. I was suggesting that the SQL function manager recognize some optional non-SQL keywords at the start of

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Dimitri Fontaine
Le mercredi 29 octobre 2008, Tom Lane a écrit : > Well, call-per-value is *necessary* for lazy evaluation, but it's not > *sufficient*. You need a function implementation that can suspend and > resume execution, and that's difficult in general. Ok, I think I begin to understand how things are tie

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Tom Lane
Dimitri Fontaine <[EMAIL PROTECTED]> writes: > Le mercredi 29 octobre 2008, Tom Lane a écrit : >> So the fact that it's possible for SQL-language functions is an >> idiosyncrasy of that language, not something we should cram into the >> general CREATE FUNCTION syntax in the vain hope that having

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Dimitri Fontaine
Le mercredi 29 octobre 2008, Tom Lane a écrit : > So the fact that it's possible for SQL-language functions is an > idiosyncrasy of that language, not something we should cram into the > general CREATE FUNCTION syntax in the vain hope that having syntax > might cause an implementation to appear som

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Tom Lane
Dimitri Fontaine <[EMAIL PROTECTED]> writes: > Done this way, the user could also choose for the function to be lazy or to > use a tuplestore whatever the language in which it's written. The odds of this ever happening for any of the PLs are not distinguishable from zero. It might be nice to hav

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Robert Haas
> So my idea would be to have the SQL function behavior choose to return values > either via tuplestore or via value-per-call, depending on the user > setting "generator" or "lazy". > Done this way, the user could also choose for the function to be lazy or to > use a tuplestore whatever the languag

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-29 Thread Dimitri Fontaine
Le mardi 28 octobre 2008, Pavel Stehule a écrit : > 2008/10/28 Dimitri Fontaine <[EMAIL PROTECTED]>: > > Hi, > > > > In the python language, functions that lazily return collections are > > called generators and use the yield keyword instead of return. > > http://www.python.org/doc/2.5.2/tut/node11

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Pavel Stehule
2008/10/28 Dimitri Fontaine <[EMAIL PROTECTED]>: > Hi, > > In the python language, functions that lazily return collections are called > generators and use the yield keyword instead of return. > http://www.python.org/doc/2.5.2/tut/node11.html#SECTION0011100 > > Maybe having such a c

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Dimitri Fontaine
Hi, In the python language, functions that lazily return collections are called generators and use the yield keyword instead of return. http://www.python.org/doc/2.5.2/tut/node11.html#SECTION0011100 Maybe having such a concept in PostgreSQL would allow the user to choose between

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Robert Haas
>> I always thought we considered that a bug though. It sure would be nice if we >> could generate results as needed instead of having to generate them in >> advance >> and store all of them. > I suppose, but short of a fundamental rethink of how PL functions work > that's not going to happen. Th

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Kenneth Marshall
On Tue, Oct 28, 2008 at 09:28:38AM -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Sun, 2008-10-26 at 21:49 -0400, Tom Lane wrote: > >> So I'm concluding that we can easily afford to switch to > >> tuplestore-always operation, especially if we are willing to put any > >> eff

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Sun, 2008-10-26 at 21:49 -0400, Tom Lane wrote: >> So I'm concluding that we can easily afford to switch to >> tuplestore-always operation, especially if we are willing to put any >> effort into tuplestore optimization. (I note that the current >> tuple

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> I suspect it doesn't help you as much as you think. It's always been >> the case that SRFs in FROM-items are fed through a tuplestore, and so >> are plpgsql SRF results. > I always thought we considered that a b

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Simon Riggs
On Sun, 2008-10-26 at 21:49 -0400, Tom Lane wrote: > So I'm concluding that we can easily afford to switch to > tuplestore-always operation, especially if we are willing to put any > effort into tuplestore optimization. (I note that the current > tuplestore code writes 24 bytes per row for this e

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-28 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > "Robert Haas" <[EMAIL PROTECTED]> writes: >> I'm pretty excited by that example. LIMIT/OFFSET is really useful as >> a way of paginating query results for display on a web page (show >> results 1-100, 101-200, etc), and I use it on potentially expensive >>

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-27 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: >>> I thought that the bad case for a tuplestore was if the set returning >>> function was expensive and the user used it with a LIMIT clause. In the >>> tuplestore case you evaluate everything then throw it away. >> >> I'm not terribly excited by that exa

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-27 Thread Robert Haas
>> I thought that the bad case for a tuplestore was if the set returning >> function was expensive and the user used it with a LIMIT clause. In the >> tuplestore case you evaluate everything then throw it away. > > I'm not terribly excited by that example --- but in any case, the real > solution to

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-27 Thread Tom Lane
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > On Sun, Oct 26, 2008 at 09:49:49PM -0400, Tom Lane wrote: >> So I'm concluding that we can easily afford to switch to tuplestore-always >> operation, especially if we are willing to put any effort into tuplestore >> optimization. > I thought tha

Re: [HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-27 Thread Martijn van Oosterhout
On Sun, Oct 26, 2008 at 09:49:49PM -0400, Tom Lane wrote: > So I'm concluding that we can easily afford to switch to tuplestore-always > operation, especially if we are willing to put any effort into tuplestore > optimization. (I note that the current tuplestore code writes 24 bytes > per row for

[HACKERS] WIP patch: convert SQL-language functions to return tuplestores

2008-10-26 Thread Tom Lane
We have an open TODO item to support SQL-language functions that return the output of a RETURNING clause attached to an INSERT/UPDATE/DELETE query within the function. This is something that was left undone in the 8.2 development cycle after this thread analyzed the problem: http://archives.postgr