Re: [HACKERS] COPY (query) TO file

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 04:47:40PM +0200, Harald Fuchs wrote: > In article <[EMAIL PROTECTED]>, > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > > Mark Woodward wrote: > >> Tom had posted a question about file compression with copy. I thought > >> about it, and I want to through this out and see i

Re: [HACKERS] COPY (query) TO file

2006-06-06 Thread Harald Fuchs
In article <[EMAIL PROTECTED]>, Andrew Dunstan <[EMAIL PROTECTED]> writes: > Mark Woodward wrote: >> Tom had posted a question about file compression with copy. I thought >> about it, and I want to through this out and see if anyone things it is a >> good idea. >> >> Currently, the COPY command o

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Andrew Dunstan
Greg Stark wrote: It would have been awfully nice to do be able to do SELECT ... FROM (VALUES (a,b,c),(d,e,f),(g,h,i)) The trouble with supporting it for any case other than INSERT is that you have to work out what the column datatypes of the construct ought to be. This is the same as

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > The interesting point here is that a is defined as a > parenthesized , which means that you ought to be able to > use a parenthesized VALUES list anyplace you could use a parenthesized > SELECT. So FROM lists, IN clauses, = ANY and friends, etc all really o

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Andrew Dunstan
Tom Lane wrote: Greg Stark <[EMAIL PROTECTED]> writes: PFC <[EMAIL PROTECTED]> writes: MySQL already does this for INSERT : INSERT INTO x (a,b) VALUES (1,2), (3,4), (5,6)...; The above syntax is SQL-standard, so we ought to support it sometime, performance benefits or no.

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> Supporting VALUES only in INSERT would be relatively trivial BTW, >> but the spec actually thinks it should be allowed as a >> in FROM ... > How does that syntax work? If you look at SQL92, INSERT ... VALUES is actu

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Supporting VALUES only in INSERT would be relatively trivial BTW, > but the spec actually thinks it should be allowed as a > in FROM ... How does that syntax work? INSERT INTO x (a,b) from select x,y,z from t from select x2,y2,z2 from t ? doesn't seem to

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Tino Wildenhain
Mark Woodward wrote: >> Mark Woodward wrote: ... >>> This runs completely in the background and can serve as a running >>> backup. >> And you are sure it would be much faster then a server local running >> psql just dumping the result of a query? > > No I can't be sure of that at all, but Th

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread PFC
MySQL already does this for INSERT : INSERT INTO x (a,b) VALUES (1,2), (3,4), (5,6)...; Does MySQL really let you stream that? Trying to do syntax like that in Postgres wouldn't work because the parser would try to build up a parse tree for the whole statement before runnin

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > PFC <[EMAIL PROTECTED]> writes: >> MySQL already does this for INSERT : >> INSERT INTO x (a,b) VALUES (1,2), (3,4), (5,6)...; > Does MySQL really let you stream that? Trying to do syntax like that in > Postgres wouldn't work because the parser would try to

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread Greg Stark
PFC <[EMAIL PROTECTED]> writes: > > I was also vaguely pondering whether all the DDL commands could be > > generalized to receive or send COPY formatted data for repeated execution. > > It would be neat to be able to prepare an UPDATE with placeholders and > > stream data in COPY format as parame

Re: [HACKERS] COPY (query) TO file

2006-06-03 Thread PFC
I was also vaguely pondering whether all the DDL commands could be generalized to receive or send COPY formatted data for repeated execution. It would be neat to be able to prepare an UPDATE with placeholders and stream data in COPY format as parameters to the UPDATE to execute it thousand

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tom Lane
Tino Wildenhain <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> You're not seriously suggesting we reimplement evaluation of WHERE clauses >> on the client side, are you? > no, did I? But what is wrong with something like: > \COPY 'SELECT foo,bar,baz FROM footable WHERE baz=5 ORDER BY foo' TO >

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Joshua D. Drake
VIEW. Not to be a sour apple or anything but I don't see how any of this is needed in the backend since we can easily use Psql to do it, or pretty much any other tool. There is an important difference between a capability in the backend vs one synthesized in the frontend. After much patience

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Joshua D. Drake
Mark Woodward wrote: Allow COPY to output from views Another idea would be to allow actual SELECT statements in a COPY. Personally I strongly favor the second option as being more flexible than the first. I second that - allowing arbitrary SELECT statements as a COPY source seems much more p

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Joshua D. Drake
Not to be a sour apple or anything but I don't see how any of this is needed in the backend since we can easily use Psql to do it, or pretty much any other tool. There is an important difference between a capability in the backend vs one synthesized in the frontend. And that would be? The su

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> >>> Allow COPY to output from views >>> Another idea would be to allow actual SELECT statements in a COPY. >>> >>> Personally I strongly favor the second option as being more flexible >>> than the first. >> >> >> I second that - allowing arbitrary SELECT statements as a COPY source >> seems muc

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Joshua D. Drake
Allow COPY to output from views Another idea would be to allow actual SELECT statements in a COPY. Personally I strongly favor the second option as being more flexible than the first. I second that - allowing arbitrary SELECT statements as a COPY source seems much more powerful and flexibl

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: > ... > pg_dump -t mytable | psql -h target -c "COPY mytable FROM STDIN" With a more selective copy, you can use pretty much this mechanism to limit a copy to a sumset of the records in a table. >>> Ok, but why not just implement this into pg_dump or psql?

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Greg Stark
Tino Wildenhain <[EMAIL PROTECTED]> writes: > Tom Lane wrote: > > Tino Wildenhain <[EMAIL PROTECTED]> writes: > >> Ok, but why not just implement this into pg_dump or psql? > >> Why bother the backend with that functionality? > > > > You're not seriously suggesting we reimplement evaluation of WH

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread David Fetter
On Fri, Jun 02, 2006 at 09:56:07AM -0400, Andrew Dunstan wrote: > Mark Woodward wrote: > >Tom had posted a question about file compression with copy. I thought > >about it, and I want to through this out and see if anyone things it is a > >good idea. > > > >Currently, the COPY command only copies a

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tino Wildenhain
Mark Woodward wrote: ... >>> pg_dump -t mytable | psql -h target -c "COPY mytable FROM STDIN" >>> >>> With a more selective copy, you can use pretty much this mechanism to >>> limit a copy to a sumset of the records in a table. >> Ok, but why not just implement this into pg_dump or psql? >> Why bo

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tino Wildenhain
Tom Lane wrote: > Tino Wildenhain <[EMAIL PROTECTED]> writes: >> Ok, but why not just implement this into pg_dump or psql? >> Why bother the backend with that functionality? > > You're not seriously suggesting we reimplement evaluation of WHERE clauses > on the client side, are you? no, did I? Bu

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: > ... > create table as select ...; followed by a copy of that table > if it really is faster then just the usual select & fetch? Why "create table?" >>> Just to simulate and time the proposal. >>> SELECT ... already works over the network and if COPY from a >>>

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tom Lane
Tino Wildenhain <[EMAIL PROTECTED]> writes: > Ok, but why not just implement this into pg_dump or psql? > Why bother the backend with that functionality? You're not seriously suggesting we reimplement evaluation of WHERE clauses on the client side, are you? regards, tom la

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tino Wildenhain
Mark Woodward wrote: ... create table as select ...; followed by a copy of that table if it really is faster then just the usual select & fetch? >>> Why "create table?" >> Just to simulate and time the proposal. >> SELECT ... already works over the network and if COPY from a >> select (wh

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Alvaro Herrera
Neil Conway wrote: > On Fri, 2006-06-02 at 09:56 -0400, Andrew Dunstan wrote: > > Allow COPY to output from views > > FYI, there is a patch for this floating around -- I believe it was > posted to -patches a few months back. I have it. The pieces of it than I can use to implement the idea belo

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Neil Conway
On Fri, 2006-06-02 at 09:56 -0400, Andrew Dunstan wrote: > Allow COPY to output from views FYI, there is a patch for this floating around -- I believe it was posted to -patches a few months back. > Another idea would be to allow actual SELECT statements in a COPY. > > Personally I strongly f

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Stefan Kaltenbrunner
Andrew Dunstan wrote: > Mark Woodward wrote: > >> Tom had posted a question about file compression with copy. I thought >> about it, and I want to through this out and see if anyone things it is a >> good idea. >> >> Currently, the COPY command only copies a table, what if it could operate >> with

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: >>> Mark Woodward wrote: Tom had posted a question about file compression with copy. I thought about it, and I want to through this out and see if anyone things it is a good idea. Currently, the COPY command only copies a table, what if it c

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tino Wildenhain
Mark Woodward wrote: >> Mark Woodward wrote: >>> Tom had posted a question about file compression with copy. I thought >>> about it, and I want to through this out and see if anyone things it is >>> a >>> good idea. >>> >>> Currently, the COPY command only copies a table, what if it could >>> opera

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Andrew Dunstan
Mark Woodward wrote: Tom had posted a question about file compression with copy. I thought about it, and I want to through this out and see if anyone things it is a good idea. Currently, the COPY command only copies a table, what if it could operate with a query, as: COPY (select * from mytable

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
> Mark Woodward wrote: >> Tom had posted a question about file compression with copy. I thought >> about it, and I want to through this out and see if anyone things it is >> a >> good idea. >> >> Currently, the COPY command only copies a table, what if it could >> operate >> with a query, as: >> >>

Re: [HACKERS] COPY (query) TO file

2006-06-02 Thread Tino Wildenhain
Mark Woodward wrote: > Tom had posted a question about file compression with copy. I thought > about it, and I want to through this out and see if anyone things it is a > good idea. > > Currently, the COPY command only copies a table, what if it could operate > with a query, as: > > COPY (select

[HACKERS] COPY (query) TO file

2006-06-02 Thread Mark Woodward
Tom had posted a question about file compression with copy. I thought about it, and I want to through this out and see if anyone things it is a good idea. Currently, the COPY command only copies a table, what if it could operate with a query, as: COPY (select * from mytable where foo='bar') as BA