Tom Lane writes:
> Applied with revisions as discussed.
Excellent ;)
Now if you wanted a small option to play with to test the extensibility
of the new system, should I propose DEFAULT '\D' (e.g.)?
Regards,
--
dim
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
Emmanuel Cecchet writes:
> [ generic copy options patch ]
Applied with revisions as discussed.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
On Mon, Sep 21, 2009 at 1:51 PM, Tom Lane wrote:
> I wrote:
>> As far as I can tell, the majority opinion is to use "format csv"
>
> BTW, if we're going to do that, shouldn't the "binary" option instead
> be spelled "format binary"?
Good catch, +1.
...Robert
--
Sent via pgsql-hackers mailing l
Tom Lane wrote:
I wrote:
As far as I can tell, the majority opinion is to use "format csv"
BTW, if we're going to do that, shouldn't the "binary" option instead
be spelled "format binary"?
Looking at the doc, it looks like FORMAT should be mandatory and be
either text, binary or cs
I wrote:
> As far as I can tell, the majority opinion is to use "format csv"
BTW, if we're going to do that, shouldn't the "binary" option instead
be spelled "format binary"?
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
Emmanuel Cecchet writes:
> The easiest for both implementation and documentation might just be to
> have a matrix of options.
> Each option has a row and a column in the matrix. The intersection of a
> row and a column is set to 0 if options are not compatible and set to 1
> if it is. This way
om Lane" ,
"Emmanuel Cecchet" , "Josh Berkus"
, "PostgreSQL-development"
Enviados: Domingo, 20 de Septiembre 2009 16:24:28 GMT -10:00 Hawai
Asunto: Re: [HACKERS] generic copy options
The easiest for both implementation and documentation might just be to
have a matrix of
The easiest for both implementation and documentation might just be to
have a matrix of options.
Each option has a row and a column in the matrix. The intersection of a
row and a column is set to 0 if options are not compatible and set to 1
if it is. This way we are sure to capture all possible
On Sun, Sep 20, 2009 at 2:25 PM, Emmanuel Cecchet wrote:
> Tom Lane wrote:
>> Emmanuel Cecchet writes:
>>>
>>> Here you will force every format to use the same set of options
>>
>> How does this "force" any such thing?
>>
>
> As far as I understand it, every format will have to handle every forma
Tom Lane writes:
> But anyhow I think we have run through all the arguments, and it's
> time for some votes from other people.
Same option names whatever the format please. We know the context is
important and people will be able to understand that header does not
refer exactly to the same proces
Emmanuel Cecchet writes:
> So maybe a tradeoff is to differentiate format specific options like in:
> (delimiter '.', format csv, format_header, format_escape...)
> This should also make clear if someone develops a new format what
> options need to be addressed.
I think that distinction would e
Tom Lane wrote:
Emmanuel Cecchet writes:
Here you will force every format to use the same set of options
How does this "force" any such thing?
As far as I understand it, every format will have to handle every format
options that may exist so that they can either implement it or th
Emmanuel Cecchet writes:
> Here you will force every format to use the same set of options
How does this "force" any such thing?
> and if
> someone introduces a new option, you will have to modify all other
> formats to make sure they throw an error telling the user that this
> option is not
Tom Lane wrote:
No, I don't think so. Suppose I write
COPY ... (xml_header on)
If HEADER isn't actually an option supported by XML format, what I will
get here is an "unknown option" error, which conveys just about nothing
--- is it really an unsupported combination, or did I just miss
Emmanuel Cecchet writes:
> That would assume that the semantic of the other options (header, quote,
> espace, ...) is exactly the same for each format. Otherwise this will be
> a nightmare to document.
Well, sure, we should pick a different name for an option that means
something significantly
Tom Lane wrote:
Robert Haas writes:
... A related question is whether we
should replace the "CSV" option with a "FORMAT" option for which one
of the possible choices is "CSV" (much as we did with EXPLAIN).
That might be a good idea --- otherwise we'd need some ad-hoc code
to check th
Robert Haas writes:
> ... A related question is whether we
> should replace the "CSV" option with a "FORMAT" option for which one
> of the possible choices is "CSV" (much as we did with EXPLAIN).
That might be a good idea --- otherwise we'd need some ad-hoc code
to check that only one format opt
On Sat, Sep 19, 2009 at 3:10 PM, Tom Lane wrote:
> Emmanuel Cecchet writes:
>> [ latest patch version ]
>
> Do we have consensus on the syntax for this patch? In particular,
> what about the question of adding CSV_ to all the CSV-specific option
> names? Emmanuel argued that this is necessary t
Emmanuel Cecchet writes:
> [ generic copy options patch ]
I went ahead and applied the psql \copy part of this, since that saves
us a couple hundred lines of code regardless of what may or may not
happen on the backend side. There were a couple of minor bugs, and
I also found a few other simplif
Emmanuel Cecchet writes:
> [ latest patch version ]
Do we have consensus on the syntax for this patch? In particular,
what about the question of adding CSV_ to all the CSV-specific option
names? Emmanuel argued that this is necessary to avoid confusion if
we someday introduce other copy formats
Josh Berkus wrote:
Nope, but it was on the checklist and I was being thorough.
That's a good thing. I was just seeing if I needed to get involved in
performance testing.
That would be good to have more people test the autopartitioning feature
in COPY. If you want to be involved in per
On Fri, Sep 18, 2009 at 10:31:21AM -0700, Josh Berkus wrote:
>
> > Nope, but it was on the checklist and I was being thorough.
>
> That's a good thing. I was just seeing if I needed to get involved in
> performance testing.
>
> --
> Josh Berkus
> PostgreSQL Experts Inc.
> www.pgexperts.com
I
> Nope, but it was on the checklist and I was being thorough.
That's a good thing. I was just seeing if I needed to get involved in
performance testing.
--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make chan
On Fri, Sep 18, 2009 at 10:21:08AM -0700, Josh Berkus wrote:
> On 9/17/09 3:54 PM, Greg Smith wrote:
> > On Thu, 17 Sep 2009, Dan Colish wrote:
> >
> >> - Performance appears to be the same although I don't have a good
> >> way for
> >> testing this at the moment
> >
> > Here's what I d
On 9/17/09 3:54 PM, Greg Smith wrote:
> On Thu, 17 Sep 2009, Dan Colish wrote:
>
>> - Performance appears to be the same although I don't have a good
>> way for
>> testing this at the moment
>
> Here's what I do to generate simple COPY performance test cases:
Is there any reason to thi
Robert Haas wrote:
On Thu, Sep 17, 2009 at 8:31 PM, Dan Colish wrote:
Ok, so I ran something like you suggested and did a simple copy from an
empty file to just test the parsing. I have the COPY statement run 3733
times in the transaction block and did the select timestamps, but I
still on
On Thu, Sep 17, 2009 at 8:31 PM, Dan Colish wrote:
> Ok, so I ran something like you suggested and did a simple copy from an
> empty file to just test the parsing. I have the COPY statement run 3733
> times in the transaction block and did the select timestamps, but I
> still only was a few millis
On Thu, Sep 17, 2009 at 07:45:45PM -0400, Andrew Dunstan wrote:
>
>
> Dan Colish wrote:
>> CREATE TABLE
>> INSERT 0 10
>> Timing is on.
>> COPY 10
>> Time: 83.273 ms
>> BEGIN
>> Time: 0.412 ms
>> TRUNCATE TABLE
>> Time: 0.357 ms
>> COPY 1000
Dan Colish wrote:
CREATE TABLE
INSERT 0 10
Timing is on.
COPY 10
Time: 83.273 ms
BEGIN
Time: 0.412 ms
TRUNCATE TABLE
Time: 0.357 ms
COPY 10
Time: 140.911 ms
COMMIT
Time: 4.909 ms
On that particular patch, as Robert mentioned, only the parsing has changed.
In the case of \copy, the parsing is much lighter than before in psql
(remains the same in the server). The bigger the COPY operation the less
you will see the impact of the parsing since it is done only once for
the e
On Thu, Sep 17, 2009 at 07:10:35PM -0400, Andrew Dunstan wrote:
>
>
> Greg Smith wrote:
>> On Thu, 17 Sep 2009, Dan Colish wrote:
>>
>>> - Performance appears to be the same although I don't have a good
>>> way for
>>> testing this at the moment
>>
>> Here's what I do to generate simple
Greg Smith wrote:
On Thu, 17 Sep 2009, Dan Colish wrote:
- Performance appears to be the same although I don't have a good
way for
testing this at the moment
Here's what I do to generate simple COPY performance test cases:
CREATE TABLE t (i integer);
INSERT INTO t SELECT x FROM
On Thu, Sep 17, 2009 at 6:54 PM, Greg Smith wrote:
> On Thu, 17 Sep 2009, Dan Colish wrote:
>
>> - Performance appears to be the same although I don't have a good
>> way for
>> testing this at the moment
>
> Here's what I do to generate simple COPY performance test cases:
>
> CREAT
On Thu, 17 Sep 2009, Dan Colish wrote:
- Performance appears to be the same although I don't have a good way
for
testing this at the moment
Here's what I do to generate simple COPY performance test cases:
CREATE TABLE t (i integer);
INSERT INTO t SELECT x FROM generate_seri
Hi,
I have read through the patch a few times and it looks OK. The
additions to the COPY syntax work as expected and as agreed upon
based on the thread. Below are some points from my checklist.
- Patch applies cleanly
- Included new tests and documentation
Here you go.
Emmanuel
Dan Colish wrote:
On Thu, Sep 17, 2009 at 02:56:07PM -0400, Emmanuel Cecchet wrote:
Dan,
My bad, I copy/pasted the hard link in output/copy.source instead of
@abs_build...@.
Here is a complete version of the patch with the fix on output/copy.source.
Emmanuel
On Thu, Sep 17, 2009 at 02:56:07PM -0400, Emmanuel Cecchet wrote:
> Dan,
>
> My bad, I copy/pasted the hard link in output/copy.source instead of
> @abs_build...@.
> Here is a complete version of the patch with the fix on output/copy.source.
>
> Emmanuel
Emmanuel,
Thanks for ge
On Thu, Sep 17, 2009 at 02:56:07PM -0400, Emmanuel Cecchet wrote:
> Dan,
>
> My bad, I copy/pasted the hard link in output/copy.source instead of
> @abs_build...@.
> Here is a complete version of the patch with the fix on output/copy.source.
>
> Emmanuel
>
Hooray, that works just fine now. I gue
Dan,
My bad, I copy/pasted the hard link in output/copy.source instead of
@abs_build...@.
Here is a complete version of the patch with the fix on output/copy.source.
Emmanuel
On Thu, Sep 17, 2009 at 11:07:33AM -0400, Emmanuel Cecchet wrote:
Tom Lane wrote:
I wonder though if we cou
On Thu, Sep 17, 2009 at 11:07:33AM -0400, Emmanuel Cecchet wrote:
> Tom Lane wrote:
>> I wonder though if we couldn't simplify matters. Offhand it seems to me
>> that psql doesn't need to validate the command's syntax fully. All it
>> really needs to do is find the target filename and replace it w
Tom Lane wrote:
I wonder though if we couldn't simplify matters. Offhand it seems to me
that psql doesn't need to validate the command's syntax fully. All it
really needs to do is find the target filename and replace it with
STDIN/STDOUT. Could we have it just treat the remainder of the line
li
Emmanuel Cecchet writes:
> Does that mean that we can drop the 7.3 syntax or should we just keep it?
I wouldn't object to dropping the 7.3 syntax now, especially if we're
about to introduce a new syntax and deprecate the old one ...
regards, tom lane
--
Sent via pgsql-h
Tom Lane wrote:
Emmanuel Cecchet writes:
Tom Lane wrote:
psql has MORE need to support old syntax than the backend does, because
it's supposed to work against old servers.
Well, I wonder how many users just upgrade psql vs upgrade the server.
We have established a proje
Emmanuel Cecchet writes:
> Tom Lane wrote:
>> psql has MORE need to support old syntax than the backend does, because
>> it's supposed to work against old servers.
>>
> Well, I wonder how many users just upgrade psql vs upgrade the server.
We have established a project policy that psql backslash
2009/9/17 Emmanuel Cecchet :
> Pavel Stehule wrote:
>>>
>>> Well, I wonder how many users just upgrade psql vs upgrade the server. I
>>> was
>>> thinking that when users perform a database upgrade their application
>>> often
>>> remain the same and therefore the server needs to support the old synt
Emmanuel Cecchet wrote:
> I just wonder how many users are using a single psql to manage
> multiple server instances of different older versions.
I do that, but I do try to keep all the active versions on my machine,
so that I can use one which exactly matches any of our 100 servers
when it m
Pavel Stehule wrote:
Well, I wonder how many users just upgrade psql vs upgrade the server. I was
thinking that when users perform a database upgrade their application often
remain the same and therefore the server needs to support the old syntax.
Unless you are upgrading a machine where a bunch
2009/9/17 Emmanuel Cecchet :
> Tom Lane wrote:
>>>
>>> While I understand the need for the server to still support the syntax,
>>> is it necessary for newer version of psql to support the old syntax?
>>>
>>
>> psql has MORE need to support old syntax than the backend does, because
>> it's supposed
Tom Lane wrote:
While I understand the need for the server to still support the syntax,
is it necessary for newer version of psql to support the old syntax?
psql has MORE need to support old syntax than the backend does, because
it's supposed to work against old servers.
Well, I wonder
Pavel,
I am not sure about syntax change. Isn't better solve this problem
well. This is too simple solution. I thinking, so we able to add new
parser for COPY statement and share this paraser between SQL and psql.
Refactoring COPY to put new parsers seems to be another project. The
idea here
Tom Lane wrote:
psql has MORE need to support old syntax than the backend does, because
it's supposed to work against old servers.
I wonder though if we couldn't simplify matters. Offhand it seems to me
that psql doesn't need to validate the command's syntax fully. All it
really needs to do
Emmanuel Cecchet writes:
> Robert Haas wrote:
>>> When we decide to drop the old syntax (in 8.6?), we will be able to clean a
>>> lot especially in psql.
>> Considering that we are still carrying syntax that was deprecated in
>> 7.3, I don't think it's likely that we'll phase out the present synt
Robert Haas wrote:
I don't think the way the doc changes are formatted is consistent with
what we've done elsewhere. I think that breaking the options out as a
separate block could be OK (because otherwise they have to be
duplicated between COPY TO and COPY FROM) but it should be done more
like
On Wed, Sep 16, 2009 at 6:43 PM, Emmanuel Cecchet wrote:
> Here is a new version of the patch with an updated doc and psql.
Thanks, that's great.
I don't think the way the doc changes are formatted is consistent with
what we've done elsewhere. I think that breaking the options out as a
separate
Robert,
Here is a new version of the patch with an updated doc and psql.
I changed the name of the CSV options to prefix them with csv_ to avoid
confusion with any future options. I also had to change the grammar to
allow '*' as a parameter (needed for cvs_force_quote).
When we decide to drop
Robert Haas writes:
> On Mon, Sep 14, 2009 at 3:25 PM, Emmanuel Cecchet wrote:
>> I have never looked at the psql code but that could be a good way to get
>> started on that. If you can point me at where to look at, I'll give it a
>> try.
> I don't know either off the top of my head, but I'll go
On Mon, Sep 14, 2009 at 3:25 PM, Emmanuel Cecchet wrote:
> Robert Haas wrote:
>>
>> On Mon, Sep 14, 2009 at 2:51 PM, Emmanuel Cecchet
>> wrote:
>>
>>>
>>> This looks good. Shoud I try to elaborate on that for the patch with
>>> error
>>> logging and autopartitioning in COPY?
>>>
>>
>> That make s
Robert Haas wrote:
On Mon, Sep 14, 2009 at 2:51 PM, Emmanuel Cecchet wrote:
This looks good. Shoud I try to elaborate on that for the patch with error
logging and autopartitioning in COPY?
That make sense to me. You shouldn't need to do anything else in
gram.y; whatever you want to a
On Mon, Sep 14, 2009 at 2:51 PM, Emmanuel Cecchet wrote:
> This looks good. Shoud I try to elaborate on that for the patch with error
> logging and autopartitioning in COPY?
That make sense to me. You shouldn't need to do anything else in
gram.y; whatever you want to add should just involve chan
This looks good. Shoud I try to elaborate on that for the patch with
error logging and autopartitioning in COPY?
manu
Robert Haas wrote:
Here's a half-baked proof of concept for the above approach. This
probably needs more testing than I've given it, and I haven't
attempted to fix the psql pa
On Fri, Sep 11, 2009 at 5:45 PM, Robert Haas wrote:
> On Fri, Sep 11, 2009 at 5:32 PM, Tom Lane wrote:
>> Robert Haas writes:
>>> The biggest problem I have with this change is that it's going to
>>> massively break anyone who is using the existing COPY syntax.
>>
>> Why? We'd certainly still s
61 matches
Mail list logo
