Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Christopher Browne
[EMAIL PROTECTED] ("Joolz") writes: > Hello everyone, > > I'm building a postgresql db which will have to get lots of data > from "the outside" (customers, that is). The db has lots of > constraints, and I'm sure that our customers will offer lots of > invalid information. We receive the informati

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Mike Rylander
On Fri, 4 Feb 2005 13:56:23 +0100 (CET), Joolz <[EMAIL PROTECTED]> wrote: > If is has to be perl, so be it, although I'm not a big fan. Do you > think this is possible in python? > Sure. I just suggested Perl since that's my QnD tool of choice. -- Mike Rylander [EMAIL PROTECTED] GPLS -- PINES

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Sean Davis
On Feb 4, 2005, at 8:27 AM, Joolz wrote: Csaba Nagy zei: [snip] I'm afraid this is a bit too indirect IMHO. As I want to know the line number in which an error occurs, I would have to traverse the error-tolerant table with limit 1 offset N, and report N when an error occurs, hoping that the row ord

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Sean Davis
On Feb 4, 2005, at 8:30 AM, Joolz wrote: Sean Davis zei: I use a trigger on tables with foreign key references to either ignore the insert row or insert an appropriate matching row in the referenced Thanks Sean, but in my situation I don't want the database to be so versatile as to handle all the e

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Joolz
Sean Davis zei: > I use a trigger on tables with foreign key references to either > ignore > the insert row or insert an appropriate matching row in the > referenced > table, if it does not exist. In the function, I just raise a notice > that I am doing this. This is a simple example: > create or

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Joolz
Csaba Nagy zei: > [snip] >> I'm afraid this is a bit too indirect IMHO. As I want to know the >> line number in which an error occurs, I would have to traverse the >> error-tolerant table with limit 1 offset N, and report N when an >> error occurs, hoping that the row order is identical to the lin

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Csaba Nagy
[snip] > I'm afraid this is a bit too indirect IMHO. As I want to know the > line number in which an error occurs, I would have to traverse the > error-tolerant table with limit 1 offset N, and report N when an > error occurs, hoping that the row order is identical to the line > order in the csv fi

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Sean Davis
I use a trigger on tables with foreign key references to either ignore the insert row or insert an appropriate matching row in the referenced table, if it does not exist. In the function, I just raise a notice that I am doing this. This is a simple example: create or replace function tgf_inser

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Joolz
Michael Glaesemann zei: > > On Feb 4, 2005, at 21:32, Joolz wrote: > >> What I need is an import where all valid lines from the csv files >> are read into the db, and I also get a logfile for all invalid >> lines, stating the line number plus the pg error message so I can >> see which constraint wa

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Michael Glaesemann
On Feb 4, 2005, at 21:32, Joolz wrote: What I need is an import where all valid lines from the csv files are read into the db, and I also get a logfile for all invalid lines, stating the line number plus the pg error message so I can see which constraint was violated. I can't think of a direct, ele

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Joolz
Mike Rylander zei: > On Fri, 4 Feb 2005 13:32:40 +0100 (CET), Joolz > <[EMAIL PROTECTED]> wrote: >> Hello everyone, >> >> I'm building a postgresql db which will have to get lots of data >> from "the outside" (customers, that is). The db has lots of >> constraints, and I'm sure that our customers w

Re: [GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Mike Rylander
On Fri, 4 Feb 2005 13:32:40 +0100 (CET), Joolz <[EMAIL PROTECTED]> wrote: > Hello everyone, > > I'm building a postgresql db which will have to get lots of data > from "the outside" (customers, that is). The db has lots of > constraints, and I'm sure that our customers will offer lots of > invalid

[GENERAL] error-tolerant COPY FROM

2005-02-04 Thread Joolz
Hello everyone, I'm building a postgresql db which will have to get lots of data from "the outside" (customers, that is). The db has lots of constraints, and I'm sure that our customers will offer lots of invalid information. We receive the information in csv format. My first thought was to read t