On Wed, Apr 17, 2024 at 4:28 PM torikoshia wrote:
>
> On 2024-04-16 13:16, Masahiko Sawada wrote:
> > On Tue, Apr 2, 2024 at 7:34 PM torikoshia
> > wrote:
> >>
> >> On 2024-04-01 11:31, Masahiko Sawada wrote:
> >> > On Fri, Mar 29, 2024 at 11:54 AM torikoshia
> >> > wrote:
> >> >>
> >> >> On 202
On 2024-04-16 13:16, Masahiko Sawada wrote:
On Tue, Apr 2, 2024 at 7:34 PM torikoshia
wrote:
On 2024-04-01 11:31, Masahiko Sawada wrote:
> On Fri, Mar 29, 2024 at 11:54 AM torikoshia
> wrote:
>>
>> On 2024-03-28 21:54, Masahiko Sawada wrote:
>> > On Thu, Mar 28, 2024 at 9:38 PM torikoshia
>>
On Tue, Apr 2, 2024 at 7:34 PM torikoshia wrote:
>
> On 2024-04-01 11:31, Masahiko Sawada wrote:
> > On Fri, Mar 29, 2024 at 11:54 AM torikoshia
> > wrote:
> >>
> >> On 2024-03-28 21:54, Masahiko Sawada wrote:
> >> > On Thu, Mar 28, 2024 at 9:38 PM torikoshia
> >> > wrote:
> >> >>
> >> >> On 202
On 2024-04-01 11:31, Masahiko Sawada wrote:
On Fri, Mar 29, 2024 at 11:54 AM torikoshia
wrote:
On 2024-03-28 21:54, Masahiko Sawada wrote:
> On Thu, Mar 28, 2024 at 9:38 PM torikoshia
> wrote:
>>
>> On 2024-03-28 10:20, Masahiko Sawada wrote:
>> > Hi,
>> >
>> > On Thu, Jan 18, 2024 at 5:33 PM
On Fri, Mar 29, 2024 at 11:54 AM torikoshia wrote:
>
> On 2024-03-28 21:54, Masahiko Sawada wrote:
> > On Thu, Mar 28, 2024 at 9:38 PM torikoshia
> > wrote:
> >>
> >> On 2024-03-28 10:20, Masahiko Sawada wrote:
> >> > Hi,
> >> >
> >> > On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada
> >> > wrote
On 2024-03-28 21:54, Masahiko Sawada wrote:
On Thu, Mar 28, 2024 at 9:38 PM torikoshia
wrote:
On 2024-03-28 10:20, Masahiko Sawada wrote:
> Hi,
>
> On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada
> wrote:
>>
>> On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov
>> wrote:
>> >
>> > On Thu, Ja
On Thu, Mar 28, 2024 at 9:38 PM torikoshia wrote:
>
> On 2024-03-28 10:20, Masahiko Sawada wrote:
> > Hi,
> >
> > On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada
> > wrote:
> >>
> >> On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov
> >> wrote:
> >> >
> >> > On Thu, Jan 18, 2024 at 4:16 AM tori
On 2024-03-28 10:20, Masahiko Sawada wrote:
Hi,
On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada
wrote:
On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov
wrote:
>
> On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote:
> > On 2024-01-18 10:10, jian he wrote:
> > > On Thu, Jan 18, 2024 at 8:5
Hi,
On Thu, Jan 18, 2024 at 5:33 PM Masahiko Sawada wrote:
>
> On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov
> wrote:
> >
> > On Thu, Jan 18, 2024 at 4:16 AM torikoshia
> > wrote:
> > > On 2024-01-18 10:10, jian he wrote:
> > > > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada
> > > > w
On 2024-01-19 22:27, Alexander Korotkov wrote:
Hi!
On Fri, Jan 19, 2024 at 2:37 PM torikoshia
wrote:
Thanks for making the patch!
The patch is pushed! The proposed changes are incorporated excluding
this.
> - /* If SAVE_ERROR_TO is specified, skip rows
> with soft
Hi!
On Fri, Jan 19, 2024 at 2:37 PM torikoshia wrote:
> Thanks for making the patch!
The patch is pushed! The proposed changes are incorporated excluding this.
> > - /* If SAVE_ERROR_TO is specified, skip rows
> > with soft errors */
> > + /* If ON_E
On 2024-01-18 23:59, jian he wrote:
Hi.
patch refactored based on "on_error {stop|ignore}"
doc changes:
--- a/doc/src/sgml/ref/copy.sgml
+++ b/doc/src/sgml/ref/copy.sgml
@@ -43,7 +43,7 @@ COPY { table_name [ ( column_name [, ...] ) | * }
FORCE_NOT_NULL { ( column_name [, ...] ) | * }
F
Hi.
patch refactored based on "on_error {stop|ignore}"
doc changes:
--- a/doc/src/sgml/ref/copy.sgml
+++ b/doc/src/sgml/ref/copy.sgml
@@ -43,7 +43,7 @@ COPY { table_name [ ( column_name [, ...] ) | * }
FORCE_NOT_NULL { ( column_name [, ...] ) | * }
FORCE_NULL { ( column_name [, ...] ) |
On 2024-01-18 16:59, Alexander Korotkov wrote:
On Thu, Jan 18, 2024 at 4:16 AM torikoshia
wrote:
On 2024-01-18 10:10, jian he wrote:
> On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada
> wrote:
>> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
>> > Kyotaro-san's suggestion isn't bad, though I
On Thu, Jan 18, 2024 at 4:59 PM Alexander Korotkov wrote:
>
> On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote:
> > On 2024-01-18 10:10, jian he wrote:
> > > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada
> > > wrote:
> > >> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
> > >> > Kyotaro-san'
čt 18. 1. 2024 v 8:59 odesílatel Alexander Korotkov
napsal:
> On Thu, Jan 18, 2024 at 4:16 AM torikoshia
> wrote:
> > On 2024-01-18 10:10, jian he wrote:
> > > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada >
> > > wrote:
> > >> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
> > >> > Kyotaro
On Thu, Jan 18, 2024 at 4:16 AM torikoshia wrote:
> On 2024-01-18 10:10, jian he wrote:
> > On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada
> > wrote:
> >> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
> >> > Kyotaro-san's suggestion isn't bad, though I might shorten it to
> >> > error_action
On 2024-01-18 10:10, jian he wrote:
On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada
wrote:
On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
>
> Alexander Korotkov writes:
> > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi
> > wrote:
> >> On the other hand, SAVE_ERROR_TO takes 'error' or '
On Thu, Jan 18, 2024 at 8:57 AM Masahiko Sawada wrote:
>
> On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
> >
> > Alexander Korotkov writes:
> > > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi
> > > wrote:
> > >> On the other hand, SAVE_ERROR_TO takes 'error' or 'none', which
> > >> indica
On Thu, Jan 18, 2024 at 6:38 AM Tom Lane wrote:
>
> Alexander Korotkov writes:
> > On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi
> > wrote:
> >> On the other hand, SAVE_ERROR_TO takes 'error' or 'none', which
> >> indicate "immediately error out" and 'just ignore the failure'
> >> respective
Alexander Korotkov writes:
> On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi
> wrote:
>> On the other hand, SAVE_ERROR_TO takes 'error' or 'none', which
>> indicate "immediately error out" and 'just ignore the failure'
>> respectively, but these options hardly seem to denote a 'location',
>> an
On Wed, Jan 17, 2024 at 9:49 AM Kyotaro Horiguchi
wrote:
> At Wed, 17 Jan 2024 14:38:54 +0900, torikoshia
> wrote in
> > Hi,
> >
> > Thanks for applying!
> >
> > > + errmsg_plural("%zd row were skipped due to data type
> > > incompatibility",
> >
> > Sorry, I just noticed it, but 'were' should b
On Wed, Jan 17, 2024 at 7:38 AM torikoshia wrote:
>
> Hi,
>
> Thanks for applying!
>
> > + errmsg_plural("%zd row were skipped due
> > to data type incompatibility",
>
> Sorry, I just noticed it, but 'were' should be 'was' here?
Sure, the fix is pushed.
--
Regar
At Wed, 17 Jan 2024 14:38:54 +0900, torikoshia
wrote in
> Hi,
>
> Thanks for applying!
>
> > + errmsg_plural("%zd row were skipped due to data type
> > incompatibility",
>
> Sorry, I just noticed it, but 'were' should be 'was' here?
>
> >> BTW I'm thinking we should add a column to pg_stat_p
Hi,
Thanks for applying!
+ errmsg_plural("%zd row were skipped due
to data type incompatibility",
Sorry, I just noticed it, but 'were' should be 'was' here?
BTW I'm thinking we should add a column to pg_stat_progress_copy that
counts soft errors. I'll suggest t
Hi,
> Thanks for updating the patch!
You're welcome!
> Here is a minor comment:
>
> > +/*
> > + * Extract a defGetCopySaveErrorToChoice value from a DefElem.
> > + */
>
> Should be Extract a "CopySaveErrorToChoice"?
Fixed.
> BTW I'm thinking we should add a column to pg_stat_progress_copy that
On 2024-01-16 00:17, Alexander Korotkov wrote:
On Mon, Jan 15, 2024 at 8:44 AM Masahiko Sawada
wrote:
On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov
wrote:
>
> On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada
wrote:
> > Thank you for updating the patch. Here are two comments:
> >
> >
On Mon, Jan 15, 2024 at 8:44 AM Masahiko Sawada wrote:
>
> On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov
> wrote:
> >
> > On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada
> > wrote:
> > > Thank you for updating the patch. Here are two comments:
> > >
> > > ---
> > > + if (cstate->opts.s
On Mon, Jan 15, 2024 at 8:21 AM Alexander Korotkov wrote:
>
> On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada
> wrote:
> > Thank you for updating the patch. Here are two comments:
> >
> > ---
> > + if (cstate->opts.save_error_to != COPY_SAVE_ERROR_TO_UNSPECIFIED &&
> > + cstate->num_err
On Sun, Jan 14, 2024 at 10:35 PM Masahiko Sawada wrote:
> Thank you for updating the patch. Here are two comments:
>
> ---
> + if (cstate->opts.save_error_to != COPY_SAVE_ERROR_TO_UNSPECIFIED &&
> + cstate->num_errors > 0)
> + ereport(WARNING,
> + errmsg("%zd rows were
On Sun, Jan 14, 2024 at 10:30 AM Alexander Korotkov
wrote:
>
> Hi!
>
> I think this is a demanding and long-waited feature. The thread is
> pretty long, but mostly it was disputes about how to save the errors.
> The present patch includes basic infrastructure and ability to ignore
> errors, thus
Hi!
I think this is a demanding and long-waited feature. The thread is
pretty long, but mostly it was disputes about how to save the errors.
The present patch includes basic infrastructure and ability to ignore
errors, thus it's pretty simple.
On Sat, Jan 13, 2024 at 4:20 PM jian he wrote:
> On
On Fri, Jan 12, 2024 at 10:59 AM torikoshia wrote:
>
>
> Thanks for reviewing!
>
> Updated the patch merging your suggestions except below points:
>
> > + cstate->num_errors = 0;
>
> Since cstate is already initialized in below lines, this may be
> redundant.
>
> | /* Allocate workspace and
On Wed, Jan 10, 2024 at 4:42 PM Masahiko Sawada
wrote:
Yeah, I'm still thinking it's better to implement this feature
incrementally. Given we're closing to feature freeze, I think it's
unlikely to get the whole feature into PG17 since there are still many
design discussions we need in addition
On Tue, Jan 9, 2024 at 10:36 PM torikoshia wrote:
>
> On Tue, Dec 19, 2023 at 10:14 AM Masahiko Sawada
> wrote:
> > If we want only such a feature we need to implement it together (the
> > patch could be split, though). But if some parts of the feature are
> > useful for users as well, I'd recomm
On Tue, Jan 9, 2024 at 11:36 PM torikoshia wrote:
>
> On Tue, Dec 19, 2023 at 10:14 AM Masahiko Sawada
> wrote:
> > If we want only such a feature we need to implement it together (the
> > patch could be split, though). But if some parts of the feature are
> > useful for users as well, I'd recomm
On Tue, Dec 19, 2023 at 10:14 AM Masahiko Sawada
wrote:
If we want only such a feature we need to implement it together (the
patch could be split, though). But if some parts of the feature are
useful for users as well, I'd recommend implementing it incrementally.
That way, the patches can get sm
On Fri, Jan 5, 2024 at 4:37 PM jian he wrote:
>
> > > > be reused for a different user.
> > > >
> > >
> > > You are right.
> > > so I changed, now the schema owner will be the error table owner.
> > > every error table tuple inserts,
> > > I switch to schema owner, do the insert, then switch back
On Fri, Jan 5, 2024 at 12:05 AM vignesh C wrote:
>
> On Thu, 28 Dec 2023 at 09:27, jian he wrote:
> >
> > On Wed, Dec 20, 2023 at 8:27 PM Masahiko Sawada
> > wrote:
> > >
> > >
> > > Why do we need to use SPI? I think we can form heap tuples and insert
> > > them to the error table. Creating th
On Thu, 28 Dec 2023 at 09:27, jian he wrote:
>
> On Wed, Dec 20, 2023 at 8:27 PM Masahiko Sawada wrote:
> >
> >
> > Why do we need to use SPI? I think we can form heap tuples and insert
> > them to the error table. Creating the error table also doesn't need to
> > use SPI.
> >
> Thanks for pointi
On Wed, Dec 20, 2023 at 8:27 PM Masahiko Sawada wrote:
>
>
> Why do we need to use SPI? I think we can form heap tuples and insert
> them to the error table. Creating the error table also doesn't need to
> use SPI.
>
Thanks for pointing it out. I figured out how to form heap tuples and
insert them
On Wed, Dec 20, 2023 at 1:07 PM jian he wrote:
>
> On Tue, Dec 19, 2023 at 9:14 AM Masahiko Sawada wrote:
> >
> >
> > The error table hub idea is still unclear to me. I assume that there
> > are error tables at least on each database. And an error table can
> > have error data that happened durin
On Tue, Dec 19, 2023 at 9:14 AM Masahiko Sawada wrote:
>
>
> The error table hub idea is still unclear to me. I assume that there
> are error tables at least on each database. And an error table can
> have error data that happened during COPY FROM, including malformed
> lines. Do the error tables
On Mon, Dec 18, 2023 at 4:41 PM jian he wrote:
>
> On Mon, Dec 18, 2023 at 1:09 PM torikoshia wrote:
> >
> > Hi,
> >
> > > save the error metadata to system catalogs would be more expensive,
> > > please see below explanation.
> > > I have no knowledge of publications.
> > > but i feel there is
On Mon, Dec 18, 2023 at 9:16 AM jian he wrote:
>
> On Fri, Dec 15, 2023 at 4:49 AM Masahiko Sawada wrote:
> >
> > Hi,
> >
> > I've read this thread and the latest patch. IIUC with SAVE_ERROR
> > option, COPY FROM creates an error table for the target table and
> > writes error information there.
On Mon, Dec 18, 2023 at 1:09 PM torikoshia wrote:
>
> Hi,
>
> > save the error metadata to system catalogs would be more expensive,
> > please see below explanation.
> > I have no knowledge of publications.
> > but i feel there is a feature request: publication FOR ALL TABLES
> > exclude regex_pa
Hi,
save the error metadata to system catalogs would be more expensive,
please see below explanation.
I have no knowledge of publications.
but i feel there is a feature request: publication FOR ALL TABLES
exclude regex_pattern.
Anyway, that would be another topic.
I think saving error metadat
On 2023-12-15 05:48, Masahiko Sawada wrote:
Thanks for joining this discussion!
I've read this thread and the latest patch. IIUC with SAVE_ERROR
option, COPY FROM creates an error table for the target table and
writes error information there.
While I agree that the final shape of this feature
On Fri, Dec 15, 2023 at 4:49 AM Masahiko Sawada wrote:
>
> Hi,
>
> I've read this thread and the latest patch. IIUC with SAVE_ERROR
> option, COPY FROM creates an error table for the target table and
> writes error information there.
>
> While I agree that the final shape of this feature would be
Hi,
On Tue, Dec 12, 2023 at 10:04 PM jian he wrote:
>
> On Mon, Dec 11, 2023 at 10:05 PM Alena Rybakina
> wrote:
> >
> > Hi! Thank you for your work. Your patch looks better!
> > Yes, thank you! It works fine, and I see that the regression tests have
> > been passed. 🙂
> > However, when I ran '
On 12.12.2023 16:04, jian he wrote:
On Mon, Dec 11, 2023 at 10:05 PM Alena Rybakina
wrote:
Hi! Thank you for your work. Your patch looks better!
Yes, thank you! It works fine, and I see that the regression tests have been
passed. 🙂
However, when I ran 'copy from with save_error' operation wit
On Mon, Dec 11, 2023 at 10:05 PM Alena Rybakina
wrote:
>
> Hi! Thank you for your work. Your patch looks better!
> Yes, thank you! It works fine, and I see that the regression tests have been
> passed. 🙂
> However, when I ran 'copy from with save_error' operation with simple csv
> files (copy_te
Hi! Thank you for your work. Your patch looks better!
On 10.12.2023 13:32, jian he wrote:
On Fri, Dec 8, 2023 at 3:09 PM Alena Rybakina wrote:
Thank you for your work. Unfortunately, your code contained errors during the
make installation:
'SAVEPOINT' after 'SAVE_ERROR' in unreserved_keyword
On Fri, Dec 8, 2023 at 3:09 PM Alena Rybakina wrote:
>
> Thank you for your work. Unfortunately, your code contained errors during the
> make installation:
>
> 'SAVEPOINT' after 'SAVE_ERROR' in unreserved_keyword list is misplaced
> 'SAVEPOINT' after 'SAVE_ERROR' in bare_label_keyword list is mis
Thank you for your work. Unfortunately, your code contained errors
during the make installation:
'SAVEPOINT' after 'SAVE_ERROR' in unreserved_keyword list is misplaced
'SAVEPOINT' after 'SAVE_ERROR' in bare_label_keyword list is misplaced
make[2]: *** [../../../src/Makefile.global:783: gram.c] E
On Tue, Dec 5, 2023 at 6:07 PM Alena Rybakina wrote:
>
> Hi!
>
> Thank you for your contribution to this thread.
>
>
> I reviewed it and have a few questions.
>
> 1. I have seen that you delete a table before creating it, to which you want
> to add errors due to a failed "copy from" operation. I
Hi!
Thank you for your contribution to this thread.
On 04.12.2023 05:23, jian he wrote:
hi.
here is my implementation based on previous discussions
add a new COPY FROM flag save_error.
save_error only works with non-BINARY flags.
save_error is easier for me to implement, if using "save error"
hi.
here is my implementation based on previous discussions
add a new COPY FROM flag save_error.
save_error only works with non-BINARY flags.
save_error is easier for me to implement, if using "save error" I
worry, 2 words, gram.y will not work.
save_error also works other flag like {csv mode, for
On 14/11/2023 17:10, Damir Belyalov wrote:
Here is a very straw-man-level sketch of what I think might work.
The option to COPY FROM looks something like
ERRORS TO other_table_name (item [, item [, ...]])
I tried to implement the patch using a table and came across a num
On Thu, Nov 9, 2023 at 4:12 AM Tom Lane wrote:
>
> Daniel Gustafsson writes:
> >> On 8 Nov 2023, at 19:18, Tom Lane wrote:
> >> I think an actually usable feature of this sort would involve
> >> copying all the failed lines to some alternate output medium,
> >> perhaps a second table with a TEXT
Damir Belyalov writes:
> Here is a very straw-man-level sketch of what I think might work.
> The option to COPY FROM looks something like
>
>ERRORS TO other_table_name (item [, item [, ...]])
>
> I tried to implement the patch using a table and came across a number of
> questions.
Hi!
On 14.11.2023 13:10, Damir Belyalov wrote:
Here is a very straw-man-level sketch of what I think might work.
The option to COPY FROM looks something like
ERRORS TO other_table_name (item [, item [, ...]])
I tried to implement the patch using a table and came across a
> Here is a very straw-man-level sketch of what I think might work.
> The option to COPY FROM looks something like
>
> ERRORS TO other_table_name (item [, item [, ...]])
>
I tried to implement the patch using a table and came across a number of
questions.
Which table should we implement f
Tom Lane writes:
> Daniel Gustafsson writes:
>>> On 8 Nov 2023, at 19:18, Tom Lane wrote:
>>> I think an actually usable feature of this sort would involve
>>> copying all the failed lines to some alternate output medium,
>>> perhaps a second table with a TEXT column to receive the original
>
Hello everyone!
Thanks for turning back to this patch.
I had already thought about storing errors in the table / separate file /
logfile and it seems to me that the best way is to output errors in
logfile. As for user it is more convenient to look for errors in the place
where they are usually ge
Hi,
On 2023-11-08 19:00:01 -0500, Tom Lane wrote:
> Andres Freund writes:
> > On 2023-11-08 13:18:39 -0500, Tom Lane wrote:
> >> I think an actually usable feature of this sort would involve
> >> copying all the failed lines to some alternate output medium,
> >> perhaps a second table with a TEXT
Andres Freund writes:
> On 2023-11-08 13:18:39 -0500, Tom Lane wrote:
>> I think an actually usable feature of this sort would involve
>> copying all the failed lines to some alternate output medium,
>> perhaps a second table with a TEXT column to receive the original
>> data line.
> If we go in
Hi,
On 2023-11-08 13:18:39 -0500, Tom Lane wrote:
> Damir writes:
> > [ v7-0002-Add-new-COPY-option-IGNORE_DATATYPE_ERRORS.patch ]
>
> Sorry for being so late to the party, but ... I don't think this
> is a well-designed feature as it stands. Simply dropping failed rows
> seems like an unusable
Daniel Gustafsson writes:
>> On 8 Nov 2023, at 19:18, Tom Lane wrote:
>> I think an actually usable feature of this sort would involve
>> copying all the failed lines to some alternate output medium,
>> perhaps a second table with a TEXT column to receive the original
>> data line. (Or maybe an
> On 8 Nov 2023, at 19:18, Tom Lane wrote:
> I think an actually usable feature of this sort would involve
> copying all the failed lines to some alternate output medium,
> perhaps a second table with a TEXT column to receive the original
> data line. (Or maybe an array of text that could receiv
Damir writes:
> [ v7-0002-Add-new-COPY-option-IGNORE_DATATYPE_ERRORS.patch ]
Sorry for being so late to the party, but ... I don't think this
is a well-designed feature as it stands. Simply dropping failed rows
seems like an unusable definition for any application that has
pretensions of robustn
Although v7 patch doesn't have commit messages on the patch, I think
leave commit message is good for reviewers.
Sure, didn't notice it. Added the commit message to the updated patch.
* Note: DON'T convert this error to "soft" style (errsave/ereturn). We
* want this data type to stay perm
On 2023-09-15 19:02, Damir Belyalov wrote:
Since v5 patch failed applying anymore, updated the patch.
Thank you for updating the patch . I made a little review on it where
corrected some formatting.
Thanks for your review and update!
I don't have objections the modification of the codes and
> Since v5 patch failed applying anymore, updated the patch.
Thank you for updating the patch . I made a little review on it where
corrected some formatting.
> - COPY with a datatype error that can't be handled as a soft error
>
> I didn't know proper way to test this, but I've found data type wi
Since v5 patch failed applying anymore, updated the patch.
On 2023-03-23 02:50, Andres Freund wrote:
I suggest adding a few more tests:
- COPY with a datatype error that can't be handled as a soft error
I didn't know proper way to test this, but I've found data type widget's
input function wi
I'm sorry I was unable to respond right away.
On 09.05.2023 17:23, torikoshia wrote:
You may already understand it, but these variable names are given in
imitation of FREEZE and BINARY cases:
--- a/src/include/commands/copy.h
+++ b/src/include/commands/copy.h
@@ -42,6 +42,7 @@ typedef st
I'm sorry I was unable to respond right away.
On 09.05.2023 17:23, torikoshia wrote:
You may already understand it, but these variable names are given in
imitation of FREEZE and BINARY cases:
--- a/src/include/commands/copy.h
+++ b/src/include/commands/copy.h
@@ -42,6 +42,7 @@ typedef st
On 2023-05-07 05:05, Alena Rybakina wrote:
Thanks for your reviewing and comments!
I noticed that you used _ignore_datatype_errors_specified_ variable in
_copy.c_ , but guc has a short name _ignore_datatype_errors_. Also you
used the short variable name in _CopyFormatOptions_ structure.
You ma
Hi!
Thank you, Damir, for your patch. It is very interesting to review it!
It seemed to me that the names of variables are not the same everywhere.
I noticed that you used /ignore_datatype_errors_specified/ variable in
/copy.c/ , but guc has a short name /ignore_datatype_errors/. Also you
use
>
> I might misunderstand something, but I believe the v5 patch uses
> copy_generic_opt_list and it does not add IGNORE_DATATYPE_ERRORS as a
> keyword.
> It modifies neither kwlist.h nor gram.y.
>
Sorry, didn't notice that. I think everything is alright now.
Regards,
Damir Belyalov
Postgres Profe
On 2023-03-27 23:28, Damir Belyalov wrote:
Hi!
I made the specified changes and my patch turned out the same as
yours. The performance measurements were the same too.
Thanks for your review and measurements.
The only thing left to do is how not to add IGNORE_DATATYPE_ERRORS as
a keyword. See
Hi!
I made the specified changes and my patch turned out the same as yours. The
performance measurements were the same too.
The only thing left to do is how not to add IGNORE_DATATYPE_ERRORS as a
keyword. See how this is done for parameters such as FORCE_NOT_NULL,
FORCE_NULL, FORCE_QUOTE. They ar
On 2023-03-23 02:50, Andres Freund wrote:
Thanks again for your review.
Attached v5 patch.
Have you measured whether this has negative performance effects when
*NOT*
using the new option?
I loaded 1000 rows of pgbench_accounts on my laptop and compared the
elapsed time.
GUCs changed from
On 2023-03-23 02:50, Andres Freund wrote:
Hi,
Tom, see below - I wonder if should provide one more piece of
infrastructure
around the saved error stuff...
Have you measured whether this has negative performance effects when
*NOT*
using the new option?
As-is this does not work with FORMAT
Hi,
Tom, see below - I wonder if should provide one more piece of infrastructure
around the saved error stuff...
Have you measured whether this has negative performance effects when *NOT*
using the new option?
As-is this does not work with FORMAT BINARY - and converting the binary input
functi
On 2023-03-17 21:23, torikoshia wrote:
On 2023-03-07 18:09, Daniel Gustafsson wrote:
On 7 Mar 2023, at 09:35, Damir Belyalov wrote:
I felt just logging "Error: %ld" would make people wonder the meaning
of
the %ld. Logging something like ""Error: %ld data type errors were
found" might be cle
On 2023-03-07 18:09, Daniel Gustafsson wrote:
On 7 Mar 2023, at 09:35, Damir Belyalov wrote:
I felt just logging "Error: %ld" would make people wonder the meaning
of
the %ld. Logging something like ""Error: %ld data type errors were
found" might be clearer.
Thanks. For more clearance change
> On 7 Mar 2023, at 09:35, Damir Belyalov wrote:
> I felt just logging "Error: %ld" would make people wonder the meaning of
> the %ld. Logging something like ""Error: %ld data type errors were
> found" might be clearer.
>
> Thanks. For more clearance change the message to: "Errors were found:
>
> FWIW, Greenplum has a similar construct (but which also logs the errors
> in the
> db) where data type errors are skipped as long as the number of errors
> don't
> exceed a reject limit. If the reject limit is reached then the COPY
> fails:
> >
> > LOG ERRORS [ SEGMENT REJECT LIMIT [ RO
On 2023-03-06 23:03, Daniel Gustafsson wrote:
On 28 Feb 2023, at 15:28, Damir Belyalov wrote:
Tested patch on all cases: CIM_SINGLE, CIM_MULTI, CIM_MULTI_CONDITION.
As expected it works.
Also added a description to copy.sgml and made a review on patch.
Thanks for your tests and improvements
> On 28 Feb 2023, at 15:28, Damir Belyalov wrote:
> Tested patch on all cases: CIM_SINGLE, CIM_MULTI, CIM_MULTI_CONDITION. As
> expected it works.
> Also added a description to copy.sgml and made a review on patch.
>
> I added 'ignored_errors' integer parameter that should be output after the
Hello
Tested patch on all cases: CIM_SINGLE, CIM_MULTI, CIM_MULTI_CONDITION. As
expected it works.
Also added a description to copy.sgml and made a review on patch.
I added 'ignored_errors' integer parameter that should be output after the
option is finished.
All errors were added to the system l
On 2023-02-06 15:00, Tom Lane wrote:
Andres Freund writes:
On February 5, 2023 9:12:17 PM PST, Tom Lane
wrote:
Damir Belyalov writes:
InputFunctionCallSafe() is good for detecting errors from
input-functions
but there are such errors from NextCopyFrom () that can not be
detected
with Input
Andres Freund writes:
> On February 5, 2023 9:12:17 PM PST, Tom Lane wrote:
>> Damir Belyalov writes:
>>> InputFunctionCallSafe() is good for detecting errors from input-functions
>>> but there are such errors from NextCopyFrom () that can not be detected
>>> with InputFunctionCallSafe(), e.g. "
Hi,
On February 5, 2023 9:12:17 PM PST, Tom Lane wrote:
>Damir Belyalov writes:
>>> I don't think this is the right approach. Creating a subtransaction for
>>> each row will cause substantial performance issues.
>
>> Subtransactions aren't created for each row. The block of rows in one
>> subtr
Damir Belyalov writes:
>> I don't think this is the right approach. Creating a subtransaction for
>> each row will cause substantial performance issues.
> Subtransactions aren't created for each row. The block of rows in one
> subtransaction is 1000 (SAFE_BUFFER_SIZE) and can be changed.
I think
Hi, Andres!
Thank you for reviewing.
> I don't think this is the right approach. Creating a subtransaction for
> each row will cause substantial performance issues.
>
Subtransactions aren't created for each row. The block of rows in one
subtransaction is 1000 (SAFE_BUFFER_SIZE) and can be chang
H,
On 2023-02-03 13:27:24 +0300, Damir Belyalov wrote:
> @@ -625,6 +628,173 @@ CopyMultiInsertInfoStore(CopyMultiInsertInfo *miinfo,
> ResultRelInfo *rri,
> miinfo->bufferedBytes += tuplen;
> }
>
> +/*
> + * Safely reads source data, converts to a tuple and fills tuple buffer.
> + * Skip
Hi, Danil and Nikita!
Thank you for reviewing.
Why is there no static keyword in the definition of the SafeCopying()
> function, but it is in the function declaration.
>
Correct this.
675: MemoryContext cxt =
> MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
> 676:
> 677: valid_row = Next
Hi!
I have looked at your patch and have a few questions.
110: static bool SafeCopying(CopyFromState cstate, ExprContext *econtext,
111: TupleTableSlot *myslot);
---
636: bool
637: SafeCopying(CopyFromState cstate, ExprContext *econtext,
TupleTableSlot *myslot)
Why is there no static keyword in
1 - 100 of 116 matches
Mail list logo
