On 2024-10-08 18:39, Fujii Masao wrote:
On 2024/10/07 21:51, torikoshia wrote:
While reviewing, I also noticed that the check for
"opts_out->binary && opts_out->on_error != COPY_ON_ERROR_STOP"
is similarly placed before setting the defaults, which might not
be correct. This check should probably
On 2024/10/07 21:51, torikoshia wrote:
While reviewing, I also noticed that the check for
"opts_out->binary && opts_out->on_error != COPY_ON_ERROR_STOP"
is similarly placed before setting the defaults, which might not
be correct. This check should probably be moved as well.
Additionally, the c
Thanks for your review!
On Thu, Oct 3, 2024 at 4:27 PM jian he
wrote:
mentioning maxerror is a
bigint type
or explicitly mentioning the maximum allowed value of "maxerror" would
be great.
Added a description that it allows positive bigint.
On Thu, Oct 3, 2024 at 11:43 PM Fujii Masao
w
On 2024/09/30 21:08, torikoshia wrote:
Since defGetInt64() checks not only whether the input value exceeds the range
of bigint but also input value is specified, attached v6 patch checks them by
directly calling defGetInt64().
Thanks for updating the patch! But the patch no longer applied
> I'm wondering if we can use the wording maxerror as in the attached
> patch.
>
+
+REJECT_LIMIT
+
+
+ Specifies the maximum number of errors tolerated while converting a
+ column's input value to its data type, when
ON_ERROR is
+ set to ignore.
+ If the input c
rds,
--
Atsushi Torikoshi
NTT DATA Group CorporationFrom 55a99fc186c263cdd7741a38a9c684c9cb8ac1d1 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi
Date: Mon, 30 Sep 2024 20:33:26 +0900
Subject: [PATCH v6] Add new COPY option REJECT_LIMIT
9e2d870 enabled the COPY FROM command to skip soft errors, but there
was no
+/*
+ * Extract REJECT_LIMIT value from a DefElem.
+ */
+static int64
+defGetCopyRejectLimitOptions(DefElem *def)
+{
+ int64 reject_limit;
+
+ if (def->arg == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("REJECT_LIMIT requires a positive integer")));
+
+ if (nodeTag(def->arg) =
re are no opposing opinions, removed 'INFINITY' as well.
+1
Regards,
--
Regards,
--
Atsushi Torikoshi
NTT DATA Group CorporationFrom e40e378ac6eb8583b8cfbac07ec459acc03fc4b1 Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi
Date: Thu, 26 Sep 2024 22:05:28 +0900
Subject: [PATCH v5] Add new COPY option REJECT_LIMI
On 2024/09/24 14:25, torikoshia wrote:
Updated the patch.
Thanks for updating the patch!
+REJECT_LIMIT { integer }
The curly braces {} seem unnecessary here.
+ When a positive integer value is specified, COPY
limits
+ the maximum tolerable number of errors while converting
ates unlimited number of errors.
--
Regards,
--
Atsushi Torikoshi
NTT DATA Group CorporationFrom 0e92a90bd9f944b7f14845e942147d11f81e3b6f Mon Sep 17 00:00:00 2001
From: Atsushi Torikoshi
Date: Tue, 24 Sep 2024 14:11:22 +0900
Subject: [PATCH v4] Add new COPY option REJECT_LIMIT number
9e2d870 en
On 2024-07-23 02:06, Kirill Reshke wrote:
Thanks for your review.
Few comments:
+ When a positive integer value is specified,
COPY limits
+ the maximum tolerable number of errors while converting a
column's input
+ value into its data type.
If nothing is specified, then the
On Tue, Jul 23, 2024 at 1:35 PM Fujii Masao
wrote:
Thanks for your review.
On 2024/07/22 21:37, torikoshia wrote:
On Fri, Jul 19, 2024 at 11:48 PM Junwang Zhao
wrote:
Thanks for the comment.
In patch 0002, the ratio is calculated by the already
skipped/processed
rows, but what if a user
On 2024/07/22 21:37, torikoshia wrote:
On Fri, Jul 19, 2024 at 11:48 PM Junwang Zhao wrote:
Thanks for the comment.
In patch 0002, the ratio is calculated by the already skipped/processed
rows, but what if a user wants to copy 1000 rows, and he/she can tolerate
10 error rows, so he/she migh
Hi! Nice feature.
Few comments:
> + When a positive integer value is specified, COPY
> limits
> + the maximum tolerable number of errors while converting a column's
> input
> + value into its data type.
If nothing is specified, then the maximum tolerable number of errors
is one,
ense.
Updated the patch.
BTW if "set_to_null" is supported someday, REJECT_LIMIT can also
apply. The command would cinsert NULL into the target table upon
encountering errors and continue, but fail if the number of errors
exceed REJECT_LIMIT.
Agreed.
--
Regards,
--
Atsushi Torikoshi
NT
On 2024/07/19 22:03, Fujii Masao wrote:
On 2024/07/17 22:21, torikoshia wrote:
On 2024-07-03 02:07, Fujii Masao wrote:
However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR option is
still necessary.
I remembered another reason for the necessity of ON_ERROR.
ON_ERROR define
Hi Torikoshia,
On Wed, Jul 17, 2024 at 9:21 PM torikoshia wrote:
>
> On 2024-07-03 02:07, Fujii Masao wrote:
> > However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR
> > option is still necessary.
>
> I remembered another reason for the necessity of ON_ERROR.
>
> ON_ERROR defines how
On 2024/07/17 22:21, torikoshia wrote:
On 2024-07-03 02:07, Fujii Masao wrote:
However, if we support REJECT_LIMIT, I'm not sure if the ON_ERROR option is
still necessary.
I remembered another reason for the necessity of ON_ERROR.
ON_ERROR defines how to behave when encountering an error
ws using REJECT_LIMIT regardless of the ON_ERROR
option value.
[1]
https://www.postgresql.org/message-id/flat/CACJufxH_OJpVra=0c4ow8fbxhj7hemcvatnepa5vaursena...@mail.gmail.com
--
Regards,
--
Atsushi Torikoshi
NTT DATA Group CorporationFrom d00e4a404fc9c6ccc8aec3885210909a21c55005 Mon Sep 17 00:00:
On 2024-07-05 12:59, Fujii Masao wrote:
On 2024/07/04 12:05, torikoshia wrote:
I'm going to update it after discussing the option format as described
below.
Thanks!
I agree that it's possible to use only REJECT_LIMIT without ON_ERROR.
I also think it's easy to understand that REJECT_LIMIT=0
On 2024/07/04 12:05, torikoshia wrote:
I'm going to update it after discussing the option format as described below.
Thanks!
I agree that it's possible to use only REJECT_LIMIT without ON_ERROR.
I also think it's easy to understand that REJECT_LIMIT=0 is ON_ERROR=stop.
However, expressing
On 2024-07-03 02:07, Fujii Masao wrote:
Thanks for your comments!
On 2024/01/26 18:49, torikoshia wrote:
Hi,
9e2d870 enabled the COPY command to skip soft error, and I think we
can add another option which specifies the maximum tolerable number of
soft errors.
I remember this was discussed
On 2024/01/26 18:49, torikoshia wrote:
Hi,
9e2d870 enabled the COPY command to skip soft error, and I think we can add
another option which specifies the maximum tolerable number of soft errors.
I remember this was discussed in [1], and feel it would be useful when loading
'dirty' data but
On 2024-01-27 00:20, David G. Johnston wrote:
Thanks for your comments!
On Fri, Jan 26, 2024 at 2:49 AM torikoshia
wrote:
Hi,
9e2d870 enabled the COPY command to skip soft error, and I think we
can
add another option which specifies the maximum tolerable number of
soft
errors.
I remember t
On Fri, Jan 26, 2024 at 2:49 AM torikoshia
wrote:
> Hi,
>
> 9e2d870 enabled the COPY command to skip soft error, and I think we can
> add another option which specifies the maximum tolerable number of soft
> errors.
>
> I remember this was discussed in [1], and feel it would be useful when
> load
Date: Fri, 26 Jan 2024 18:32:40 +0900
Subject: [PATCH v1] Add new COPY option REJECT_LIMIT
REJECT_LIMIT specifies the maximum tolerable number of malformed rows.
If input data has more malformed errors than this value, entire COPY fails.
This option must be used with ON_ERROR to be set to other
26 matches
Mail list logo
