On 05/11/2026 16:24, Fujii Masao wrote:
On Thu, Nov 5, 2026 at 9:40 AM Ayoub Kazar <[email protected]> wrote:
However, ANALYZE silently ignores reject_limit, skips all
malformed rows, and finishes with a NOTICE.
Personally, I'd prefer to keep this behavior. Statistics based on valid rows
are still useful even when the number of errors exceeds reject_limit.
Queries may still succeed, for example, when using LIMIT, whereas
aborting ANALYZE would leave the statistics outdated. So I think
ANALYZE should continue to ignore reject_limit. Thought?
Yes, very reasonable.
The NOTICE itself also provides good info about what's going on, so
that's okay.
If we agree, how about documenting this exception and adding a comment
in file_acquire_sample_rows() explaining why reject_limit is not enforced?
Attached is v2 documenting this in file-fdw.sgml and adding a comment in
file_acquire_sample_rows().
Regards,
Ayoub Kazar
From 315f05f1f6986e48a72c971a26544d4fbacc78f2 Mon Sep 17 00:00:00 2001
From: AyoubKAZ <[email protected]>
Date: Thu, 5 Nov 2026 16:56:52 +0100
Subject: [PATCH v2] file_fdw: Document why ANALYZE does not enforce
REJECT_LIMIT
When a file_fdw foreign table is configured with on_error 'ignore' and
reject_limit, foreign table scans abort with an error once the number
of skipped rows exceeds reject_limit. However, ANALYZE intentionally
ignores reject_limit while sampling rows so that statistics can still be
gathered from valid rows, rather than failing and leaving table
statistics outdated.
Document this exception in the file_fdw documentation and add an
explanatory comment in file_acquire_sample_rows().
---
contrib/file_fdw/file_fdw.c | 6 ++++++
doc/src/sgml/file-fdw.sgml | 2 ++
2 files changed, 8 insertions(+)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 7d9dc3baf74..91863dcc243 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -1261,6 +1261,12 @@ file_acquire_sample_rows(Relation onerel, int elevel,
* ErrorSaveContext ready for the next NextCopyFrom. Since we
* don't set details_wanted and error_data is not to be filled,
* just resetting error_occurred is enough.
+ *
+ * Note that unlike fileIterateForeignScan(), we intentionally do
+ * not enforce reject_limit here. Statistics collected from valid
+ * rows remain useful even when the number of errors exceeds
+ * reject_limit, whereas aborting ANALYZE would leave the table
+ * with outdated statistics.
*/
cstate->escontext->error_occurred = false;
diff --git a/doc/src/sgml/file-fdw.sgml b/doc/src/sgml/file-fdw.sgml
index 3638689436f..012985994bd 100644
--- a/doc/src/sgml/file-fdw.sgml
+++ b/doc/src/sgml/file-fdw.sgml
@@ -157,6 +157,8 @@
Specifies the maximum number of errors tolerated while converting a column's
input value to its data type, the same as <command>COPY</command>'s
<literal>REJECT_LIMIT</literal> option.
+ This option is not enforced when <command>ANALYZE</command> samples rows
+ from the foreign table.
</para>
</listitem>
</varlistentry>
--
2.34.1
