Hi hackers,
Commit 23a27b039d94ba359286694831eafe03cd970eef changed the type of
numbers-of-tuples-processed counters to uint64 and adjusted various PLs
to cope with this. I noticed the PL/Perl changes did not take full
advantage of what Perl is capable of handling, so here's a patch that
improves that.
1) Perl's integers are at least pointer-sized and either signed or
unsigned, so can potentially hold up to 2⁶⁴-1. Floating point numbers
can also be larger than double (up to 128bit), allowing for exact
representation of integers beyond 2⁵³. Hence, adjust the setting of
the "processed" hash item to use UV_MAX for the limit and (NV) or
(UV) for the casts.
2) Perl 5.20 and later use SSize_t for array indices, so can cope with
up to SSize_t_max items in an array (if you have the memory).
3) To be able to actually return such result sets from sp_exec_query(),
I had to change the repalloc() in spi_printtup() to repalloc_huge().
--
"A disappointingly low fraction of the human race is,
at any given time, on fire." - Stig Sandbeck Mathisen
>From f6d38f1a5c7d6135dd5f580aa8ed38bb74162a0f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilm...@ilmari.org>
Date: Sun, 13 Mar 2016 01:21:43 +0000
Subject: [PATCH] Use correct types and limits for PL/Perl SPI query results
Perl's integers are pointer-sized, so can hold more than INT_MAX on LP64
platforms, and come in both signed (IV) and unsigned (UV). Floating
point values (NV) may also be larger than double.
Since Perl 5.19.4 array indices are SSize_t instead of I32, so allow up
to SSize_t_max on those versions. The limit is not imposed just by
av_extend's argument type, but all the array handling code, so remove
the speculative comment.
Finally, change spi_printtup() to use repalloc_huge() to be able to
return a tuple table of more than 1GiB.
---
src/backend/executor/spi.c | 2 +-
src/pl/plperl/plperl.c | 13 ++++---------
src/pl/plperl/plperl.h | 7 +++++++
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 3d04c23..fd94179 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -1800,7 +1800,7 @@ spi_printtup(TupleTableSlot *slot, DestReceiver *self)
/* Double the size of the pointer array */
tuptable->free = tuptable->alloced;
tuptable->alloced += tuptable->free;
- tuptable->vals = (HeapTuple *) repalloc(tuptable->vals,
+ tuptable->vals = (HeapTuple *) repalloc_huge(tuptable->vals,
tuptable->alloced * sizeof(HeapTuple));
}
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 269f7f3..d405179 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -3104,9 +3104,9 @@ plperl_spi_execute_fetch_result(SPITupleTable *tuptable, uint64 processed,
hv_store_string(result, "status",
cstr2sv(SPI_result_code_string(status)));
hv_store_string(result, "processed",
- (processed > (uint64) INT_MAX) ?
- newSVnv((double) processed) :
- newSViv((int) processed));
+ (processed > (uint64) UV_MAX) ?
+ newSVnv((NV) processed) :
+ newSVuv((UV) processed));
if (status > 0 && tuptable)
{
@@ -3114,12 +3114,7 @@ plperl_spi_execute_fetch_result(SPITupleTable *tuptable, uint64 processed,
SV *row;
uint64 i;
- /*
- * av_extend's 2nd argument is declared I32. It's possible we could
- * nonetheless push more than INT_MAX elements into a Perl array, but
- * let's just fail instead of trying.
- */
- if (processed > (uint64) INT_MAX)
+ if (processed > AV_SIZE_MAX)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("query result has too many rows to fit in a Perl array")));
diff --git a/src/pl/plperl/plperl.h b/src/pl/plperl/plperl.h
index 9179bbd..2f376ee 100644
--- a/src/pl/plperl/plperl.h
+++ b/src/pl/plperl/plperl.h
@@ -100,4 +100,11 @@ void plperl_spi_freeplan(char *);
void plperl_spi_cursor_close(char *);
char *plperl_sv_to_literal(SV *, char *);
+/* Perl 5.19.4 changed array indices from I32 to SSize_t */
+#if PERL_BCDVERSION >= 0x5019004
+#define AV_SIZE_MAX ((uint64) SSize_t_MAX)
+#else
+#define AV_SIZE_MAX ((uint64) I32_MAX)
+#endif
+
#endif /* PL_PERL_H */
--
2.7.0
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
