Alexander Lakhin writes:
> 08.04.2024 18:08, Tom Lane wrote:
>> Hmm, the point about recursion is still valid isn't it? I agree the
>> reference to ExecQueryUsingCursor is obsolete, but I think we need to
>> reconstruct what this comment is actually talking about. It's
>> certainly pretty obscur
Alexander Lakhin wrote:
> >> Now that ExecQueryUsingCursor() is gone, it's not clear, what does
> >> the following comment mean:?
> >> * We must turn off gexec_flag to avoid infinite recursion. Note that
> >> * this allows ExecQueryUsingCursor to be applied to the individual
>
08.04.2024 18:08, Tom Lane wrote:
Alexander Lakhin writes:
Now that ExecQueryUsingCursor() is gone, it's not clear, what does
the following comment mean:?
* We must turn off gexec_flag to avoid infinite recursion. Note that
* this allows ExecQueryUsingCursor to be applied to the indi
Alexander Lakhin writes:
> Now that ExecQueryUsingCursor() is gone, it's not clear, what does
> the following comment mean:?
> * We must turn off gexec_flag to avoid infinite recursion. Note that
> * this allows ExecQueryUsingCursor to be applied to the individual query
> * results.
Hello Daniel and Tom,
08.04.2024 17:25, Daniel Verite wrote:
So I whacked the patch around till I liked it better, and pushed it.
Thanks for taking care of this!
Now that ExecQueryUsingCursor() is gone, it's not clear, what does
the following comment mean:?
* We must turn off gexec_flag
Tom Lane wrote:
> I've reconsidered after realizing that implementing FETCH_COUNT
> atop traditional single-row mode would require either merging
> single-row results into a bigger PGresult or persuading psql's
> results-printing code to accept an array of PGresults not just
> one. Either
So what was really bothering me about this patchset was that I
didn't think marginal performance gains were a sufficient reason
to put a whole different operating mode into libpq. However,
I've reconsidered after realizing that implementing FETCH_COUNT
atop traditional single-row mode would requir
Tom Lane wrote:
> > I should say that I've noticed significant latency improvements with
> > FETCH_COUNT retrieving large resultsets, such that it would benefit
> > non-interactive use cases.
>
> Do you have a theory for why that is? It's pretty counterintuitive
> that it would help at a
"Daniel Verite" writes:
> Tom Lane wrote:
>> I do not buy that psql's FETCH_COUNT mode is a sufficient reason
>> to add it. FETCH_COUNT mode is not something you'd use
>> non-interactively
> I should say that I've noticed significant latency improvements with
> FETCH_COUNT retrieving large
Tom Lane wrote:
> I do not buy that psql's FETCH_COUNT mode is a sufficient reason
> to add it. FETCH_COUNT mode is not something you'd use
> non-interactively
I should say that I've noticed significant latency improvements with
FETCH_COUNT retrieving large resultsets, such that it would
"Daniel Verite" writes:
> Updated patches are attached.
I started to look through this, and almost immediately noted
-
- Retrieving Query Results Row-by-Row
+
+ Retrieving Query Results in chunks
This is a bit problematic, because changing the sect1 ID will
change the page's URL, eg
https:
Laurenz Albe wrote:
> Here is the code review for patch number 2:
> +static void
> +CloseGOutput(FILE *gfile_fout, bool is_pipe)
>
> It makes sense to factor out this code.
> But shouldn't these functions have a prototype at the beginning of the file?
Looking at the other static functio
Laurenz Albe wrote:
> I had a look at patch 0001 (0002 will follow).
Thanks for reviewing this!
I've implemented the suggested doc changes. A patch update
will follow with the next part of the review.
> > --- a/src/interfaces/libpq/fe-exec.c
> > +++ b/src/interfaces/libpq/fe-exec.c
> >
On Fri, 2024-03-29 at 14:07 +0100, Laurenz Albe wrote:
> I had a look at patch 0001 (0002 will follow).
Here is the code review for patch number 2:
> diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
[...]
+static bool
+SetupGOutput(PGresult *result, FILE **gfile_fout, bool *is_pipe)
[.
On Tue, 2024-01-30 at 15:29 +0100, Daniel Verite wrote:
> PFA a rebased version.
I had a look at patch 0001 (0002 will follow).
> -
> - Retrieving Query Results Row-by-Row
> +
> + Retrieving Query Results by chunks
That should be "in chunks".
> +
> +
> +
> + PQsetChunkedRows
Jakub Wartak wrote:
> when I run with default pager (more or less):
> \set FETCH_COUNT 1000
> WITH data AS (SELECT generate_series(1, 2000) as Total) select
> repeat('a',100) || data.Total || repeat('b', 800) as total_pat from
> data;
> -- it enters pager, a skip couple of pages and t
Hi Daniel,
On Tue, Jan 30, 2024 at 3:29 PM Daniel Verite wrote:
> PFA a rebased version.
Thanks for the patch! I've tested it using my original reproducer and
it works great now against the original problem description. I've
taken a quick look at the patch, it looks good for me. I've tested
us
vignesh C wrote:
> patching file src/interfaces/libpq/exports.txt
> Hunk #1 FAILED at 191.
> 1 out of 1 hunk FAILED -- saving rejects to file
> src/interfaces/libpq/exports.txt.rej
>
> Please post an updated version for the same.
PFA a rebased version.
Best regards,
--
Daniel Vérité
h
On Tue, 2 Jan 2024 at 20:28, Daniel Verite wrote:
>
> Hi,
>
> PFA a rebased version.
CFBot shows that the patch does not apply anymore as in [1]:
=== Applying patches on top of PostgreSQL commit ID
a3a836fb5e51183eae624d43225279306c2285b8 ===
=== applying patch
./v5-0001-Implement-retrieval-of-
Hi,
PFA a rebased version.
Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite
From cd0fe1d517a0e31e031fbbea1e603a715c77ea97 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20V=C3=A9rit=C3=A9?=
Date: Tue, 2 Jan 2024 14:15:48 +0100
Subject: [PATCH v5 1/2] Imple
Hi,
Here's a new version to improve the performance of FETCH_COUNT
and extend the cases when it can be used.
Patch 0001 adds a new mode in libpq to allow the app to retrieve
larger chunks of results than the single row of the row-by-row mode.
The maximum number of rows per PGresult is set by the
Tom Lane wrote:
> This gives me several "-Wincompatible-pointer-types" warnings
> [...]
> I think what you probably ought to do to avoid all that is to change
> the arguments of PrintQueryResult and nearby routines to be "const
> PGresult *result" not just "PGresult *result".
The const-ne
"Daniel Verite" writes:
> PFA an updated patch.
This gives me several "-Wincompatible-pointer-types" warnings
(as are also reported by the cfbot):
common.c: In function 'ExecQueryAndProcessResults':
common.c:1686:24: warning: passing argument 1 of 'PrintQueryTuples' from
incompatible pointer ty
I wrote:
> Here's a POC patch implementing row-by-row fetching.
PFA an updated patch.
Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8..ad5e8a5de9 100644
--- a/src/bin/psql/common.
Tom Lane wrote:
> I agree that it seems like a good idea to try.
> There will be more per-row overhead, but the increase in flexibility
> is likely to justify that.
Here's a POC patch implementing row-by-row fetching.
If it wasn't for the per-row overhead, we could probably get rid of
Ex
On Wed, Jan 4, 2023 at 6:38 PM Robert Haas wrote:
>
> On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote:
> > As you well know, psql's FETCH_COUNT mechanism is far older than
> > single-row mode. I don't think anyone's tried to transpose it
> > onto that. I agree that it seems like a good idea to t
On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote:
> As you well know, psql's FETCH_COUNT mechanism is far older than
> single-row mode. I don't think anyone's tried to transpose it
> onto that. I agree that it seems like a good idea to try.
> There will be more per-row overhead, but the increase i
Robert Haas writes:
> On Wed, Jan 4, 2023 at 10:22 AM Daniel Verite wrote:
>> A solution would be for psql to use PQsetSingleRowMode() to retrieve
>> results row-by-row, as opposed to using a cursor, and then allocate
>> memory for only FETCH_COUNT rows at a time.
> Is there any reason that some
On Wed, Jan 4, 2023 at 10:22 AM Daniel Verite wrote:
> A solution would be for psql to use PQsetSingleRowMode() to retrieve
> results row-by-row, as opposed to using a cursor, and then allocate
> memory for only FETCH_COUNT rows at a time. Incidentally it solves
> other problems like queries conta
Jakub Wartak wrote:
> It might be a not so well known fact (?) that CTEs are not executed
> with cursor when asked to do so, but instead silently executed with
> potential huge memory allocation going on. Patch is attached. My one
> doubt is that not every statement starting with "WITH" is
Hi -hackers,
I've spent some time fighting against "out of memory" errors coming
out of psql when trying to use the cursor via FETCH_COUNT. It might be
a not so well known fact (?) that CTEs are not executed with cursor
when asked to do so, but instead silently executed with potential huge
memory
31 matches
Mail list logo
