Alexander Lakhin writes:
> 08.04.2024 18:08, Tom Lane wrote:
>> Hmm, the point about recursion is still valid isn't it? I agree the
>> reference to ExecQueryUsingCursor is obsolete, but I think we need to
>> reconstruct what this comment is actually talking about. It's
>> certainly pretty obscur
Alexander Lakhin wrote:
> >> Now that ExecQueryUsingCursor() is gone, it's not clear, what does
> >> the following comment mean:?
> >> * We must turn off gexec_flag to avoid infinite recursion. Note that
> >> * this allows ExecQueryUsingCursor to be applied to the individual
>
08.04.2024 18:08, Tom Lane wrote:
Alexander Lakhin writes:
Now that ExecQueryUsingCursor() is gone, it's not clear, what does
the following comment mean:?
* We must turn off gexec_flag to avoid infinite recursion. Note that
* this allows ExecQueryUsingCursor to be applied to the indi
Alexander Lakhin writes:
> Now that ExecQueryUsingCursor() is gone, it's not clear, what does
> the following comment mean:?
> * We must turn off gexec_flag to avoid infinite recursion. Note that
> * this allows ExecQueryUsingCursor to be applied to the individual query
> * results.
Hello Daniel and Tom,
08.04.2024 17:25, Daniel Verite wrote:
So I whacked the patch around till I liked it better, and pushed it.
Thanks for taking care of this!
Now that ExecQueryUsingCursor() is gone, it's not clear, what does
the following comment mean:?
* We must turn off gexec_flag
Tom Lane wrote:
> I've reconsidered after realizing that implementing FETCH_COUNT
> atop traditional single-row mode would require either merging
> single-row results into a bigger PGresult or persuading psql's
> results-printing code to accept an array of PGresults not just
> one. Either
So what was really bothering me about this patchset was that I
didn't think marginal performance gains were a sufficient reason
to put a whole different operating mode into libpq. However,
I've reconsidered after realizing that implementing FETCH_COUNT
atop traditional single-row mode would requir
Tom Lane wrote:
> > I should say that I've noticed significant latency improvements with
> > FETCH_COUNT retrieving large resultsets, such that it would benefit
> > non-interactive use cases.
>
> Do you have a theory for why that is? It's pretty counterintuitive
> that it would help at a
"Daniel Verite" writes:
> Tom Lane wrote:
>> I do not buy that psql's FETCH_COUNT mode is a sufficient reason
>> to add it. FETCH_COUNT mode is not something you'd use
>> non-interactively
> I should say that I've noticed significant latency improvements with
> FETCH_COUNT retrieving large
Tom Lane wrote:
> I do not buy that psql's FETCH_COUNT mode is a sufficient reason
> to add it. FETCH_COUNT mode is not something you'd use
> non-interactively
I should say that I've noticed significant latency improvements with
FETCH_COUNT retrieving large resultsets, such that it would
"Daniel Verite" writes:
> Updated patches are attached.
I started to look through this, and almost immediately noted
-
- Retrieving Query Results Row-by-Row
+
+ Retrieving Query Results in chunks
This is a bit problematic, because changing the sect1 ID will
change the page's URL, eg
https:
Laurenz Albe wrote:
> Here is the code review for patch number 2:
> +static void
> +CloseGOutput(FILE *gfile_fout, bool is_pipe)
>
> It makes sense to factor out this code.
> But shouldn't these functions have a prototype at the beginning of the file?
Looking at the other static functio
Laurenz Albe wrote:
> I had a look at patch 0001 (0002 will follow).
Thanks for reviewing this!
I've implemented the suggested doc changes. A patch update
will follow with the next part of the review.
> > --- a/src/interfaces/libpq/fe-exec.c
> > +++ b/src/interfaces/libpq/fe-exec.c
> >
On Fri, 2024-03-29 at 14:07 +0100, Laurenz Albe wrote:
> I had a look at patch 0001 (0002 will follow).
Here is the code review for patch number 2:
> diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
[...]
+static bool
+SetupGOutput(PGresult *result, FILE **gfile_fout, bool *is_pipe)
[.
On Tue, 2024-01-30 at 15:29 +0100, Daniel Verite wrote:
> PFA a rebased version.
I had a look at patch 0001 (0002 will follow).
> -
> - Retrieving Query Results Row-by-Row
> +
> + Retrieving Query Results by chunks
That should be "in chunks".
> +
> +
> +
> + PQsetChunkedRows
Jakub Wartak wrote:
> when I run with default pager (more or less):
> \set FETCH_COUNT 1000
> WITH data AS (SELECT generate_series(1, 2000) as Total) select
> repeat('a',100) || data.Total || repeat('b', 800) as total_pat from
> data;
> -- it enters pager, a skip couple of pages and t
Hi Daniel,
On Tue, Jan 30, 2024 at 3:29 PM Daniel Verite wrote:
> PFA a rebased version.
Thanks for the patch! I've tested it using my original reproducer and
it works great now against the original problem description. I've
taken a quick look at the patch, it looks good for me. I've tested
us
vignesh C wrote:
> patching file src/interfaces/libpq/exports.txt
> Hunk #1 FAILED at 191.
> 1 out of 1 hunk FAILED -- saving rejects to file
> src/interfaces/libpq/exports.txt.rej
>
> Please post an updated version for the same.
PFA a rebased version.
Best regards,
--
Daniel Vérité
h
On Tue, 2 Jan 2024 at 20:28, Daniel Verite wrote:
>
> Hi,
>
> PFA a rebased version.
CFBot shows that the patch does not apply anymore as in [1]:
=== Applying patches on top of PostgreSQL commit ID
a3a836fb5e51183eae624d43225279306c2285b8 ===
=== applying patch
./v5-0001-Implement-retrieval-of-
Hi,
PFA a rebased version.
Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite
From cd0fe1d517a0e31e031fbbea1e603a715c77ea97 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20V=C3=A9rit=C3=A9?=
Date: Tue, 2 Jan 2024 14:15:48 +0100
Subject: [PATCH v5 1/2] Imple
Hi,
Here's a new version to improve the performance of FETCH_COUNT
and extend the cases when it can be used.
Patch 0001 adds a new mode in libpq to allow the app to retrieve
larger chunks of results than the single row of the row-by-row mode.
The maximum number of rows per PGresult is set by the
Tom Lane wrote:
> This gives me several "-Wincompatible-pointer-types" warnings
> [...]
> I think what you probably ought to do to avoid all that is to change
> the arguments of PrintQueryResult and nearby routines to be "const
> PGresult *result" not just "PGresult *result".
The const-ne
"Daniel Verite" writes:
> PFA an updated patch.
This gives me several "-Wincompatible-pointer-types" warnings
(as are also reported by the cfbot):
common.c: In function 'ExecQueryAndProcessResults':
common.c:1686:24: warning: passing argument 1 of 'PrintQueryTuples' from
incompatible pointer ty
I wrote:
> Here's a POC patch implementing row-by-row fetching.
PFA an updated patch.
Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index f907f5d4e8..ad5e8a5de9 100644
--- a/src/bin/psql/common.
Tom Lane wrote:
> I agree that it seems like a good idea to try.
> There will be more per-row overhead, but the increase in flexibility
> is likely to justify that.
Here's a POC patch implementing row-by-row fetching.
If it wasn't for the per-row overhead, we could probably get rid of
Ex
On Wed, Jan 4, 2023 at 6:38 PM Robert Haas wrote:
>
> On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote:
> > As you well know, psql's FETCH_COUNT mechanism is far older than
> > single-row mode. I don't think anyone's tried to transpose it
> > onto that. I agree that it seems like a good idea to t
On Wed, Jan 4, 2023 at 11:36 AM Tom Lane wrote:
> As you well know, psql's FETCH_COUNT mechanism is far older than
> single-row mode. I don't think anyone's tried to transpose it
> onto that. I agree that it seems like a good idea to try.
> There will be more per-row overhead, but the increase i
Robert Haas writes:
> On Wed, Jan 4, 2023 at 10:22 AM Daniel Verite wrote:
>> A solution would be for psql to use PQsetSingleRowMode() to retrieve
>> results row-by-row, as opposed to using a cursor, and then allocate
>> memory for only FETCH_COUNT rows at a time.
> Is there any reason that some
On Wed, Jan 4, 2023 at 10:22 AM Daniel Verite wrote:
> A solution would be for psql to use PQsetSingleRowMode() to retrieve
> results row-by-row, as opposed to using a cursor, and then allocate
> memory for only FETCH_COUNT rows at a time. Incidentally it solves
> other problems like queries conta
Jakub Wartak wrote:
> It might be a not so well known fact (?) that CTEs are not executed
> with cursor when asked to do so, but instead silently executed with
> potential huge memory allocation going on. Patch is attached. My one
> doubt is that not every statement starting with "WITH" is
30 matches
Mail list logo
