Proposal: Better generation of values in GENERATED columns.

2019-08-26 Thread Daniel Migowski
s can be implemented without affecting performance negatively when one just does extra stuff in the error case, so I wouldn't do table scans when creating the insert initially. Any reasons why this isn't a good idea to try? Regards, Daniel Migowski

Re: Patch: New GUC prepared_statement_limit to limit memory used by prepared statements

2019-08-18 Thread Daniel Migowski
Am 19.08.2019 um 05:57 schrieb Kyotaro Horiguchi: At Sun, 18 Aug 2019 09:43:09 +0200, Daniel Migowski wrote in <6e25ca12-9484-8994-a1ee-40fdbe6af...@ikoffice.de> Am 17.08.2019 um 19:10 schrieb Ibrar Ahmed: On Sat, Aug 17, 2019 at 6:58 PM Daniel Migowski mailto:dmigow...@ikoffice.de&g

Re: Patch: New GUC prepared_statement_limit to limit memory used by prepared statements

2019-08-18 Thread Daniel Migowski
Am 17.08.2019 um 19:10 schrieb Ibrar Ahmed: On Sat, Aug 17, 2019 at 6:58 PM Daniel Migowski <mailto:dmigow...@ikoffice.de>> wrote: attached you find a patch that adds a new GUC: Quick questions before looking at the patch. prepared_statement_limit:  - Do we have a consen

Patch: New GUC prepared_statement_limit to limit memory used by prepared statements

2019-08-17 Thread Daniel Migowski
also fully released to free memory here. Regards, Daniel Migowski PS@Konstantin: This patch also includes the CachedPlanMemoryUsage function you like, maybe you like the review the patch for me? diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index cdc30fa..2da4ba8 100644

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Daniel Migowski
Am 05.08.2019 um 19:16 schrieb Andres Freund: On 2019-07-28 06:20:40 +, Daniel Migowski wrote: how do you want to generalize it? Are you thinking about a view solely for the display of the memory usage of different objects? I'm not quite sure. I'm just not sure that addin

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-08-05 Thread Daniel Migowski
Am 05.08.2019 um 18:30 schrieb Konstantin Knizhnik: On 31.07.2019 1:38, Daniel Migowski wrote: Am 31.07.2019 um 00:29 schrieb Tom Lane: Daniel Migowski writes: Ok, just have read about the commitfest thing. Is the patch OK for that? Or is there generally no love for a mem_usage column here

Re: Patch to clean Query after rewrite-and-analyze - reduces memusage up to 50% - increases TPS by up to 50%

2019-08-03 Thread Daniel Migowski
Am 03.08.2019 um 18:38 schrieb Tom Lane: Daniel Migowski writes: While examining the reasons for excessive memory usage in prepared statements I noticed that RTE_JOIN-kind RTEs contain a bunch of columnNames and joinaliasvars, that are irrelevant after the Query after has been rewritten. Uh

Patch to clean Query after rewrite-and-analyze - reduces memusage up to 50% - increases TPS by up to 50%

2019-08-03 Thread Daniel Migowski
, but everything should have been in memory when I ran the test. Regards, Daniel Migowski diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c index b945b15..af8dbc9 100644 --- a/src/backend/commands/prepare.c +++ b/src/backend/commands/prepare.c @@ -51,6 +51,36 @@ static

Re: [HACKERS] Cached plans and statement generalization

2019-08-02 Thread Daniel Migowski
Am 02.08.2019 um 10:57 schrieb Konstantin Knizhnik: On 02.08.2019 11:25, Daniel Migowski wrote: I have two suggestions however: 1. Please allow to gather information about the autoprepared statements by returning them in pg_prepared_statements view. I would love to monitor usage of them as

Re: [HACKERS] Cached plans and statement generalization

2019-08-02 Thread Daniel Migowski
list, so it is kept a bit longer. Thanks, Daniel Migowski

Proposal: Clean up RangeTblEntry nodes after query preparation

2019-08-02 Thread Daniel Migowski
drastically. *Final questions:* Is there a reason we don't just null the unused values from the RTEs? I would love to implement such a cleanup step. Or if null is not possible, just replace stuff with a simpler NOVAR node and replace names with empty strings? I believe this would reduce mem usage for PreparedStatements by >90% at least here. Regards, Daniel Migowski

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
Am 31.07.2019 um 00:29 schrieb Tom Lane: Daniel Migowski writes: Ok, just have read about the commitfest thing. Is the patch OK for that? Or is there generally no love for a mem_usage column here? If it was, I really would add some memory monitoring in our app regarding this. You should

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
's my companies mail client but switching to Thunderbird now for communication with the list, trying to be a good citizen now. Regards, Daniel Migowski

AW: AW: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
00:09 An: Daniel Migowski Cc: Andres Freund ; pgsql-hackers@lists.postgresql.org Betreff: Re: AW: Adding column "mem_usage" to view pg_prepared_statements Daniel Migowski writes: > Will my patch be considered for 12.0? The calculation of the mem_usage value > might be impro

AW: Adding column "mem_usage" to view pg_prepared_statements

2019-07-30 Thread Daniel Migowski
Hello, Will my patch be considered for 12.0? The calculation of the mem_usage value might be improved later on but because the system catalog is changed I would love to add it before 12.0 becomes stable. Regards, Daniel Migowski -Ursprüngliche Nachricht- Von: Daniel Migowski

Re: Adding column "mem_usage" to view pg_prepared_statements

2019-07-27 Thread Daniel Migowski
o when I try to build a more compact MemoryContext implementation and see how that works out. Thanks for pointing out the relevant information in the statement column of the view. Regards, Daniel Migowski -Ursprüngliche Nachricht- Von: Andres Freund Gesendet: Samstag, 27. Juli 2019 2

Adding column "mem_usage" to view pg_prepared_statements

2019-07-27 Thread Daniel Migowski
tement that is prepared as far as I would assume. Would prefer to just have the "select *..." in that column. Kind regards, Daniel Migowski

AW: Question about MemoryContexts / possible memory leak in CachedPlanSource usage

2019-07-25 Thread Daniel Migowski
Ah, you are right, I looked in fe_memutils.c. Makes sense now, thanks!! -Ursprüngliche Nachricht- Von: Andres Freund Gesendet: Donnerstag, 25. Juli 2019 22:31 An: Daniel Migowski Cc: pgsql-hack...@postgresql.org Betreff: Re: Question about MemoryContexts / possible memory leak in

Question about MemoryContexts / possible memory leak in CachedPlanSource usage

2019-07-25 Thread Daniel Migowski
created in the current MemoryContext? It so, why do I need to use palloc() instead of malloc()? Kind regards, Daniel Migowski