On Thu, Aug 31, 2023 at 9:19 AM John Morris
wrote:
> Here is an updated version of the earlier work.
>
> This version:
>1) Tracks memory as requested by the backend.
>2) Includes allocations made during program startup.
>3) Optimizes the "fast path" to only update two local variables.
On Thu, 2023-01-05 at 14:13 -0600, Justin Pryzby wrote:
>
> I suggest to close the associated CF entry.
Closed with status of Withdrawn. If that is invalid, please advise.
> (Also, the people who participated in this thread may want to be
> included in the other thread going forward.)
Explicitl
On Thu, Jan 05, 2023 at 01:58:33PM -0500, Reid Thompson wrote:
> On Tue, 2023-01-03 at 16:25 +0530, vignesh C wrote:
> > ...
> > The patch does not apply on top of HEAD as in [1], please post a
> > rebased patch:
> >...
> > Regards,
> > Vignesh
>
> Per conversation in thread listed below, patches
On Tue, 2023-01-03 at 16:25 +0530, vignesh C wrote:
> ...
> The patch does not apply on top of HEAD as in [1], please post a
> rebased patch:
>...
> Regards,
> Vignesh
Per conversation in thread listed below, patches have been submitted to the
"Add the ability to limit the amount of memory that
On Thu, 8 Dec 2022 at 19:44, Reid Thompson
wrote:
>
> On Sun, 2022-11-27 at 09:40 -0600, Justin Pryzby wrote:
> > > BTW, these should have some kind of prefix, like PG_ALLOC_* to
> > > avoid causing the same kind of problem for someone else that
> > > another header caused for you by defining some
On Sun, 2022-11-27 at 09:40 -0600, Justin Pryzby wrote:
> > BTW, these should have some kind of prefix, like PG_ALLOC_* to
> > avoid causing the same kind of problem for someone else that
> > another header caused for you by defining something somewhere
> > called IGNORE (ignore what, I don't know)
Hi,
On 2022-12-06 08:47:55 -0500, Reid Thompson wrote:
> The intent was to capture and display all the memory allocated to the
> backends, for admins/users/max_total_backend_memory/others to utilize.
It's going to be far less accurate than that. For one, there's a lot of
operating system resource
On Fri, 2022-12-02 at 09:18 -0800, Andres Freund wrote:
> Hi,
>
> On 2022-12-02 11:09:30 -0500, Reid Thompson wrote:
> > It appears to me that Postmaster populates the local variable that
> > *my_allocated_bytes points to. That allocation is passed to forked
> > children, and if not zeroed out, wi
Hi,
On 2022-12-02 11:09:30 -0500, Reid Thompson wrote:
> It appears to me that Postmaster populates the local variable that
> *my_allocated_bytes points to. That allocation is passed to forked
> children, and if not zeroed out, will be double counted as part of
> the child allocation. Is this inva
On Mon, 2022-11-28 at 10:59 -0800, Andres Freund wrote:
> On 2022-11-26 22:10:06 -0500, Reid Thompson wrote:
> > - zero allocated bytes after fork to avoid double counting
> > postmaster allocations
>
> I still don't understand this - postmaster shouldn't be counted at
> all. It
> doesn't have
On 2022-11-26 22:10:06 -0500, Reid Thompson wrote:
>- zero allocated bytes after fork to avoid double counting postmaster
> allocations
I still don't understand this - postmaster shouldn't be counted at all. It
doesn't have a PgBackendStatus. There simply shouldn't be any tracked
allocations
On Sun, Nov 27, 2022 at 7:41 AM Justin Pryzby wrote:
> On Sun, Nov 27, 2022 at 12:32:19AM -0500, Reid Thompson wrote:
> > @@ -32,6 +33,12 @@ typedef enum BackendState
> > STATE_DISABLED
> > } BackendState;
> >
> > +/* Enum helper for reporting memory allocated bytes */
> > +enum allocation
On Sun, Nov 27, 2022 at 12:32:19AM -0500, Reid Thompson wrote:
> @@ -32,6 +33,12 @@ typedef enum BackendState
> STATE_DISABLED
> } BackendState;
>
> +/* Enum helper for reporting memory allocated bytes */
> +enum allocation_direction
> +{
> + DECREASE = -1,
> + INCREASE = 1,
> +};
On Sat, Nov 26, 2022 at 9:32 PM Reid Thompson
wrote:
> On Sat, 2022-11-26 at 22:10 -0500, Reid Thompson wrote:
> > Code rebased to current master.
> > Updated to incorporate additional recommendations from the the list
> >- add units to variables in view
> >- remove 'backend_' prefix from
On Sun, Nov 27, 2022 at 12:32:19AM -0500, Reid Thompson wrote:
> attempt to remedy cfbot windows build issues
You can trigger those tests under your own/private repo by pushing a
branch to github. See src/tools/ci/README
I suppose the cfbot page ought to point that out.
--
Justin
On Sat, 2022-11-26 at 22:10 -0500, Reid Thompson wrote:
> Code rebased to current master.
> Updated to incorporate additional recommendations from the the list
> - add units to variables in view
> - remove 'backend_' prefix from variables/functions
> - update documentation
> - add funct
On Wed, 2022-11-09 at 08:54 -0500, Reid Thompson wrote:
> Hi Andres,
> Thanks for looking at this and for the feedback. Responses inline
> below.
>
>> On Fri, 2022-11-04 at 19:41 -0700, Andres Freund wrote:
> > Hi,
> >
> On 2022-11-04 08:56:13 -0400, Reid Thompson wrote:
> >
> > I'm not convince
On Wed, 2022-11-09 at 09:23 -0500, Reid Thompson wrote:
> Hi Melanie,
> Thank you for looking at this and for the feedback. Responses inline
> below.
>
> On Mon, 2022-11-07 at 16:17 -0500, Melanie Plageman wrote:
> >
> > It doesn't seem like you need the backend_ prefix in the view since
> > that
Code rebased to current master.
Updated to incorporate additional recommendations from the the list
- add units to variables in view
- remove 'backend_' prefix from variables/functions
- update documentation
- add functional test for allocated_bytes
- refactor allocation reporting to
Hi,
2022-11-09 08:54:54 -0500, Reid Thompson wrote:
> Thanks for looking at this and for the feedback. Responses inline below.
> > > +void
> > > +pgstat_report_backend_allocated_bytes_decrease(uint64
> > > deallocation)
> > > +{
> > > + volatile PgBackendStatus *beentry = MyBEEntry;
> > > +
Hi Melanie,
Thank you for looking at this and for the feedback. Responses inline
below.
On Mon, 2022-11-07 at 16:17 -0500, Melanie Plageman wrote:
> On Fri, Nov 04, 2022 at 08:56:13AM -0400, Reid Thompson wrote:
> > From a8de5d29c0c6f10962181926a49ad4fec1e52bd1 Mon Sep 17 00:00:00
> > 2001
> > Fro
Hi Andres,
Thanks for looking at this and for the feedback. Responses inline below.
On Fri, 2022-11-04 at 19:41 -0700, Andres Freund wrote:
> Hi,
>
> On 2022-11-04 08:56:13 -0400, Reid Thompson wrote:
>
> I'm not convinced that counting DSM values this way is quite right.
> There are a few uses
On Fri, Nov 04, 2022 at 08:56:13AM -0400, Reid Thompson wrote:
> From a8de5d29c0c6f10962181926a49ad4fec1e52bd1 Mon Sep 17 00:00:00 2001
> From: Reid Thompson
> Date: Thu, 11 Aug 2022 12:01:25 -0400
> Subject: [PATCH] Add tracking of backend memory allocated to pg_stat_activity
>
> This new field
Hi,
On 2022-11-04 08:56:13 -0400, Reid Thompson wrote:
> From a8de5d29c0c6f10962181926a49ad4fec1e52bd1 Mon Sep 17 00:00:00 2001
> From: Reid Thompson
> Date: Thu, 11 Aug 2022 12:01:25 -0400
> Subject: [PATCH] Add tracking of backend memory allocated to pg_stat_activity
>
> This new field display
On Fri, 2022-11-04 at 11:06 +0100, Drouvot, Bertrand wrote:
> Hi,
>
> It looks like the patch does not apply anymore since b1099eca8f.
>
> Regards,
>
Thanks,
rebased to current master attached.
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
reid.thomp...@crunchydata.com
www.cr
Hi,
On 10/25/22 8:59 PM, Reid Thompson wrote:
On Tue, 2022-10-25 at 14:51 -0400, Reid Thompson wrote:
patch rebased to current master
actually attach the patch
It looks like the patch does not apply anymore since b1099eca8f.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS O
On Tue, 2022-10-25 at 14:51 -0400, Reid Thompson wrote:
> patch rebased to current master
>
actually attach the patch
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
reid.thomp...@crunchydata.com
www.crunchydata.com
From ab654a48ec7bfbc3bc377c5757a04f1756e72e79 Mon Sep 17 00:00:00
patch rebased to current master
--
Reid Thompson
Senior Software Engineer
Crunchy Data, Inc.
reid.thomp...@crunchydata.com
www.crunchydata.com
Greetings,
* Drouvot, Bertrand (bdrou...@amazon.com) wrote:
> On 9/9/22 7:08 PM, Justin Pryzby wrote:
> >On Fri, Sep 09, 2022 at 12:34:15PM -0400, Stephen Frost wrote:
> >>>While we are at it, what do you think about also recording the max memory
> >>>allocated by a backend? (could be useful and w
On Fri, Sep 09, 2022 at 12:34:15PM -0400, Stephen Frost wrote:
> > While we are at it, what do you think about also recording the max memory
> > allocated by a backend? (could be useful and would avoid sampling for which
> > there is no guarantee to sample the max anyway).
FYI, that's already kind
Greetings,
* Kyotaro Horiguchi (horikyota@gmail.com) wrote:
> At Tue, 06 Sep 2022 17:10:49 -0400, Reid Thompson
> wrote in
> > I'm open to guidance on testing for performance degradation. I did
> > note some basic pgbench comparison numbers in the thread regarding
> > limiting backend m
Greetings,
* Drouvot, Bertrand (bdrou...@amazon.com) wrote:
> On 9/1/22 3:28 AM, Kyotaro Horiguchi wrote:
> >At Wed, 31 Aug 2022 12:05:55 -0500, Justin Pryzby
> >wrote in
> >>On Wed, Aug 31, 2022 at 12:03:06PM -0400, Reid Thompson wrote:
> >>>Attached is a patch to
> >>>Add tracking of backend m
At Wed, 07 Sep 2022 17:08:41 +0900 (JST), Kyotaro Horiguchi
wrote in
> At Tue, 06 Sep 2022 17:10:49 -0400, Reid Thompson
> wrote in
> > On Thu, 2022-09-01 at 13:43 +0900, Kyotaro Horiguchi wrote:
> > >
> > > > @@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
> > > >
At Tue, 06 Sep 2022 17:10:49 -0400, Reid Thompson
wrote in
> On Thu, 2022-09-01 at 13:43 +0900, Kyotaro Horiguchi wrote:
> >
> > > @@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
> > > return NULL;
> > >
> > > context->mem_alloc
On Thu, 2022-09-01 at 13:43 +0900, Kyotaro Horiguchi wrote:
>
> > @@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
> > return NULL;
> >
> > context->mem_allocated += blksize;
> > + pgstat_report_backend_mem_allocated_increase(blksi
>
On Wed, 2022-08-31 at 12:05 -0500, Justin Pryzby wrote:
> > + proargmodes =>
> > '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}'
> > ,
>
> In the past, there was concern about making pg_stat_activity wider by
> adding information that's less-essential than what's been there fo
At Wed, 31 Aug 2022 12:03:06 -0400, Reid Thompson
wrote in
> Attached is a patch to
> Add tracking of backend memory allocated to pg_stat_activity
> @@ -916,6 +930,7 @@ AllocSetAlloc(MemoryContext context, Size size)
> return NULL;
>
> context->mem_allocat
At Wed, 31 Aug 2022 12:05:55 -0500, Justin Pryzby wrote
in
> On Wed, Aug 31, 2022 at 12:03:06PM -0400, Reid Thompson wrote:
> > Hi Hackers,
> >
> > Attached is a patch to
> > Add tracking of backend memory allocated to pg_stat_activity
>
> > + proargmodes =>
> > '{i,o,o,o,o,o,o,o,o,o,o,o,o,
On Wed, Aug 31, 2022 at 12:03:06PM -0400, Reid Thompson wrote:
> Hi Hackers,
>
> Attached is a patch to
> Add tracking of backend memory allocated to pg_stat_activity
> + proargmodes =>
> '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
In the past, there was concern about
39 matches
Mail list logo
