On 2022-May-18, Jan Wieck wrote: > Maybe I'm missing something, but what is it that you would actually consider > a solution? Knowing your current memory consumption doesn't make the need > for allocating some right now go away. What do you envision the response of > PostgreSQL to be if we had that information about resource pressure?
What was mentioned in the talk where this issue was presented, is that people would like malloc() to return NULL when there's memory pressure, even if Linux has been configured indicating that memory overcommit is OK. The reason they can't set overcommit off is that it prevents other services in the same system from running properly. As I understand, setrlimit() sets the memory limit for any single process. But that isn't useful -- the limit needed is for the whole set of processes under postmaster. Limiting any individual process does no good. Now that's where cgroup's memory limiting features would prove useful, if they weren't totally braindead: https://www.kernel.org/doc/Documentation/cgroup-v2.txt Apparently, if the cgroup goes over the "high" limit, the processes are *throttled*. Then if the group goes over the "max" limit, OOM-killer is invoked. (I can't see any way to make this even more counterproductive to the database use case. Making the database work more slowly doesn't fix anything.) So ditch cgroups. What they (Timescale) do, is have a LD_PRELOAD library that checks status of memory pressure, and return NULL from malloc(). This then leads to clean abort of transactions and all is well. There's nothing that Postgres needs to do different than today. I suppose that what they would like, is a way to inquire into the memory pressure status at MemoryContextAlloc() time and return NULL if it is too high. How exactly this would work is unclear to me; maybe one process keeps an eye on it in an OS-specific manner, and if it does get near the maximum, set a bit in shared memory that other processes can examine when MemoryContextAlloc is called. It doesn't have to be exactly accurate; an approximation is probably okay. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
