Unsure what the MariaDB change was but I suspect it was a combination of
OS upgrade to Ubuntu 24.04 (which meant the oom killer started doing
it's thing) and MariaDB upgrade to 11.4. The standard MALLOC libraries
then just didn't work as they had, causing memory to be consumed but
never released.
It's been a year since we switched over to the config as described and
the issue hasn't resurfaced.
Kind regards
Derick
On 10/04/2026 16:20, Ruben Safir via discuss wrote:
On Fri, Mar 27, 2026 at 11:28:07AM +0000, Maria wrote:
Hi Ruben,
I went through a similar experience a while back trying to deal with
the OOM Killer on Ubuntu server and database servers which had been
running fine for a long time. Underlying issue came about after
upgrading to 24.04 as this is when the OOM killer started to have an
impact on the running processes. As has been previously mentioned,
the obvious process for it to kill and free up memory is the DB one,
as it's the single highest use memory application. However, it may
not have been the DB which caused it to hit the threshold, that
could have been everything else on the server (e.g. Apache)
One of the very useful bit's of information I got out of it was this
bit of SQL, which calculates the actual maximum footprint of memory
MariaDB would need:
|SELECT ( ( @@binlog_file_cache_size + @@innodb_buffer_pool_size +
@@innodb_log_buffer_size + @@key_buffer_size + @@query_cache_size +
( @@max_connections * ( @@binlog_cache_size +
@@binlog_stmt_cache_size + @@bulk_insert_buffer_size +
@@join_buffer_size + @@net_buffer_length + @@read_buffer_size +
@@read_rnd_buffer_size + @@sort_buffer_size + @@thread_stack ) ) + (
@@slave_parallel_threads * ( @@slave_parallel_max_queued ) ) + (
@@open_files_limit * 1024 ) ) / 1024 / 1024 / 1024) AS
max_memory_GB;|
There is a full thread on my journey here -
https://dba.stackexchange.com/questions/345743/why-does-my-mariadb-application-use-more-memory-than-configured
The system has crashed again just when I thought to myself that it has
been very sable of late which is good because I've been limited to
remote access to the system. This moring the entire system has now
stalled. I can ping it, but DNS, SSh and Aphace is now all
no-responsive. I'm betting I've run out of memory again.
I will look at TC_malloc, I think as is says in that thread. This is so
agravating because this has been ROCK SOLID stable for decades and now
it is completely unstable.
What changed in the last mariadb updates on Arch?
I found btop to be extremely useful as well, as it would show be the
change in time of CPU/Memory as well as the current top processes.
Good luck!
Kind regards
Derick
On 27/03/2026 04:47, Ruben Safir via discuss wrote:
On Thu, Mar 26, 2026 at 09:21:09AM +0000, Maria wrote:
FWIW, I've seen that happen when ibps has been set too high for the host.
(Either a typo or when I've not allocated enough ram in a vm) and ooms
immediately on start.
Check my.cnf (and any .d/fragments) for this line and ensure the value is sane.
innodb_buffer_pool_size = 1G
Thanks I will look at this
-----Original Message-----
From: Ruben Safir via discuss<[email protected]> Sent:
25 March 2026 23:20
To: Ruben Safir<[email protected]>
Cc: Gordan Bobic<[email protected]>;[email protected]
Subject: [MariaDB discuss] Re: mariadb connected to apache regularly crashing
this is what it gets when it won't even rstart Mar 25 19:13:34 www3 kernel:
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=openrc.mariadb,mems_allowed=0,global_oom,task_memcg=/68,task=/usr/local/apac,pid=32686,uid=2
Mar 25 19:13:34 www3 kernel: mariadbd invoked oom-killer:
gfp_mask=0x140dca(GFP_HIGHUSER_MOVABLE|__GFP_COMP|__GFP_ZERO), order=0,
oom_score_adj=0
Mar 25 19:13:34 www3 kernel: CPU: 2 UID: 60 PID: 6030 Comm: mariadbd Not
tainted 6.12.71-1-lts #1 0c6f64fd677899b1f215535073935a6275af35f3
Mar 25 19:13:34 www3 kernel: [ 6030] 60 6030 427423 43148
43104 44 0 462848 224 0 mariadbd
Mar 25 19:13:34 www3 kernel:
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=openrc.mariadb,mems_allowed=0,global_oom,task_memcg=/openrc.mariadb,task=mariadbd,pid=6030,uid=60
Mar 25 19:13:34 www3 kernel: Out of memory: Killed process 6030
(mariadbd)total-vm:1709692kB,anon-rss:172416kB,file-rss:176kB,shmem-rss:0kB,UID:60
pgtables:452kB oom_score_adj:0 Mar 25 19:13:54 www3 mariadb[6042]: WARNING:
mariadb has already been started Mar 25 19:13:58 www3 start-stop-daemon[6062]:
Will stop /usr/bin/mariadbd
look at that vm of 1709692kb
why is it attempting to grab so much mmemory?
_______________________________________________
discuss mailing list [email protected]
To unsubscribe send an email [email protected]
--
Derick Turner - He/Him
Managing Director
e-Learn Design Ltd | Scotland's Moodle Partner
T +44 (0)845 474 4512 | M +44 (0)7808 068 797 |www.e-learndesign.co.uk
_______________________________________________
discuss mailing list -- [email protected]
To unsubscribe send an email to [email protected]
--
Derick Turner - He/Him
Managing Director
e-Learn Design Ltd | Scotland's Moodle Partner
T +44 (0)845 474 4512 | M +44 (0)7808 068 797 | www.e-learndesign.co.uk
_______________________________________________
discuss mailing list -- [email protected]
To unsubscribe send an email to [email protected]
