Hi
Our host "bib" owns 4 virtual CPU, 8 GB RAM and 18 GB swap space. It
runs on Debian GNU/Linux 12 with Koha 24.05.01 and around 78'000
bibliographic records.
Since some time Koha sporadically just stops (today already twice), in
such cases log file "/var/log/syslog" contains messages as attached at
the end of this e-mail (the "oom-killer" is not always invoked by
memcached). The messages say the host is out of memory so the Koha
process was killed... I have already increased the swap space from 2 GB
to 18 GB but to no avail.
How can we find out what is consuming so much memory causing Koha to get
killed? How can we avoid this?
Best wishes: Michael
--
Geschäftsführer · Diplombibliothekar BBS, Informatiker eidg. Fachausweis
Admin Kuhn GmbH · Pappelstrasse 20 · 4123 Allschwil · Schweiz
T 0041 (0)61 261 55 61 · E m...@adminkuhn.ch · W www.adminkuhn.ch
2025-03-10T12:48:00.668764+01:00 bib kernel: [769596.762201] memcached
invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP),
order=0, oom_score_adj=0
2025-03-10T12:48:00.670036+01:00 bib kernel: [769596.762885] CPU: 3 PID:
2332685 Comm: memcached Not tainted 6.1.0-31-amd64 #1 Debian 6.1.128-1
2025-03-10T12:48:00.670038+01:00 bib kernel: [769596.763487] Hardware
name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
2025-03-10T12:48:00.670039+01:00 bib kernel: [769596.764019] Call Trace:
2025-03-10T12:48:00.670039+01:00 bib kernel: [769596.764528] <TASK>
2025-03-10T12:48:00.670040+01:00 bib kernel: [769596.765027]
dump_stack_lvl+0x44/0x5c
2025-03-10T12:48:00.670040+01:00 bib kernel: [769596.765539]
dump_header+0x4a/0x211
2025-03-10T12:48:00.670041+01:00 bib kernel: [769596.766045]
oom_kill_process.cold+0xb/0x10
2025-03-10T12:48:00.670041+01:00 bib kernel: [769596.766535]
out_of_memory+0x1fd/0x4c0
2025-03-10T12:48:00.670042+01:00 bib kernel: [769596.767046]
__alloc_pages_slowpath.constprop.0+0xc83/0xe40
2025-03-10T12:48:00.670042+01:00 bib kernel: [769596.767537]
__alloc_pages+0x305/0x330
2025-03-10T12:48:00.670043+01:00 bib kernel: [769596.768014]
folio_alloc+0x17/0x50
2025-03-10T12:48:00.670043+01:00 bib kernel: [769596.768499]
__filemap_get_folio+0x155/0x340
2025-03-10T12:48:00.670044+01:00 bib kernel: [769596.768967]
filemap_fault+0x139/0x910
2025-03-10T12:48:00.670044+01:00 bib kernel: [769596.769395] ?
filemap_map_pages+0x153/0x700
2025-03-10T12:48:00.670044+01:00 bib kernel: [769596.769820]
__do_fault+0x33/0x110
2025-03-10T12:48:00.670045+01:00 bib kernel: [769596.770303]
do_fault+0x1b9/0x410
2025-03-10T12:48:00.670157+01:00 bib kernel: [769596.770734]
__handle_mm_fault+0x660/0xfa0
2025-03-10T12:48:00.670298+01:00 bib kernel: [769596.771189]
handle_mm_fault+0xdb/0x2d0
2025-03-10T12:48:00.670299+01:00 bib kernel: [769596.771594]
do_user_addr_fault+0x191/0x550
2025-03-10T12:48:00.670300+01:00 bib kernel: [769596.771982]
exc_page_fault+0x70/0x170
2025-03-10T12:48:00.670300+01:00 bib kernel: [769596.772411]
asm_exc_page_fault+0x22/0x30
2025-03-10T12:48:00.670301+01:00 bib kernel: [769596.772829] RIP:
0033:0x557b12827140
2025-03-10T12:48:00.670302+01:00 bib kernel: [769596.773207] Code:
Unable to access opcode bytes at 0x557b12827116.
2025-03-10T12:48:00.670302+01:00 bib kernel: [769596.773589] RSP:
002b:00007ffc7f74a9c8 EFLAGS: 00010246
2025-03-10T12:48:00.670303+01:00 bib kernel: [769596.773991] RAX:
00007fc1802bd440 RBX: 0000557b44749ac0 RCX: 0000000000000669
2025-03-10T12:48:00.670303+01:00 bib kernel: [769596.774387] RDX:
0000000000000000 RSI: 0000000000000001 RDI: 00000000ffffffff
2025-03-10T12:48:00.670303+01:00 bib kernel: [769596.774723] RBP:
0000000000000001 R08: 0000557b12827140 R09: 00007fc1802dd000
2025-03-10T12:48:00.670304+01:00 bib kernel: [769596.775111] R10:
0000000000000000 R11: 0000000000000000 R12: 0000000000000001
2025-03-10T12:48:00.670305+01:00 bib kernel: [769596.775506] R13:
0000000000000000 R14: 000000007fffffff R15: 0000557b1285cb80
2025-03-10T12:48:00.670305+01:00 bib kernel: [769596.775831] </TASK>
2025-03-10T12:48:00.670305+01:00 bib kernel: [769596.776226] Mem-Info:
2025-03-10T12:48:00.670365+01:00 bib kernel: [769596.776592]
active_anon:550829 inactive_anon:1391840 isolated_anon:0
2025-03-10T12:48:00.670366+01:00 bib kernel: [769596.776592]
active_file:0 inactive_file:94 isolated_file:0
2025-03-10T12:48:00.670366+01:00 bib kernel: [769596.776592]
unevictable:0 dirty:0 writeback:0
2025-03-10T12:48:00.670367+01:00 bib kernel: [769596.776592]
slab_reclaimable:9935 slab_unreclaimable:13188
2025-03-10T12:48:00.670367+01:00 bib kernel: [769596.776592]
mapped:1134 shmem:5697 pagetables:5740
2025-03-10T12:48:00.670367+01:00 bib kernel: [769596.776592]
sec_pagetables:0 bounce:0
2025-03-10T12:48:00.670368+01:00 bib kernel: [769596.776592]
kernel_misc_reclaimable:0
2025-03-10T12:48:00.670368+01:00 bib kernel: [769596.776592] free:25298
free_pcp:0 free_cma:0
2025-03-10T12:48:00.670368+01:00 bib kernel: [769596.789186] Node 0
active_anon:2203316kB inactive_anon:5567360kB active_file:80kB
inactive_file:156kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB mapped:4536kB dirty:0kB writeback:0kB shmem:22788kB
shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 876544kB writeback_tmp:0kB
kernel_stack:9056kB pagetables:22960kB sec_pagetables:0kB
all_unreclaimable? no
2025-03-10T12:48:00.670369+01:00 bib kernel: [769596.790069] Node 0 DMA
free:14336kB boost:0kB min:128kB low:160kB high:192kB
reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB
active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB
present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB
local_pcp:0kB free_cma:0kB
2025-03-10T12:48:00.670370+01:00 bib kernel: [769596.790924]
lowmem_reserve[]: 0 2899 7843 7843 7843
2025-03-10T12:48:00.670370+01:00 bib kernel: [769596.791252] Node 0
DMA32 free:44376kB boost:0kB min:24936kB low:31168kB high:37400kB
reserved_highatomic:0KB active_anon:842432kB inactive_anon:2119316kB
active_file:0kB inactive_file:84kB unevictable:0kB writepending:0kB
present:3129196kB managed:3018408kB mlocked:0kB bounce:0kB free_pcp:0kB
local_pcp:0kB free_cma:0kB
2025-03-10T12:48:00.670370+01:00 bib kernel: [769596.792175]
lowmem_reserve[]: 0 0 4943 4943 4943
2025-03-10T12:48:00.670371+01:00 bib kernel: [769596.792455] Node 0
Normal free:42480kB boost:0kB min:42516kB low:53144kB high:63772kB
reserved_highatomic:0KB active_anon:1360884kB inactive_anon:3447664kB
active_file:120kB inactive_file:0kB unevictable:0kB writepending:0kB
present:5242880kB managed:5062604kB mlocked:0kB bounce:0kB free_pcp:0kB
local_pcp:0kB free_cma:0kB
2025-03-10T12:48:00.670371+01:00 bib kernel: [769596.793398]
lowmem_reserve[]: 0 0 0 0 0
2025-03-10T12:48:00.670372+01:00 bib kernel: [769596.793716] Node 0 DMA:
0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB
1*2048kB (M) 3*4096kB (M) = 14336kB
2025-03-10T12:48:00.670372+01:00 bib kernel: [769596.794404] Node 0
DMA32: 115*4kB (UME) 68*8kB (UME) 110*16kB (UME) 94*32kB (UME) 78*64kB
(UME) 56*128kB (UE) 40*256kB (UME) 15*512kB (E) 3*1024kB (UE) 3*2048kB
(E) 0*4096kB = 45068kB
2025-03-10T12:48:00.670373+01:00 bib kernel: [769596.795118] Node 0
Normal: 412*4kB (UME) 307*8kB (UME) 166*16kB (UME) 167*32kB (UME)
83*64kB (UME) 50*128kB (UME) 23*256kB (E) 13*512kB (ME) 7*1024kB (ME)
0*2048kB 0*4096kB = 43528kB
2025-03-10T12:48:00.670373+01:00 bib kernel: [769596.795916] Node 0
hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
2025-03-10T12:48:00.670373+01:00 bib kernel: [769596.796271] Node 0
hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
2025-03-10T12:48:00.670374+01:00 bib kernel: [769596.796713] 155111
total pagecache pages
2025-03-10T12:48:00.670374+01:00 bib kernel: [769596.797122] 149255
pages in swap cache
2025-03-10T12:48:00.670374+01:00 bib kernel: [769596.797463] Free swap
= 0kB
2025-03-10T12:48:00.670375+01:00 bib kernel: [769596.797848] Total swap
= 1998844kB
2025-03-10T12:48:00.670922+01:00 bib kernel: [769596.798238] 2097017
pages RAM
2025-03-10T12:48:00.670926+01:00 bib kernel: [769596.798577] 0 pages
HighMem/MovableOnly
2025-03-10T12:48:00.670926+01:00 bib kernel: [769596.798954] 72924 pages
reserved
2025-03-10T12:48:00.671386+01:00 bib kernel: [769596.799356] 0 pages
hwpoisoned
2025-03-10T12:48:00.671388+01:00 bib kernel: [769596.799693] Tasks state
(memory values in pages):
2025-03-10T12:48:00.671388+01:00 bib kernel: [769596.800070] [ pid ]
uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
2025-03-10T12:48:00.671389+01:00 bib kernel: [769596.800463] [ 274]
0 274 30075 22 241664 274 -250 systemd-journal
2025-03-10T12:48:00.671389+01:00 bib kernel: [769596.800839] [ 295]
0 295 6770 278 86016 221 -1000 systemd-udevd
2025-03-10T12:48:00.671390+01:00 bib kernel: [769596.801224] [ 524]
997 524 22521 154 77824 82 0 systemd-timesyn
2025-03-10T12:48:00.671390+01:00 bib kernel: [769596.801603] [ 609]
0 609 2372 35 53248 28 0 cron
2025-03-10T12:48:00.671390+01:00 bib kernel: [769596.801997] [ 610]
100 610 2011 143 61440 8 -900 dbus-daemon
2025-03-10T12:48:00.671391+01:00 bib kernel: [769596.802387] [ 612]
0 612 55447 184 81920 116 0 rsyslogd
2025-03-10T12:48:00.671391+01:00 bib kernel: [769596.802749] [ 613]
0 613 4185 203 73728 73 0 systemd-logind
2025-03-10T12:48:00.671391+01:00 bib kernel: [769596.803147] [ 629]
0 629 895 40 49152 3 0 atd
2025-03-10T12:48:00.671392+01:00 bib kernel: [769596.803558] [ 666]
0 666 1499 202 53248 45 0 dhclient
2025-03-10T12:48:00.671392+01:00 bib kernel: [769596.803913] [ 715]
0 715 1435 100 49152 105 0 dhclient
2025-03-10T12:48:00.671393+01:00 bib kernel: [769596.804313] [ 747]
104 747 2208 550 61440 34 0 epmd
2025-03-10T12:48:00.671393+01:00 bib kernel: [769596.804716] [ 753]
0 753 173229 3280 430080 880 0 fail2ban-server
2025-03-10T12:48:00.671393+01:00 bib kernel: [769596.805078] [ 774]
106 774 920414 28673 1294336 23431 -500 beam.smp
2025-03-10T12:48:00.671393+01:00 bib kernel: [769596.805494] [ 797]
0 797 2188 20 53248 2 0 agetty
2025-03-10T12:48:00.671394+01:00 bib kernel: [769596.806006] [ 812]
0 812 3859 213 65536 120 -1000 sshd
2025-03-10T12:48:00.671394+01:00 bib kernel: [769596.806567] [ 841]
106 841 616 19 53248 6 -500 erl_child_setup
2025-03-10T12:48:00.671395+01:00 bib kernel: [769596.807045] [ 1774]
0 1774 10665 108 69632 53 0 master
2025-03-10T12:48:00.671395+01:00 bib kernel: [769596.807442] [ 1779]
102 1779 10700 127 81920 48 0 qmgr
2025-03-10T12:48:00.671396+01:00 bib kernel: [769596.807848] [ 2023]
106 2023 848 11 45056 14 -500 inet_gethost
2025-03-10T12:48:00.671396+01:00 bib kernel: [769596.808192] [ 2026]
106 2026 848 24 45056 8 -500 inet_gethost
2025-03-10T12:48:00.671397+01:00 bib kernel: [769596.808560] [ 2034]
106 2034 644 14 40960 14 -500 sh
2025-03-10T12:48:00.671397+01:00 bib kernel: [769596.808953] [ 3262]
103 3262 2058793 518769 7716864 392357 0 mariadbd
2025-03-10T12:48:00.671397+01:00 bib kernel: [769596.809287] [ 392666]
102 392666 12109 128 90112 225 0 tlsmgr
2025-03-10T12:48:00.671398+01:00 bib kernel: [769596.809647] [2332677]
0 2332677 3456 434 61440 303 0 apache2
2025-03-10T12:48:00.671398+01:00 bib kernel: [769596.810031] [2332685]
105 2332685 107470 2677 159744 2787 0 memcached
2025-03-10T12:48:00.671399+01:00 bib kernel: [769596.810358] [2332735]
1003 2332735 2128 21 49152 15 0 daemon
2025-03-10T12:48:00.671399+01:00 bib kernel: [769596.810703] [2332737]
1003 2332737 14826 161 110592 256 0 zebrasrv
2025-03-10T12:48:00.671399+01:00 bib kernel: [769596.811075] [2332802]
1003 2332802 6211 2612 94208 287 0 starman
master
2025-03-10T12:48:00.671400+01:00 bib kernel: [769596.811400] [2332846]
1003 2332846 2128 37 49152 0 0 daemon
2025-03-10T12:48:00.671400+01:00 bib kernel: [769596.811734] [2332847]
1003 2332847 29751 13047 221184 414 0
background_jobs
2025-03-10T12:48:00.671401+01:00 bib kernel: [769596.812076] [2332875]
1003 2332875 2128 21 45056 14 0 daemon
2025-03-10T12:48:00.671401+01:00 bib kernel: [769596.812396] [2332877]
1003 2332877 41320 19112 323584 4526 0
background_jobs
2025-03-10T12:48:00.671401+01:00 bib kernel: [769596.812718] [2332902]
1003 2332902 2128 13 49152 22 0 daemon
2025-03-10T12:48:00.671402+01:00 bib kernel: [769596.813044] [2332904]
1003 2332904 73759 36200 540672 6622 0
rebuild_zebra.p
2025-03-10T12:48:00.671402+01:00 bib kernel: [769596.813369] [1011996]
102 1011996 10671 154 73728 0 0 pickup
2025-03-10T12:48:00.671402+01:00 bib kernel: [769596.813651] [1014020]
33 1014020 3517 422 61440 316 0 apache2
2025-03-10T12:48:00.671403+01:00 bib kernel: [769596.813976] [1015250]
33 1015250 3517 426 61440 311 0 apache2
2025-03-10T12:48:00.671403+01:00 bib kernel: [769596.814307] [1018044]
33 1018044 3517 472 61440 265 0 apache2
2025-03-10T12:48:00.671403+01:00 bib kernel: [769596.814583] [1018109]
33 1018109 3517 465 61440 272 0 apache2
2025-03-10T12:48:00.671404+01:00 bib kernel: [769596.814882] [1018113]
33 1018113 3517 473 61440 265 0 apache2
2025-03-10T12:48:00.671404+01:00 bib kernel: [769596.815171] [1020113]
33 1020113 3517 473 61440 264 0 apache2
2025-03-10T12:48:00.671404+01:00 bib kernel: [769596.815506] [1024635]
33 1024635 3517 474 61440 264 0 apache2
2025-03-10T12:48:00.671405+01:00 bib kernel: [769596.815796] [1026139]
33 1026139 3517 474 61440 264 0 apache2
2025-03-10T12:48:00.671405+01:00 bib kernel: [769596.816089] [1026170]
33 1026170 3517 473 61440 264 0 apache2
2025-03-10T12:48:00.671405+01:00 bib kernel: [769596.816378] [1026641]
1003 1026641 1185289 1122584 9445376 29621 0
/usr/share/koha
2025-03-10T12:48:00.671406+01:00 bib kernel: [769596.816688] [1026652]
33 1026652 3517 473 61440 264 0 apache2
2025-03-10T12:48:00.671406+01:00 bib kernel: [769596.816972] [1027075]
1003 1027075 99496 34733 749568 33424 0
/usr/share/koha
2025-03-10T12:48:00.671407+01:00 bib kernel: [769596.817289] [1027426]
1003 1027426 3594 624 65536 241 0 apache2
2025-03-10T12:48:00.671416+01:00 bib kernel: [769596.817615] [1027453]
1003 1027453 3592 669 61440 240 0 apache2
2025-03-10T12:48:00.671416+01:00 bib kernel: [769596.817890] [1027749]
1003 1027749 16877 179 114688 2268 0 zebrasrv
2025-03-10T12:48:00.671416+01:00 bib kernel: [769596.818218] [1027761]
1003 1027761 3538 563 61440 244 0 apache2
2025-03-10T12:48:00.671417+01:00 bib kernel: [769596.818501] [1027769]
1003 1027769 16966 2354 114688 229 0 zebrasrv
2025-03-10T12:48:00.671417+01:00 bib kernel: [769596.818812] [1027770]
1003 1027770 3523 559 61440 241 0 apache2
2025-03-10T12:48:00.671417+01:00 bib kernel: [769596.819085]
oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/koha-common.service,task=/usr/share/koha,pid=1026641,uid=1003
2025-03-10T12:48:00.671418+01:00 bib kernel: [769596.819717] Out of
memory: Killed process 1026641 (/usr/share/koha) total-vm:4741156kB,
anon-rss:4490332kB, file-rss:4kB, shmem-rss:0kB, UID:1003
pgtables:9224kB oom_score_adj:0
2025-03-10T12:48:00.721396+01:00 bib systemd[1]: koha-common.service: A
process of this unit has been killed by the OOM killer.
2025-03-10T12:48:03.702481+01:00 bib koha-common[1027819]: Stopping
Plack daemon for bib:.
2025-03-10T12:48:03.755927+01:00 bib koha-common[1027861]: Stopping Koha
worker daemon for bib (default):.
2025-03-10T12:48:03.789372+01:00 bib koha-common[1027886]: Stopping Koha
worker daemon for bib (long_tasks):.
2025-03-10T12:48:03.825612+01:00 bib koha-common[1027911]: Stopping Koha
indexing daemon for bib:.
2025-03-10T12:48:05.057975+01:00 bib systemd[1]: koha-common.service:
Failed with result 'oom-kill'.
2025-03-10T12:48:05.058217+01:00 bib systemd[1]: koha-common.service:
Consumed 2d 22h 25min 610ms CPU time.
_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/
