descartin created an issue (kamailio/kamailio#4313)
<!--
Kamailio Project uses GitHub Issues only for bugs in the code or feature
requests. Please use this template only for bug reports.
If you have questions about using Kamailio or related to its configuration
file, ask on sr-users mailing list:
*
https://lists.kamailio.org/mailman3/postorius/lists/sr-users.lists.kamailio.org/
If you have questions about developing extensions to Kamailio or its existing C
code, ask on sr-dev mailing list:
*
https://lists.kamailio.org/mailman3/postorius/lists/sr-dev.lists.kamailio.org/
Please try to fill this template as much as possible for any issue. It helps
the developers to troubleshoot the issue.
Note that an issue report may be closed automatically after about 2 months
if there is no interest from developers or community users on pursuing it, being
considered expired. In such case, it can be reopened by writing a comment that
includes
the token `/notexpired`. About two weeks before considered expired, the issue is
marked with the label `stale`, trying to notify the submitter and everyone else
that might be interested in it. To remove the label `stale`, write a comment
that
includes the token `/notstale`. Also, any comment postpone the `expire`
timeline,
being considered that there is interest in pursuing the issue.
If there is no content to be filled in a section, the entire section can be
removed.
You can delete the comments from the template sections when filling.
You can delete next line and everything above before submitting (it is a
comment).
-->
### Description
Hello how are you, we are deploying a kamailio instance on 5.5.6 version
```
version: kamailio 5.5.6 (x86_64/linux) 0125f8-dirty
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES,
TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024,
BUF_SIZE 65535, DEFAULT PKG_SIZE 32MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 0125f8 -dirty
compiled on 12:01:31 May 21 2025 with gcc 12.2.0
```
and after some minutes processing calls we are seeing some core like this
```
(gdb) bt
#0 __strlen_evex () at ../sysdeps/x86_64/multiarch/strlen-evex.S:79
#1 0x00007f5753edc258 in __vfprintf_internal (s=s@entry=0x7ffc848f6aa0,
format=format@entry=0x5570eb867da8 "%s: %.*s%s%s%sBUG: qm: fragm. %p
(address %p) beginning overwritten (%lx)! Memory allocator was called from
%s:%u. Fragment marked by %s:%lu. Exec from %s:%u.\n",
ap=ap@entry=0x7ffc848f6cc8, mode_flags=mode_flags@entry=0) at
./stdio-common/vfprintf-process-arg.c:397
#2 0x00007f5753efc758 in __vsnprintf_internal (
string=0x7ffc848f6d40 "CRITICAL: <core> [core/mem/q_malloc.c:123]:
qm_debug_check_frag(): BUG: qm: fragm. 0x7f569357aae0 (address 0x7f569357ab18)
beginning overwritten (663d610a0d303030)! Memory allocator was called from
co"..., maxlen=<optimized out>, maxlen@entry=960,
format=format@entry=0x5570eb867da8 "%s: %.*s%s%s%sBUG: qm: fragm. %p
(address %p) beginning overwritten (%lx)! Memory allocator was called from
%s:%u. Fragment marked by %s:%lu. Exec from %s:%u.\n",
args=args@entry=0x7ffc848f6cc8, mode_flags=mode_flags@entry=0) at
./libio/vsnprintf.c:114
#3 0x00007f5753f7ef2c in __vsyslog_internal (pri=138, fmt=<optimized out>,
ap=ap@entry=0x7ffc848f7150, mode_flags=mode_flags@entry=0) at
./misc/syslog.c:218
#4 0x00007f5753f7f536 in __syslog (pri=<optimized out>, fmt=<optimized out>)
at ./misc/syslog.c:91
#5 0x00005570eb6d8f25 in qm_debug_check_frag (qm=0x7f5692cd8000,
f=0x7f569357aae0, file=0x5570eb861995 "core: core/usr_avp.c", line=627,
efile=0x5570eb867c25 "core/mem/q_malloc.c", eline=511) at
core/mem/q_malloc.c:123
#6 0x00005570eb6dd50f in qm_free (qmp=0x7f5692cd8000, p=0x7f569357ab18,
file=0x5570eb861995 "core: core/usr_avp.c", func=0x5570eb863260 <__func__.8>
"destroy_avp_list_unsafe", line=627, mname=0x5570eb861990 "core")
at core/mem/q_malloc.c:511
#7 0x00005570eb6b32c3 in destroy_avp_list_unsafe (list=0x7f5693545238) at
core/usr_avp.c:627
#8 0x00007f57524b3ba9 in free_cell_helper (dead_cell=0x7f5693545038, silent=1,
fname=0x7f57525b7bdf "h_table.c", fline=466) at h_table.c:255
#9 0x00007f57524b49a2 in free_hash_table () at h_table.c:466
#10 0x00007f57524e8772 in tm_shutdown () at t_funcs.c:88
#11 0x00005570eb6081a0 in destroy_modules () at core/sr_module.c:842
#12 0x00005570eb3d8161 in cleanup (show_status=1) at main.c:575
#13 0x00005570eb3d9da8 in shutdown_children (sig=15, show_status=1) at
main.c:718
#14 0x00005570eb3dd0bc in handle_sigs () at main.c:816
#15 0x00005570eb3ea5d1 in main_loop () at main.c:1903
#16 0x00005570eb3f3f9c in main (argc=15, argv=0x7ffc848f8168) at main.c:3061
```
we are using debian12, and we see this log on the system when the crash
```
2025-07-08T06:21:55.874670+00:00 mad-proxy-4 kernel: [4744966.979706] traps:
kamailio[3282034] general protection fault ip:7f18c2a719d8 sp:7ffde5c84d68
error:0 in libc.so.6[7f18c2930000+155000]
2025-07-08T06:21:55.910639+00:00 mad-proxy-4 kernel: [4744967.018138] traps:
kamailio[3282015] general protection fault ip:561437233f64 sp:7ffde5c859a0
error:0 in kamailio[561436f31000+425000]
2025-07-08T06:21:55.919781+00:00 mad-proxy-4 systemd[1]: theseus.service: Main
process exited, code=killed, status=11/SEGV
2025-07-08T06:21:55.919880+00:00 mad-proxy-4 systemd[1]: theseus.service:
Failed with result 'signal'.
```
The traffic which seems is causing the issue may be using sipt module,
functions sipt_set_calling and sipt_destination
the libc version we have in the system is
```
ii libc6:amd64 2.36-9+deb12u10
amd64 GNU C Library: Shared libraries
ii libc6-dbg:amd64 2.36-9+deb12u10
amd64 GNU C Library: detached debugging symbols
ii libc6-dev:amd64 2.36-9+deb12u10
amd64 GNU C Library: Development Libraries and Header Files
ii libc6-i386 2.36-9+deb12u10
amd64 GNU C Library: 32-bit shared libraries for AMD64
```
could be possible any issue with the module sipt related to the library we are
using?
I checked commits between 5.5.6 and 6.0 and I think the difference is related
to kemi support and some format changes only
thanks a lot and regards
david escartin
<!--
Explain what you did, what you expected to happen, and what actually happened.
-->
### Troubleshooting
#### Reproduction
<!--
If the issue can be reproduced, describe how it can be done.
-->
#### Debugging Data
<!--
If you got a core dump, use gdb to extract troubleshooting data - full
backtrace,
local variables and the list of the code at the issue location.
gdb /path/to/kamailio /path/to/corefile
bt full
info locals
list
If you are familiar with gdb, feel free to attach more of what you consider to
be relevant.
-->
```
#0 __strlen_evex () at ../sysdeps/x86_64/multiarch/strlen-evex.S:79
79 ../sysdeps/x86_64/multiarch/strlen-evex.S: No existe el fichero o el
directorio.
(gdb) bt
#0 __strlen_evex () at ../sysdeps/x86_64/multiarch/strlen-evex.S:79
#1 0x00007f5753edc258 in __vfprintf_internal (s=s@entry=0x7ffc848f6aa0,
format=format@entry=0x5570eb867da8 "%s: %.*s%s%s%sBUG: qm: fragm. %p
(address %p) beginning overwritten (%lx)! Memory allocator was called from
%s:%u. Fragment marked by %s:%lu. Exec from %s:%u.\n",
ap=ap@entry=0x7ffc848f6cc8, mode_flags=mode_flags@entry=0) at
./stdio-common/vfprintf-process-arg.c:397
#2 0x00007f5753efc758 in __vsnprintf_internal (
string=0x7ffc848f6d40 "CRITICAL: <core> [core/mem/q_malloc.c:123]:
qm_debug_check_frag(): BUG: qm: fragm. 0x7f569357aae0 (address 0x7f569357ab18)
beginning overwritten (663d610a0d303030)! Memory allocator was called from
co"..., maxlen=<optimized out>, maxlen@entry=960,
format=format@entry=0x5570eb867da8 "%s: %.*s%s%s%sBUG: qm: fragm. %p
(address %p) beginning overwritten (%lx)! Memory allocator was called from
%s:%u. Fragment marked by %s:%lu. Exec from %s:%u.\n",
args=args@entry=0x7ffc848f6cc8, mode_flags=mode_flags@entry=0) at
./libio/vsnprintf.c:114
#3 0x00007f5753f7ef2c in __vsyslog_internal (pri=138, fmt=<optimized out>,
ap=ap@entry=0x7ffc848f7150, mode_flags=mode_flags@entry=0) at
./misc/syslog.c:218
#4 0x00007f5753f7f536 in __syslog (pri=<optimized out>, fmt=<optimized out>)
at ./misc/syslog.c:91
#5 0x00005570eb6d8f25 in qm_debug_check_frag (qm=0x7f5692cd8000,
f=0x7f569357aae0, file=0x5570eb861995 "core: core/usr_avp.c", line=627,
efile=0x5570eb867c25 "core/mem/q_malloc.c", eline=511) at
core/mem/q_malloc.c:123
#6 0x00005570eb6dd50f in qm_free (qmp=0x7f5692cd8000, p=0x7f569357ab18,
file=0x5570eb861995 "core: core/usr_avp.c", func=0x5570eb863260 <__func__.8>
"destroy_avp_list_unsafe", line=627, mname=0x5570eb861990 "core")
at core/mem/q_malloc.c:511
#7 0x00005570eb6b32c3 in destroy_avp_list_unsafe (list=0x7f5693545238) at
core/usr_avp.c:627
#8 0x00007f57524b3ba9 in free_cell_helper (dead_cell=0x7f5693545038, silent=1,
fname=0x7f57525b7bdf "h_table.c", fline=466) at h_table.c:255
#9 0x00007f57524b49a2 in free_hash_table () at h_table.c:466
#10 0x00007f57524e8772 in tm_shutdown () at t_funcs.c:88
#11 0x00005570eb6081a0 in destroy_modules () at core/sr_module.c:842
#12 0x00005570eb3d8161 in cleanup (show_status=1) at main.c:575
#13 0x00005570eb3d9da8 in shutdown_children (sig=15, show_status=1) at
main.c:718
#14 0x00005570eb3dd0bc in handle_sigs () at main.c:816
#15 0x00005570eb3ea5d1 in main_loop () at main.c:1903
#16 0x00005570eb3f3f9c in main (argc=15, argv=0x7ffc848f8168) at main.c:3061
```
#### Log Messages
<!--
Check the syslog file and if there are relevant log messages printed by
Kamailio, add them next, or attach to issue, or provide a link to download them
(e.g., to a pastebin site).
-->
```
(paste your log messages here)
```
#### SIP Traffic
<!--
If the issue is exposed by processing specific SIP messages, grab them with
ngrep or save in a pcap file, then add them next, or attach to issue, or
provide a link to download them (e.g., to a pastebin site).
-->
```
(paste your sip traffic here)
```
### Possible Solutions
<!--
If you found a solution or workaround for the issue, describe it. Ideally,
provide a pull request with a fix.
-->
### Additional Information
* **Kamailio Version** - output of `kamailio -v`
```
version: kamailio 5.5.6 (x86_64/linux) 0125f8-dirty
flags: USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
USE_MCAST, DNS_IP_HACK, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC,
DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLOCKLIST, HAVE_RESOLV_RES,
TLS_PTHREAD_MUTEX_SHARED
ADAPTIVE_WAIT_LOOPS 1024, MAX_RECV_BUFFER_SIZE 262144, MAX_URI_SIZE 1024,
BUF_SIZE 65535, DEFAULT PKG_SIZE 32MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: 0125f8 -dirty
compiled on 12:01:31 May 21 2025 with gcc 12.2.0
```
* **Operating System**:
<!--
Details about the operating system, the type: Linux (e.g.,: Debian 8.4, Ubuntu
16.04, CentOS 7.1, ...), MacOS, xBSD, Solaris, ...;
Kernel details (output of `lsb_release -a` and `uname -a`)
-->
```
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm
Linux mad-proxy-4.bts.io 6.1.0-30-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.124-1
(2025-01-12) x86_64 GNU/Linux
```
--
Reply to this email directly or view it on GitHub:
https://github.com/kamailio/kamailio/issues/4313
You are receiving this because you are subscribed to this thread.
Message ID: <kamailio/kamailio/issues/4...@github.com>_______________________________________________
Kamailio - Development Mailing List -- sr-dev@lists.kamailio.org
To unsubscribe send an email to sr-dev-le...@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the
sender!
