Working dir can be provided with -w command line parameter. If not set, the corefile should be generated in / or a folder specific for the OS settings (e.g., macos has /corefiles).
Root is needed because some OSes have restrictions on which app can dump core files. You can try with a normal user and if not working, then using root might be the solution. Cheers, Daniel On 16/11/16 15:06, Alexandru Covalschi wrote: > Thanks for response Daniel, > > Can you point me to the location where the core dump should be generated? > Afaik kamailio was compiled in /usr/local, what is the workdir for that > setup? > I can't figure it out from systemd config file. Also, is it really necessary > to run as root? What if I set enough permissions on workdir? >> 16 нояб. 2016 г., в 11:32, Daniel-Constantin Mierla <mico...@gmail.com> >> написал(а): >> >> Hello, >> >> the plan is to freeze development of v5.0 before the Chirstmas of the >> first week of January, then it will be a 1-1.5 months of testing, >> followed by the release. So expect like 2-3 months till the stable >> release of 5.0. >> >> As for the issue, it can be a buffer overflow somewhere, not related to >> mysql module code, but triggered by use of it. >> >> It is important to get the core file for such case, before starting >> kamailio do 'ulimit -c unlimited'. You may need to run kamailio as root >> to be able to write the core file. Enable also the option for one core >> file per process, typically is: >> >> echo "1" > /proc/sys/kernel/core_uses_pid >> >> Once you get core files, extract the output of 'bt full' with gdb and >> send it over here. >> >> Cheers, >> Daniel >> >> >> On 15/11/16 22:35, Alexandru Covalschi wrote: >>> Hello list, >>> >>> We’re using dev version of Kamailio: >>> version: kamailio 5.0.0-dev4 (x86_64/linux) ff63e5 >>> flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, >>> DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, >>> Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, >>> FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, >>> USE_DST_BLACKLIST, HAVE_RESOLV_RES >>> ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, >>> MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB >>> poll method support: poll, epoll_lt, epoll_et, sigio_rt, select. >>> id: ff63e5 >>> compiled on 15:46:49 May 31 2016 with gcc 4.9.2 >>> >>> Sometimes we encounter such issue: >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: <core> >>> [db_row.c:114]: db_allocate_row(): no private memory left >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: db_mysql >>> [km_row.c:57]: db_mysql_convert_row(): could not allocate row >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: db_mysql >>> [km_res.c:188]: db_mysql_convert_rows(): error while converting row #16 >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: db_mysql >>> [km_res.c:217]: db_mysql_convert_result(): error while converting rows >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: db_mysql >>> [km_dbase.c:261]: db_mysql_store_result(): error while converting result >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: <core> >>> [db_query.c:139]: db_do_query_internal(): error while storing result >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: permissions >>> [trusted.c:91]: reload_trusted_table(): failed to query database >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: <core> >>> [db_row.c:114]: db_allocate_row(): no private memory left >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: db_mysql >>> [km_row.c:57]: db_mysql_convert_row(): could not allocate row >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: db_mysql >>> [km_dbase.c:444]: db_mysql_fetch_result(): error while converting row #15 >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: ERROR: htable >>> [ht_db.c:234]: ht_db_load_table(): Error while fetching result >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12276]: : <core> >>> [mem/q_malloc.c:468]: qm_free(): BUG: qm_free: freeing already freed >>> pointer (0x7f5ebda8ae18), called from db_mysql: km_dbase.c: >>> db_mysql_free_result(305), first free db_mysql: km_dbase.c: >>> db_mysql_free_result(305) - aborting >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12281]: CRITICAL: <core> >>> [pass_fd.c:275]: receive_fd(): EOF on 16 >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12268]: ALERT: <core> >>> [main.c:739]: handle_sigs(): child process 12276 exited by a signal 6 >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12268]: ALERT: <core> >>> [main.c:742]: handle_sigs(): core was not generated >>> Nov 9 23:30:01 sbc01 /usr/local/sbin/kamailio[12268]: INFO: <core> >>> [main.c:754]: handle_sigs(): terminating due to SIGCHLD >>> >>> The thing is we heavily use mysql module, but only to update the in-memory >>> tables by kamcmd. Each N minutes a special script updates the >>> trusted,address and htable executing kamcmd. Kamailio (and kamcmd as well) >>> talks only with localhost mysql server. >>> What I saw when encountered that issue on a live machine is that issue >>> happens only with one of child processes, any other are ok. >>> Interesting thing is that happens at the same time with machines on the >>> same «set», I mean that issue happened simultaneously with two our test >>> machines which actually didn’t have any load on them. >>> The common thing between those machines is that they are in same subnet and >>> local mysql databases are filled by scripts which query same external db. >>> I can’t confirm if there were or there weren’t any networking issues at >>> that time with those machines, but as soon as kamcmd queries localhost that >>> shouldn’t be the source of the issue. >>> >>> So my questions are: >>> 1. Has anyone encountered such thing? >>> 2. Maybe the issue is already localized so it has sense to update? We >>> actually use that on production (pls don’t throw too much rocks at me), so >>> maintenance should be properly planned and I must be sure update won’t >>> break anything. >>> 3. If update is proposed - how to do it? I mean - follow the guide >>> https://www.kamailio.org/wiki/install/devel/git or there are some other >>> tips? I suppose in ideal world I don’t even stop the binary, only restart >>> after make all && make install are done, as everything is in-memory. Am I >>> correct? >>> 4. When can we expect stable 5.0 version? (at least tell if it’s >>> months/years) >>> >>> Thanks in advance! >>> _______________________________________________ >>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list >>> sr-users@lists.sip-router.org >>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users >> -- >> Daniel-Constantin Mierla >> http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda >> Kamailio Advanced Training, Berlin, Nov 28-30, 2016 - http://www.asipto.com >> >> >> _______________________________________________ >> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list >> sr-users@lists.sip-router.org >> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio Advanced Training, Berlin, Nov 28-30, 2016 - http://www.asipto.com _______________________________________________ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
