https://bugs.kde.org/show_bug.cgi?id=433778
Bug ID: 433778
Summary: Valgrind crashes when analyzing glusterfs client
Product: valgrind
Version: unspecified
Platform: FreeBSD Ports
OS: FreeBSD
Status: REPORTED
Severity: crash
Priority: NOR
Component: memcheck
Assignee: jsew...@acm.org
Reporter: claude.e.stei...@gmail.com
Target Milestone: ---
SUMMARY
Glusterfs client on freebsd is suffering a memory leak and gets OOM killed,
even with most recent glusterfs 8.3 on most recent freebsd 12.2/13.
Statedump of glusterfs doesn't show anything helpfull, no over-sized objects
are shown.
To further analyze this issue, I wanted to start the glusterfs client with
valgrind.
Starting glusterfs client with valgrind results in some warings, an error
message from valgrind and then shuts down.
(see logs below)
STEPS TO REPRODUCE
1. Install valgrind 3.17.0 from freebsd pkg manager/ports
2. Install glusterfs from pkg manager/ports (there is still the older 8.0, 8.3
can be compiled from ports by switching 8.0 with the newer 8.3)
3. Start glusterfs under valgrind
(valgrind /usr/local/sbin/glusterfs --process-name fuse --no-daemon
--volfile-server=gluster2 --volfile-id=/volume1 /mnt/glusterfs)
OBSERVED RESULT
Warnings about unhandlet amd64-freebsd syscalls are shown (570)
The following message is shown:
valgrind: m_syswrap/syswrap-main.c:2172 (void vgPlain_client_syscall(ThreadId,
UInt)): Assertion 'VG_(iseqsigset)(&tst->sig_mask, &tst->tmp_sig_mask)' failed.
valgrind/glusterfs shuts down => filesystem is not mounted and the memory leak
can not be tested.
EXPECTED RESULT
Correctly start glusterfs under valgrind without crash should mount the
glusterfs filesystem,
filesystem load tests can be started, output of memory usage of glusterfs
should be shown after unmounting the filesystem after the loadtest.
SOFTWARE/OS VERSIONS
FreeBSD: FreeBSD Webserver5 12.2-RELEASE-p3 FreeBSD 12.2-RELEASE-p3 GENERIC
amd64
GlusterFS: glusterfs-8.3
Valgrind: valgrind-devel-3.17.0.g20200723,1
(same happens with the forked valgrind here:
https://github.com/paulfloyd/freebsd_valgrind)
ADDITIONAL INFORMATION
Full log of test run:
==20181== Memcheck, a memory error detector
==20181== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==20181== Using Valgrind-3.17.0.GIT and LibVEX; rerun with -h for copyright
info
==20181== Command: /usr/local/sbin/glusterfs --process-name fuse --no-daemon
--volfile-server=gluster2 --volfile-id=/volume1 /mnt/glusterfs
==20181==
--20181-- WARNING: unhandled amd64-freebsd syscall: 570
--20181-- You may be able to write your own handler.
--20181-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--20181-- Nevertheless we consider this a bug. Please report
--20181-- it at http://valgrind.org/support/bug_reports.html.
--20181-- WARNING: unhandled amd64-freebsd syscall: 570
--20181-- You may be able to write your own handler.
--20181-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--20181-- Nevertheless we consider this a bug. Please report
--20181-- it at http://valgrind.org/support/bug_reports.html.
==20592==
==20592== HEAP SUMMARY:
==20592== in use at exit: 668,125 bytes in 90 blocks
==20592== total heap usage: 130 allocs, 40 frees, 880,421 bytes allocated
==20592==
==20673==
==20673== HEAP SUMMARY:
==20673== in use at exit: 668,399 bytes in 98 blocks
==20673== total heap usage: 146 allocs, 48 frees, 881,543 bytes allocated
==20673==
==20592== LEAK SUMMARY:
==20592== definitely lost: 0 bytes in 0 blocks
==20592== indirectly lost: 0 bytes in 0 blocks
==20592== possibly lost: 620,589 bytes in 81 blocks
==20592== still reachable: 14,768 bytes in 8 blocks
==20592== suppressed: 32,768 bytes in 1 blocks
==20592== Rerun with --leak-check=full to see details of leaked memory
==20592==
==20592== For lists of detected and suppressed errors, rerun with: -s
==20592== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==20673== LEAK SUMMARY:
==20673== definitely lost: 224 bytes in 1 blocks
==20673== indirectly lost: 50 bytes in 7 blocks
==20673== possibly lost: 620,589 bytes in 81 blocks
==20673== still reachable: 14,768 bytes in 8 blocks
==20673== suppressed: 32,768 bytes in 1 blocks
==20673== Rerun with --leak-check=full to see details of leaked memory
==20673==
==20673== For lists of detected and suppressed errors, rerun with: -s
==20673== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
valgrind: m_syswrap/syswrap-main.c:2172 (void vgPlain_client_syscall(ThreadId,
UInt)): Assertion 'VG_(iseqsigset)(&tst->sig_mask, &tst->tmp_sig_mask)' failed.
host stacktrace:
==20181== at 0x38102002: ??? (in
/usr/local/lib/valgrind/memcheck-amd64-freebsd)
==20181== by 0x40536CFDF: ???
==20181== by 0x380FAB38: ??? (in
/usr/local/lib/valgrind/memcheck-amd64-freebsd)
==20181== by 0x38102001: ??? (in
/usr/local/lib/valgrind/memcheck-amd64-freebsd)
==20181== by 0x40536C83F: ???
==20181== by 0x402010F6F: ???
sched status:
running_tid=6
Thread 1: status = VgTs_WaitSys syscall 209 (lwpid 100118)
==20181== at 0x4EB728A: _poll (in /lib/libc.so.7)
==20181== by 0x4A4CA25: ??? (in /lib/libthr.so.3)
==20181== by 0x49080D1: ??? (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x20EDDD: main (in /usr/local/sbin/glusterfsd)
client stack range: [0x7FBFFC000 0x7FC000FFF] client SP: 0x7FC000658
valgrind stack range: [0x4029AE000 0x402AADFFF] top usage: 7152 of 1048576
Thread 2: status = VgTs_WaitSys syscall 454 (lwpid 100370)
==20181== at 0x4A5868C: ??? (in /lib/libthr.so.3)
==20181== by 0x4A4BEAF: ??? (in /lib/libthr.so.3)
==20181== by 0x4A55CAA: ??? (in /lib/libthr.so.3)
==20181== by 0x48BEC2D: ??? (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x4A49FAB: ??? (in /lib/libthr.so.3)
client stack range: [0x7FFFDFDFE000 0x7FFFDFFFDFFF] client SP: 0x7FFFDFFFDEC8
valgrind stack range: [0x404E49000 0x404F48FFF] top usage: 3088 of 1048576
Thread 3: status = VgTs_WaitSys syscall 429 (lwpid 100371)
==20181== at 0x4EA4C4A: _sigwait (in /lib/libc.so.7)
==20181== by 0x4A4F97A: ??? (in /lib/libthr.so.3)
==20181== by 0x20D9EA: glusterfs_sigwaiter (in /usr/local/sbin/glusterfsd)
==20181== by 0x4A49FAB: ??? (in /lib/libthr.so.3)
client stack range: [0x7FFFDFBFD000 0x7FFFDFDFCFFF] client SP: 0x7FFFDFDFCF38
valgrind stack range: [0x404F4D000 0x40504CFFF] top usage: 2928 of 1048576
Thread 4: status = VgTs_WaitSys syscall 240 (lwpid 100372)
==20181== at 0x4EA4E2A: _nanosleep (in /lib/libc.so.7)
==20181== by 0x4A4C8FB: ??? (in /lib/libthr.so.3)
==20181== by 0x4E0D58A: sleep (in /lib/libc.so.7)
==20181== by 0x48D4277: ??? (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x4A49FAB: ??? (in /lib/libthr.so.3)
client stack range: [0x7FFFDF9FC000 0x7FFFDFBFBFFF] client SP: 0x7FFFDFBF9F08
valgrind stack range: [0x405055000 0x405154FFF] top usage: 3024 of 1048576
Thread 5: status = VgTs_WaitSys syscall 454 (lwpid 100373)
==20181== at 0x4A5868C: ??? (in /lib/libthr.so.3)
==20181== by 0x4A4BEAF: ??? (in /lib/libthr.so.3)
==20181== by 0x4A55CAA: ??? (in /lib/libthr.so.3)
==20181== by 0x48E7337: ??? (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x48E7944: ??? (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x4A49FAB: ??? (in /lib/libthr.so.3)
client stack range: [0x7FFFDF7FB000 0x7FFFDF9FAFFF] client SP: 0x7FFFDF9FAEA8
valgrind stack range: [0x405161000 0x405260FFF] top usage: 3024 of 1048576
Thread 6: status = VgTs_Runnable syscall 232 (lwpid 100374)
==20181== at 0x4EC5C4A: _clock_gettime (in /lib/libc.so.7)
==20181== by 0x48DD8D8: timespec_now (in
/usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x6A9AB85: ??? (in
/usr/local/lib/glusterfs/8.3/xlator/cluster/afr.so)
==20181== by 0x6ABA33A: ??? (in
/usr/local/lib/glusterfs/8.3/xlator/cluster/afr.so)
==20181== by 0x48E6695: ??? (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x4E04448: ??? (in /lib/libc.so.7)
client stack range: ??????? client SP: 0x773DED8
valgrind stack range: [0x40526D000 0x40536CFFF] top usage: 4312 of 1048576
Thread 7: status = VgTs_WaitSys syscall 93 (lwpid 100375)
==20181== at 0x4F138DA: _select (in /lib/libc.so.7)
==20181== by 0x4A4CCB1: ??? (in /lib/libthr.so.3)
==20181== by 0x49237C2: runner (in /usr/local/lib/libglusterfs.so.0.0.1)
==20181== by 0x4A49FAB: ??? (in /lib/libthr.so.3)
client stack range: [0x7FFFDF3F9000 0x7FFFDF5F8FFF] client SP: 0x7FFFDF5F8F18
valgrind stack range: [0x405379000 0x405478FFF] top usage: 3184 of 1048576
Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.
If that doesn't help, please report this bug to: www.valgrind.org
In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.
--
You are receiving this mail because:
You are watching all bug changes.
