hi, Wolfgang, Thank you for your reply.
I did manage to run valgrind memcheck by launching job (--ntask-per-node=64, --cpu-per-task=2) as " srun valgrind --tool=memcheck --leak-check=full --track-origins=yes --suppressions=openmpi-valgrind.supp /my/project/path/binary > log.log " and this is typical output of memcheck for one task: " ==1024886== HEAP SUMMARY: ==1024886== in use at exit: 10,109,942 bytes in 20,666 blocks ==1024886== total heap usage: 8,070,458 allocs, 8,049,792 frees, 10,241,574,631 bytes allocated ==1024886== ==1024855== ==1024906== HEAP SUMMARY: ==1024906== in use at exit: 10,129,660 bytes in 20,721 blocks ==1024906== total heap usage: 6,674,001 allocs, 6,653,280 frees, 50,454,219,932 bytes allocated ==1024906== ==1024910== HEAP SUMMARY: ==1024910== in use at exit: 10,110,344 bytes in 20,671 blocks ==1024910== total heap usage: 7,738,278 allocs, 7,717,607 frees, 9,563,207,155 bytes allocated " I'm not sure if the mpi-valgrind suppress ever works, but there are about 10M leaks for one task. Tim, Sincerely On Mon, Oct 2, 2023 at 1:40 AM Wolfgang Bangerth <bange...@colostate.edu> wrote: > On 9/30/23 02:01, timo Hyvärinen wrote: > > > > So my questions here are (i) Did this issue ever happened for other > deal.II > > applications, how to solve it expect increase the number of nodes or > memory > > requirements; (ii) What kind of profiling/debugger tools > nowaday's deal.II > > experts are using to dress memory issue. Should I build Valgrind by > myself? > > Does Valgrind only support MPI 2, my openMPI is v.3. > > Valgrind doesn't care. > > 6.5*10^5 unknowns with a quadratic element in 3d can probably be expected > to > take in the range of 2-5 GB. That should fit into most machines. But at > the > same time, this is a small enough problem that you can run it under > valgrind's > memory profilers on any workstation or laptop you have access to. You > could > also talk to the system administrators of the cluster you work on to see > whether they are willing to give you a more up to date version of valgrind. > > Best > W. > > -- > ------------------------------------------------------------------------ > Wolfgang Bangerth email: bange...@colostate.edu > www: http://www.math.colostate.edu/~bangerth/ > > > -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/CAArwj0He3R8QutTg6sREUFjtDpWGUxFTQvtF--yFmVCm5mHZFw%40mail.gmail.com.
