Hi Martin, that didn't solve my problem. What I have done in the meantime is replace the check in line 247 of trilinos_vector.cc with true. I don't know if this causes memory leaks or anything but my code seems to be working fine with that change. To your suggestion: Would I have also had to call the templated version for BlockVectors or only for Vectors? I only tried the latter. Would I have had to also apply some patch to my dealii library for it to work or is the patch you talked about simply that you included the functionality of the call GrowingVectorMemory<TrilinosWrappers::MPI::Vector>::release_unused_memory() in some places? I have also wanted to try MPICH instead of OpenMPI because of a post about an internal error in OpenMPI and one of the functions appearing in the call stacks sometimes not blocking properly.
Thank you for your time and your fast responses - the whole library and the people developing it and making it available are simply awesome ;) Pascal Am Mittwoch, 15. März 2017 17:26:23 UTC+1 schrieb Martin Kronbichler: > > Dear Pascal, > > This problem seems related to a problem we recently worked around in > https://github.com/dealii/dealii/pull/4043 > > Can you check what happens if you call > GrowingVectorMemory<TrilinosWrappers::MPI::Vector>::release_unused_memory() > > between your optimization steps? If a communicator gets stack in those > places it is likely a stale object somewhere that we fail to work around > for some reason. > > Best, > Martin > > On 15.03.2017 14:10, Pascal Kraft wrote: > > Dear Timo, > > I have done some more digging and found out the following. The problems > seem to happen in trilinos_vector.cc between the lines 240 and 270. > What I see on the call stacks is, that one process reaches line 261 ( ierr > = vector->GlobalAssemble (last_action); ) and then waits inside this call > at an MPI_Barrier with the following stack: > 20 <symbol is not available> 7fffd4d18f56 > 19 opal_progress() 7fffdc56dfca > 18 ompi_request_default_wait_all() 7fffddd54b15 > 17 ompi_coll_tuned_barrier_intra_recursivedoubling() 7fffcf9abb5d > 16 PMPI_Barrier() 7fffddd68a9c > 15 Epetra_MpiDistributor::DoPosts() 7fffe4088b4f > 14 Epetra_MpiDistributor::Do() 7fffe4089773 > 13 Epetra_DistObject::DoTransfer() 7fffe400a96a > 12 Epetra_DistObject::Export() 7fffe400b7b7 > 11 int Epetra_FEVector::GlobalAssemble<int>() 7fffe4023d7f > 10 Epetra_FEVector::GlobalAssemble() 7fffe40228e3 > > The other (in my case three) processes are stuck in the head of the > if/else-f statement leading up to this point, namely in the line > if (vector->Map().SameAs(v.vector > <https://www.dealii.org/8.4.0/doxygen/deal.II/classTrilinosWrappers_1_1VectorBase.html#afa80df228813b5bd94a6e780a4f5e6ae>->Map()) > > == false) > inside the call to SameAs(...) with stacks like > 15 opal_progress() 7fffdc56dfbc 14 ompi_request_default_wait_all() > 7fffddd54b15 13 ompi_coll_tuned_allreduce_intra_recursivedoubling() > 7fffcf9a4913 12 PMPI_Allreduce() 7fffddd6587f 11 Epetra_MpiComm::MinAll() > 7fffe408739e 10 Epetra_BlockMap::SameAs() 7fffe3fb9d74 > Maybe this helps. Producing a smaller example will likely not be possible > in the coming two weeks but if there are no solutions until then I can try. > Greetings, > Pascal > -- > The deal.II project is located at http://www.dealii.org/ > For mailing list/forum options, see > https://groups.google.com/d/forum/dealii?hl=en > --- > You received this message because you are subscribed to the Google Groups > "deal.II User Group" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to dealii+un...@googlegroups.com <javascript:>. > For more options, visit https://groups.google.com/d/optout. > > > -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
