I am still working on a small example that shows the problem,
our problematic call is part of a fairly extensive framework so its not
easy to post that part, but see below.
As you can see the subroutine is recursive and will be calling itself
again depending on the outcome posted here.
The MPI_ALLREDUCE of dum(3) is the part that causes the ompi_free_list
to grow.
Is there an MCA parameter to limit the groth of the ompi_free_list ?
Max
---
RECURSIVE SUBROUTINE setup(l,n,listrank)
!
!
USE dagmgpar_mem
IMPLICIT NONE
INTEGER :: l,n
INTEGER, OPTIONAL :: listrank(n+1:*)
INTEGER :: nc,ierr,i,j,k,nz
LOGICAL :: slcoarse
INTEGER, POINTER, DIMENSION(:) :: jap
REAL(kind(0.0d0)), POINTER, DIMENSION(:) :: ap
LOGICAL, SAVE :: slowcoarse
REAL(kind(0.0d0)) :: fw,eta,dum(3),dumsend(3)
#ifdef WITHOUTINPLACE
REAL(kind(0.0d0)) :: dumbuffer(3)
#endif
CHARACTER(len=13) :: prtint
REAL (kind(0.0d0)) :: fff(1)
!
nn(l)=n
nlc(1)=n
IF (n > 0) THEN
nlc(2)=dt(l)%ia(n+1)-dt(l)%ia(1)
ELSE
nlc(2)=0
END IF
ngl=nlc
IF (l==2) slowcoarse=.FALSE.
slcoarse = 2*nlcp(1) < 3*nlc(1) .AND. 2*nlcp(2) < 3*nlc(2)
IF( l == nstep+1 .OR. l == maxlev &
.OR. ( ngl(1) <= maxcoarset) &
.OR. ( nglp(1) < 2*ngl(1) .AND. nglp(2) < 2*ngl(2) &
.AND. ngl(1) <= maxcoarseslowt ) &
.OR. ( slowcoarse .AND. slcoarse ) &
.OR. nglp(1) == ngl(1) ) THEN
nlev=l
dumsend(3)=-1.0d0
ELSE
dumsend(3)=dble(NPROC)
END IF
dumsend(1:2)=dble(nlc)
#ifdef WITHOUTINPLACE
dumbuffer = dum
CALL MPI_ALLREDUCE(dumbuffer,dum,3,MPI_DOUBLE_PRECISION, &
MPI_SUM,ICOMM,ierr)
#else
CALL MPI_ALLREDUCE(dumsend,dum,3,MPI_DOUBLE_PRECISION,&
MPI_SUM,ICOMM,ierr)
#endif
ngl=dum(1:2)
IF (dum(3) .LE. 0.0d0) nlev=l
slowcoarse=slcoarse
...
Yes, the number of elements each freelist accepts to allocate can be bounded.
However, we need to know which freelist we should act upon.
What exactly you means by "MPI_ALLREDUCE is called in a recursive way"? You
mean inside a loop right?
George.
On Sep 8, 2013, at 21:36 , Max Staufer wrote:
I will post a small example for testing.
It is interesting to note though that this happens only
when MPI_ALLREDUCE is called in a recursive kind of way.
Is there a possibility to limit the OMPI_free_list groth, via an --mca
parameter ?
Date: Sun, 08 Sep 2013 14:51:44 +0200
From: Max Staufer
To: us...@open-mpi.org
Subject: [OMPI users] OMPI_LIST_GROW keeps allocating memory
Message-ID: <522c72e0.9000...@gmx.net>
Content-Type: text/plain; charset=ISO-8859-15
Hi All,
using ompi 1.4.5 or 1.6.5 for that matter, I came across an
interesting thing
when an MPI function is called from in a recusivly called subroutine
(Fortran Interface)
the MPI_ALLREDUCE function allocates memory in the OMPI_LIST_GROW functions.
It does this indefinitly. In our case OMPI allocated 100GB.
is there a method to limit this behaviour ?
thanks
Max
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
