On Fri, 12 Mar 2010 15:06:33 -0500, Gus Correa wrote:
> Hi Cole, Jed
>
> I don't have much direct experience with PETSc.
Disclaimer: I've been using PETSc for several years and also work on the
library itself.
> I mostly troubleshooted other people's PETSc programs,
> and observed their perform
Gus Correa wrote:
> At each time step you exchange halo/ghost sections across
> neighbor subdomains, using MPI_Send/MPI_Recv,
> or MPI_SendRecv.
> Even better if you use non-blocking calls
> MPI_ISend/MPI_[I]Recv/MPI_Wait[all].
> Read about the advantages of non-blocking communication
> in the "M
Sent: Thursday, March 11, 2010 1:00 PM
To: Cole, Derek E; us...@open-mpi.org
Subject: Re: [OMPI users] 3D domain decomposition with MPI
On Thu, 11 Mar 2010 12:44:01 -0500, "Cole, Derek E"
wrote:
I am replying to this via the daily-digest message I got. Sorry it
wasn't sooner...
E; us...@open-mpi.org
Subject: Re: [OMPI users] 3D domain decomposition with MPI
On Thu, 11 Mar 2010 12:44:01 -0500, "Cole, Derek E"
wrote:
> I am replying to this via the daily-digest message I got. Sorry it
> wasn't sooner... I didn't realize I was getting replies
On Thu, 11 Mar 2010 12:44:01 -0500, "Cole, Derek E"
wrote:
> I am replying to this via the daily-digest message I got. Sorry it
> wasn't sooner... I didn't realize I was getting replies until I got
> the digest. Does anyone know how to change it so I get the emails as
> you all send them?
Log in
I am replying to this via the daily-digest message I got. Sorry it wasn't
sooner... I didn't realize I was getting replies until I got the digest. Does
anyone know how to change it so I get the emails as you all send them?
>>Unless your computation is so "embarrassingly parallel" that each proc
On Wed, 10 Mar 2010 22:25:43 -0500, Gus Correa wrote:
> Ocean dynamics equations, at least in the codes I've seen,
> normally use "pencil" decomposition, and are probably harder to
> handle using 3D "chunk" decomposition (due to the asymmetry imposed by
> gravity).
There is also a lot to be said
Gus Correa wrote:
I am not an MPI expert, just a user/programmer,
so I may be dead wrong in what I wrote.
Please, correct me if I am wrong.
It all sounds not only right to me but also well put.
Hi Eugene
Your argument is correct.
I would cast the conclusion slightly different, though:
The "pencil" decomposition scales better with the number of
processors than the "book" decomposition.
Your example shows this well for the extreme case
where the number of processes
is equal to the array
Eugene Loh wrote:
I don't understand what "waste" or "redundant calculations" you're
referring to here. Let's say each cell is updated based on itself and
neighboring values. Each cell has a unique "owner" who computes the
updated value. That value is shared with neighbors if the cell is n
Gus Correa wrote:
The redundant calculations of overlap points on neighbor subdomains
in general cannot be avoided.
Exchanging the overlap data across neighbor subdomain processes
cannot be avoided either.
However, **full overlap slices** are exchanged after each computational
step (in our case
Gus Correa wrote:
Also, I wonder why you want to decompose on both X and Y ("pencils"),
and not only X ("books"),
which may give you a smaller/simpler domain decomposition
and communication footprint.
Whether you can or cannot do this way depends on your
computation, which I don't know about.
Hi Derek
PS - The same book "MPI the complete reference" has a thorough
description of MPI types in Chapter 3.
You may want to create and use a MPI_TYPE_VECTOR with the
appropriate count, blocklength, and stride, to exchange all the
"0..Z" overlap slices in a single swoop.
(If I understood right,
Hi Derek
Typically in the domain decomposition codes we have here
(atmosphere, oceans, climate)
there is an overlap across the boundaries of subdomains.
Unless your computation is so "embarrassingly parallel" that
each process can operate from start to end totally independent from
the others, y
It already seems that you have a good idea of what challenges you're
facing. So, I'm unclear which part you're asking about.
Which cells do you need to update [x][y][z]? It sounds like you need
nearest neighbors. So, one technique is to allocate on each process
not just the subsection of da
Hi all. I am relatively new to MPI, and so this may be covered somewhere else,
but I can't seem to find any links to tutorials mentioning any specifics, so
perhaps someone here can help.
In C, I have a 3D array that I have dynamically allocated and access like
Array[x][y][z]. I was hoping to ca
16 matches
Mail list logo
