I've been working for the MVAPICH project for around three years.
Since
this thread is discussing MVAPICH, I thought I should post to this
thread. Galen's description of MVAPICH is not accurate. MVAPICH uses
RDMA for short message to deliver performance benefits to the
applications. However, i
Galen Shipman wrote:
>Hi Jean,
>
>You probably are not seeing overhead costs so much as you are seeing
>the difference between using send/recv for small messages, which Open
>MPI uses, and RDMA for small messages. If you are comparing against
>another implementation that uses RDMA for small messa
On Thu, 2006-02-09 at 16:37 -0700, Brightwell, Ronald wrote:
> I apologize if it seems like I'm picking on you.
No offense taken.
> I'm hypersensitive to
> people trying to make judgements based on micro-benchmark performance.
> I've been trying to make an argument that two-node ping-pong laten
>
> No, I assumed it based on comparisions between doing and not doing small
> msg rdma at various scales, from a paper Galen pointed out to me.
> http://www.cs.unm.edu/~treport/tr/05-10/Infiniband.pdf
>
Actually, I wasn't so much concerned with how you jumped to your conclusion.
I just wanted t
On Feb 9, 2006, at 3:03 PM, Jean-Christophe Hugly wrote:
On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote:
[...]
From an adoption perspective, though, the ability to shine in
micro-benchmarks is important, even if it means using an ad-hoc
tuning.
There is some justification for it af
On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote:
> > [...]
> >
> > >From an adoption perspective, though, the ability to shine in
> > micro-benchmarks is important, even if it means using an ad-hoc tuning.
> > There is some justification for it after all. There are small clusters
> > out t
When do you plan on having the small-msg rdma option available ?
I would expect this in the very near future, we will be discussing
schedules next week.
Thanks,
Galen
J-C
--
Jean-Christophe Hugly
PANTA
> [...]
>
> >From an adoption perspective, though, the ability to shine in
> micro-benchmarks is important, even if it means using an ad-hoc tuning.
> There is some justification for it after all. There are small clusters
> out there (many more than big ones, in fact) so taking maximum advantage
>
>
> > So far, the best latency I got from ompi is 5.24 us, and the best I
> > got from mvapich is 3.15.
> > I am perfectly ready to accept that ompi scales better and that may be
> > more important (except to the marketing dept :-) ), but I do not
> > understand your explanation based on small-
I would recommend reading the following tech report, it should shed
some light to how these things work :
http://www.cs.unm.edu/research/search_technical_reports_by_keyword/?
string=infiniband
1 - It does not seem that mvapich does RDMA for small messages. It will
do RDMA for any message
> As we plan to drop all support for the old
> > generation of PML/PTL, I don't think is a wise idea to spend time on
> > the openib PTL to make it working with uniq ...
> >
> >Thanks,
> > george.
> >
>
> With the change to ob1/BTLs, there was also a refactoring of data
> structures that
>> For Heroic latencies on IB we would need to use small message RDMA and
>> poll each peers dedicated memory region for completion.
>
> Well, I tried to play around with the eager_limit, min_rdma, etc. I did
> not see the latency of messages of a given size be lowered by changing
> the tresholds
>
> On Feb 8, 2006, at 7:06 PM, Jean-Christophe Hugly wrote:
>
>> But should I understand from all this that the "direct" mode will
>> never
>> actually work ? It seems that if you need at least two transports,
>> then
>> none of them can be the hardwired unique one, right ? Unless there's a
>> bui
On Feb 8, 2006, at 7:06 PM, Jean-Christophe Hugly wrote:
But should I understand from all this that the "direct" mode will
never
actually work ? It seems that if you need at least two transports,
then
none of them can be the hardwired unique one, right ? Unless there's a
built-in switch bet
> you need to specify both the transport and self, such as:
> mpirun -mca btl self,tcp
I found that the reason why I was no-longer able to run without openib
was that I had some openib-specific tunables on the command line. I
expected the params would get ignored, but instead it just sat there.
Sorry, more questions to answer:
On the other hand I am not sure it could even work at all, as whenever
I
tried at run-time to limit the list to just one transport (be it tcp or
openib, btw), mpi apps would not start.
you need to specify both the transport and self, such as:
mpirun -mca btl s
Hi Jean,
You probably are not seeing overhead costs so much as you are seeing
the difference between using send/recv for small messages, which Open
MPI uses, and RDMA for small messages. If you are comparing against
another implementation that uses RDMA for small messages then yes, you
will
Hi guys,
Does someone know what the framework costs in term of latency ?
Righ now the latency I get with the openib btl is not great: 5.35 us. I
was looking at what I could do to get it down. I tried to get openib be
the only btl but the build process refused.
On the other hand I am not sure it
18 matches
Mail list logo
