[O-MPI users] ABI or API?

2005-03-26 Thread Jonathan Day
Hi,

With this debate over a common interface, I've noticed
that some have been talking about a common API (ie:
code-level compatibility) and others an ABI (ie:
binary-level compatibility). There's a big difference,
so I think it might be helpful if it was cleared up as
to what it was that was wanted. :)

Assuming an ABI, I suggest calling it IPM (for
Implementation-Proof MPI - yes, another recursive
acronym! :)

An ABI could be written as:

1) A wrapper that the program linked to, where the
wrapper could connect to any one of a number of MPI
libraries

2) A wrapper around the MPI library which provided a
standard interface programs could be compiled against

3) An environment that dlopen()ed both the program AND
an MPI library

Any of these would allow you to have a high degree of
independence of language-specific quirks, (1) in
particular as you could both generic and
language-specific transliteration code. You'd simply
link to what was most appropriate.

(1) would also have the benefit that you could
effectively exploit compiler-specific features (eg:
OpenMP) or language-specific parallelisms (eg: Occam)
because the wrapper would (in part or in whole) be
inside the application and therefore would have access
to all of these bug^H^H^Hfeatures.

(2) has the benefit that the library presents a common
interface, no matter what. Any variation can be
handled inside the wrapper. That means you could use
anything that provides essentially the same
capabilities, not just MPI. It might also make it
easier to handle mixed MPI-1 and MPI-2 environments,
as the differences at the application layer would be
transparent.

(3) has the advantage that you could hot-swap MPI
implementations, while an application is running. This
might be interesting in network research and possibly
useful for really exotic environments, I can't see it
would have much to offer otherwise.





__ 
Do you Yahoo!? 
Yahoo! Small Business - Try our new resources site!
http://smallbusiness.yahoo.com/resources/ 


Re: [O-MPI users] Re: [Beowulf] Alternative to MPI ABI

2005-03-26 Thread Jeff Squyres

On Mar 25, 2005, at 6:26 PM, Greg Lindahl wrote:

Making even 2 MPI implementations agree on an ABI is an enormous 
amount

of work.  Given that two major MPI implementations take opposite sides
on the pointers-vs.integers for MPI handles debate (and I suspect that
neither is willing to change), just getting them to agree on one of
them will be a major amount of work.  Then changing the internals of
one of those MPIs to match the other is another enormous amount of 
work

(death by a million cuts).


You yourself said how MPI implementers would actually implement this
without needing to change any internals: Make the C interface routines
do the same thing that F77 does today. Elapsed time: a few months,
same as MorphMPI. No internals need to be changed.

I suppose the good news is that if this is your main objection,
then it's gone.


Er... no.

Interesting: it seems that you are assuming that mpi.h should use 
integers for MPI handles.


Regardless of which way you choose, your statement "No internals have 
to change" is inaccurate.   At a minimum, *EVERY* MPI API function in 
somebody's implementation will have to change.  I'm not splitting hairs 
on what "internals" means; what I'm saying is that code in the MPI 
implementations [who have chosen "wrong"] have to change.  It doesn't 
matter whether it's code in the API calls or down in the progress 
engine;  a lot of code has to change.  And potentially a bunch of other 
infrastructure has to be changed to match (depending on how the MPI 
works).


And let's not forget that this issue is actually one of the essential 
elements of the pointers-vs-integers debate.  Some MPI implementations 
(both of mine included) have very good reasons for not having the C API 
calls do the same thing as the Fortran API calls.  But that's a 
different conversation (one that I unfortunately do not have time to 
have via e-mail).



Also, as I pointed out in my original alternate proposal, with
PatrickMPI, only those who want to use an ABI will use it.  Those who
do *not* want an ABI do not have to have it forced upon them.


I missed where anyone was being forced to do anything. MPI
implementers can support the ABI alongside their current interface or
not, it's their choice.


Er... no.

Well, let's think about this for a minute.  For an MPI implementation 
to support two interfaces, it will need 2 mpi.h's, 2 sets of MPI API 
functions, and 2 corresponding sets of infrastructure to match.  I have 
difficulty seeing MPI implementors wanting to support this -- the 
software engineering issues alone are tremendously unattractive (e.g., 
the testing scenarios have -- at least -- doubled).


It'll also lead to user confusion.  "Oh, yes, our product supports ABC 
MPI." / "But I'm using ABC MPI, and it doesn't work." / "Oh, you need 
to use the MPI ABI of ABC MPI..."  To have a single MPI implementation 
support multiple instances of its API, it [at least effectively] needs 
to be installed twice.  Users therefore have to choose which to 
compile/link against, etc.  So from the user's perspective, MPI ABC API 
is effectively Yet Another MPI (as compared to MPI ABC non-ABI).


In short: if an MPI implementation wants to support an MPI ABI, I have 
difficulty believing that it will be anything other than its 
one-and-only main interface.  So, sure, I guess an MPI implementation 
isn't *forced* to only support an MPI ABI, it's just *strongly 
recommended*.  ;-)


-

I guess I don't understand your reluctance to accept a MorphMPI-like 
solution:


- it will work
- it will take far less time to implement
- it does not require a committee (there's no need to standardize its 
mpi.h)

- no MPI implementors have to agree on anything
- no existing MPI implementations need to change
- no software engineering issues or practices for existing MPI 
implementations need to change

- people who want it will use it (and those who don't, won't)

Are you trying to jump start MPI-3?

-

Sidenote: I'll try to keep up with this conversation, but I can't 
promise anything -- it's reaching a frequency that is difficult for me 
to match.


--
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/



Re: [O-MPI users] Re: [Beowulf] Alternative to MPI ABI

2005-03-26 Thread Greg Lindahl
On Sat, Mar 26, 2005 at 06:47:41AM -0500, Jeff Squyres wrote:

> Regardless of which way you choose, your statement "No internals have 
> to change" is inaccurate.   At a minimum, *EVERY* MPI API function in 
> somebody's implementation will have to change.

That's what I call the interface, yes. It's a similar size effort to
creating an F77 interface in an MPI implementation. We've agreed on
this several times already; I'm not sure why we need to agree on it
again, or why the exact definition of the word "internals" is so
important.

> I guess I don't understand your reluctance to accept a MorphMPI-like 
> solution:

You have repeated your original MorphMPI attributes. I responded to
them, and I don't see any sign that you've read my response. This is
not the way discussions are usually held.

-- greg