there was a proof-of-concept implementation of MPI for Plan 9 at LANL which relied on primitives specific to the OS in order to implement the communication (mounted remote file servers and namespaces). The cool thing about it was that it allowed one to write MPI programs using shell scripts. The funding for the project didn't materialize and it wasn't pursued further.
i don't have any copies of the two technical reports that discussed it (only references), nor do i have the code anymore, so i'm not sure how useful this whole email is, sorry.