I'm assuming that you are referring to the fact that MPI uses an "int" to specify the "length" of a message (i.e., the number of items to be sent and received), like this:
MPI_Send(buffer, count, datatype, ...) "count" is an int, which means that it has a max value of INT_MAX, which, on today's 64 bit machines, is about 2 billion. Note, however, that the message that is sent is really "count times length of datatype", and datatype can represent something more than 1 byte. For example, if you use (count=2B, datatype=MPI_INT), that represents ~2 billion MPI_INTs, and an "int" is typically 4 bytes, so you're sending an ~8 billion byte message. And you can use the MPI datatype constructors to create a datatype that is longer than that (google around; you'll find some good MPI datatype constructor tutorials). E.g., you can send (count=2B, datatype=my_datatype), where my_datatype represents, say, 128 bytes. Then you'll be sending a message that is (2 billion * 128) bytes. And so on. So there's two typical ways to send "extra large" messages in MPI: 1. Use a (count, datatype) tuple that will represent more than 2 billion bytes. 2. Break your message up into multiple parts. E.g., if you have 16GB message to send, send it in 16 1GB chunks (which I assume is what Changsheng Jiang's code snipit was doing). On Jul 18, 2013, at 4:54 AM, mohammad assadsolimani <m.assadsolim...@jesus.ch> wrote: > Dear Mike, > > Thank you for your response. I use this program > http://reduze.hepforge.org/ > > I did not rite its code myself.Did I understand you right? > You mean I have to use the new version of OMPI namely: > > OMPI 1.7 > > because the version which I use is: mpirun (Open MPI) 1.5.5 > What is IB ? > > thank you in advanced > > Cheers, > Mohammad > > >> >> Message: 5 >> Date: Wed, 17 Jul 2013 16:15:41 +0300 >> From: Mike Dubman <mi...@dev.mellanox.co.il> >> Subject: Re: [OMPI users] max. message size >> To: Open MPI Users <us...@open-mpi.org> >> Message-ID: >> <CAAO1KyakymR7wqOMwW31ntZrbYShMzkhh9Wvm7yy1Swofmh=1...@mail.gmail.com> >> Content-Type: text/plain; charset="iso-8859-1" >> >> do you use IB as a transport? max message size in IB/RDMA is limited to >> 2G, but OMPI 1.7 splits large buffers during RDMA into 2G chunks. >> >> >> >> On Wed, Jul 17, 2013 at 11:51 AM, mohammad assadsolimani < >> m.assadsolim...@jesus.ch> wrote: >> >>> >>> Dear all, >>> >>> I do my PhD in physics and use a program, which uses openmpi for >>> a sophisticated calculation. >>> But there is a Problem with "max. message size ". That is limited to ~2GB. >>> Someone suggested that I have to use chunks i.e. I have to disassemble >>> the massages >>> in smaller massages. That might be nice, but I do not know how? >>> I often was searching the last time in internet however I did not get an >>> example. >>> Is there any other possibility to increase this volume without >>> manipulation of >>> the code? >>> >>> The version of my ompi: mpirun (Open MPI) 1.5.5 >>> >>> >>> I am very grateful for all of your help and thank you in advanced >>> Mohammad >>> >>> ------------------------------**------------------------------**-- >>> Webmail: http://mail.livenet.ch >>> Glauben entdecken: http://www.jesus.ch >>> Christliches Webportal: http://www.livenet.ch >>> >>> >>> >>> ______________________________**_________________ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/**mailman/listinfo.cgi/users<http://www.open-mpi.org/mailman/listinfo.cgi/users> >>> >> -------------- next part -------------- >> HTML attachment scrubbed and removed >> >> ------------------------------ >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> End of users Digest, Vol 2631, Issue 1 >> ************************************** >> > > > > -------------------------------------------------------------- > Webmail: http://mail.livenet.ch > Glauben entdecken: http://www.jesus.ch > Christliches Webportal: http://www.livenet.ch > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
