[OMPI users] dual Gigabit ethernet support
Hi, In between two nodes I have dual Gigabit ethernet full duplex links. I was doing benchmarking using non-blocking MPI send and receive. But I am getting only speed corresponds to one Gigabit ethernet full duplex link (< 2Gbps). I have checked using ifconfig, this transfer is using both the ports, then why I am not getting full throughput from dual Gigabit ethernet ports? Can anyone please help me in this? Regards, Jayanta Jayanta Roy National Centre for Radio Astrophysics | Phone : +91-20-25697107 Tata Institute of Fundamental Research | Fax: +91-20-25692149 Pune University Campus, Pune 411 007| e-mail : j...@ncra.tifr.res.in India
Re: [OMPI users] dual Gigabit ethernet support
Jayanta, What is your bus on this machine? If it is PCI-X 133 you are going to be limited, also memory bandwidth could also be the bottleneck. Thanks, Galen Jayanta Roy wrote: Hi, In between two nodes I have dual Gigabit ethernet full duplex links. I was doing benchmarking using non-blocking MPI send and receive. But I am getting only speed corresponds to one Gigabit ethernet full duplex link (< 2Gbps). I have checked using ifconfig, this transfer is using both the ports, then why I am not getting full throughput from dual Gigabit ethernet ports? Can anyone please help me in this? Regards, Jayanta Jayanta Roy National Centre for Radio Astrophysics | Phone : +91-20-25697107 Tata Institute of Fundamental Research | Fax: +91-20-25692149 Pune University Campus, Pune 411 007| e-mail : j...@ncra.tifr.res.in India ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] dual Gigabit ethernet support
I am also guessing you might be actually using only one of the gigabit links even though you have two available. I assume you have configured the equal-cost-multi-path (ECMP) IP routes between the two hosts correctly; even then, ECMP, as implemented in most IP stacks (not sure if there is an RFC for this), really works if you are reaching the same destination network but two different hosts. For example, ECMP would use one link to reach 10.0.0.1and another to reach 10.0.0.2, both on the 10.0.0.0/24 network. In your case, there is only one end host, so your IP stack might be actually using one link all the time. Also, the bus might be a bottleneck as Galen pointed out Thanks Durga On 8/22/06, Galen M. Shipman wrote: Jayanta, What is your bus on this machine? If it is PCI-X 133 you are going to be limited, also memory bandwidth could also be the bottleneck. Thanks, Galen Jayanta Roy wrote: >Hi, > >In between two nodes I have dual Gigabit ethernet full duplex links. I was >doing benchmarking using non-blocking MPI send and receive. But I am >getting only speed corresponds to one Gigabit ethernet full duplex link >(< 2Gbps). I have checked using ifconfig, this transfer is using both the >ports, then why I am not getting full throughput from dual Gigabit >ethernet ports? Can anyone please help me in this? > >Regards, >Jayanta > > > >Jayanta Roy >National Centre for Radio Astrophysics | Phone : +91-20-25697107 >Tata Institute of Fundamental Research | Fax: +91-20-25692149 Pune >University Campus, Pune 411 007| e-mail : j...@ncra.tifr.res.in >India > >___ >users mailing list >us...@open-mpi.org >http://www.open-mpi.org/mailman/listinfo.cgi/users > > ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users -- Devil wanted omnipresence; He therefore created communists.
Re: [OMPI users] dual Gigabit ethernet support
Hi Galen, The GigE is on the ESB2. This lives on a 4GB/sec link to the MCH. I believe we aren't really running close to the I/O bandwidth limit. This MPI transfer uses both the ports as there are increment in RX and TX bytes of both eth0 and eth1. But I am getting the same bandwidth with or without using "--mca btl_tcp_if_include eth0" in the mpirun command line. Are we missing something? Or I have to go for linux ethernet channel bonding? Regards, Jayanta On Tue, 22 Aug 2006, Galen M. Shipman wrote: Jayanta, What is your bus on this machine? If it is PCI-X 133 you are going to be limited, also memory bandwidth could also be the bottleneck. Thanks, Galen Jayanta Roy wrote: Hi, In between two nodes I have dual Gigabit ethernet full duplex links. I was doing benchmarking using non-blocking MPI send and receive. But I am getting only speed corresponds to one Gigabit ethernet full duplex link (< 2Gbps). I have checked using ifconfig, this transfer is using both the ports, then why I am not getting full throughput from dual Gigabit ethernet ports? Can anyone please help me in this? Regards, Jayanta Jayanta Roy National Centre for Radio Astrophysics | Phone : +91-20-25697107 Tata Institute of Fundamental Research | Fax: +91-20-25692149 Pune University Campus, Pune 411 007| e-mail : j...@ncra.tifr.res.in India ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users Jayanta Roy National Centre for Radio Astrophysics | Phone : +91-20-25697107 Tata Institute of Fundamental Research | Fax: +91-20-25692149 Pune University Campus, Pune 411 007| e-mail : j...@ncra.tifr.res.in India
[OMPI users] Dynamically loading mpi on Mac OS X
HI, I am trying to dynamically load mpi.dylib on Mac OS X. It seems to load fine, but when I call MPI_Init(), I get the error shown below. I can call other functions jsut fine (like MPI_Initialized). Also, my mpi install is seeing all the needed components and I can load them myself without error using dlopen. I can also compile and run mpi programs and I build openmpi with shared library support. [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_allocator_basic.so, 9): Symbol not found: _ompi_free_list_item_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_allocator_basic.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_rcache_rb.so, 9): Symbol not found: _ompi_free_list_item_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_rcache_rb.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_mpool_sm.so, 9): Symbol not found: _mca_allocator_base_components Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_mpool_sm.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_pml_ob1.so, 9): Symbol not found: _ompi_free_list_item_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_pml_ob1.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_basic.so, 9): Symbol not found: _mca_pml Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_basic.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_hierarch.so, 9): Symbol not found: _ompi_mpi_op_max Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_hierarch.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_sm.so, 9): Symbol not found: _ompi_mpi_local_convertor Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_sm.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_tuned.so, 9): Symbol not found: _mca_pml Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_tuned.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_osc_pt2pt.so, 9): Symbol not found: _ompi_request_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_osc_pt2pt.so Expected in: flat namespace (ignored) -- No available pml components were found! This means that there are no components of this type installed on your system or all the components reported that they could not be used. This is a fatal error; your MPI process is likely to abort. Check the output of the "ompi_info" command and ensure that components of this type are available on your system. You may also wish to check the value of the "component_path" MCA parameter and ensure that it has at least one directory that contains valid MCA components. -- [localhost:00973] PML ob1 cannot be selected Any Ideas? Thanks Brian Granger
[OMPI users] Dynamic loading of libmpi.dylib on Mac OS X
HI, I am trying to dynamically load mpi.dylib on Mac OS X (using ctypes in python). It seems to load fine, but when I call MPI_Init(), I get the error shown below. I can call other functions just fine (like MPI_Initialized). Also, my mpi install is seeing all the needed components and I can load them myself without error using dlopen. I can also compile and run mpi programs and I build openmpi with shared library support. [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_allocator_basic.so, 9): Symbol not found: _ompi_free_list_item_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_allocator_basic.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_rcache_rb.so, 9): Symbol not found: _ompi_free_list_item_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_rcache_rb.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_mpool_sm.so, 9): Symbol not found: _mca_allocator_base_components Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_mpool_sm.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_pml_ob1.so, 9): Symbol not found: _ompi_free_list_item_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_pml_ob1.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_basic.so, 9): Symbol not found: _mca_pml Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_basic.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_hierarch.so, 9): Symbol not found: _ompi_mpi_op_max Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_hierarch.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_sm.so, 9): Symbol not found: _ompi_mpi_local_convertor Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_sm.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_coll_tuned.so, 9): Symbol not found: _mca_pml Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_coll_tuned.so Expected in: flat namespace (ignored) [localhost:00973] mca: base: component_find: unable to open: dlopen(/usr/local/openmpi-1.1/lib/openmpi/mca_osc_pt2pt.so, 9): Symbol not found: _ompi_request_t_class Referenced from: /usr/local/openmpi-1.1/lib/openmpi/mca_osc_pt2pt.so Expected in: flat namespace (ignored) -- No available pml components were found! This means that there are no components of this type installed on your system or all the components reported that they could not be used. This is a fatal error; your MPI process is likely to abort. Check the output of the "ompi_info" command and ensure that components of this type are available on your system. You may also wish to check the value of the "component_path" MCA parameter and ensure that it has at least one directory that contains valid MCA components. -- [localhost:00973] PML ob1 cannot be selected Any Ideas? Thanks Brian Granger
