Hi, what ofed version do you use? (ofed_info -s)
On Sun, Aug 17, 2014 at 7:16 PM, Rio Yokota <rioyok...@mac.com> wrote: > I have recently upgraded from Ubuntu 12.04 to 14.04 and OpenMPI gives the > following warning upon execution, which did not appear before the upgrade. > > WARNING: It appears that your OpenFabrics subsystem is configured to only > allow registering part of your physical memory. This can cause MPI jobs to > run with erratic performance, hang, and/or crash. > > Everything that I could find on google suggests to change log_num_mtt, but > I cannot do this for the following reasons: > 1. There is no log_num_mtt in /sys/module/mlx4_core/parameters/ > 2. Adding "options mlx4_core log_num_mtt=24" to /etc/modprobe.d/mlx4.conf > doesn't seem to change anything > 3. I am not sure how I can restart the driver because there is no > "/etc/init.d/openibd" file (I've rebooted the system but it didn't do > anything to create log_num_mtt) > > [Template information] > 1. OpenFabrics is from the Ubuntu distribution using "apt-get install > infiniband-diags ibutils ibverbs-utils libmlx4-dev" > 2. OS is Ubuntu 14.04 LTS > 3. Subnet manager is from the Ubuntu distribution using "apt-get install > opensm" > 4. Output of ibv_devinfo is: > hca_id: mlx4_0 > transport: InfiniBand (0) > fw_ver: 2.10.600 > node_guid: 0002:c903:003d:52b0 > sys_image_guid: 0002:c903:003d:52b3 > vendor_id: 0x02c9 > vendor_part_id: 4099 > hw_ver: 0x0 > board_id: MT_1100120019 > phys_port_cnt: 1 > port: 1 > state: PORT_ACTIVE (4) > max_mtu: 4096 (5) > active_mtu: 4096 (5) > sm_lid: 1 > port_lid: 1 > port_lmc: 0x00 > link_layer: InfiniBand > 5. Output of ifconfig for IB is > ib0 Link encap:UNSPEC HWaddr > 80-00-00-48-FE-80-00-00-00-00-00-00-00-00-00-00 > inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0 > inet6 addr: fe80::202:c903:3d:52b1/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1 > RX packets:26 errors:0 dropped:0 overruns:0 frame:0 > TX packets:34 errors:0 dropped:16 overruns:0 carrier:0 > collisions:0 txqueuelen:256 > RX bytes:5843 (5.8 KB) TX bytes:4324 (4.3 KB) > 6. ulimit -l is "unlimited" > > Thanks, > Rio > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2014/08/25048.php >
