I suggest you look at the current offload settings on the NIC There have been quite a few bugs when bonding, vlans, bridges etc. are involved - sometimes you have to set the offload settings on the logical interface (like bond0.vlan_id) and not on the NIC. Even then, what I’ve seen is that if the interface goes down and up, those settings are _reset_ and you need to set them again, but YMMV as that depends on what the driver does (and in our case has been fixed in newer EL6 kernels).
The behaviour was almost exactly as what you see and we use Intel 82599ES (very close to X540, not sure if the same driver). run: ethtool -K $dev tso off gso off gro off lro off on the f18 and see if it makes a difference. Jan > On 18 Jul 2015, at 02:01, Alex Gorbachev <a...@iss-integration.com> wrote: > > May I suggest checking also the error counters on your network switch? > > Check speed and duplex. Is bonding in use? Is flow control on? Can you swap > the network cable? Can you swap a NIC with another node and does the problem > follow? > > Hth, Alex > > On Friday, July 17, 2015, Steve Thompson <s...@vgersoft.com > <mailto:s...@vgersoft.com>> wrote: > On Fri, 17 Jul 2015, J David wrote: > > f16 inbound: 6Gbps > f16 outbound: 6Gbps > f17 inbound: 6Gbps > f17 outbound: 6Gbps > f18 inbound: 6Gbps > f18 outbound: 1.2Mbps > > Unless the network was very busy when you did this, I think that 6 Gb/s may > not be very good either. Usually iperf will give you much more than that. For > example, between two of my OSD's, I get 9.4 Gb/s, or up to 9.9 Gb/s when > nothing else is happening. > > Steve > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com <> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
