Hi Simon, I didn't want to hijack the other thread so I have started a new one.
The issue I was seeing was with tcp packets to media servers behind LVS. When the media streams are going out to the clients, everything is fine as the packets going through LVS are tiny. When I try to send incoming streams through LVS, I start seeing issues sometimes where an icmp is sent back to the client stating the packet is too large to forward to the real server. This causes the incoming stream to get further and further behind. Originally, on all the servers, I had mtu set to 1500 and tcpdump would be showing some packets that were larger than this and would be rejected. I increased the mtu to 9000 but then tcpdump simply reported that the same clients were sending packets larger than 9000 bytes. The last tests I did was on a server that I no longer have access to so I cannot be sure of the settings then. My current servers are running Centos 5.7 2.6.18-274.7.1.el5 and have GRO off. LRO is not listed. [root@laxlb01 ~]# ethtool -k eth0 Offload parameters for eth0: Cannot get device udp large send offload settings: Operation not supported rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: off generic-receive-offload: off I see that tcp segmentation offload is set to on. Will this cause the same effect? I would really like to get to the bottom of what is causing this problem so that I can run the incoming streams through the LVS. At the moment, we are balancing these with DNS which is not ideal. Thanks, Roger. On 24/11/2011, at 1:09 PM, Simon Horman wrote: > Hi Roger, > > if you are seeing that problem then I suggest turning off > both GRO and LRO on the interface on the director that receives > the packets. > > e.g. > # ethtool -k eth0 | egrep '(receive-offload)' > generic-receive-offload: on > large-receive-offload: on > > # ethtool -K eth0 lro off > # ethtool -K eth0 gro off > > # ethtool -k eth0 | egrep '(receive-offload)' > generic-receive-offload: off > large-receive-offload: off > > > IPVS may be used in conjunction with GRO since 2.6.39, > so it should not be necessary to disable GRO if > a 2.6.39 is used. > > IPVS is not compatible with LRO as of 3.1 (or 3.2, when it is released) > > LRO is specific to TCP. > GRO affects both TCP and UDP (and other protocols IIRC). _______________________________________________ Please read the documentation before posting - it's available at: http://www.linuxvirtualserver.org/ LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org Send requests to lvs-users-requ...@linuxvirtualserver.org or go to http://lists.graemef.net/mailman/listinfo/lvs-users
