alexandremattioli commented on code in PR #444: URL: https://github.com/apache/cloudstack-documentation/pull/444#discussion_r1794108901
########## source/plugins/vxlan.rst: ########## @@ -87,101 +66,28 @@ have MTU of 1500 bytes, meaning that your physical interface/bridge must have MT In order to configure "jumbo frames" you can i.e. make physical interface/bridge with 9000 bytes MTU, then all the VXLAN interfaces will be created with MTU of 8950 bytes, and then MTU size inside Instance can be set to 8950 bytes. -Important note on max number of multicast groups (and thus VXLAN interfaces) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host). -Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host. -On Linux kernel 3.x you actually can provision more than 20, but ARP request will silently fail and cause client's networking problems -On Linux kernel 4.x you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces. -Increase needed parameter to sane value (i.e. 100 or 200) as required. -If you need to operate more than 20 Instances from different client's Network, this change above is required. - -Advanced: Build kernel and iproute2 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Even if your system doesn't support VXLAN, you can compile the kernel -and iproute2 by yourself. The following procedure is an example for -CentOS 6.4. - - -Build kernel -^^^^^^^^^^^^ - -:: - - $ sudo yum groupinstall "Development Tools" - $ sudo yum install ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel bc - - $ KERNEL_VERSION=3.10.4 - # Declare the kernel version you want to build. - - $ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-${KERNEL_VERSION}.tar.xz - $ tar xvf linux-${KERNEL_VERSION}.tar.xz - $ cd linux-${KERNEL_VERSION} - $ cp /boot/config-`uname -r` .config - $ make oldconfig - # You may keep hitting enter and choose the default. - - $ make menuconfig - # Dig into "Device Drivers" -> "Network device support", - # then select "Virtual eXtensible Local Area Network (VXLAN)" and hit space. - # Make sure it indicates "<M>" (build as module), then Save and Exit. - - # You may also want to check "IPv4 NAT" and its child nodes in "IP: Netfilter Configuration" - # and "IPv6 NAT" and its child nodes in "IPv6: Netfilter Configuration". - # In 3.10.4, you can find the options in - # "Networking support" -> "Networking options" - # -> "Network packet filtering framework (Netfilter)". - - $ make # -j N - # You may use -j N option to make the build process parallel and faster, - # generally N = 1 + (cores your machine have). - - $ sudo make modules_install - $ sudo make install - # You would get an error like "ERROR: modinfo: could not find module XXXX" here. - # This happens mainly due to config structure changes between kernel versions. - # You can ignore this error, until you find you need the kernel module. - # If you feel uneasy, you can go back to make menuconfig, - # find module XXXX by using '/' key, enable the module, build and install the kernel again. - - $ sudo vi /etc/grub.conf - # Make sure the new kernel isn't set as the default and the timeout is long enough, - # so you can select the new kernel during boot process. - # It's not a good idea to set the new kernel as the default until you confirm the kernel works fine. - - $ sudo reboot - # Select the new kernel during the boot process. +In general it's recommend to use an MTU of at least 9000 bytes or larger. Most VXLAN capable network cards and switch support an MTU of up to 9216. +Using an MTU of 9216 bytes allows for using Jumbo Frames (9000) within guest networks. -Build iproute2 -^^^^^^^^^^^^^^ -:: - - $ sudo yum install db4-devel +VXLAN using Multicast +--------------------- +The default mode for using VXLAN is Multicast. The required configuration is described below. - $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git - $ cd iproute2 - $ git tag - # Find the version that matches the kernel. - # If you built kernel 3.10.4 as above, it would be v3.10.0. +Important note on max number of multicast groups +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - $ git checkout v3.10.0 - $ ./configure - $ make # -j N - $ sudo make install +Default value of "net.ipv4.igmp_max_memberships" (cat /proc/sys/net/ipv4/igmp_max_memberships) is "20", which means that host can be joined to max 20 multicast groups (attach max 20 multicast IPs on the host). +Since all VXLAN (VTEP) interfaces provisioned on host are multicast-based (belong to certain multicast group, and thus has it's own multicast IP that is used as VTEP), this means that you can not provision more than 20 (working) VXLAN interfaces per host. -.. note:: Please use rebuild kernel and tools at your own risk. +Under Linux you can NOT provision (start) more than 20 VXLAN interfaces and error message "No buffer space available" can be observed in Cloudstack Agent logs after provisioning required bridges and VXLAN interfaces. Review Comment: ```suggestion Under Linux you can NOT by default provision (start) more than 20 VXLAN interfaces and the error message "No buffer space available" will appear in the Cloudstack Agent logs after provisioning the required bridges and VXLAN interfaces. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@cloudstack.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
