VXLAN on Linux (Debian)

The code that  implements the VXLAN RFC (draft) has been included in the Linux kernel mainline since version 3.7.0. Thus all that is required to implement VXLANs on a Linux system is to install a kernel with version 3.7.0 or higher and the respective iproute2 package. For this guide kernel version 3.7.4 and iproute2 version 3.7.0 are used to implement VXLANs on a Debian 6.0.6 system.

Initially download and compile a kernel with version 3.7.0 or higher. Note that the bzip2 and kernel-package packages need to be added on a default Debian 6.0.6 installation. Alternatively you can download a pre-compiled kernel 3.7.4 deb package here. Here are the commands:

apt-get install bzip2 kernel-package

Download the kernel sources and respective SHA sumfile:

cd /usr/src/
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.7.4.tar.bz2
wget http://www.kernel.org/pub/linux/kernel/v3.0/sha256sums.asc
grep `shasum -a 256 ./linux-3.7.4.tar.bz2` sha256sums.asc

The last command will compare the SHA sum of the downloaded file to the one listed in the sums file. If the SHA sum is correct you should get an output like this:

sha256sums.asc:fc3116e4ff90fabc0e9382bc69653fe6c146c373faa2f1f2b406e180670822e5  ./linux-3.7.4.tar.bz2

Now uncompress, compile, and package the new kernel image. Note that you will need to enable the VXLAN functionality when you configure the kernel with menuconfig. The option can be found under “Device Drivers -> Network device support -> Virtual eXtensible Local Area Network (VXLAN)”.

cd /usr/src/
tar jxvf linux-3.7.4.tar.bz2
cp /boot/config-$(uname -r) /usr/src/linux-3.7.4/.config
cd linux-3.7.4; make menuconfig
make-kpkg clean
fakeroot make-kpkg --initrd --revision=custom.1.0 kernel_image
sudo dpkg -i ../linux-image-3.7.4_custom.1.0_i386.deb

Note:
Several versions of the 3.x.x series have a bug and will fail when compiling with the following error:

make: *** Documentation/lguest: No such file or directory.  Stop.
make: *** [debian/stamp/build/kernel] Error 2

To work around this just do a “make menuconfig” and disable all options in the Paravirtualization section. This is a known bug for more info see here.
Lastly install the newly created package and reboot.

dpkg -i /usr/src/linux-image-3.7.4_custom.1.0_i386.deb

This will install the new kernel as the default entry in GRUB. The original kernel will still be available in the GRUB menu as the second choice.

To be able to configure VXLANs through the ip command the iproute2 package version 3.7.0 is required. First install the pkg-config, iptables-dev, libdb4.8-dev, bison and flex packages, get the matching iproute2 sources:

apt-get install pkg-config iptables-dev libdb4.8-dev bison flex
cd /usr/src
wget http://kernel.org/pub/linux/utils/net/iproute2/iproute2-3.7.0.tar.bz2
wget http://kernel.org/pub/linux/utils/net/iproute2/sha256sums.asc
grep `shasum -a 256 ./iproute2-3.7.0.tar.bz2` sha256sums.asc

The last command will compare the SHA sum of the downloaded file to the one listed in the sums file. If the SHA sum is correct you should get an output like this:

sha256sums.asc:9c9b6460539d40c6bef2cc673d7ef4d87776e69d0ae1da13e0aab11a3317025e  ./iproute2-3.7.0.tar.bz2

Now configure, compile and install the iproute2 utils:

cd /usr/src/iproute2-3.7.0/
./configure
make
make install

Now if everything went well the command ip -V will return:

ip utility, iproute2-ss121211

To configure an VXLAN with VNID 10 and moulticast address 239.0.0.10 over interface eth1 use the following command:

ip link add vxlan10 type vxlan id 10 group 239.0.0.10 ttl 4 dev eth1

This will create a network interface named vxlan10 which can be configured as a standard network interface using ifconfig or ip. For example you can assign it an IP address with the following command:

ip addr add 192.168.1.1/24 broadcast 192.168.1.255 dev vxlan10

The following command is useful to view the details of a VXLAN interface:

root@debian:~# ip -d link show vxlan10
4: vxlan10: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN mode DEFAULT 
    link/ether 52:bb:24:5f:ef:29 brd ff:ff:ff:ff:ff:ff
    vxlan id 10 group 239.0.0.10 dev eth1 port 32768 61000 ageing 300

The following command will remove a VXLAN interface and its associated configuration:

ip link delete vxlan10

At this point fully functional VXLAN interfaces can be created and configured on this Linux system.

Advertisements

6 thoughts on “VXLAN on Linux (Debian)

  1. Pingback: (Not) Connecting Docker containers between VMs with VXLAN | Chris Swan's Weblog

  2. Pingback: Connecting Docker containers between VMs with VXLAN | Chris Swan's Weblog

    • Hi Tao.

      Yes I did try that and worked fine. I also setup two KVM VMs on two hosts and bridged the VM’s interface with a VXLAN interface on each host and it also worked fine. The VMs communicated on the same subnet although the hosts themselves were connectet to a router, each host connected to a different subnet.

  3. Now finding a simple solution to using vxlan w/out multicast is what I’d like to see. VXFLD seems a possibility, same with ETCD and Flannel.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s