LACP FreeBSD

31.6 Link Aggregation and Failover

Written by Andrew Thompson.

31.6.1 Introduction

The lagg(4) interface allows aggregation of multiple network interfaces as one virtual interface for the purpose of providing fault-tolerance and high-speed links.

31.6.2 Operating Modes

Failover

Sends and receives traffic only through the master port. If the master port becomes unavailable, the next active port is used. The first interface added is the master port; any interfaces added after that are used as failover devices.

Cisco® Fast EtherChannel®

Cisco Fast EtherChannel (FEC), is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link. If the switch supports LACP then that should be used instead. FEC balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IPv4/IPv6 source and destination address.

LACP

The IEEE® 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. LACP will negotiate a set of aggregable links with the peer in to one or more Link Aggregated Groups (LAG). Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed, in most cases there will only be one LAG which contains all ports. In the event of changes in physical connectivity, Link Aggregation will quickly converge to a new configuration. LACP balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IPv4/IPv6 source and destination address.

Loadbalance

This is an alias of FEC mode.

Round-robin

Distributes outgoing traffic using a round-robin scheduler through all active ports and accepts incoming traffic from any active port. This mode violates Ethernet Frame ordering and should be used with caution.

31.6.3 Examples

Example 31-1. LACP aggregation with a Cisco® Switch
This example connects two interfaces on a FreeBSD machine to the switch as a single load balanced and fault tolerant link. More interfaces can be added to increase throughput and fault tolerance. Since frame ordering is mandatory on Ethernet links then any traffic between two stations always flows over the same physical link limiting the maximum speed to that of one interface. The transmit algorithm attempts to use as much information as it can to distinguish different traffic flows and balance across the available interfaces.
On the Cisco switch add the FastEthernet0/1 and FastEthernet0/2 interfaces to the channel-group 1:

interface FastEthernet0/1
 channel-group 1 mode active
 channel-protocol lacp
!
interface FastEthernet0/2
 channel-group 1 mode active
 channel-protocol lacp

On the FreeBSD machine create the lagg(4) interface using fxp0 and fxp1:

# ifconfig lagg0 create 
# ifconfig lagg0 up laggproto lacp laggport fxp0 laggport fxp1

View the interface status by running:

# ifconfig lagg0

Ports marked as ACTIVE are part of the active aggregation group that has been negotiated with the remote switch and traffic will be transmitted and received. Use the verbose output of ifconfig(8) to view the LAG identifiers.

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:05:5d:71:8d:b8
        media: Ethernet autoselect
        status: active
        laggproto lacp
        laggport: fxp1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: fxp0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

To see the port status on the switch, use show lacp neighbor:

switch# show lacp neighbor 
Flags:  S - Device is requesting Slow LACPDUs 
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode     

Channel group 1 neighbors

Partner's information:

                  LACP port                        Oper    Port     Port
Port      Flags   Priority  Dev ID         Age     Key     Number   State
Fa0/1     SA      32768     0005.5d71.8db8  29s    0x146   0x3      0x3D  
Fa0/2     SA      32768     0005.5d71.8db8  29s    0x146   0x4      0x3D

For more detail use the show lacp neighbor detail command.

Example 31-2. Failover mode
Failover mode can be used to switch over to a secondary interface if the link is lost on the master interface. Create and configure the lagg0 interface, with fxp0 as the master interface and fxp1 as the secondary interface:

# ifconfig lagg0 create
# ifconfig lagg0 up laggproto failover laggport fxp0 laggport fxp1

The interface will look something like this, the major differences will be the MAC address and the device names:

# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:05:5d:71:8d:b8
        media: Ethernet autoselect
        status: active
        laggproto failover
        laggport: fxp1 flags=0<>
        laggport: fxp0 flags=5<MASTER,ACTIVE>

Traffic will be transmitted and received on fxp0. If the link is lost on fxp0 then fxp1 will become the active link. If the link is restored on the master interface then it will once again become the active link.

Example 31-3. Failover mode between wired and wireless interfaces
For laptop users, it is usually desirable to make wireless as a secondary interface, which is to be used when the wired connection is not available. With lagg(4), it is possible to use one IP address, prefer the wired connection for both performance and security reasons, while maintaining the ability to transfer data over the wireless connection.
In this setup, we will need to override the underlying wireless interface's MAC address to match the lagg(4)'s, which is inherited from the master interface being used, the wired interface.
In this setup, we will treat the wired interface, bge0, as the master, and the wireless interface, wlan0, as the failover interface. The wlan0 was created from iwn0 which we will set up with the wired connection's MAC address. The first step would be to obtain the MAC address from the wired interface:

# ifconfig bge0
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
    ether 00:21:70:da:ae:37
    inet6 fe80::221:70ff:feda:ae37%bge0 prefixlen 64 scopeid 0x2 
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active

You can replace the bge0 to match your reality, and will get a different ether line which is the MAC address of your wired interface. Now, we change the underlying wireless interface, iwn0:

# ifconfig iwn0 ether 00:21:70:da:ae:37

Bring up the wireless interface but don't set up any IP address on it:

# ifconfig wlan0 create wlandev iwn0 ssid my_router up

Create the lagg(4) interface with bge0 as master, and failover to wlan0 if necessary:

# ifconfig lagg0 create
# ifconfig lagg0 up laggproto failover laggport bge0 laggport wlan0

The interface will look something like this, the major differences will be the MAC address and the device names:

# ifconfig lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 00:21:70:da:ae:37
        media: Ethernet autoselect
        status: active
        laggproto failover
        laggport: wlan0 flags=0<>
        laggport: bge0 flags=5<MASTER,ACTIVE>

To avoid having to do this after every reboot, one can add something like the following lines to the /etc/rc.conf file:

ifconfig_bge0="up"
ifconfig_iwn0="ether 00:21:70:da:ae:37"
wlans_iwn0="wlan0"
ifconfig_wlan0="WPA"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport bge0 laggport wlan0 DHCP"

FreeBSD: NIC Bonding / Link Aggregation / Trunking / Link Failover Tutorial

by Vivek Gite on February 13, 2009 · 15 comments

I've two Intel gigabit network card installed in HP server. I know how to setup bounding under CentOS Linux, but I'd like to do same under FreeBSD. How do I setup link aggregation of multiple network interfaces as one virtual trunk interface for the purpose of providing fault-tolerance and high-speed links under FreeBSD 7.x server?

FreeBSD has lagg - link aggregation and link failover interface. The lagg interface allows aggregation of multiple network interfaces as one virtual lagg interface for the purpose of providing fault-tolerance and high-speed links.

How do I load lagg driver?

First, you need to load the lagg driver into the memory. To load the driver as a module at boot time, place the following line in loader.conf:
if_lagg_load="YES"
Type the following command to add line to /boot/loader.conf file, enter:
# echo 'if_lagg_load="YES"' >> /boot/loader.conf
To load driver for current session use kldload command, type:
# kldload if_lagg
# kldstat

A note about custom FreeBSD kernels

If you have custom compiled kernel, you need to compile this driver into the kernel, place the following line in your FreeBSD kernel configuration file:
device lagg

How do I create a lagg interface?

A lagg interface can be created using the following command:
# ifconfig laggN create
# ifconfig lagg0 create
It can use different link aggregation protocols specified using the laggproto proto option. The driver currently supports the following aggregation protocols:

Aggregation Protocols	Description
failover	Sends and receives traffic only through the master port. If the master port becomes unavailable, the next active port is used. The first interface added is the master port; any interfaces added after that are used as failover devices.
fec	Supports Cisco EtherChannel. This is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link.
lacp	Supports the IEEE 802.3ad Link Aggregation Control Protocol (LACP) and the Marker Protocol. LACP will negotiate a set of aggregable links with the peer in to one or more Link Aggregated Groups. Each LAG is composed of ports of the same speed, set to full-duplex operation. The traffic will be balanced across the ports in the LAG with the greatest total speed, in most cases there will only be one LAG which contains all ports. In the event of changes in physical connectivity, Link Aggregation will quickly converge to a new configuration.
loadbalance	Balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. This is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link. The hash includes the Ethernet source and destination address, and, if available, the VLAN tag, and the IP source and destination address.
roundrobin	Distributes outgoing traffic using a round-robin scheduler through all active ports and accepts incoming traffic from any active port.
none	This protocol is intended to do nothing: it disables any traffic without disabling the lagg interface itself.

An Example - Create link aggregation using LACP

To create a 802.3ad link aggregation using LACP with two em Intel PRO/1000 Gigabit Ethernet adapter driver Gigabit Ethernet interfaces use the following procedure. First, you do not need to assign any IPs to em0 and em1 interfaces. Do not configure them via /etc/rc.conf. Next, bring up both interfaces, enter:
# ifconfig em0 up
# ifconfig em1 up
Create a lagg interface called lagg0, enter:
# ifconfig lagg0 create
Finally, assign an IP address to lagg0:
# ifconfig lagg0 laggproto lacp laggport em0 laggport em1 10.24.116.2 netmask 255.255.255.192
If you need to create failover between NIC instead of lacp , enter:
# ifconfig lagg0 up laggproto failover laggport em0 laggport em1 10.24.116.2 netmask 255.255.255.192
To verify status of your links, enter:
# ifconfig
You may also need to add a default gateway, enter:
# route add default 10.24.116.100
# netstat -nr
Where,

• lagg0 : Bounding interface name.
• laggproto lacp : Bounding protocol. See above tables for possible values.
• laggport em0 : Your first Ethernet interface name.
• laggport em1 : Your second Ethernet interface name.
• 10.24.116.2 : IP address for lagg0 interface.
• netmask 255.255.255.192 : Netmask address for lagg0 interface.

How do I create an active failover interface?

The following example uses an active failover interface to set up roaming between wired and wireless networks using two network devices. Whenever the wired master interface is unplugged, the wireless failover device will be used:
# ifconfig em0 up
# ifconfig ath0 nwid my_ssid up
# ifconfig lagg0 laggproto failover laggport em0 laggport ath0 192.168.1.1 netmask 255.255.255.0

How do I make link aggregation configuration persistent?

To make link aggregation configuration persistent use cloned_interfaces variable in /etc/rc.conf. Open /etc/rc.conf file, enter:
# vi /etc/rc.conf
Append the following configuration:
ifconfig_em0="up"
ifconfig_em1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport em0 laggport em1"
ipv4_addrs_lagg0="10.24.116.2/32"
defaultrouter="10.24.116.100"
Save and close the file. Now, FreeBSD will remember your configuration after the system reboot.