β

在 Linux 服务器配置 LACP 与 VLAN

Phoenix's island 55 阅读

存储服务器不想放在 OVH 了。所以自己来托管一台机器,顺便折腾下 2x1Gbps 组 LACP Bonding。

前提:服务器需要至少 2 个千兆物理网卡,上联交换机支持 802.3ad。

配置交换机

这里使用的是 Cisco Nexus 3064PQ-10GE 交换机,我们的接口在 Eth1/21-22 ,port-channel 的配置如下:

# show interface trunk

--------------------------------------------------------------------------------
Port Native Status Port
Vlan Channel
--------------------------------------------------------------------------------
Eth1/21 1 trnk-bndl Po100
Eth1/22 1 trnk-bndl Po100
Po100 1 trunking --
show port-channel database
port-channel100
Last membership update is successful
2 ports in total, 2 ports up
First operational port is Ethernet1/21
Age of the port-channel is 0d:00h:02m:16s
Time since last bundle is 0d:00h:02m:04s
Last bundled member is Ethernet1/22
Ports: Ethernet1/21 [active ] [up] *
Ethernet1/22 [active ] [up]

配置服务器

服务器操作系统是 Arch Linux,由于蜜汁问题 netctl 无法启动网卡,就只好用 systemd-networkd 啦。

麻烦一些,但是也还算顺利。与往常一样,折腾服务器网络的时候需要备着 IPMI 以防 connection lost。

内核模块

需要加载 bonding 模块。将模块名写入列表,文件 /etc/modules-load.d/bonding.conf ,内容只需要一行:

bonding

先别急着加载模块,为了防止模块自动建立一个默认网卡影响后续配置,以及设置 LACP Mode=4 … 等等,先加入一行参数。文件 /etc/modprobe.d/bonding.conf

options bonding mode=4 miimon=100 max_bonds=0

然后安装 ifenslave 包,再 modprobe bonding 即可。

bonding 虚拟网卡

首先创建一个虚拟网卡的设备。文件 /etc/systemd/network/bond0.netdev 内容为

[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=802.3ad
TransmitHashPolicy=layer2+3
LACPTransmitRate=fast
AdSelect=bandwidth

然后在此虚拟网卡上创建网络。这里使用两个物理网卡 eth0 eth1 作为 bundle,交换机上的 VLAN id 是 113 。文件 /etc/systemd/network/bond0.network 内容为

[Match]
Name=bond0

[Network]
VLAN=vlan113
BindCarrier=eth0 eth1

接下来分别为 eth0 eth1 建立网络设置。

[Match]
Name=eth0

[Network]
Bond=bond0
[Match]
Name=eth1

[Network]
Bond=bond0

最后是 VLAN 的设置。前面设置了上联 VLAN id 是 113,这里分别建立 VLAN 的虚拟网卡(based on bond0) 并设置网络(IP, etc)。

[NetDev]
Name=vlan113
Kind=vlan

[VLAN]
Id=113
[Match]
Name=vlan113

[Network]
VLAN=vlan113

[Address]
Address=10.1.0.100/24

[Route]
Destination=0.0.0.0/0
Gateway=10.1.0.1
DNS=1.1.1.1

[Address]
Address=2600:x:x:x::2/64

[Route]
Gateway=2600:x:x:x::1

多个地址、IPv6 等可以写多个 [Address] [Route]

至此就完成啦。开启 systemd-networkd 的自启动:

systemctl enable systemd-networkd.service

然后重启网络:

systemctl restart systemd-networkd.service

如果配置都没有问题,网络会中断十几秒然后恢复。现在查看网卡列表已经可以看到组合的网卡了:

# ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
link/ether <REDACTED> brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
link/ether <REDACTED> brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether <REDACTED> brd ff:ff:ff:ff:ff:ff
5: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether <REDACTED> brd ff:ff:ff:ff:ff:ff
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether <REDACTED> brd ff:ff:ff:ff:ff:ff
7: vlan113@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether <REDACTED> brd ff:ff:ff:ff:ff:ff

ethtool 查看 bond0 的速率显示 2000Mb/s

# ethtool bond0
Settings for bond0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 2000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes

搞定收工(‘・ω・’)

Reference:

存储服务器不想放在 OVH 了。所以自己来托管一台机器,顺便折腾下 2x1Gbps 组 LACP Bonding。

作者:Phoenix's island
Sun will shine on the horizon.
原文地址:在 Linux 服务器配置 LACP 与 VLAN, 感谢原作者分享。

发表评论