br_netfilter模块未发现

安装该模块

重新执行加载命令，结果还是原来的错误

考虑到跟内核相关配置，重启了服务器，恢复正常

1 centos版本信息查看

[root@localhost ~]# uname -a

Linux localhost.localdomain 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

k8s集群机器关闭防火墙

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# systemctl disable firewalld

Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.

Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

2 安装必要的包

k8s集群安装以下安装包

[root@k8s-node ~]# yum -y install net-tools wget vim ntpd

[root@k8s-node ~]# systemctl enable ntpd

[root@k8s-node ~]# systemctl start ntpd

3配置hosts

[root@k8s-node ~]# cat /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.248.141 k8s-master

192.168.248.142 k8s-node

4 部署master节点

4.1 Master节点安装必要的安装包

[root@k8s-master ~]# yum -y install etcd

4.2更改/etc/etcd/etcd.conf配置文件

[root@k8s-master etcd]# cat /etc/etcd/etcd.conf | grep -v "^#"

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"

ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379,http://0.0.0.0:4001"

ETCD_NAME="master"

ETCD_ADVERTISE_CLIENT_URLS= http://k8s-master:2379,http://k8s-master:4001

4.3设置开机启动并验证状态

[root@k8s-master ~]#systemctl enable etcd

[root@k8s-master ~]#systemctl start etcd

etcd检查

[root@k8s_master ~]# etcdctl -C http://k8s-master:4001 cluster-health

member 8e9e05c52164694d is healthy: got healthy result from http://k8s_master:2379 cluster is healthy

[root@k8s_master ~]# etcdctl -C http://k8s-master:2379 cluster-health

member 8e9e05c52164694d is healthy: got healthy result from http://k8s_master:2379 cluster is healthy

5 安装部署docker环境（略）

6 安装kubernetes

[root@k8s_master ~]# yum install kubernetes

安装过程中报错如下

Error: docker-ce conflicts with 2:docker-1.13.1-75.git8633870.el7.centos.x86_64

可以执行如下命令解决

1、查看安装过的docker：yum list installed | grep docker

2、卸载docker：yum remove -y docker-ce.x86_64 0:18.03.0.ce-1.el7.centos

3、删除容器镜像：rm -rf /var/lib/docker

再次安装kubernetes，安装成功，而且会自动安装docker

6.1修改apiserver服务的配置文件

[root@k8s-master kubernetes]# cat /etc/kubernetes/apiserver | grep -v "^#"

KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"

KUBE_API_PORT="--port=8080"

KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.248.141:2379"

KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

KUBE_API_ARGS=""

6.2修改config配置文件:

[root@k8s-master kubernetes]# cat /etc/kubernetes/config | grep -v "^#"

KUBE_LOGTOSTDERR="--logtostderr=true"

KUBE_LOG_LEVEL="--v=0"

KUBE_ALLOW_PRIV="--allow-privileged=false"

KUBE_MASTER="--master=http://192.168.248.141:8080"

设置开机启动，开启服务

[root@k8s-master ~]#systemctl enable kube-apiserver kube-controller-manager kube-scheduler docker

[root@k8s-master ~]#systemctl start kube-apiserver kube-controller-manager kube-scheduler docker

6.3查看服务端口:

[root@k8s-master ~]# netstat –tnlp

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name   

tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1273/master        

tcp        0      0 127.0.0.1:2380          0.0.0.0:*               LISTEN      2126/etcd          

tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      571/rpcbind        

tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1362/dnsmasq       

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      998/sshd           

tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      996/cupsd          

tcp6       0      0 ::1:25                  :::*                    LISTEN      1273/master        

tcp6       0      0 :::4001                 :::*                    LISTEN      2126/etcd          

tcp6       0      0 :::6443                 :::*                    LISTEN      3216/kube-apiserver

tcp6       0      0 :::10251                :::*                    LISTEN      3222/kube-scheduler

tcp6       0      0 :::2379                 :::*                    LISTEN      2126/etcd          

tcp6       0      0 :::10252                :::*                    LISTEN      3221/kube-controlle

tcp6       0      0 :::111                  :::*                    LISTEN      571/rpcbind        

tcp6       0      0 :::8080                 :::*                    LISTEN      3216/kube-apiserver

tcp6       0      0 :::22                   :::*                    LISTEN      998/sshd           

tcp6       0      0 ::1:631                 :::*                    LISTEN      996/cupsd 

7部署node节点

7.1安装docker（略）安装k8s（略）

7.2 Node节点主机做以下配置：

修改config配置文件

[root@k8s-node kubernetes]# cat /etc/kubernetes/config | grep -v "^#"

KUBE_LOGTOSTDERR="--logtostderr=true"

KUBE_LOG_LEVEL="--v=0"

KUBE_ALLOW_PRIV="--allow-privileged=false"

KUBE_MASTER="--master=http://192.168.248.141:8080"

修改kubelet配置文件

[root@k8s-node kubernetes]# cat /etc/kubernetes/kubelet | grep -v "^#"

KUBELET_ADDRESS="--address=0.0.0.0"

KUBELET_HOSTNAME="--hostname-override=192.168.248.142"

KUBELET_API_SERVER="--api-servers=http://192.168.248.141:8080"

KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

KUBELET_ARGS=""

设置开机启动、开启服务

[root@k8s_client1 ~]# systemctl enable kubelet kube-proxy

[root@k8s_client1 ~]# systemctl start kubelet kube-proxy

查看端口：

[root@k8s_client1 ~]# netstat –ntlp

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name   

tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1277/master        

tcp        0      0 127.0.0.1:10248         0.0.0.0:*               LISTEN      3246/kubelet       

tcp        0      0 127.0.0.1:10249         0.0.0.0:*               LISTEN      3133/kube-proxy    

tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      575/rpcbind        

tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1332/dnsmasq       

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1000/sshd          

tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      998/cupsd          

tcp6       0      0 ::1:25                  :::*                    LISTEN      1277/master        

tcp6       0      0 :::4194                 :::*                    LISTEN      3246/kubelet       

tcp6       0      0 :::10250                :::*                    LISTEN      3246/kubelet       

tcp6       0      0 :::10255                :::*                    LISTEN      3246/kubelet       

tcp6       0      0 :::111                  :::*                    LISTEN      575/rpcbind        

tcp6       0      0 :::22                   :::*                    LISTEN      1000/sshd          

tcp6       0      0 ::1:631                 :::*                    LISTEN      998/cupsd      

Master上查看集群中的节点及节点状态

[root@k8s-master kubernetes]# kubectl get node

NAME              STATUS    AGE

192.168.248.142   Ready     2m

[root@k8s-master kubernetes]# kubectl -s http://k8s-master:8080 get node

NAME              STATUS    AGE

192.168.248.142   Ready     2m

kubernetes集群搭建完成。

在k8s集群中创建pod，如果出现如下错误

其中最主要的问题是：details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)

解决方案：

查看/etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt （该链接就是上图中的说明） 是一个软链接，但是链接过去后并没有真实的/etc/rhsm，所以需要使用yum安装：

yum install *rhsm*

安装完成后，执行一下docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest

如果依然报错，可参考下面的方案：

wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm

rpm2cpio python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm | cpio -iv --to-stdout ./etc/rhsm/ca/redhat-uep.pem | tee /etc/rhsm/ca/redhat-uep.pem

 注释：rpm2cpio命令用于将rpm软件包转换为cpio格式的文件

cpio命令主要是用来建立或者还原备份档的工具程序，cpio命令可以复制文件到归档包中，或者从归档包中复文件。          

           -i   还原备份档  

           -v  详细显示指令的执行过程

这两个命令会生成/etc/rhsm/ca/redhat-uep.pem文件.

[root@k8s-node ~]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest

在master节点新建pod并查看起状态为正常

flannel是CoreOS提供用于解决Dokcer集群跨主机通讯的覆盖网络工具。它的主要思路是：预先留出一个网段，每个主机使用其中一部分，然后每个容器被分配不同的ip；让所有的容器认为大家在同一个直连的网络，底层通过UDP/VxLAN等进行报文的封装和转发。