β

kubernetes 搭建

nosa.me 228 阅读

三台机器( Centos 7 ):

docker-test-master0.hy01  10.0.24.206,作为 kubernetes master;

docker-test-etcd0.hy01  10.0.27.165,作为 etcd server;

docker-test-node0.hy01  10.0.41.200,作为 kubernetes node。

在 docker-test-master0.hy01 上安装:

yum -y install kubernetes-master

在 docker-test-etcd0.hy01 上安装:

yum -y install etcd

在 docker-test-node0.hy01 机器上安装:

yum -y install bridge-utils docker flannel kubernetes-node kubernetes-client

修改 docker-test-master0.hy01 配置。

修改 /etc/kubernetes/apiserver:



###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#

# The address on the local server to listen to.
KUBE_API_ADDRESS="--address=0.0.0.0"

# The port on the local server to listen on.
KUBE_API_PORT="--port=8080"

# Port minions listen on
KUBELET_PORT="--kubelet_port=10250"

# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd_servers=http://docker-test-etcd0.hy01:2379"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.253.0.0/16"

# default admission control policies
# KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

# Add your own!
KUBE_API_ARGS=""

修改 docker-test-etcd0.hy01 配置。

修改 /etc/etcd/etcd.conf,如下:



# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_LISTEN_PEER_URLS="http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
#ETCD_INITIAL_CLUSTER="default=http://localhost:2380"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#
#[proxy]
#ETCD_PROXY="off"
#
#[security]
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_CLIENT_CERT_AUTH="false"
#ETCD_TRUSTED_CA_FILE=""
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
#ETCD_PEER_CLIENT_CERT_AUTH="false"
#ETCD_PEER_TRUSTED_CA_FILE=""
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""

修改 docker-test-node0.hy01 的配置。

1. 修改 docker 配置。

修改 /etc/sysconfig/docker,如下:



# /etc/sysconfig/docker

# Modify these options if you want to change the way the docker daemon runs
OPTIONS="--selinux-enabled -s btrfs"

DOCKER_CERT_PATH=/etc/docker

# If you want to add your own registry to be used for docker search and docker
# pull use the ADD_REGISTRY option to list a set of registries, each prepended
# with --add-registry flag. The first registry added will be the first registry
# searched.
ADD_REGISTRY=--add-registry dockerhub.internal

# If you want to block registries from being used, uncomment the BLOCK_REGISTRY
# option and give it a set of registries, each prepended with --block-registry
# flag. For example adding docker.io will stop users from downloading images
# from docker.io
BLOCK_REGISTRY=--block-registry docker.io

# If you have a registry secured with https but do not have proper certs
# distributed, you can tell docker to not look for full authorization by
# adding the registry to the INSECURE_REGISTRY line and uncommenting it.
INSECURE_REGISTRY=--insecure-registry dockerhub.internal

# On an SELinux system, if you remove the --selinux-enabled option, you
# also need to turn on the docker_transition_unconfined boolean.
# setsebool -P docker_transition_unconfined 1

# Location used for temporary files, such as those created by
# docker load and build operations. Default is /var/lib/docker/tmp
# Can be overriden by setting the following environment variable.
# DOCKER_TMPDIR=/var/tmp

# Controls the /etc/cron.daily/docker-logrotate cron job status.
# To disable, uncomment the line below.
# LOGROTATE=false

dockerhub.internal 是我们私有的 registry。

docker 的目录是 /var/lib/docker,最好单独挂载出来,并格式化成 btrfs。

修改 /etc/sysconfig/flanneld,如下:



# Flanneld configuration options

# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD="http://docker-test-etcd0.hy01:2379"

# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_KEY="/nosa.me/network"

# Any additional options that you want to pass
#FLANNEL_OPTIONS=""

3. 修改 kubernetes node 的配置。

1). 修改 /etc/kubernetes/config,如下:



###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow_privileged=false"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://docker-test-master0.hy01:8080"

2). 修改 /etc/kubernetes/kubelet,如下:



###
# kubernetes kubelet (minion) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override=docker-test-node0.hy01"

# location of the api-server
KUBELET_API_SERVER="--api_servers=http://docker-test-master0.hy01:8080"

# Add your own!
KUBELET_ARGS="--pod-infra-container-image=dockerhub.internal/kubernetes/pause:latest"

默认 kubelet 会访问公网的 gcr.io/google_containers/pause,修改 KUBELET_ARGS 从我们自己的 registry 下载。

在 docker-test-master0.hy01 机器上启动服务:

systemctl start kube-apiserver.service

systemctl start kube-controller-manager.service

systemctl start kube-scheduler.service

在 docker-test-etcd0.hy01 创建 kubernetes 所用的网段:

systemctl start etcd

etcdctl set /nosa.me/network/config '{ "Network": "10.253.0.0/16" }'

在 docker-test-node0.hy01 启动服务:

systemctl start flanneld

systemctl start docker

systemctl start kubelet

看看 docker-test-node0.hy01  的情况:

907A6D4C-C3B5-4AB5-A48B-28A117B88105 BB790C4E-8648-4826-B8D4-BF0234CD4DE6 47AA3762-EBCD-45A5-A38D-40C33B1421B1

看上面第一张图,有两个网卡 docker0 和 flannel0,还有物理网卡 em2,它们三个的路由表决定了数据包如何转发。

docker0 通过虚拟交换机和 docker 实例相连,docker 实例访问其他 node 机器上的实例时,使用 flannel0 ( VXLAN ) 出去,访问非 docker  实例时,使用 nat,也就是第二张和第三张图设置的 iptables 规则( docker 服务起来后会自动设置 )。

我们启动一个 docker 实例看看情况:

# docker run -t -i centos6-base /bin/bash

从下两张图看出,IP 地址属于 10.253.21.0/24( 在 10.254.0.0/16 之内 ),flanneld 使得 docker 在启动的时候加上了

--bip=10.253.21.1/24

,使得 docker 实例地址都在这个子网内,而且实例的网关是 docker0 的地址(10.253.21.1 )。

这个实例可以 ping通 外面非 node 机器的 IP ( 使用 nat 转发,如果用 iptables -F 和 iptables -F -t nat 把规则刷掉就 ping 不通了,重启 docker 服务后恢复 ),由于我们只有一台 node,没测试 ping 其他 node 机器( 使用 nat 转发 )和其上面的 docker 实例 ( 使用 VXLAN 封装 ),不过也是可以通的。

另外可以看出,/etc/resolv.conf 和 node 机器一样( 所以解析没问题 ),/etc/hosts 和 /etc/hostname 则和 node 不一样。

D1EE5DAC-6A1B-4B8F-8C62-0FB5136FE565 9F09797B-8405-44BC-889A-0FBC6DC3BE0C ADF314AB-4178-4E29-A146-9502B43AA1DA

http://kubernetes.io/v1.0/docs/getting-started-guides/centos/centos_manual_config.html

http://dockone.io/article/618

https://github.com/coreos/flannel/blob/master/README.md

http://dockone.io/article/520

No related posts.

作者:nosa.me
未来不会有sa
原文地址:kubernetes 搭建, 感谢原作者分享。

发表评论