Table of Contents
1. kubernetes集群升级 #
1.1 kubernetes升级概述 #
kubernetes版本升级迭代非常快，每三个月更新一个版本,很多新的功能在新版本中快速迭代，为了与社区版本功能保持一致，升级kubernetes集群，社区已通过kubeadm工具统一升级集群，升级步骤简单易行。首先来看下升级kubernetes集群需要升级那些组件：
- 升级管理节点，管理节点上的kube-apiserver，kuber-controller-manager，kube-scheduler，etcd等；
- 其他管理节点，管理节点如果以高可用的方式部署，多个高可用节点需要一并升级；
- worker工作节点，升级工作节点上的Container Runtime如docker，kubelet和kube-proxy。
版本升级通常分为两类：小版本升级和跨版本升级，小版本升级如1.14.1升级只1.14.2，小版本之间可以跨版本升级如1.14.1直接升级至1.14.3；跨版本升级指大版本升级，如1.14.x升级至1.15.x。本文以离线的方式将1.14.1升级至1.1.5.1版本，升级前需要满足条件如下：
- 当前集群版本需要大于1.14.x，可升级至1.14.x和1.15.x版本，小版本和跨版本之间升级；
- 关闭swap空间；
- 备份数据，将etcd数据备份，以及一些重要目录如/etc/kubernetes,/var/lib/kubelet；
- 升级过程中pod需要重启，确保应用使用RollingUpdate滚动升级策略，避免业务有影响。
1.2 升级前准备工作 #
1、查看当前版本,系统上部署的版本是1.1.4.1
1[root@node-1 ~]# kubectl version
2Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
3Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:02:58Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
4
5[root@node-1 ~]# kubeadm version
6kubeadm version: &version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:08:49Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
2、查看node节点的版本,node上的kubelet和kube-proxy使用1.1.4.1版本
1[root@node-1 ~]# kubectl get nodes
2NAME STATUS ROLES AGE VERSION
3node-1 Ready master 25h v1.14.1
4node-2 Ready <none> 25h v1.14.1
5node-3 Ready <none> 25h v1.14.1
3、其他组件状态，确保当前组件，应用状态正常
1[root@node-1 ~]# kubectl get componentstatuses
2NAME STATUS MESSAGE ERROR
3scheduler Healthy ok
4controller-manager Healthy ok
5etcd-0 Healthy {"health":"true"}
6[root@node-1 ~]#
7[root@node-1 ~]# kubectl get deployments --all-namespaces
8NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
9default demo 3/3 3 3 37m
10kube-system coredns 2/2 2 2 25h
4、查看kubernetes最新版本（配置kubernetes的yum源，需要合理上网才可以访问），使用yum list --showduplicates kubeadm --disableexcludes=kubernetes查看当前能升级版本，绿色为当前版本，蓝色为可以升级的版本，如下图：
%E7%A6%BB%E7%BA%BF%E5%8D%87%E7%BA%A7kubernetes%E9%9B%86%E7%BE%A4/1%20-%201620.jpg)
kubernetes可升级版本列表
1.3 升级master节点 #
1、倒入安装镜像，先从cos中下载安装镜像并通过docker load导入到系统中，下载地址，解压并进入到v1.15.1目录下，将镜像倒入到三个节点中,以node-2为例倒入镜像：
1倒入镜像：
2[root@node-2 v1.15.1]# docker image load -i kube-apiserver\:v1.15.1.tar
3[root@node-2 v1.15.1]# docker image load -i kube-scheduler\:v1.15.1.tar
4[root@node-2 v1.15.1]# docker image load -i kube-controller-manager\:v1.15.1.tar
5[root@node-2 v1.15.1]# docker image load -i kube-proxy\:v1.15.1.tar
6
7查看当前系统导入镜像列表：
8[root@node-1 ~]# docker image list
9REPOSITORY TAG IMAGE ID CREATED SIZE
10k8s.gcr.io/kube-proxy v1.15.1 89a062da739d 8 weeks ago 82.4MB
11k8s.gcr.io/kube-controller-manager v1.15.1 d75082f1d121 8 weeks ago 159MB
12k8s.gcr.io/kube-scheduler v1.15.1 b0b3c4c404da 8 weeks ago 81.1MB
13k8s.gcr.io/kube-apiserver v1.15.1 68c3eb07bfc3 8 weeks ago 207MB
14k8s.gcr.io/kube-proxy v1.14.1 20a2d7035165 5 months ago 82.1MB
15k8s.gcr.io/kube-apiserver v1.14.1 cfaa4ad74c37 5 months ago 210MB
16k8s.gcr.io/kube-scheduler v1.14.1 8931473d5bdb 5 months ago 81.6MB
17k8s.gcr.io/kube-controller-manager v1.14.1 efb3887b411d 5 months ago 158MB
18quay.io/coreos/flannel v0.11.0-amd64 ff281650a721 7 months ago 52.6MB
19k8s.gcr.io/coredns 1.3.1 eb516548c180 8 months ago 40.3MB
20k8s.gcr.io/etcd 3.3.10 2c4adeb21b4f 9 months ago 258MB
21k8s.gcr.io/pause 3.1 da86e6ba6ca1 21 months ago 742kB
2、更新kubeadm版本至1.15.1，国内可以参考https://blog.51cto.com/2157217/1983992设置kubernetes源
%E7%A6%BB%E7%BA%BF%E5%8D%87%E7%BA%A7kubernetes%E9%9B%86%E7%BE%A4/2%20-%201620.jpg)
更新kubeadm版本至1.15.1
3、校验kubeadm版本,已升级至1.1.5.1版本
1[root@node-1 ~]# kubeadm version
2kubeadm version: &version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T09:15:32Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
4、查看升级计划，通过kubeadm可以查看当前集群的升级计划，会显示当前小版本最新的版本以及社区最新的版本
1[root@node-1 ~]# kubeadm upgrade plan
2[upgrade/config] Making sure the configuration is correct:
3[upgrade/config] Reading configuration from the cluster...
4[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
5[preflight] Running pre-flight checks.
6[upgrade] Making sure the cluster is healthy:
7[upgrade] Fetching available versions to upgrade to
8[upgrade/versions] Cluster version: v1.14.1 #当前集群版本
9[upgrade/versions] kubeadm version: v1.15.1 #当前kubeadm版本
10[upgrade/versions] Latest stable version: v1.15.3 #社区最新版本
11[upgrade/versions] Latest version in the v1.14 series: v1.14.6 #1.14.x中最新的版本
12
13Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
14COMPONENT CURRENT AVAILABLE
15Kubelet 3 x v1.14.1 v1.14.6
16
17Upgrade to the latest version in the v1.14 series:
18
19COMPONENT CURRENT AVAILABLE #可升级的版本信息，当前可从1.14.1升级至1.14.6版本
20API Server v1.14.1 v1.14.6
21Controller Manager v1.14.1 v1.14.6
22Scheduler v1.14.1 v1.14.6
23Kube Proxy v1.14.1 v1.14.6
24CoreDNS 1.3.1 1.3.1
25Etcd 3.3.10 3.3.10
26
27You can now apply the upgrade by executing the following command:
28
29 kubeadm upgrade apply v1.14.6 #升级至1.14.6执行的操作命令
30
31_____________________________________________________________________
32
33Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
34COMPONENT CURRENT AVAILABLE
35Kubelet 3 x v1.14.1 v1.15.3
36
37Upgrade to the latest stable version:
38
39COMPONENT CURRENT AVAILABLE #跨版本升级的版本，当前最新的版本是1.15.3
40API Server v1.14.1 v1.15.3
41Controller Manager v1.14.1 v1.15.3
42Scheduler v1.14.1 v1.15.3
43Kube Proxy v1.14.1 v1.15.3
44CoreDNS 1.3.1 1.3.1
45Etcd 3.3.10 3.3.10
46
47You can now apply the upgrade by executing the following command:
48
49 kubeadm upgrade apply v1.15.3 #升级至社区最新的版本执行的操作
50
51Note: Before you can perform this upgrade, you have to update kubeadm to v1.15.3.
52
53_____________________________________________________________________
5、当前镜像没有下载最新镜像，本文以升级1.1.5.1版本为例，升级其他版本相类似，需要确保当前集群已获取到相关镜像，升级过程中也会更新证书，可通过--certificate-renewal=false关闭证书升级，升级至1.15.1版本操作如下：
1[root@node-1 ~]# kubeadm upgrade apply v1.15.1
2[upgrade/config] Making sure the configuration is correct:
3[upgrade/config] Reading configuration from the cluster...
4[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
5[preflight] Running pre-flight checks.
6[upgrade] Making sure the cluster is healthy:
7[upgrade/version] You have chosen to change the cluster version to "v1.15.1"
8[upgrade/versions] Cluster version: v1.14.1
9[upgrade/versions] kubeadm version: v1.15.1
10[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y 版本升级信息，确认操作
11[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler etcd]
12[upgrade/prepull] Prepulling image for component etcd.
13[upgrade/prepull] Prepulling image for component kube-apiserver.
14[upgrade/prepull] Prepulling image for component kube-controller-manager.
15[upgrade/prepull] Prepulling image for component kube-scheduler.
16[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
17[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
18[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-etcd
19[apiclient] Found 0 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
20[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-controller-manager
21[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-scheduler
22[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-etcd
23[apiclient] Found 1 Pods for label selector k8s-app=upgrade-prepull-kube-apiserver
24[upgrade/prepull] Prepulled image for component kube-controller-manager.
25[upgrade/prepull] Prepulled image for component kube-scheduler.
26[upgrade/prepull] Prepulled image for component etcd.
27[upgrade/prepull] Prepulled image for component kube-apiserver. #拉取镜像的步骤
28[upgrade/prepull] Successfully prepulled the images for all the control plane components
29[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.15.1"...
30Static pod: kube-apiserver-node-1 hash: bdf7ffba48feb2fc4c7676e7525066fd
31Static pod: kube-controller-manager-node-1 hash: ecf9c37413eace225bc60becabeddb3b
32Static pod: kube-scheduler-node-1 hash: f44110a0ca540009109bfc32a7eb0baa
33[upgrade/etcd] Upgrading to TLS for etcd #开始更新静态pod，及master上的每个节点
34[upgrade/staticpods] Writing new Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests122291749"
35[upgrade/staticpods] Preparing for "kube-apiserver" upgrade
36[upgrade/staticpods] Renewing apiserver certificate
37[upgrade/staticpods] Renewing apiserver-kubelet-client certificate
38[upgrade/staticpods] Renewing front-proxy-client certificate
39[upgrade/staticpods] Renewing apiserver-etcd-client certificate #更新证书
40[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2019-09-15-12-41-40/kube-apiserver.yaml"
41[upgrade/staticpods] Waiting for the kubelet to restart the component
42[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
43Static pod: kube-apiserver-node-1 hash: bdf7ffba48feb2fc4c7676e7525066fd
44Static pod: kube-apiserver-node-1 hash: bdf7ffba48feb2fc4c7676e7525066fd
45Static pod: kube-apiserver-node-1 hash: bdf7ffba48feb2fc4c7676e7525066fd
46Static pod: kube-apiserver-node-1 hash: 4cd1e2acc44e2d908fd2c7b307bfce59
47[apiclient] Found 1 Pods for label selector component=kube-apiserver
48[upgrade/staticpods] Component "kube-apiserver" upgraded successfully! #更新成功
49[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade
50[upgrade/staticpods] Renewing controller-manager.conf certificate
51[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2019-09-15-12-41-40/kube-controller-manager.yaml"
52[upgrade/staticpods] Waiting for the kubelet to restart the component
53[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
54Static pod: kube-controller-manager-node-1 hash: ecf9c37413eace225bc60becabeddb3b
55Static pod: kube-controller-manager-node-1 hash: 17b23c8c6fcf9b9f8a3061b3a2fbf633
56[apiclient] Found 1 Pods for label selector component=kube-controller-manager
57[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!#更新成功
58[upgrade/staticpods] Preparing for "kube-scheduler" upgrade
59[upgrade/staticpods] Renewing scheduler.conf certificate
60[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2019-09-15-12-41-40/kube-scheduler.yaml"
61[upgrade/staticpods] Waiting for the kubelet to restart the component
62[upgrade/staticpods] This might take a minute or longer depending on the component/version gap (timeout 5m0s)
63Static pod: kube-scheduler-node-1 hash: f44110a0ca540009109bfc32a7eb0baa
64Static pod: kube-scheduler-node-1 hash: 18859150495c74ad1b9f283da804a3db
65[apiclient] Found 1 Pods for label selector component=kube-scheduler
66[upgrade/staticpods] Component "kube-scheduler" upgraded successfully! #更新成功
67[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
68[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
69[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace
70[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
71[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
72[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
73[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
74[addons] Applied essential addon: CoreDNS
75[addons] Applied essential addon: kube-proxy
76
77[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.15.1". Enjoy! #更新成功提示
78[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
6、上述可看到master升级成功的信息，升级指特定版本只需要在apply的后面指定具体的版本即可，升级完完master后可以升级各个组件的plugin，详情参考不同网络的升级步骤，如flannel，calico等，升级过程升级对应的DaemonSets即可。
7、升级kubelet版本并重启kubelet服务，至此，master节点版本升级完毕。
1[root@node-1 ~]# yum install -y kubelet-1.15.1-0 kubectl-1.15.1-0 --disableexcludes=kubernetes
2[root@node-1 ~]# systemctl daemon-reload
3[root@node-1 ~]# systemctl restart kubelet
1.4 升级worker节点 #
1、升级kubeadm和kubelet软件包
1[root@node-2 ~]# yum install -y kubelet-1.15.1-0 --disableexcludes=kubernetes
2[root@node-2 ~]# yum install -y kubeadm-1.15.1-0 --disableexcludes=kubernetes
3[root@node-2 ~]# yum install -y kubectl-1.15.1-0 --disableexcludes=kubernetes
2、设置节点进入维护模式并驱逐worker节点上的应用，会将出了DaemonSets之外的其他应用迁移到其他节点上
1设置维护和驱逐：
2[root@node-1 ~]# kubectl drain node-2 --ignore-daemonsets
3node/node-2 cordoned
4WARNING: ignoring DaemonSet-managed Pods: kube-system/kube-flannel-ds-amd64-tm6wj, kube-system/kube-proxy-2wqhj
5evicting pod "coredns-5c98db65d4-86gg7"
6evicting pod "demo-7b86696648-djvgb"
7pod/demo-7b86696648-djvgb evicted
8pod/coredns-5c98db65d4-86gg7 evicted
9node/node-2 evicted
10
11查看node的情况，此时node-2多了SchedulingDisabled标识位，即新的node都不会调度到该节点上
12[root@node-1 ~]# kubectl get nodes
13NAME STATUS ROLES AGE VERSION
14node-1 Ready master 26h v1.15.1
15node-2 Ready,SchedulingDisabled <none> 26h v1.14.1
16node-3 Ready <none> 26h v1.14.1
17
18查看node-2上的pods，pod都已经迁移到其他node节点上
19[root@node-1 ~]# kubectl get pods --all-namespaces -o wide
20NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
21default demo-7b86696648-6f22r 1/1 Running 0 30s 10.244.2.5 node-3 <none> <none>
22default demo-7b86696648-fjmxn 1/1 Running 0 116m 10.244.2.2 node-3 <none> <none>
23default demo-7b86696648-nwwxf 1/1 Running 0 116m 10.244.2.3 node-3 <none> <none>
24kube-system coredns-5c98db65d4-cqbbl 1/1 Running 0 30s 10.244.0.6 node-1 <none> <none>
25kube-system coredns-5c98db65d4-g59qt 1/1 Running 2 28m 10.244.2.4 node-3 <none> <none>
26kube-system etcd-node-1 1/1 Running 0 13m 10.254.100.101 node-1 <none> <none>
27kube-system kube-apiserver-node-1 1/1 Running 0 13m 10.254.100.101 node-1 <none> <none>
28kube-system kube-controller-manager-node-1 1/1 Running 0 13m 10.254.100.101 node-1 <none> <none>
29kube-system kube-flannel-ds-amd64-99tjl 1/1 Running 1 26h 10.254.100.101 node-1 <none> <none>
30kube-system kube-flannel-ds-amd64-jp594 1/1 Running 0 26h 10.254.100.103 node-3 <none> <none>
31kube-system kube-flannel-ds-amd64-tm6wj 1/1 Running 0 26h 10.254.100.102 node-2 <none> <none>
32kube-system kube-proxy-2wqhj 1/1 Running 0 28m 10.254.100.102 node-2 <none> <none>
33kube-system kube-proxy-k7c4f 1/1 Running 1 27m 10.254.100.101 node-1 <none> <none>
34kube-system kube-proxy-zffgq 1/1 Running 0 28m 10.254.100.103 node-3 <none> <none>
35kube-system kube-scheduler-node-1 1/1 Running 0 13m 10.254.100.101 node-1 <none> <none>
3、升级worker节点
1[root@node-2 ~]# kubeadm upgrade node
2[upgrade] Reading configuration from the cluster...
3[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
4[upgrade] Skipping phase. Not a control plane node[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace
5[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
6[upgrade] The configuration for this node was successfully updated!
7[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.
4、重启kubelet服务
1[root@node-2 ~]# systemctl daemon-reload
2[root@node-2 ~]# systemctl restart kubelet
5、取消节点调度标志，确保worker节点可正常调度
1[root@node-1 ~]# kubectl uncordon node-2
2node/node-2 uncordoned
3[root@node-1 ~]# kubectl get nodes
4NAME STATUS ROLES AGE VERSION
5node-1 Ready master 27h v1.15.1
6node-2 Ready <none> 27h v1.15.1 #已升级成功
7node-3 Ready <none> 27h v1.14.1
按照上述步骤升级node-3节点，如下是升级完成后所有节点版本状态：
1[root@node-1 ~]# kubectl get nodes
2NAME STATUS ROLES AGE VERSION
3node-1 Ready master 27h v1.15.1
4node-2 Ready <none> 27h v1.15.1
5node-3 Ready <none> 27h v1.15.1
1.5 升级原理 #
1、kubeadm upgrade apply执行动作
- 检查集群是否具备更新条件，检查apiserver处于可用状态，所有node处于ready状态，确保cs组件正常
- 强制版本更新策略
- 检查更新所需镜像是否下载或者可拉取
- 更新所有控制节点组件，确保异常时能回滚到原有状态
- 更新kube-dns和kube-proxy的配置文件，确保所需的的RBAC授权配置正常
- 生成新的证书文件并备份证书（当证书超时超过180天）
2、kubeadm upgrade node执行动作
- 从kubeadm中获取ClusterConfiguration，即从集群中获取到更新集群的配置文件并应用
- 更新node节点上的kubelet配置信息和软件
2. 更新集群至1.15.3 #
截止至2019.9.15，当前kubernetes社区最新版本是1.15.3，本文演示以在线的方式升级kubernetes集群至1.15.3版本，步骤和前文操作类似。
1、安装最新软件包，kubeadm，kubelet，kubectl,三个节点均需要安装
1[root@node-1 ~]# yum install kubeadm kubectl kubelet
2已加载插件：fastestmirror, langpacks
3Loading mirror speeds from cached hostfile
4正在解决依赖关系
5--> 正在检查事务
6---> 软件包 kubeadm.x86_64.0.1.15.1-0 将被 升级
7---> 软件包 kubeadm.x86_64.0.1.15.3-0 将被 更新
8---> 软件包 kubectl.x86_64.0.1.15.1-0 将被 升级
9---> 软件包 kubectl.x86_64.0.1.15.3-0 将被 更新
10---> 软件包 kubelet.x86_64.0.1.15.1-0 将被 升级
11---> 软件包 kubelet.x86_64.0.1.15.3-0 将被 更新
12--> 解决依赖关系完成
13
14依赖关系解决
15
16========================================================================================================================================================================
17 Package 架构 版本 源 大小
18========================================================================================================================================================================
19正在更新:
20 kubeadm x86_64 1.15.3-0 kubernetes 8.9 M
21 kubectl x86_64 1.15.3-0 kubernetes 9.5 M
22 kubelet x86_64 1.15.3-0 kubernetes 22 M
23
24事务概要
25========================================================================================================================================================================
26升级 3 软件包
2、升级master节点
1查看升级计划
2[root@node-1 ~]# kubeadm upgrade plan
3[upgrade/config] Making sure the configuration is correct:
4[upgrade/config] Reading configuration from the cluster...
5[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
6[preflight] Running pre-flight checks.
7[upgrade] Making sure the cluster is healthy:
8[upgrade] Fetching available versions to upgrade to
9[upgrade/versions] Cluster version: v1.15.1
10[upgrade/versions] kubeadm version: v1.15.3
11[upgrade/versions] Latest stable version: v1.15.3
12[upgrade/versions] Latest version in the v1.15 series: v1.15.3
13
14Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
15COMPONENT CURRENT AVAILABLE
16Kubelet 3 x v1.15.1 v1.15.3
17
18Upgrade to the latest version in the v1.15 series:
19
20COMPONENT CURRENT AVAILABLE
21API Server v1.15.1 v1.15.3
22Controller Manager v1.15.1 v1.15.3
23Scheduler v1.15.1 v1.15.3
24Kube Proxy v1.15.1 v1.15.3
25CoreDNS 1.3.1 1.3.1
26Etcd 3.3.10 3.3.10
27
28You can now apply the upgrade by executing the following command:
29
30 kubeadm upgrade apply v1.15.3
31
32_____________________________________________________________________
33
34升级master节点
35[root@node-1 ~]# kubeadm upgrade apply v1.15.3
3、升级worker节点,以此升级node-2和node-3节点
1设置五污点去驱逐
2[root@node-1 ~]# kubectl drain node-2 --ignore-daemonsets
3
4执行升级操作
5[root@node-2 ~]# kubeadm upgrade node
6[root@node-2 ~]# systemctl daemon-reload
7[root@node-2 ~]# systemctl restart kubelet
8
9取消调度标志位
10[root@node-1 ~]# kubectl uncordon node-2
11node/node-2 uncordoned
12
13确认版本升级
14[root@node-1 ~]# kubectl get nodes
15NAME STATUS ROLES AGE VERSION
16node-1 Ready master 27h v1.15.3
17node-2 Ready <none> 27h v1.15.3
18node-3 Ready <none> 27h v1.15.1
4、所有节点升级后的状态
1所有node状态
2[root@node-1 ~]# kubectl get nodes
3NAME STATUS ROLES AGE VERSION
4node-1 Ready master 27h v1.15.3
5node-2 Ready <none> 27h v1.15.3
6node-3 Ready <none> 27h v1.15.3
7
8查看组件状态
9[root@node-1 ~]# kubectl get componentstatuses
10NAME STATUS MESSAGE ERROR
11scheduler Healthy ok
12controller-manager Healthy ok
13etcd-0 Healthy {"health":"true"}
14
15查看应用状态
16[root@node-1 ~]# kubectl get deployments --all-namespaces
17NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
18default demo 3/3 3 3 160m
19kube-system coredns 2/2 2 2 27h
20
21查看DaemonSets状态
22[root@node-1 ~]# kubectl get daemonsets --all-namespaces
23NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
24kube-system kube-flannel-ds-amd64 3 3 3 3 3 beta.kubernetes.io/arch=amd64 27h
25kube-system kube-proxy 3 3 3 3 3 beta.kubernetes.io/os=linux 27h
写在最后 #
至此，通过上述的两个案例介绍了kubernetes离线升级（无法连接外网）和在线升级（需合理上网）的操作，升级master节点，升级node节点的流程和实现细节，对于体验新功能和线上版本升级提供指导。
参考文献 #
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15/
『 转载 』该文章来源于网络，侵删。