目录
一.系统环境
二.前言
三.Kubernetes 控制器
四.Deployment概览
五.创建deployment
六.修改deploy副本数
6.1 kubectl edit deploy 修改副本数
6.2 修改yaml文件更改副本数
6.3 kubectl scale修改副本数
七.HorizontalPodAutoscaler(HPA)
7.1 Pod 水平自动扩缩HorizontalPodAutoscaler概览
7.2 HPA自动伸缩pod副本数实战
八.k8s升级镜像
8.1 kubectl edit deployment升级镜像
8.2 kubectl set image deploy升级镜像
8.3 镜像升级历史记录

一.系统环境

服务器版本	docker软件版本	Kubernetes(k8s)集群版本	CPU架构
CentOS Linux release 7.4.1708 (Core)	Docker version 20.10.12	v1.21.9	x86_64

Kubernetes集群架构：k8scloude1作为master节点，k8scloude2，k8scloude3作为worker节点

服务器	操作系统版本	CPU架构	进程	功能描述
k8scloude1/192.168.110.130	CentOS Linux release 7.4.1708 (Core)	x86_64	docker，kube-apiserver，etcd，kube-scheduler，kube-controller-manager，kubelet，kube-proxy，coredns，calico	k8s master节点
k8scloude2/192.168.110.129	CentOS Linux release 7.4.1708 (Core)	x86_64	docker，kubelet，kube-proxy，calico	k8s worker节点
k8scloude3/192.168.110.128	CentOS Linux release 7.4.1708 (Core)	x86_64	docker，kubelet，kube-proxy，calico	k8s worker节点

二.前言

使用deployment的前提是已经有一套可以正常运行的Kubernetes集群，关于Kubernetes(k8s)集群的安装部署，可以查看博客《Centos7 安装部署Kubernetes(k8s)集群》https://www.cnblogs.com/renshengdezheli/p/16686769.html。

三.Kubernetes 控制器

在 Kubernetes 中，控制器通过监控集群的公共状态，并致力于将当前状态转变为期望的状态。

一个控制器至少追踪一种类型的 Kubernetes 资源。这些对象有一个代表期望状态的 spec 字段。 该资源的控制器负责确保其当前状态接近期望状态。

控制器可能会自行执行操作；在 Kubernetes 中更常见的是一个控制器会发送信息给 API 服务器。

Kubernetes 内置一组控制器，运行在 kube-controller-manager 内。这些内置的控制器提供了重要的核心功能。

Deployment 控制器和 Job 控制器是 Kubernetes 内置控制器的典型例子。 Kubernetes 允许你运行一个稳定的控制平面，这样即使某些内置控制器失败了，控制平面的其他部分会接替它们的工作。

四.Deployment概览

一个 Deployment 为 Pod 和 ReplicaSet 提供声明式的更新能力。

你负责描述 Deployment 中的目标状态，而 Deployment 控制器（Controller）以受控速率更改实际状态，使其变为期望状态。你可以定义 Deployment 以创建新的 ReplicaSet，或删除现有 Deployment，并通过新的 Deployment 收养其资源。

注意：不要管理 Deployment 所拥有的 ReplicaSet 。

以下是 Deployments 的典型用例：

创建 Deployment 以将 ReplicaSet 上线。ReplicaSet 在后台创建 Pod。检查 ReplicaSet 的上线状态，查看其是否成功。
通过更新 Deployment 的 PodTemplateSpec，声明 Pod 的新状态。新的 ReplicaSet 会被创建，Deployment 以受控速率将 Pod 从旧 ReplicaSet 迁移到新 ReplicaSet。每个新的 ReplicaSet 都会更新 Deployment 的修订版本。
如果 Deployment 的当前状态不稳定，回滚到较早的 Deployment 版本。每次回滚都会更新 Deployment 的修订版本。
扩大 Deployment 规模以承担更多负载。
暂停 Deployment 的上线以应用对 PodTemplateSpec 所作的多项修改，然后恢复其执行以启动新的上线版本。
使用 Deployment 状态来判定上线过程是否出现停滞。
清理较旧的不再需要的 ReplicaSet 。

五.创建deployment

创建存放deployment配置文件的目录和namespace

[root@k8scloude1 ~]# mkdir deployment

[root@k8scloude1 ~]# cd deployment/

[root@k8scloude1 deployment]# pwd

/root/deployment

#创建namespace

[root@k8scloude1 deployment]# kubectl create ns deployment

namespace/deployment created

#切换命名空间为deployment

[root@k8scloude1 deployment]# kubens deployment

Context "kubernetes-admin@kubernetes" modified.

Active namespace is "deployment".

[root@k8scloude1 deployment]# kubectl get pod

No resources found in deployment namespace.

Kubernetes里有很多简写，比如deployment简称deploy，pod简称po

查看pod

[root@k8scloude1 deployment]# kubectl get po

No resources found in deployment namespace.

查看对资源的操作有哪些

[root@k8scloude1 deployment]# kubectl --help | grep resource

  create        Create a resource from a file or from stdin.

  get           显示一个或更多 resources

  delete        Delete resources by filenames, stdin, resources and names, or by resources and label selector

  rollout       Manage the rollout of a resource

  describe      显示一个指定 resource 或者 group 的 resources 详情

  patch         Update field(s) of a resource

  wait          Experimental: Wait for a specific condition on one or many resources.

  api-resources Print the supported API resources on the server

可以查看k8s支持的简写

[root@k8scloude1 deployment]# kubectl api-resources

NAME                              SHORTNAMES   APIVERSION                             NAMESPACED   KIND

bindings                                       v1                                     true         Binding

componentstatuses                 cs           v1                                     false        ComponentStatus

configmaps                        cm           v1                                     true         ConfigMap

endpoints                         ep           v1                                     true         Endpoints

events                            ev           v1                                     true         Event

limitranges                       limits       v1                                     true         LimitRange

namespaces                        ns           v1                                     false        Namespace

nodes                             no           v1                                     false        Node

persistentvolumeclaims            pvc          v1                                     true         PersistentVolumeClaim

persistentvolumes                 pv           v1                                     false        PersistentVolume

pods                              po           v1                                     true         Pod

podtemplates                                   v1                                     true         PodTemplate

replicationcontrollers            rc           v1                                     true         ReplicationController

resourcequotas                    quota        v1                                     true         ResourceQuota

secrets                                        v1                                     true         Secret

serviceaccounts                   sa           v1                                     true         ServiceAccount

services                          svc          v1                                     true         Service

mutatingwebhookconfigurations                  admissionregistration.k8s.io/v1        false        MutatingWebhookConfiguration

validatingwebhookconfigurations                admissionregistration.k8s.io/v1        false        ValidatingWebhookConfiguration

customresourcedefinitions         crd,crds     apiextensions.k8s.io/v1                false        CustomResourceDefinition

apiservices                                    apiregistration.k8s.io/v1              false        APIService

controllerrevisions                            apps/v1                                true         ControllerRevision

daemonsets                        ds           apps/v1                                true         DaemonSet

deployments                       deploy       apps/v1                                true         Deployment

replicasets                       rs           apps/v1                                true         ReplicaSet

statefulsets                      sts          apps/v1                                true         StatefulSet

tokenreviews                                   authentication.k8s.io/v1               false        TokenReview

localsubjectaccessreviews                      authorization.k8s.io/v1                true         LocalSubjectAccessReview

selfsubjectaccessreviews                       authorization.k8s.io/v1                false        SelfSubjectAccessReview

selfsubjectrulesreviews                        authorization.k8s.io/v1                false        SelfSubjectRulesReview

subjectaccessreviews                           authorization.k8s.io/v1                false        SubjectAccessReview

horizontalpodautoscalers          hpa          autoscaling/v1                         true         HorizontalPodAutoscaler

cronjobs                          cj           batch/v1                               true         CronJob

jobs                                           batch/v1                               true         Job

certificatesigningrequests        csr          certificates.k8s.io/v1                 false        CertificateSigningRequest

leases                                         coordination.k8s.io/v1                 true         Lease

bgpconfigurations                              crd.projectcalico.org/v1               false        BGPConfiguration

bgppeers                                       crd.projectcalico.org/v1               false        BGPPeer

blockaffinities                                crd.projectcalico.org/v1               false        BlockAffinity

caliconodestatuses                             crd.projectcalico.org/v1               false        CalicoNodeStatus

clusterinformations                            crd.projectcalico.org/v1               false        ClusterInformation

felixconfigurations                            crd.projectcalico.org/v1               false        FelixConfiguration

globalnetworkpolicies                          crd.projectcalico.org/v1               false        GlobalNetworkPolicy

globalnetworksets                              crd.projectcalico.org/v1               false        GlobalNetworkSet

hostendpoints                                  crd.projectcalico.org/v1               false        HostEndpoint

ipamblocks                                     crd.projectcalico.org/v1               false        IPAMBlock

ipamconfigs                                    crd.projectcalico.org/v1               false        IPAMConfig

ipamhandles                                    crd.projectcalico.org/v1               false        IPAMHandle

ippools                                        crd.projectcalico.org/v1               false        IPPool

ipreservations                                 crd.projectcalico.org/v1               false        IPReservation

kubecontrollersconfigurations                  crd.projectcalico.org/v1               false        KubeControllersConfiguration

networkpolicies                                crd.projectcalico.org/v1               true         NetworkPolicy

networksets                                    crd.projectcalico.org/v1               true         NetworkSet

endpointslices                                 discovery.k8s.io/v1                    true         EndpointSlice

events                            ev           events.k8s.io/v1                       true         Event

ingresses                         ing          extensions/v1beta1                     true         Ingress

flowschemas                                    flowcontrol.apiserver.k8s.io/v1beta1   false        FlowSchema

prioritylevelconfigurations                    flowcontrol.apiserver.k8s.io/v1beta1   false        PriorityLevelConfiguration

nodes                                          metrics.k8s.io/v1beta1                 false        NodeMetrics

pods                                           metrics.k8s.io/v1beta1                 true         PodMetrics

ingressclasses                                 networking.k8s.io/v1                   false        IngressClass

ingresses                         ing          networking.k8s.io/v1                   true         Ingress

networkpolicies                   netpol       networking.k8s.io/v1                   true         NetworkPolicy

runtimeclasses                                 node.k8s.io/v1                         false        RuntimeClass

poddisruptionbudgets              pdb          policy/v1                              true         PodDisruptionBudget

podsecuritypolicies               psp          policy/v1beta1                         false        PodSecurityPolicy

clusterrolebindings                            rbac.authorization.k8s.io/v1           false        ClusterRoleBinding

clusterroles                                   rbac.authorization.k8s.io/v1           false        ClusterRole

rolebindings                                   rbac.authorization.k8s.io/v1           true         RoleBinding

roles                                          rbac.authorization.k8s.io/v1           true         Role

priorityclasses                   pc           scheduling.k8s.io/v1                   false        PriorityClass

csidrivers                                     storage.k8s.io/v1                      false        CSIDriver

csinodes                                       storage.k8s.io/v1                      false        CSINode

csistoragecapacities                           storage.k8s.io/v1beta1                 true         CSIStorageCapacity

storageclasses                    sc           storage.k8s.io/v1                      false        StorageClass

volumeattachments                              storage.k8s.io/v1                      false        VolumeAttachment

查看deployment控制器

[root@k8scloude1 deployment]# kubectl get deploy

No resources found in deployment namespace.

建议使用yaml文件创建deploy,命令行的方式，支持的选项比较少，

生成deployment的yaml文件

#nginx为deploy的名字    --image=nginx使用Nginx镜像

[root@k8scloude1 deployment]# kubectl create deploy nginx --image=nginx --dry-run=client -o yaml >nginx.yaml

[root@k8scloude1 deployment]# ll -h nginx.yaml

-rw-r--r-- 1 root root 384 1月  25 15:39 nginx.yaml

查看生成的deploy yaml配置文件

[root@k8scloude1 deployment]# cat nginx.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  creationTimestamp: null

  labels:

    app: nginx

  name: nginx

spec:

  replicas: 1

  selector:

    matchLabels:

      app: nginx

  strategy: {}

  template:

    metadata:

      creationTimestamp: null

      labels:

        app: nginx

    spec:

      containers:

      - image: nginx

        name: nginx

        resources: {}

status: {}

pod模板可以有多个标签，只要其中一个标签和matchLabels里的标签一致即可

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  creationTimestamp: null

  #这个labels是deployment的标签

  labels:

    app: nginx

  #deployment名字

  name: nginx

spec:

  #replicas: 1 表示副本数为1，只生成一个pod

  replicas: 1

  #selector标签选择器

  selector:

    matchLabels:

      appx: xyz

  strategy: {}

  #template下面是pod的模板

  template:

    metadata:

      creationTimestamp: null

      #这个labels表示pod标签

      labels:

         #标签

        app: nginx

        appx: xyz

    spec:

      containers:

      - image: nginx

        name: nginx

        resources: {}

status: {}

创建deploy

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml

deployment.apps/nginx created

[root@k8scloude1 deployment]# kubectl get deploy

NAME    READY   UP-TO-DATE   AVAILABLE   AGE

nginx   0/1     1            0           11s

直接下载nginx镜像太慢，重新修改下yaml文件

[root@k8scloude1 deployment]# kubectl delete deploy nginx

deployment.apps "nginx" deleted

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  creationTimestamp: null

  labels:

    app: nginx

  name: nginx

spec:

  replicas: 1

  selector:

    matchLabels:

      appx: xyz

  strategy: {}

  template:

    metadata:

      creationTimestamp: null

      labels:

        app: nginx

        appx: xyz

    spec:

      #优雅删除pod的期限时间

      terminationGracePeriodSeconds: 0

      containers:

      - image: nginx

        name: nginx

        #修改镜像下载策略：IfNotPresent表示本地有镜像就不下载

        imagePullPolicy: IfNotPresent

        resources: {}

status: {}

创建deploy

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml

deployment.apps/nginx created

查看deployment

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR

nginx   1/1     1            1           11s   nginx        nginx    appx=xyz

#--show-labels 显示标签

[root@k8scloude1 deployment]# kubectl get deploy -o wide --show-labels

NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR   LABELS

nginx   1/1     1            1           60s   nginx        nginx    appx=xyz   app=nginx

可以看到pod也被创建出来了

[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels

NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES   LABELS

nginx-6fcb7995f5-c8lpc   1/1     Running   0          2m25s   10.244.112.146   k8scloude2   <none>           <none>            app=nginx,appx=xyz,pod-template-hash=6fcb7995f5

注意：删除pod之后，deploy会根据副本数replicas，创建缺失的pod,所以要想真正删除pod，需要删除deployment控制器

[root@k8scloude1 deployment]# kubectl delete pod nginx-6fcb7995f5-c8lpc --force

warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.

pod "nginx-6fcb7995f5-c8lpc" force deleted

#pod又重新创建了

[root@k8scloude1 deployment]# kubectl get pod -o wide --show-labels

NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES   LABELS

nginx-6fcb7995f5-s6smg   1/1     Running   0          5s    10.244.112.147   k8scloude2   <none>           <none>            app=nginx,appx=xyz,pod-template-hash=6fcb7995f5

六.修改deploy副本数

修改deployment副本数有三种方法：

注意

6.1 kubectl edit deploy 修改副本数

查看deploy

[root@k8scloude1 deployment]# kubectl get deploy

NAME    READY   UP-TO-DATE   AVAILABLE   AGE

nginx   1/1     1            1           10m

第一种：kubectl edit deploy deployname ,直接修改replicas的数目即可，实时生效，注意：可以在线修改deploy但是不能在线修改pod

[root@k8scloude1 deployment]# kubectl edit deploy nginx

deployment.apps/nginx edited

查看deploy，可以发现副本数变了，READY为3/3

[root@k8scloude1 deployment]# kubectl get deploy

NAME    READY   UP-TO-DATE   AVAILABLE   AGE

nginx   3/3     3            3           11m

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR

nginx   3/3     3            3           11m   nginx        nginx    appx=xyz

6.2 修改yaml文件更改副本数

第二种：可以直接修改yaml文件： replicas: 6，修改为6个副本

[root@k8scloude1 deployment]# vim nginx.yaml 

[root@k8scloude1 deployment]# cat nginx.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  creationTimestamp: null

  labels:

    app: nginx

  name: nginx

spec:

  replicas: 6

  selector:

    matchLabels:

      appx: xyz

  strategy: {}

  template:

    metadata:

      creationTimestamp: null

      labels:

        app: nginx

        appx: xyz

    spec:

      terminationGracePeriodSeconds: 0

      containers:

      - image: nginx

        name: nginx

        imagePullPolicy: IfNotPresent

        resources: {}

status: {}

kubectl edit可以立即生效，但是修改yaml文件需要apply才能生效

[root@k8scloude1 deployment]# kubectl get deploy

NAME    READY   UP-TO-DATE   AVAILABLE   AGE

nginx   3/3     3            3           14m

[root@k8scloude1 deployment]# kubectl apply -f nginx.yaml

deployment.apps/nginx configured

[root@k8scloude1 deployment]# kubectl get deploy

NAME    READY   UP-TO-DATE   AVAILABLE   AGE

nginx   6/6     6            6           15m

可以看到deploy生成了6个pod

[root@k8scloude1 deployment]# kubectl get pod -o wide

NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES

nginx-6fcb7995f5-64gm2   1/1     Running   0          22s     10.244.251.204   k8scloude3   <none>           <none>

nginx-6fcb7995f5-75jfd   1/1     Running   0          22s     10.244.251.206   k8scloude3   <none>           <none>

nginx-6fcb7995f5-8c75n   1/1     Running   0          4m23s   10.244.251.205   k8scloude3   <none>           <none>

nginx-6fcb7995f5-cs6w4   1/1     Running   0          4m23s   10.244.112.143   k8scloude2   <none>           <none>

nginx-6fcb7995f5-s6smg   1/1     Running   0          11m     10.244.112.147   k8scloude2   <none>           <none>

nginx-6fcb7995f5-tpwr4   1/1     Running   0          22s     10.244.112.148   k8scloude2   <none>           <none>

6.3 kubectl scale修改副本数

kubectl scale在线伸缩deploy，--replicas=10修改副本数为10个

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10

deployment.apps/nginx scaled

[root@k8scloude1 deployment]# kubectl get pod -o wide

NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES

nginx-6fcb7995f5-5sv29   1/1     Running   0          4s      10.244.112.152   k8scloude2   <none>           <none>

nginx-6fcb7995f5-64gm2   1/1     Running   0          2m12s   10.244.251.204   k8scloude3   <none>           <none>

nginx-6fcb7995f5-75jfd   1/1     Running   0          2m12s   10.244.251.206   k8scloude3   <none>           <none>

nginx-6fcb7995f5-8c75n   1/1     Running   0          6m13s   10.244.251.205   k8scloude3   <none>           <none>

nginx-6fcb7995f5-cs6w4   1/1     Running   0          6m13s   10.244.112.143   k8scloude2   <none>           <none>

nginx-6fcb7995f5-f6nz9   1/1     Running   0          4s      10.244.112.150   k8scloude2   <none>           <none>

nginx-6fcb7995f5-hx224   1/1     Running   0          4s      10.244.112.149   k8scloude2   <none>           <none>

nginx-6fcb7995f5-s6smg   1/1     Running   0          13m     10.244.112.147   k8scloude2   <none>           <none>

nginx-6fcb7995f5-tpwr4   1/1     Running   0          2m12s   10.244.112.148   k8scloude2   <none>           <none>

nginx-6fcb7995f5-zmsfx   1/1     Running   0          4s      10.244.251.208   k8scloude3   <none>           <none>

修改pod副本为1

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1

deployment.apps/nginx scaled

[root@k8scloude1 deployment]# kubectl get pod -o wide

NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES

nginx-6fcb7995f5-8c75n   1/1     Running   0          6m33s   10.244.251.205   k8scloude3   <none>           <none>

七.HorizontalPodAutoscaler(HPA)

7.1 Pod 水平自动扩缩HorizontalPodAutoscaler概览

在 Kubernetes 中，HorizontalPodAutoscaler 自动更新工作负载资源（例如 Deployment 或者 StatefulSet）， 目的是自动扩缩工作负载以满足需求。

水平扩缩意味着对增加的负载的响应是部署更多的 Pod。这与 “垂直（Vertical）” 扩缩不同，对于 Kubernetes， 垂直扩缩意味着将更多资源（例如：内存或 CPU）分配给已经为工作负载运行的 Pod。

如果负载减少，并且 Pod 的数量高于配置的最小值， HorizontalPodAutoscaler 会指示工作负载资源（Deployment、StatefulSet 或其他类似资源）缩减。

水平 Pod 自动扩缩不适用于无法扩缩的对象（例如：DaemonSet。）

HorizontalPodAutoscaler 被实现为 Kubernetes API 资源和控制器。

资源决定了控制器的行为。在 Kubernetes 控制平面内运行的水平 Pod 自动扩缩控制器会定期调整其目标（例如：Deployment）的所需规模，以匹配观察到的指标，例如，平均 CPU 利用率、平均内存利用率或你指定的任何其他自定义指标。

7.2 HPA自动伸缩pod副本数实战

HPA（horizontal pod autoscalers）水平自动伸缩通过检测pod CPU的负载，解决deployment里某pod负载太重，动态伸缩pod的数量来负载均衡。

现在deploy只生成了一个pod

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-8c75n   1/1     Running   0          45m

查看HPA

[root@k8scloude1 deployment]# kubectl get hpa

No resources found in deployment namespace.

设置HPA：--min=2 最小pod副本数为2，--max=5 最大pod副本数为5

[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --max=5 --min=2

horizontalpodautoscaler.autoscaling/nginx autoscaled

观察pod副本数，发现pod副本数自动变为2了，因为HPA设置了最小pod副本数为2

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

nginx-6fcb7995f5-zrdpj   1/1     Running   0          22s

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-8c75n   1/1     Running   0          49m

nginx-6fcb7995f5-zrdpj   1/1     Running   0          39s

使用kubectl scale 设置pod副本数为1

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=1

deployment.apps/nginx scaled

因为autoscale是实时扩展的，所以就算scale修改为1，autoscale也会自动把副本数修改为2

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS              RESTARTS   AGE

nginx-6fcb7995f5-5jl6h   0/1     ContainerCreating   0          0s

nginx-6fcb7995f5-8c75n   1/1     Running             0          50m

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-5jl6h   1/1     Running   0          4s

nginx-6fcb7995f5-8c75n   1/1     Running   0          50m

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-6fcb7995f5-5jl6h   1/1     Running   0          16s

nginx-6fcb7995f5-8c75n   1/1     Running   0          50m

查看hpa

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS         MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   <unknown>/80%   2         5         2          4m18s

删除HPA

[root@k8scloude1 deployment]# kubectl delete hpa nginx

horizontalpodautoscaler.autoscaling "nginx" deleted

[root@k8scloude1 deployment]# kubectl get hpa

No resources found in deployment namespace.

删除nginx deployment

[root@k8scloude1 deployment]# kubectl delete -f nginx.yaml

deployment.apps "nginx" deleted

[root@k8scloude1 deployment]# kubectl get deploy

No resources found in deployment namespace.

接下来给pod设置资源限制，对pod的资源限制可以通过pod里的resource字段来限制，resources里的requests字段表示容器所在节点资源的最小值，最低要求，满足不了这个要求pod创建不成功。

requests:cpu: 400m 表示pod需要400个微核心才能创建成功，注意：m代表微核心，1个核等于1000个微核心。

[root@k8scloude1 deployment]# vim nginxcpulimit.yaml 

[root@k8scloude1 deployment]# cat nginxcpulimit.yaml

apiVersion: apps/v1

kind: Deployment

metadata:

  creationTimestamp: null

  labels:

    app: nginx

  name: nginx

spec:

  replicas: 1

  selector:

    matchLabels:

      appx: xyz

  strategy: {}

  template:

    metadata:

      creationTimestamp: null

      labels:

        app: nginx

        appx: xyz

    spec:

      terminationGracePeriodSeconds: 0

      containers:

      - image: nginx

        name: nginx

        imagePullPolicy: IfNotPresent

        resources:

          #设置资源限制：CPU：400m    #m代表微核心，1个核等于1000个微核心

          requests:

            cpu: 400m

status: {}

创建deployment

[root@k8scloude1 deployment]# kubectl apply -f nginxcpulimit.yaml

deployment.apps/nginx created

[root@k8scloude1 deployment]# kubectl get deployments -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES   SELECTOR

nginx   1/1     1            1           13s   nginx        nginx    appx=xyz

查看node节点的负载

[root@k8scloude1 deployment]# kubectl top node

W0125 17:06:20.377474   15650 top_node.go:119] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag

NAME         CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%

k8scloude1   230m         11%    1375Mi          42%

k8scloude2   100m         5%     669Mi           34%

k8scloude3   113m         5%     737Mi           38%

设置HPA：最小pod副本数为1，最大pod副本数为15，--cpu-percent=80表示当deployment资源对象的CPU使用率高达80%时，就会进行扩容，最多扩容到15个

[root@k8scloude1 deployment]# kubectl autoscale deployment nginx --min=1 --max=15 --cpu-percent=80

horizontalpodautoscaler.autoscaling/nginx autoscaled

查看hpa

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   0%/80%    1         15        1          69s

安装一个压力测试的包：ab工具用于压力测试

[root@k8scloude1 deployment]# yum -y install httpd-tools

创建一个service服务，服务端口为80，service类型为NodePort

[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80 --type=NodePort

service/nginxsvc exposed

[root@k8scloude1 deployment]# kubectl get svc -o wide

NAME       TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE   SELECTOR

nginxsvc   NodePort   10.96.231.54   <none>        80:31085/TCP   10s   appx=xyz

查看pod

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-79b8d956bd-wbcq5   1/1     Running   0          12m

使用ab进行压力测试，一直给nginx服务发请求，Nginx服务的网址为：http://k8s节点ip:31085/，也就是http://192.168.110.130:31085/index.html。

ab压力测试命令参数解释：

-t：测试所进行的最大秒数。其内部隐含值是-n 50000，它可以使对服务器的测试限制在一个固定的总时间以内。默认时，没有时间限制。
-n：在测试会话中所执行的请求个数。默认时，仅执行一个请求。
-c：一次产生的请求个数。默认是一次一个。

下面一直给Nginx服务发请求

[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://192.168.110.130:31085/index.html

This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.110.130 (be patient)

Completed 100000 requests

^C

Server Software:        nginx/1.21.5

Server Hostname:        192.168.110.130

Server Port:            31085

Document Path:          /index.html

Document Length:        615 bytes

Concurrency Level:      1000

Time taken for tests:   27.096 seconds

Complete requests:      112145

Failed requests:        2029

   (Connect: 0, Receive: 0, Length: 0, Exceptions: 2029)

Write errors:           0

Total transferred:      95814989 bytes

HTML transferred:       69487620 bytes

Requests per second:    4138.87 [#/sec] (mean)

Time per request:       241.612 [ms] (mean)

Time per request:       0.242 [ms] (mean, across all concurrent requests)

Transfer rate:          3453.31 [Kbytes/sec] received

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0  121 219.7     73    2693

Processing:     1  113  84.3     96    5111

Waiting:        0   95  78.9     80    5063

Total:          2  235 241.9    178    7568

Percentage of the requests served within a certain time (ms)

  50%    178

  66%    208

  75%    230

  80%    247

  90%    319

  95%    469

  98%   1188

  99%   1233

 100%   7568 (longest request)

ab压力测试之后，查看pod的变化，pod数自动变为了3个

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-79b8d956bd-622fw   1/1     Running   0          14s

nginx-79b8d956bd-8z5h7   1/1     Running   0          14s

nginx-79b8d956bd-wbcq5   1/1     Running   0          14m

查看hpa的变化

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   31%/80%   1         15        3          11m

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   61%/80%   1         15        3          12m

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-79b8d956bd-622fw   1/1     Running   0          101s

nginx-79b8d956bd-8z5h7   1/1     Running   0          101s

nginx-79b8d956bd-wbcq5   1/1     Running   0          15m

删除服务并重新创建一个service服务，这次的service服务类型为ClusterIP。

[root@k8scloude1 deployment]# kubectl delete svc nginxsvc

[root@k8scloude1 deployment]# kubectl get svc

No resources found in deployment namespace.

[root@k8scloude1 deployment]# kubectl expose --name=nginxsvc deployment nginx --port=80

service/nginxsvc exposed

[root@k8scloude1 deployment]# kubectl get svc

NAME       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE

nginxsvc   ClusterIP   10.109.240.56   <none>        80/TCP    3s

Nginx服务的网址为：http://10.109.240.56:80/index.html，继续进行ab压力测试，给Nginx服务发请求。

#继续ab压力测试

[root@k8scloude1 ~]# ab -t 600 -n 1000000 -c 1000 http://10.109.240.56:80/index.html

This is ApacheBench, Version 2.3 <$Revision: 1430300 %sSourceCode%gt;

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.109.240.56 (be patient)

Completed 100000 requests

Completed 200000 requests

Completed 300000 requests

Completed 400000 requests

Completed 500000 requests

Completed 600000 requests

Completed 700000 requests

Completed 800000 requests

Completed 900000 requests

apr_pollset_poll: The timeout specified has expired (70007)

Total of 985106 requests completed

执行ab压力测试的同时观察pod的变化

查看pod的负载

[root@k8scloude1 deployment]# kubectl top pods

W0125 17:25:03.753728   30784 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag

NAME                     CPU(cores)   MEMORY(bytes)

nginx-79b8d956bd-5hq5j   179m         4Mi

nginx-79b8d956bd-622fw   366m         5Mi

nginx-79b8d956bd-6wjsl   248m         5Mi

nginx-79b8d956bd-8z5h7   360m         5Mi

nginx-79b8d956bd-wbcq5   254m         5Mi

查看hpa，现在已经有5个pod了

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   71%/80%   1         15        5          16m

继续观察pod

[root@k8scloude1 deployment]# kubectl top pods

W0125 17:25:49.908634   31380 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag

NAME                     CPU(cores)   MEMORY(bytes)

nginx-79b8d956bd-5hq5j   254m         5Mi

nginx-79b8d956bd-622fw   326m         5Mi

nginx-79b8d956bd-6wjsl   322m         5Mi

nginx-79b8d956bd-8z5h7   255m         5Mi

nginx-79b8d956bd-wbcq5   264m         5Mi             

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   70%/80%   1         15        5          18m

[root@k8scloude1 deployment]# kubectl top pods

W0125 17:27:46.476364   32773 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag

NAME                     CPU(cores)   MEMORY(bytes)

nginx-79b8d956bd-5hq5j   285m         5Mi

nginx-79b8d956bd-622fw   331m         5Mi

nginx-79b8d956bd-6wjsl   351m         5Mi

nginx-79b8d956bd-8z5h7   277m         5Mi

nginx-79b8d956bd-wbcq5   264m         6Mi    

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   75%/80%   1         15        5          19m

停止给nginx发送请求之后hpa的TARGETS降下来了

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   0%/80%    1         15        5          20m

但是pod数没有降下来

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   0%/80%    1         15        5          20m

[root@k8scloude1 deployment]# kubectl top pods

W0125 17:29:32.004502   34116 top_pod.go:140] Using json format to get metrics. Next release will switch to protocol-buffers, switch early by passing --use-protocol-buffers flag

NAME                     CPU(cores)   MEMORY(bytes)

nginx-79b8d956bd-5hq5j   0m           5Mi

nginx-79b8d956bd-622fw   0m           5Mi

nginx-79b8d956bd-6wjsl   0m           5Mi

nginx-79b8d956bd-8z5h7   0m           5Mi

nginx-79b8d956bd-wbcq5   0m           6Mi             

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   0%/80%    1         15        5          21m

注意：pod负载降下来之后，pod副本数不会立即减少，过一段时间pod副本数才会减少，默认时间间隔为5分钟，这样的目的是为了防止pod副本数的抖动。

10分钟之后观察pod，pod副本降下来了。

[root@k8scloude1 deployment]# kubectl get hpa

NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE

nginx   Deployment/nginx   0%/80%    1         15        1          46m

[root@k8scloude1 deployment]# kubectl get pod

NAME                     READY   STATUS    RESTARTS   AGE

nginx-79b8d956bd-wbcq5   1/1     Running   0          50m

[root@k8scloude1 deployment]# kubectl delete hpa nginx

horizontalpodautoscaler.autoscaling "nginx" deleted

八.k8s升级镜像

8.1 kubectl edit deployment升级镜像

查看deploy，可以看到镜像为nginx

[root@k8scloude1 deployment]# kubectl get deploy

NAME    READY   UP-TO-DATE   AVAILABLE   AGE

nginx   1/1     1            1           179m

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES   SELECTOR

nginx   1/1     1            1           179m   nginx        nginx    appx=xyz

第一种方法：使用kubectl edit deployment deployname升级镜像。把deploy的镜像由nginx升级为hub.c.163.com/library/nginx:1.12.0，修改 - image: nginx 为 - image: hub.c.163.com/library/nginx:1.12.0

[root@k8scloude1 deployment]# kubectl edit deployment nginx

deployment.apps/nginx edited

修改之后查看deploy，IMAGES变了。

修改deploy的镜像的时候，本质上是删除现有的pod，然后用新镜像创建新pod。

[root@k8scloude1 deployment]# kubectl get deployments -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                               SELECTOR

nginx   1/1     1            1           3h3m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

[root@k8scloude1 deployment]# kubectl get pods -o wide

NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES

nginx-66bcf4489b-ztvnc   1/1     Running   0          77s   10.244.251.211   k8scloude3   <none>           <none>

8.2 kubectl set image deploy升级镜像

查看deploy

[root@k8scloude1 deployment]# kubectl get deployment -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR

nginx   1/1     1            1           6h41m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

第二种方式：使用kubectl set image deploy deployname 容器名=镜像名 升级镜像。

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0

deployment.apps/nginx image updated

查看deploy，可以发现镜像升级为hub.c.163.com/library/nginx:1.13.0了

[root@k8scloude1 deployment]# kubectl get deployment -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR

nginx   1/1     1            1           6h42m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

继续升级镜像

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest

deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl get deployment -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR

nginx   1/1     1            1           6h43m   nginx        hub.c.163.com/library/nginx:latest   appx=xyz

[root@k8scloude1 deployment]# kubectl get pod -o wide

NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES

nginx-75698d6db5-bzv9d   1/1     Running   0          17s   10.244.251.212   k8scloude3   <none>           <none>

8.3 镜像升级历史记录

查看镜像替换的历史记录 kubectl rollout history deployment nginx，现在镜像替换的历史记录是不清楚的，是因为升级镜像的时候没有record记录。

[root@k8scloude1 deployment]# kubectl rollout history deployment nginx

deployment.apps/nginx

REVISION  CHANGE-CAUSE

1         <none>

2         <none>

3         <none>

4         <none>

替换镜像并记录

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record

deployment.apps/nginx image updated

可以看到镜像替换成功

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR

nginx   1/1     1            1           6h47m   nginx        hub.c.163.com/library/nginx:1.12.0   appx=xyz

多进行几次镜像替换并记录

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record

deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record

deployment.apps/nginx image updated

[root@k8scloude1 deployment]# kubectl set image deploy nginx nginx=nginx --record

deployment.apps/nginx image updated

查看镜像替换的历史记录

[root@k8scloude1 deployment]# kubectl rollout history deployment nginx

deployment.apps/nginx

REVISION  CHANGE-CAUSE

5         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.12.0 --record=true

6         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true

7         kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:latest --record=true

8         kubectl set image deploy nginx nginx=nginx --record=true

查看deploy和pod

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES   SELECTOR

nginx   1/1     1            1           6h48m   nginx        nginx    appx=xyz

[root@k8scloude1 deployment]# kubectl get pod -o wide

NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE         NOMINATED NODE   READINESS GATES

nginx-79b8d956bd-xblj9   1/1     Running   0          61s   10.244.251.214   k8scloude3   <none>           <none>

kubectl set image deploy nginx nginx=hub.c.163.com/library/nginx:1.13.0 --record=true是第6条修改记录，我们可以进行回滚，回滚到镜像是hub.c.163.com/library/nginx:1.13.0的时候。

[root@k8scloude1 deployment]# kubectl rollout undo deployment nginx --to-revision=6

deployment.apps/nginx rolled back

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                               SELECTOR

nginx   1/1     1            1           6h51m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

变更镜像的本质是删除旧版本的pod，然后用新镜像创建pod。

修改deploy副本数为10个

[root@k8scloude1 deployment]# kubectl scale deployment nginx --replicas=10

deployment.apps/nginx scaled

编辑deploy，修改镜像

[root@k8scloude1 deployment]# kubectl edit deployment nginx

deployment.apps/nginx edited

[root@k8scloude1 deployment]# kubectl get deploy -o wide

NAME    READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES                               SELECTOR

nginx   5/5     5            5           7h1m   nginx        hub.c.163.com/library/nginx:1.13.0   appx=xyz

 #滚动更新配置说明：

strategy:

    rollingUpdate:

          #maxSurge：在升级过程中一次升级几个，可以是数字也可以是百分比

      maxSurge: 25%

          #maxUnavailable 在升级过程中，只能有几个不可用， 一次性删除多少个pod，可以是数字也可以是百分比

      maxUnavailable: 25%

    type: RollingUpdate