>>分享SPSS,Hadoop等大数据处理技术,以及分布式架构以及集群系统的构建 书籍支持  卫琴直播  品书摘要  在线测试  资源下载  联系我们
发表一个新主题 开启一个新投票 回复文章 您是本文章第 20520 个阅读者 刷新本主题
 * 贴子主题:  K8S使用dashboard管理集群 回复文章 点赞(0)  收藏  
作者:sunshine    发表时间:2020-03-11 18:06:54     消息  查看  搜索  好友  邮件  复制  引用

  
  今年3月份在公司的内部k8s培训会上,开发同事表示使用dashboard的可以满足日常开发需求,例如查看pod的日志,执行exec指令,查看pod的运行状态等,但对basic认证的权限控制表示担忧。
之前介绍过在1.5.2版本上部署dashboard服务,在1.9.1版本离线部署中,也介绍过dashboard服务的RBAC配置和使用技巧。因此本文将在前文基础上完善Heapster的整合与利用token对用户权限进行控制。
dashboard的特点主要如下:
1、能够直观的看到rc、deployment、pod、services等k8s组件的运行情况和日志信息。
2、结合heapster和influxdb后,dashboard的监控图表上可以看到pod的cpu和内存消耗情况。

Heapster介绍

1、Heapster是容器集群监控和性能分析工具,支持Kubernetes和CoreOS。?
2、K8S集群的HPA功能的实现就依赖于这些metric数据,HPA将Heapster作为Resource Metrics API,向其获取metric。
3、Kubernetes有个cAdvisor监控(在1.9版本里面,cAdvisor已经和kubelet整合在一起)。
在每个kubernetes Node上都会运行cAdvisor,它会收集本机以及容器的监控数据(cpu,memory,filesystem,network,uptime)。Heapster是一个收集者,Heapster可以收集Node节点上的cAdvisor数据,将每个Node上的cAdvisor的数据进行汇总,还可以按照kubernetes的资源类型来集合资源,比如Pod、Namespace,可以分别获取它们的CPU、内存、网络和磁盘的metric。默认的metric数据聚合时间间隔是1分钟。还可以把数据导入到第三方工具(如InfluxDB)。

Influxdb数据库介绍

2、Influxdb数据库的相关知识介绍,可参考文档:https://www.jianshu.com/p/d2935e99006e
2、如果对Heapster收集到的metric数据没有持久化的需求,可以不配置Influxdb数据库
3、本文Influxdb数据库的存储采用emptydir的方式实现,实际使用过程中,可以选择吧Influxdb数据库部署在k8s集群外部,或者使用其他存储方案。
4、如果有需要的话,还可以集成一个grafana做web展示。Grafana配置可参考文档:https://blog.51cto.com/ylw6006/2084403

一、获取相关镜像

需要科学上网方式获取到dashboard相关的镜像文件,仓库可纳入本地仓库统一管理    

# cat /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://192.168.115.2:1080"
# systemctl  daemon-reload
# systemctl restart docker
# docker pull k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
# docker pull k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
# docker pull k8s.gcr.io/heapster-amd64:v1.4.2

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小

二、准备配置文件

1、k8s-dashborad-sa.yaml文件,secrct和serviceaccount配置    

# cat k8s-dashborad-sa.yaml
# ------------------- Dashboard Secret ------------------- #
apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kube-system
type: Opaque
---
# ------------------- Dashboard Service Account ------------------- #
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system

  2、k8s-dashborad-rbac.yaml文件,配置 Role和Role Binding    

# cat k8s-dashborad-rbac.yaml
# ------------------- Dashboard Role & Role Binding ------------------- #
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: kubernetes-dashboard-minimal
  namespace: kube-system
rules:
  # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create"]
  # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["create"]
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
  verbs: ["get", "update", "delete"]
  # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["kubernetes-dashboard-settings"]
  verbs: ["get", "update"]
  # Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
  resources: ["services"]
  resourceNames: ["heapster"]
  verbs: ["proxy"]
- apiGroups: [""]
  resources: ["services/proxy"]
  resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kubernetes-dashboard-minimal
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

  3、k8s-dashborad-deployment.yaml配置文件,定义创建pod的模板和副本数    

# cat k8s-dashborad-deployment.yaml
# ------------------- Dashboard Deployment ------------------- #
kind: Deployment
apiVersion: apps/v1beta2
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: kubernetes-dashboard
  template:
    metadata:
      labels:
        k8s-app: kubernetes-dashboard
    spec:
      containers:
      - name: kubernetes-dashboard
        image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
        ports:
        - containerPort: 8443
          protocol: TCP
        args:
          - --auto-generate-certificates
          # Uncomment the following line to manually specify Kubernetes API server Host
          # If not specified, Dashboard will attempt to auto discover the API server and connect
          # to it. Uncomment only if the default does not work.
          # - --apiserver-host=http://my-address:port
        volumeMounts:
        - name: kubernetes-dashboard-certs
          mountPath: /certs
          # Create on-disk volume to store exec logs
        - mountPath: /tmp
          name: tmp-volume
        livenessProbe:
          httpGet:
            scheme: HTTPS
            path: /
            port: 8443
          initialDelaySeconds: 30
          timeoutSeconds: 30
      volumes:
      - name: kubernetes-dashboard-certs
        secret:
          secretName: kubernetes-dashboard-certs
      - name: tmp-volume
        emptyDir: {}
      serviceAccountName: kubernetes-dashboard
      # Comment the following tolerations if Dashboard must not be deployed on master
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule

  4、 k8s-dashborad-service.yaml配置文件,定义service    

# cat k8s-dashborad-service.yaml  
# ------------------- Dashboard Service ------------------- #
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system
spec:
  ports:
    - port: 443
      targetPort: 8443
      nodePort: 8490
  type: NodePort
  selector:
    k8s-app: kubernetes-dashboard

三、通过配置文件创建dashboard

# kubectl create -f .
# kubectl get pod,deployment,svc -n kube-system

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小

四、配置使用basic认证方式

默认情况下只支持kubeconfig和令牌认证
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小    

# echo 'admin,admin,1' > /etc/kubernetes/basic_auth_file
# grep 'auth' /usr/lib/systemd/system/kube-apiserver.service  
  --authorization-mode=Node,RBAC \
  --runtime-config=rbac.authorization.k8s.io/v1alpha1 \
  --enable-bootstrap-token-auth=true \
  --token-auth-file=/etc/kubernetes/token.csv \
  --basic-auth-file=/etc/kubernetes/basic_auth_file \

# grep  ‘basic’  k8s-dashborad-deployment.yaml   (配置在args下面)
     - --authentication-mode=basic

# systemctl daemon-reload
# systemctl restart kube-apiserver
# kubectl apply -f k8s-dashborad-deployment.yaml

  将admin用户和cluter-admin role进行角色绑定    

# curl --insecure https://vm1:6443 -basic -u admin:admin  
# kubectl create clusterrolebinding  \
login-on-dashboard-with-cluster-admin  \
--clusterrole=cluster-admin --user=admin
# curl --insecure https://vm1:6443 -basic -u admin:admin  

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小

五、访问测试

点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小

六、整合heapster和influxdb

在没有配置heapster和influxdb的情况下,pod的metric信息是无法获取到的,而早前版本K8S的HPA特性依赖的metric数据来源恰巧就是heapster和influxdb。

     1、准备yaml配置文件    

# cat heapster-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: heapster
  namespace: kube-system

# cat heapster-rbac.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: heapster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:heapster
subjects:
- kind: ServiceAccount
  name: heapster
  namespace: kube-system

# cat heapster-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: heapster
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: heapster
    spec:
      serviceAccountName: heapster
      containers:
      - name: heapster
        image: k8s.gcr.io/heapster-amd64:v1.4.2
        imagePullPolicy: IfNotPresent
        command:
        - /heapster
        - --source=kubernetes:https://kubernetes.default
        - --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086

# cat heapster-service.yaml  
apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: Heapster
  name: heapster
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: 8082
  selector:
k8s-app: heapster

# cat influxdb-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: monitoring-influxdb
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        task: monitoring
        k8s-app: influxdb
    spec:
      containers:
      - name: influxdb
        image: k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
        volumeMounts:
        - mountPath: /data
          name: influxdb-storage
      volumes:
      - name: influxdb-storage
        emptyDir: {}

# cat influxdb-service.yaml  
apiVersion: v1
kind: Service
metadata:
  labels:
    task: monitoring
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: monitoring-influxdb
  name: monitoring-influxdb
  namespace: kube-system
spec:
  ports:
  - port: 8086
    targetPort: 8086
  selector:
    k8s-app: influxdb

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
获取heapster中的获取支持的metrics    

# kubectl run -i --tty curl --namespace=kube-system  \
--image=registry.59iedu.com/webwurst/curl-utils /bin/sh
# curl http://heapster/api/v1/model/metrics
# curl http://heapster/api/v1/model/debug/allkeys

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小    

# kubectl get node
# kubectl top node

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
当heapster和influxdb pod都正常运行的时候,在dashboard里面就可以看到CPU和内存的监控数据了。
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小

七、配置用户权限

1、删除apiserver里面basic认证相关的配置后重启apiserver
--basic-auth-file=/etc/kubernetes/basic_auth_file      

# systemctl daemon-reload
# systemctl  restart kube-apiserver

  2、删除clusterrolebinding    

# kubectl delete  clusterrolebinding  login-on-dashboard-with-cluster-admin

  3、修改k8s-dashborad-deployment.yaml文件
去掉- --authentication-mode=basic参数

     4、创建普通用户,赋予所有namespace下资源的get、watch和list权限。
这里通过clusterrole和culsterrolebinding赋予所有namespace相关资源的get、watch、list权限,实际应用环境建议使用创建role和rolebinding指定特定的namespace相关资源权限,各资源权限的赋予规则遵循最小权限原则。    

# cat rbac-yang.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: role-yang
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["get","watch","list" ]
- apiGroups: ["storage.k8s.io"]
  resources: ["*"]
  verbs: ["get","watch","list" ]
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["*"]
  verbs: ["get","watch","list" ]
- apiGroups: ["batch"]
  resources: ["*"]
  verbs: ["get","watch","list" ]
- apiGroups: ["apps"]
  resources: ["*"]
  verbs: ["get","watch","list" ]
- apiGroups: ["extensions"]
  resources: ["*"]
  verbs: ["get","watch","list" ]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: role-bind-yang
subjects:
- kind: ServiceAccount
  name: yang
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: role-yang
  apiGroup: rbac.authorization.k8s.io

# kubectl create sa yang -n kube-system
# kubectl create -f rbac-yang.yaml
# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep yang | awk '{print $1}')

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小

     5、测试普通用户的权限
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
6、创建super用户admin    

# kubectl create sa admin -n kube-system
# cat rbac-admin.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin
  namespace: kube-system

# kubectl create -f rbac-admin.yaml
# kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin | awk '{print $1}')

  点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
使用admin用户的token登陆后继承cluster-admin的权限
点击在新窗口中浏览原图
CTRL+鼠标滚轮放大或缩小
参考:
https://github.com/kubernetes/dashboard/wiki/Creating-sample-user
https://github.com/kubernetes/dashboard/wiki/Access-control
https://github.com/kubernetes/heapster/blob/master/docs/model.md



----------------------------
原文链接:https://blog.51cto.com/ylw6006/2113542

程序猿的技术大观园:www.javathinker.net



[这个贴子最后由 flybird 在 2020-03-13 12:16:22 重新编辑]
  Java面向对象编程-->操作符
  JavaWeb开发-->访问数据库(Ⅰ)
  JSP与Hibernate开发-->映射组成关系
  Java网络编程-->用Spring整合CXF发布Web服务
  精通Spring-->Vue CLI脚手架工具
  Vue3开发-->创建综合购物网站应用
  30岁女IT工程师感叹:靠这工具,把报表做成养老工作,月薪快...
  害阿里程序员差点被当场开除的P0事故
  Spark批量读取Redis数据-Pipeline(Scala)
  基于spark-streaming实时推荐系统
  kubeadm安装k8s集群1.17版本
  Hadoop安装过程
  Kubernetes(K8S)集群管理Docker容器(部署篇)
  深入玩转K8S之使用kubeadm安装Kubernetes v1.10以及常见问题...
  Spark SQL常见4种数据源(详细)
  通过Spark Streaming的foreachRDD把处理后的数据写入外部存储...
  kafka作为流式处理的上一层,为什么吞吐量那么大?
  大数据的学习方向
  学习大数据处理需要掌握的技能
  深入剖析Hadoop HBase
  大数据的概念、作用和处理流程
  更多...
 IPIP: 已设置保密
树形列表:   
1页 1条记录 当前第1
发表一个新主题 开启一个新投票 回复文章


中文版权所有: JavaThinker技术网站 Copyright 2016-2026 沪ICP备16029593号-2
荟萃Java程序员智慧的结晶,分享交流Java前沿技术。  联系我们
如有技术文章涉及侵权,请与本站管理员联系。