tags: addons, heapster
Heapster是一个收集者，将每个Node上的cAdvisor的数据进行汇总，然后导到第三方工具(如InfluxDB)。
Heapster 是通过调用 kubelet 的 http API 来获取 cAdvisor 的 metrics 数据的。
由于 kublet 只在 10250 端口接收 https 请求，故需要修改 heapster 的 deployment 配置。同时，需要赋予 kube-system:heapster ServiceAccount 调用 kubelet API 的权限。
注意：如果没有特殊指明，本文档的所有操作均在 m7-autocv-gpu01 节点上执行。
到 heapster release 页面 下载最新版本的 heapster
cd /opt/k8s/work
wget https://github.com/kubernetes/heapster/archive/v1.5.4.tar.gz
tar -xzvf v1.5.4.tar.gz
mv v1.5.4.tar.gz heapster-1.5.4.tar.gz官方文件目录： heapster-1.5.4/deploy/kube-config/influxdb
$ cd heapster-1.5.4/deploy/kube-config/influxdb
$ cp grafana.yaml{,.orig}
$ diff grafana.yaml.orig grafana.yaml
67c67
< # type: NodePort
---
> type: NodePort- 开启 NodePort；
$ cp heapster.yaml{,.orig}
$ diff heapster.yaml.orig heapster.yaml
< - --source=kubernetes:https://kubernetes.default
---
> - --source=kubernetes:https://kubernetes.default?kubeletHttps=true&kubeletPort=10250- 由于 kubelet 只在 10250 监听 https 请求，故添加相关参数；
$ cd /opt/k8s/work/heapster-1.5.4/deploy/kube-config/influxdb
$ ls *.yaml
grafana.yaml heapster.yaml influxdb.yaml
$ kubectl create -f .
$ cd ../rbac/
$ cp heapster-rbac.yaml{,.orig}
$ diff heapster-rbac.yaml.orig heapster-rbac.yaml
12a13,26
> ---
> kind: ClusterRoleBinding
> apiVersion: rbac.authorization.k8s.io/v1beta1
> metadata:
> name: heapster-kubelet-api
> roleRef:
> apiGroup: rbac.authorization.k8s.io
> kind: ClusterRole
> name: system:kubelet-api-admin
> subjects:
> - kind: ServiceAccount
> name: heapster
> namespace: kube-system
$ kubectl create -f heapster-rbac.yaml- 将 serviceAccount kube-system:heapster 与 ClusterRole system:kubelet-api-admin 绑定，授予它调用 kubelet API 的权限；
如果不修改，默认的 ClusterRole system:heapster 权限不足：
E1128 10:00:05.010716 1 manager.go:101] Error in scraping containers from kubelet:172.27.128.150:10250: failed to get all container stats from Kubelet URL "https://172.27.128.150:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)" E1128 10:00:05.018556 1 manager.go:101] Error in scraping containers from kubelet:172.27.128.149:10250: failed to get all container stats from Kubelet URL "https://172.27.128.149:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)" E1128 10:00:05.022664 1 manager.go:101] Error in scraping containers from kubelet:172.27.128.148:10250: failed to get all container stats from Kubelet URL "https://172.27.128.148:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:heapster, verb=create, resource=nodes, subresource=stats)" W1128 10:00:25.000467 1 manager.go:152] Failed to get all responses in time (got 0/3)
$ kubectl get pods -n kube-system | grep -E 'heapster|monitoring'
heapster-56c9dc749-j7hvz 1/1 Running 0 1m
monitoring-grafana-c797777db-lnwnc 1/1 Running 0 1m
monitoring-influxdb-cf9d95766-5wd28 1/1 Running 0 1m检查 kubernets dashboard 界面，可以正确显示各 Nodes、Pods 的 CPU、内存、负载等统计数据和图表：
-
通过 kube-apiserver 访问：
获取 monitoring-grafana 服务 URL：
$ kubectl cluster-info Kubernetes master is running at https://172.27.128.252:8443 Heapster is running at https://172.27.128.252:8443/api/v1/namespaces/kube-system/services/heapster/proxy CoreDNS is running at https://172.27.128.252:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy kubernetes-dashboard is running at https://172.27.128.252:8443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy monitoring-grafana is running at https://172.27.128.252:8443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy monitoring-influxdb is running at https://172.27.128.252:8443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.浏览器访问 URL：
https://172.27.128.252:8443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy对于 virtuabox 做了端口映射：http://127.0.0.1:8080/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy -
通过 kubectl proxy 访问：
创建代理
kubectl proxy --address='172.27.128.150' --port=8086 --accept-hosts='^*$' Starting to serve on 172.27.129.150:8086
浏览器访问 URL：
http://172.27.128.150:8086/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy/?orgId=1对于 virtuabox 做了端口映射：http://127.0.0.1:8086/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy/?orgId=1 -
通过 NodePort 访问：
$ kubectl get svc -n kube-system|grep -E 'monitoring|heapster' heapster ClusterIP 10.254.199.65 <none> 80/TCP 3m monitoring-grafana NodePort 10.254.116.161 <none> 80:31470/TCP 3m monitoring-influxdb ClusterIP 10.254.250.185 <none> 8086/TCP 3m
- grafana 监听 NodePort 31470；
浏览器访问 URL：
http://172.27.128.150:31470/?orgId=1
