Linkerd服务网格安装部署详解

Linkerd 介绍

Linkerd 是 Kubernetes 的一个完全开源的服务网格实现。它通过为你提供运行时调试、可观测性、可靠性和安全性，使运行服务更轻松、更安全，所有这些都不需要对你的代码进行任何更改。

Linkerd 通过在每个服务实例旁边安装一组超轻、透明的代理来工作。这些代理会自动处理进出服务的所有流量。由于它们是透明的，这些代理充当高度仪表化的进程外网络堆栈，向控制平面发送遥测数据并从控制平面接收控制信号。这种设计允许 Linkerd 测量和操纵进出你的服务的流量，而不会引入过多的延迟。为了尽可能小、轻和安全，Linkerd 的代理采用 Rust 编写。

功能概述

  • 自动 mTLS：Linkerd 自动为网格应用程序之间的所有通信启用相互传输层安全性 (TLS)。
  • 自动代理注入：Linkerd 会自动将数据平面代理注入到基于 annotations 的 pod 中。
  • 容器网络接口插件：Linkerd 能被配置去运行一个 CNI 插件，该插件自动重写每个 pod 的 iptables 规则。
  • 仪表板和 Grafana：Linkerd 提供了一个 Web 仪表板，以及预配置的 Grafana 仪表板。
  • 分布式追踪：您可以在 Linkerd 中启用分布式跟踪支持。
  • 故障注入：Linkerd 提供了以编程方式将故障注入服务的机制。
  • 高可用性：Linkerd 控制平面可以在高可用性 (HA) 模式下运行。
  • HTTP、HTTP/2 和 gRPC 代理：Linkerd 将自动为 HTTP、HTTP/2 和 gRPC 连接启用高级功能（包括指标、负载平衡、重试等）。
  • Ingress：Linkerd 可以与您选择的 ingress controller 一起工作。
  • 负载均衡：Linkerd 会自动对 HTTP、HTTP/2 和 gRPC 连接上所有目标端点的请求进行负载平衡。
  • 多集群通信：Linkerd 可以透明且安全地连接运行在不同集群中的服务。
  • 重试和超时：Linkerd 可以执行特定于服务的重试和超时。
  • 服务配置文件：Linkerd 的服务配置文件支持每条路由指标以及重试和超时。
  • TCP 代理和协议检测：Linkerd 能够代理所有 TCP 流量，包括 TLS 连接、WebSockets 和 HTTP 隧道。
  • 遥测和监控：Linkerd 会自动从所有通过它发送流量的服务收集指标。
  • 流量拆分(金丝雀、蓝/绿部署)：Linkerd 可以动态地将一部分流量发送到不同的服务。

架构

整体上来看 Linkerd 由一个控制平面和一个数据平面组成。

  • 控制平面是一组服务，提供对 Linkerd 整体的控制。
  • 数据平面由在每个服务实例旁边运行的透明微代理(micro-proxies)组成，作为 Pod 中的 Sidecar 容器运行，这些代理会自动处理进出服务的所有 TCP 流量，并与控制平面进行通信以进行配置。

此外 Linkerd 还提供了一个 CLI 工具，可用于控制平面和数据平面进行交互。

架构

控制平面(control plane)

Linkerd 控制平面是一组在专用 Kubernetes 命名空间（默认为 linkerd）中运行的服务，控制平面有几个组件组成：

  • 目标服务(destination)：数据平面代理使用 destination 服务来确定其行为的各个方面。它用于获取服务发现信息；获取有关允许哪些类型的请求的策略信息；获取用于通知每条路由指标、重试和超时的服务配置文件信息和其它有用信息。
  • 身份服务(identity)：identity 服务充当 TLS 证书颁发机构，接受来自代理的 CSR 并返回签名证书。这些证书在代理初始化时颁发，用于代理到代理连接以实现 mTLS。
  • 代理注入器(proxy injector)：proxy injector 是一个 Kubernetes admission controller，它在每次创建 pod 时接收一个 webhook 请求。 此 injector 检查特定于 Linkerd 的 annotation（linkerd.io/inject: enabled）的资源。 当该 annotation 存在时，injector 会改变 pod 的规范， 并将 proxy-init 和 linkerd-proxy 容器以及相关的启动时间配置添加到 pod 中。

数据平面(data plane)

Linkerd 数据平面包含超轻型微代理，这些微代理部署为应用程序 Pod 内的 sidecar 容器。 由于由 linkerd-init（或者，由 Linkerd 的 CNI 插件）制定的 iptables 规则， 这些代理透明地拦截进出每个 pod 的 TCP 连接。

  • 代理(Linkerd2-proxy)：Linkerd2-proxy 是一个用 Rust 编写的超轻、透明的微代理。Linkerd2-proxy 专为 service mesh 用例而设计，并非设计为通用代理。代理的功能包括：
    • HTTP、HTTP/2 和任意 TCP 协议的透明、零配置代理。
    • HTTP 和 TCP 流量的自动 Prometheus 指标导出。
    • 透明、零配置的 WebSocket 代理。
    • 自动、延迟感知、第 7 层负载平衡。
    • 非 HTTP 流量的自动第 4 层负载平衡。
    • 自动 TLS。
    • 按需诊断 Tap API。
    • 代理支持通过 DNS 和目标 gRPC API 进行服务发现。
  • Linkerd init 容器：linkerd-init 容器作为 Kubernetes 初始化容器添加到每个网格 Pod 中，该容器在任何其他容器启动之前运行。它使用 iptables 通过代理将所有 TCP 流量，路由到 Pod 和从 Pod 发出。

Linkerd 安装部署

Linkerd命令安装

我们可以通过在本地安装一个 Linkerd 的 CLI 命令行工具，通过该 CLI 可以将 Linkerd 的控制平面安装到 Kubernetes 集群上。

所以首先需要在本地运行 kubectl 命令，确保可以访问一个可用的 Kubernetes 集群，如果没有集群，可以使用 KinD 在本地快速创建一个。

1
2
3
kubectl version --short
Client Version: v1.23.6
Server Version: v1.23.6

可以使用下面的命令在本地安装 Linkerd 的 CLI 工具：

1
$ curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/install | sh

如果是 Mac 系统，同样还可以使用 Homebrew 工具一键安装：

1
brew install linkerd

同样直接前往 Linkerd Release 页面 https://github.com/linkerd/linkerd2/releases/ 下载安装即可。

安装后使用下面的命令可以验证 CLI 工具是否安装成功：

1
2
3
 linkerd  version
Client version: stable-2.13.1
Server version: unavailable

正常我们可以看到 CLI 的版本信息，但是会出现 Server version: unavailable 信息，这是因为我们还没有在 Kubernetes 集群上安装控制平面造成的，所以接下来我们就来安装 Server 端。

Kubernetes 集群可以通过多种不同的方式进行配置，在安装 Linkerd 控制平面之前，我们需要检查并验证所有配置是否正确，要检查集群是否已准备好安装 Linkerd，可以执行下面的命令：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ linkerd check --pre      
kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version

pre-kubernetes-setup
--------------------
√ control plane namespace does not already exist
√ can create non-namespaced resources
√ can create ServiceAccounts
√ can create Services
√ can create Deployments
√ can create CronJobs
√ can create ConfigMaps
√ can create Secrets
√ can read Secrets
√ can read extension-apiserver-authentication configmap
√ no clock skew detected

linkerd-version
---------------
√ can determine the latest version
√ cli is up-to-date

Status check results are √

如果一切检查都 OK 则可以开始安装 Linkerd 的控制平面了，直接执行下面的命令即可一键安装：

1
2
3
4
# 新版需要先安装crd资源，才能执行安装linkerd
linkerd install --crds |kubectl apply -f -

linkerd install |kubectl apply -f -

在此命令中，linkerd install 会生成一个 Kubernetes 资源清单文件，其中包含所有必要的控制平面资源，然后使用 kubectl apply 命令即可将其安装到 Kubernetes 集群中。

可以看到会将 Linkerd 控制面安装到一个名为 linkerd 的命名空间之下，安装完成后会有如下几个 Pod 运行：

1
2
3
4
5
$ kubectl get pods -n linkerd
NAME                                      READY   STATUS    RESTARTS   AGE
linkerd-destination-7fcf6649dd-h2ztv      4/4     Running   0          92s
linkerd-identity-68d57d5-5trt8            2/2     Running   0          93s
linkerd-proxy-injector-759c487549-lqvjm   2/2     Running   0          92s

安装完成后通过运行以下命令等待控制平面准备就绪，并可以验证安装结果是否正常：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
$ linkerd check
kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version

linkerd-existence
-----------------
√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
√ no unschedulable pods
√ control plane pods are ready
√ cluster networks contains all pods
√ cluster networks contains all services

linkerd-config
--------------
√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
√ control plane CustomResourceDefinitions exist
√ control plane MutatingWebhookConfigurations exist
√ control plane ValidatingWebhookConfigurations exist
√ proxy-init container runs as root user if docker container runtime is used

linkerd-identity
----------------
√ certificate config is valid
√ trust anchors are using supported crypto algorithm
√ trust anchors are within their validity period
√ trust anchors are valid for at least 60 days
√ issuer cert is using supported crypto algorithm
√ issuer cert is within its validity period
√ issuer cert is valid for at least 60 days
√ issuer cert is issued by the trust anchor

linkerd-webhooks-and-apisvc-tls
-------------------------------
√ proxy-injector webhook has valid cert
√ proxy-injector cert is valid for at least 60 days
√ sp-validator webhook has valid cert
√ sp-validator cert is valid for at least 60 days
√ policy-validator webhook has valid cert
√ policy-validator cert is valid for at least 60 days

linkerd-version
---------------
√ can determine the latest version
√ cli is up-to-date

control-plane-version
---------------------
√ can retrieve the control plane version
√ control plane is up-to-date
√ control plane and cli versions match

linkerd-control-plane-proxy
---------------------------
√ control plane proxies are healthy
√ control plane proxies are up-to-date
√ control plane proxies and cli versions match

Status check results are √

当出现上面的 Status check results are √ 信息后表示 Linkerd 的控制平面安装成功了。

Helm安装

除了使用 CLI 工具的方式安装控制平面之外，我们也可以通过 Helm Chart 的方式来安装，如下所示：

1
2
3
4
5
6
7
8
9
$ helm repo add linkerd https://helm.linkerd.io/stable
$ exp=$(date -d '+8760 hour' +"%Y-%m-%dT%H:%M:%SZ")
#$ exp=$(date -v+8760H +"%Y-%m-%dT%H:%M:%SZ") set expiry date one year from now, in Mac:
helm install linkerd2 \
  --set-file identityTrustAnchorsPEM=ca.crt \
  --set-file identity.issuer.tls.crtPEM=issuer.crt \
  --set-file identity.issuer.tls.keyPEM=issuer.key \
  --set identity.issuer.crtExpiry=$exp \
  linkerd/linkerd2

此外该 chart 包含一个 values-ha.yaml 文件， 它覆盖了一些默认值，以便在高可用性场景下进行设置， 类似于 linkerd install 中的 --ha 选项。我们可以通过获取 chart 文件来获得 values-ha.yaml

采用哪种方式进行安装均可，到这里我们现在就完成了 Linkerd 的安装，重新执行 linkerd version 命令就可以看到 Server 端版本信息了：

1
2
3
$ linkerd version
Client version: stable-2.13.1
Server version: stable-2.13.1

Linkerd Viz 扩展安装部署

另外还有一个需要注意的是 viz 插件，在最新版本中已经没有内置 grafana 了，由于新版本已经没有内置 Grafana 了，我们安装的使用可以通过 --set grafana.url 来指定外部的 Grafana 地址（如果是集群外的地址可以通过 grafana.externalUrl 参数指定）。通过dashboard.enforcedHostRegexp来指定允许那些域名、IP进行访问dashboard。

**官方参数介绍**：https://github.com/linkerd/linkerd2/blob/main/viz/charts/linkerd-viz/README.md

1
2
3
4
linkerd viz install --set grafana.externalUrl=http://grafana.od.com,dashboard.enforcedHostRegexp='^(localhost|127\\.0\\.0\\.1|web\\.linkerd-viz\\.svc\\.cluster\\.local|web\\.linkerd-viz\\.svc|viz\\.od\\.com|\\[::1\\])(:\d+)?$'|kubectl apply -f -

#  使用外部Prometheus
linkerd viz install --set grafana.externalUrl=http://grafana.od.com,prometheusUrl=http://prometheus-k8s.monitoring.svc:9090,prometheus.enabled=false,dashboard.enforcedHostRegexp='^(localhost|127\\.0\\.0\\.1|web\\.linkerd-viz\\.svc\\.cluster\\.local|web\\.linkerd-viz\\.svc|viz\\.od\\.com|\\[::1\\])(:\d+)?$'|kubectl apply -f -

安装完成后通过运行以下命令等待Viz 扩展平面准备就绪，并可以验证安装结果是否正常：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
$ linkerd check
kubernetes-api
--------------
√ can initialize the client
√ can query the Kubernetes API

kubernetes-version
------------------
√ is running the minimum Kubernetes API version

linkerd-existence
-----------------
√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
√ no unschedulable pods
√ control plane pods are ready
√ cluster networks contains all pods
√ cluster networks contains all services

linkerd-config
--------------
√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
√ control plane CustomResourceDefinitions exist
√ control plane MutatingWebhookConfigurations exist
√ control plane ValidatingWebhookConfigurations exist
√ proxy-init container runs as root user if docker container runtime is used

linkerd-identity
----------------
√ certificate config is valid
√ trust anchors are using supported crypto algorithm
√ trust anchors are within their validity period
√ trust anchors are valid for at least 60 days
√ issuer cert is using supported crypto algorithm
√ issuer cert is within its validity period
√ issuer cert is valid for at least 60 days
√ issuer cert is issued by the trust anchor

linkerd-webhooks-and-apisvc-tls
-------------------------------
√ proxy-injector webhook has valid cert
√ proxy-injector cert is valid for at least 60 days
√ sp-validator webhook has valid cert
√ sp-validator cert is valid for at least 60 days
√ policy-validator webhook has valid cert
√ policy-validator cert is valid for at least 60 days

linkerd-version
---------------
√ can determine the latest version
√ cli is up-to-date

control-plane-version
---------------------
√ can retrieve the control plane version
√ control plane is up-to-date
√ control plane and cli versions match

linkerd-control-plane-proxy
---------------------------
√ control plane proxies are healthy
√ control plane proxies are up-to-date
√ control plane proxies and cli versions match

linkerd-viz
-----------
√ linkerd-viz Namespace exists
√ can initialize the client
√ linkerd-viz ClusterRoles exist
√ linkerd-viz ClusterRoleBindings exist
√ tap API server has valid cert
√ tap API server cert is valid for at least 60 days
√ tap API service is running
√ linkerd-viz pods are injected
√ viz extension pods are running
√ viz extension proxies are healthy
√ viz extension proxies are up-to-date
√ viz extension proxies and cli versions match
√ prometheus is installed and configured correctly
√ viz extension self-check

Status check results are √

此外我们也可以通过 Ingress 来暴露 viz 服务，创建如下所示的资源对象：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
cat > viz-ing.yaml <<EOF
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: websocket
  namespace: linkerd-viz
spec:
  headers: # 配置websocket访问
    customRequestHeaders:
      Connection: keep-alive, Upgrade
      Upgrade: WebSocket
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: linkerd-dashboard
  namespace: linkerd-viz
  annotations:
    ingress.kubernetes.io/custom-request-headers: l5d-dst-override:web.linkerd-viz.svc.cluster.local:8084
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`viz.od.com`)  # 指定域名
    kind: Rule
    services:
    - name: web
      port: 8084
  - match: Host(`viz.od.com`) && Path(`/api/tap`)
    middlewares:
      - name: websocket
        namespace: linkerd-viz
    kind: Rule
    services:
    - name: web
      port: 8084
EOF

浏览器打开https://viz.od.com/

在对应的资源后面包含一个 Grafana 的图标，点击可以自动跳转到 Grafana 的监控页面。

grafana部署yaml

官方文档：https://linkerd.io/2.13/tasks/grafana/index.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
cat > grafana.yaml <<EOF
# Source: grafana/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
  name: grafana
  namespace: infra
---
# Source: grafana/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana
  namespace: infra
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
data:
  grafana.ini: |
    [analytics]
    check_for_updates = false
    [auth]
    disable_login_form = true
    [auth.anonymous]
    enabled = true
    org_role = Editor
    [auth.basic]
    enabled = false
    [grafana_net]
    url = https://grafana.net
    [log]
    mode = console
    [log.console]
    format = text
    level = info
    [panels]
    disable_sanitize_html = true
    [paths]
    data = /var/lib/grafana/
    logs = /var/log/grafana
    plugins = /var/lib/grafana/plugins
    provisioning = /etc/grafana/provisioning
    [server]
    domain = ''
    root_url = %(protocol)s://%(domain)s:/grafana/
    serve_from_sub_path = true
  datasources.yaml: |
    apiVersion: 1
    datasources:
    - access: proxy
      editable: true
      isDefault: true
      jsonData:
        timeInterval: 5s
      name: prometheus
      orgId: 1
      type: prometheus
      url: http://prometheus.linkerd-viz.svc.cluster.local:9090
  dashboardproviders.yaml: |
    apiVersion: 1
    providers:
    - disableDeletion: false
      editable: true
      folder: ""
      name: default
      options:
        path: /var/lib/grafana/dashboards/default
      orgId: 1
      type: file
  download_dashboards.sh: |
    #!/usr/bin/env sh
    grafana_dir="/var/lib/grafana/dashboards/default"
    
    if [ ! -d "\$grafana_dir" ]; then
      mkdir -p "\$grafana_dir"
    fi

    download_dashboard() {
      local url="\$1"
      local file="\$2"
      if [ ! -f "\$file" ]; then
        echo "Downloading \$url..."
        curl -skf --connect-timeout 60 --max-time 60 \
          -H "Accept: application/json" \
          -H "Content-Type: application/json;charset=UTF-8" \
          "\$url" | sed '/-- .* --/! s/"datasource":.*,/"datasource": "prometheus",/g' \
          > "\$file"
      else
        echo "File \$file already exists"
      fi
    }

    # Download dashboards 如果文件存在而不会下载，不存在就会下载
    download_dashboard "https://grafana.com/api/dashboards/15482/revisions/2/download" \
      "\$grafana_dir/authority.json"
    download_dashboard "https://grafana.com/api/dashboards/15483/revisions/2/download" \
      "\$grafana_dir/cronjob.json"
    download_dashboard "https://grafana.com/api/dashboards/15484/revisions/2/download" \
      "\$grafana_dir/daemonset.json"
    download_dashboard "https://grafana.com/api/dashboards/15475/revisions/5/download" \
      "\$grafana_dir/deployment.json"
    download_dashboard "https://grafana.com/api/dashboards/15486/revisions/2/download" \
      "\$grafana_dir/health.json"
    download_dashboard "https://grafana.com/api/dashboards/15487/revisions/2/download" \
      "\$grafana_dir/job.json"
    download_dashboard "https://grafana.com/api/dashboards/15479/revisions/2/download" \
      "\$grafana_dir/kubernetes.json"
    download_dashboard "https://grafana.com/api/dashboards/15488/revisions/2/download" \
      "\$grafana_dir/multicluster.json"
    download_dashboard "https://grafana.com/api/dashboards/15478/revisions/2/download" \
      "\$grafana_dir/namespace.json"
    download_dashboard "https://grafana.com/api/dashboards/15477/revisions/2/download" \
      "\$grafana_dir/pod.json"
    download_dashboard "https://grafana.com/api/dashboards/15489/revisions/2/download" \
      "\$grafana_dir/prometheus.json"
    download_dashboard "https://grafana.com/api/dashboards/15490/revisions/2/download" \
      "\$grafana_dir/prometheus-benchmark.json"
    download_dashboard "https://grafana.com/api/dashboards/15491/revisions/2/download" \
      "\$grafana_dir/replicaset.json"
    download_dashboard "https://grafana.com/api/dashboards/15492/revisions/2/download" \
      "\$grafana_dir/replicationcontroller.json"
    download_dashboard "https://grafana.com/api/dashboards/15481/revisions/2/download" \
      "\$grafana_dir/route.json"
    download_dashboard "https://grafana.com/api/dashboards/15480/revisions/2/download" \
      "\$grafana_dir/service.json"
    download_dashboard "https://grafana.com/api/dashboards/15493/revisions/2/download" \
      "\$grafana_dir/statefulset.json"
    download_dashboard "https://grafana.com/api/dashboards/15474/revisions/3/download" \
      "\$grafana_dir/top-line.json"
---
# Source: grafana/templates/dashboards-json-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-dashboards-default
  namespace: infra
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
    dashboard-provider: default
data:
  {}
---
# Source: grafana/templates/clusterrole.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
  name: grafana-clusterrole
rules: []
---
# Source: grafana/templates/clusterrolebinding.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: grafana-clusterrolebinding
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
subjects:
  - kind: ServiceAccount
    name: grafana
    namespace: infra
roleRef:
  kind: ClusterRole
  name: grafana-clusterrole
  apiGroup: rbac.authorization.k8s.io
---
# Source: grafana/templates/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: grafana
  namespace: infra
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
rules: []
---
# Source: grafana/templates/rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: grafana
  namespace: infra
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: grafana
subjects:
- kind: ServiceAccount
  name: grafana
  namespace: infra
---
# Source: grafana/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: grafana
  namespace: infra
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
spec:
  type: ClusterIP
  ports:
    - name: service
      port: 80
      protocol: TCP
      targetPort: 3000
  selector:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
---
# Source: grafana/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana
  namespace: infra
  labels:
    app.kubernetes.io/name: grafana
    app.kubernetes.io/instance: grafana
    app.kubernetes.io/version: "9.4.7"
spec:
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/name: grafana
      app.kubernetes.io/instance: grafana
  template:
    metadata:
      labels:
        app.kubernetes.io/name: grafana
        app.kubernetes.io/instance: grafana
      annotations:
        linkerd.io/inject: enabled
    spec:
      nodeName: k8s-node1
      serviceAccountName: grafana
      automountServiceAccountToken: true
      securityContext:
        runAsUser: 0
      initContainers: # 通过初始化认为下载所需要的dashboards
        - name: download-dashboards
          image: "docker.io/curlimages/curl:7.85.0"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: [ "-c", "mkdir -p /var/lib/grafana/dashboards/default && /bin/sh -x /etc/grafana/download_dashboards.sh" ]
          volumeMounts:
            - name: config
              mountPath: "/etc/grafana/download_dashboards.sh"
              subPath: download_dashboards.sh
            - name: storage
              mountPath: "/var/lib/grafana"
      enableServiceLinks: true
      containers:
        - name: grafana
          image: "docker.io/grafana/grafana:9.4.7"
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: config
              mountPath: "/etc/grafana/grafana.ini"
              subPath: grafana.ini
            - name: storage
              mountPath: "/var/lib/grafana"
            - name: config
              mountPath: "/etc/grafana/provisioning/datasources/datasources.yaml"
              subPath: "datasources.yaml"
            - name: config
              mountPath: "/etc/grafana/provisioning/dashboards/dashboardproviders.yaml"
              subPath: "dashboardproviders.yaml"
          ports:
            - name: grafana
              containerPort: 3000
              protocol: TCP
          env:
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: GF_PATHS_DATA
              value: /var/lib/grafana/
            - name: GF_PATHS_LOGS
              value: /var/log/grafana
            - name: GF_PATHS_PLUGINS
              value: /var/lib/grafana/plugins
            - name: GF_PATHS_PROVISIONING
              value: /etc/grafana/provisioning
          livenessProbe:
            failureThreshold: 10
            httpGet:
              path: /api/health
              port: 3000
            initialDelaySeconds: 60
            timeoutSeconds: 30
          readinessProbe:
            httpGet:
              path: /api/health
              port: 3000
      volumes:
        - name: config
          configMap:
            name: grafana
        - name: dashboards-default
          configMap:
            name: grafana-dashboards-default
        - name: storage  # 挂载到本地
          hostPath:
            path: /data/volumes/grafana
EOF

通过traefik实现外部访问

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cat > k8s-yaml/grafana/ing.yaml <EOF
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: grafana-web
  namespace: infra
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`grafana.od.com`)  # 指定域名
    kind: Rule
    services:
    - name: grafana
      port: 80
EOF