Linkerd を試すメモ。
コンポーネント | バージョン | 備考 |
---|---|---|
eksctl | 0.80.0 | |
Kubernetes バージョン | 1.21 | |
プラットフォームのバージョン | eks.4 | |
Linkerd CLI | 2.11.1 |
クラスターの作成
1.21 のクラスターをノードなしで作成する。
CLUSTER_NAME="linkerd" cat << EOF > cluster.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: ${CLUSTER_NAME} region: ap-northeast-1 version: "1.21" vpc: cidr: "10.0.0.0/16" availabilityZones: - ap-northeast-1a - ap-northeast-1c cloudWatch: clusterLogging: enableTypes: ["*"] iam: withOIDC: true EOF
eksctl create cluster -f cluster.yaml
ノードを作成する。
cat << EOF > managed-ng-1.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: ${CLUSTER_NAME} region: ap-northeast-1 managedNodeGroups: - name: managed-ng-1 minSize: 2 maxSize: 10 desiredCapacity: 2 privateNetworking: true iam: attachPolicyARNs: - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore EOF
eksctl create nodegroup -f managed-ng-1.yaml
Admin ロールにも権限をつけておく。
USER_NAME="Admin:{{SessionName}}" AWS_ACCOUNT_ID=$(aws sts get-caller-identity --output text --query Account) ROLE_ARN="arn:aws:iam::${AWS_ACCOUNT_ID}:role/Admin" eksctl create iamidentitymapping --cluster ${CLUSTER_NAME} --arn ${ROLE_ARN} --username ${USER_NAME} --group system:masters
Linkerd CLI のインストール
Linkerd CLI をインストールする。
brew install linkerd
バージョンを確認する。
$ linkerd version Client version: stable-2.11.1 Server version: unavailable
クラスターのチェック
クラスターの前提条件をチェックする。
$ linkerd check --pre Linkerd core checks =================== kubernetes-api -------------- √ can initialize the client √ can query the Kubernetes API kubernetes-version ------------------ √ is running the minimum Kubernetes API version √ is running the minimum kubectl version pre-kubernetes-setup -------------------- √ control plane namespace does not already exist √ can create non-namespaced resources √ can create ServiceAccounts √ can create Services √ can create Deployments √ can create CronJobs √ can create ConfigMaps √ can create Secrets √ can read Secrets √ can read extension-apiserver-authentication configmap √ no clock skew detected linkerd-version --------------- √ can determine the latest version √ cli is up-to-date Status check results are √
コントロールプレーンのデプロイ
コントロールプレーンをデプロイする。Helm チャートもある。
$ linkerd install | kubectl apply -f - namespace/linkerd created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-identity created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-identity created serviceaccount/linkerd-identity created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-destination created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-destination created serviceaccount/linkerd-destination created secret/linkerd-sp-validator-k8s-tls created validatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-sp-validator-webhook-config created secret/linkerd-policy-validator-k8s-tls created validatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-policy-validator-webhook-config created clusterrole.rbac.authorization.k8s.io/linkerd-policy created clusterrolebinding.rbac.authorization.k8s.io/linkerd-destination-policy created role.rbac.authorization.k8s.io/linkerd-heartbeat created rolebinding.rbac.authorization.k8s.io/linkerd-heartbeat created clusterrole.rbac.authorization.k8s.io/linkerd-heartbeat created clusterrolebinding.rbac.authorization.k8s.io/linkerd-heartbeat created serviceaccount/linkerd-heartbeat created customresourcedefinition.apiextensions.k8s.io/servers.policy.linkerd.io created customresourcedefinition.apiextensions.k8s.io/serverauthorizations.policy.linkerd.io created customresourcedefinition.apiextensions.k8s.io/serviceprofiles.linkerd.io created customresourcedefinition.apiextensions.k8s.io/trafficsplits.split.smi-spec.io created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-proxy-injector created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-proxy-injector created serviceaccount/linkerd-proxy-injector created secret/linkerd-proxy-injector-k8s-tls created mutatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-proxy-injector-webhook-config created configmap/linkerd-config created secret/linkerd-identity-issuer created configmap/linkerd-identity-trust-roots created service/linkerd-identity created service/linkerd-identity-headless created deployment.apps/linkerd-identity created service/linkerd-dst created service/linkerd-dst-headless created service/linkerd-sp-validator created service/linkerd-policy created service/linkerd-policy-validator created deployment.apps/linkerd-destination created Warning: batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob cronjob.batch/linkerd-heartbeat created deployment.apps/linkerd-proxy-injector created service/linkerd-proxy-injector created secret/linkerd-config-overrides created
デプロイをチェックする。
$ linkerd check Linkerd core checks =================== kubernetes-api -------------- √ can initialize the client √ can query the Kubernetes API kubernetes-version ------------------ √ is running the minimum Kubernetes API version √ is running the minimum kubectl version linkerd-existence ----------------- √ 'linkerd-config' config map exists √ heartbeat ServiceAccount exist √ control plane replica sets are ready √ no unschedulable pods √ control plane pods are ready linkerd-config -------------- √ control plane Namespace exists √ control plane ClusterRoles exist √ control plane ClusterRoleBindings exist √ control plane ServiceAccounts exist √ control plane CustomResourceDefinitions exist √ control plane MutatingWebhookConfigurations exist √ control plane ValidatingWebhookConfigurations exist linkerd-identity ---------------- √ certificate config is valid √ trust anchors are using supported crypto algorithm √ trust anchors are within their validity period √ trust anchors are valid for at least 60 days √ issuer cert is using supported crypto algorithm √ issuer cert is within its validity period √ issuer cert is valid for at least 60 days √ issuer cert is issued by the trust anchor linkerd-webhooks-and-apisvc-tls ------------------------------- √ proxy-injector webhook has valid cert √ proxy-injector cert is valid for at least 60 days √ sp-validator webhook has valid cert √ sp-validator cert is valid for at least 60 days √ policy-validator webhook has valid cert √ policy-validator cert is valid for at least 60 days linkerd-version --------------- √ can determine the latest version √ cli is up-to-date control-plane-version --------------------- √ can retrieve the control plane version √ control plane is up-to-date √ control plane and cli versions match linkerd-control-plane-proxy --------------------------- √ control plane proxies are healthy √ control plane proxies are up-to-date √ control plane proxies and cli versions match Status check results are √
Pod を確認する。
$ k get po -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system aws-node-ltd8k 1/1 Running 0 7h6m kube-system aws-node-xdblx 1/1 Running 0 7h6m kube-system coredns-76f4967988-p5njc 1/1 Running 0 7h34m kube-system coredns-76f4967988-xgjss 1/1 Running 0 7h34m kube-system kube-proxy-dmc2v 1/1 Running 0 7h6m kube-system kube-proxy-s5vvz 1/1 Running 0 7h6m linkerd linkerd-destination-685b865bdb-t54s9 4/4 Running 0 117s linkerd linkerd-identity-6b78ff444f-h28v7 2/2 Running 0 117s linkerd linkerd-proxy-injector-67475764f4-wgpm6 2/2 Running 0 117s
デモアプリのインストール
デモアプリをインストールする。
$ curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/emojivoto.yml \ | kubectl apply -f - namespace/emojivoto created serviceaccount/emoji created serviceaccount/voting created serviceaccount/web created service/emoji-svc created service/voting-svc created service/web-svc created deployment.apps/emoji created deployment.apps/vote-bot created deployment.apps/voting created deployment.apps/web created
確認する。
$ k -n emojivoto get po,svc NAME READY STATUS RESTARTS AGE pod/emoji-66ccdb4d86-m7rpm 1/1 Running 0 3m25s pod/vote-bot-69754c864f-vp4x7 1/1 Running 0 3m25s pod/voting-f999bd4d7-p7cgr 1/1 Running 0 3m25s pod/web-79469b946f-bnwwv 1/1 Running 0 3m25s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/emoji-svc ClusterIP 172.20.117.164 <none> 8080/TCP,8801/TCP 3m26s service/voting-svc ClusterIP 172.20.27.217 <none> 8080/TCP,8801/TCP 3m26s service/web-svc ClusterIP 172.20.238.239 <none> 80/TCP 3m26s
外に公開されていないので、ポートフォワードしてから、http://localhost:8080 にアクセスする。
kubectl -n emojivoto port-forward svc/web-svc 8080:80
サイドカーのインジェクション
デモアプリに、サイドカーをインジェクトする。
$ kubectl get -n emojivoto deploy -o yaml \ | linkerd inject - \ | kubectl apply -f - deployment "emoji" injected deployment "vote-bot" injected deployment "voting" injected deployment "web" injected deployment.apps/emoji configured deployment.apps/vote-bot configured deployment.apps/voting configured deployment.apps/web configured
このコマンドは Deployment の PodTemplate にアノテーションを追加しているだけ。サイドカーをインジェクトするのは Pod のデプロイのタイミングの模様。
$ k -n emojivoto get deploy web -o yaml apiVersion: apps/v1 kind: Deployment spec: ... template: metadata: annotations: linkerd.io/inject: enabled ...
Pod は 2/2 になっておりサイドカーが入っている。
$ k -n emojivoto get po NAME READY STATUS RESTARTS AGE emoji-696d9d8f95-mcgff 2/2 Running 0 9m47s vote-bot-6d7677bb68-tlbf4 2/2 Running 0 9m47s voting-ff4c54b8d-wb9kk 2/2 Running 0 9m47s web-5f86686c4d-s5kf8 2/2 Running 0 9m47s
プロキシがインジェクトされていることをチェックする。
$ linkerd -n emojivoto check --proxy Linkerd core checks =================== kubernetes-api -------------- √ can initialize the client √ can query the Kubernetes API kubernetes-version ------------------ √ is running the minimum Kubernetes API version √ is running the minimum kubectl version linkerd-existence ----------------- √ 'linkerd-config' config map exists √ heartbeat ServiceAccount exist √ control plane replica sets are ready √ no unschedulable pods √ control plane pods are ready linkerd-config -------------- √ control plane Namespace exists √ control plane ClusterRoles exist √ control plane ClusterRoleBindings exist √ control plane ServiceAccounts exist √ control plane CustomResourceDefinitions exist √ control plane MutatingWebhookConfigurations exist √ control plane ValidatingWebhookConfigurations exist linkerd-identity ---------------- √ certificate config is valid √ trust anchors are using supported crypto algorithm √ trust anchors are within their validity period √ trust anchors are valid for at least 60 days √ issuer cert is using supported crypto algorithm √ issuer cert is within its validity period √ issuer cert is valid for at least 60 days √ issuer cert is issued by the trust anchor linkerd-webhooks-and-apisvc-tls ------------------------------- √ proxy-injector webhook has valid cert √ proxy-injector cert is valid for at least 60 days √ sp-validator webhook has valid cert √ sp-validator cert is valid for at least 60 days √ policy-validator webhook has valid cert √ policy-validator cert is valid for at least 60 days linkerd-identity-data-plane --------------------------- √ data plane proxies certificate match CA linkerd-version --------------- √ can determine the latest version √ cli is up-to-date linkerd-control-plane-proxy --------------------------- √ control plane proxies are healthy √ control plane proxies are up-to-date √ control plane proxies and cli versions match linkerd-data-plane ------------------ √ data plane namespace exists √ data plane proxies are ready √ data plane is up-to-date √ data plane and cli versions match √ data plane pod labels are configured correctly √ data plane service labels are configured correctly √ data plane service annotations are configured correctly √ opaque ports are properly annotated Status check results are √
Linkerd を探索する
コントロールプレーンのコア機能は機能が最小限なので、エクステンションを追加する。
クラスター上にメトリクススタックをデプロイする viz エクステンションを追加する。
$ linkerd viz install | kubectl apply -f - namespace/linkerd-viz created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-metrics-api created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-metrics-api created serviceaccount/metrics-api created serviceaccount/grafana created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-prometheus created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-prometheus created serviceaccount/prometheus created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-admin created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-auth-delegator created serviceaccount/tap created rolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-tap-auth-reader created secret/tap-k8s-tls created apiservice.apiregistration.k8s.io/v1alpha1.tap.linkerd.io created role.rbac.authorization.k8s.io/web created rolebinding.rbac.authorization.k8s.io/web created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-check created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-check created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-admin created clusterrole.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-api created clusterrolebinding.rbac.authorization.k8s.io/linkerd-linkerd-viz-web-api created serviceaccount/web created server.policy.linkerd.io/admin created serverauthorization.policy.linkerd.io/admin created server.policy.linkerd.io/proxy-admin created serverauthorization.policy.linkerd.io/proxy-admin created service/metrics-api created deployment.apps/metrics-api created server.policy.linkerd.io/metrics-api created serverauthorization.policy.linkerd.io/metrics-api created configmap/grafana-config created service/grafana created deployment.apps/grafana created server.policy.linkerd.io/grafana created serverauthorization.policy.linkerd.io/grafana created configmap/prometheus-config created service/prometheus created deployment.apps/prometheus created service/tap created deployment.apps/tap created server.policy.linkerd.io/tap-api created serverauthorization.policy.linkerd.io/tap created clusterrole.rbac.authorization.k8s.io/linkerd-tap-injector created clusterrolebinding.rbac.authorization.k8s.io/linkerd-tap-injector created serviceaccount/tap-injector created secret/tap-injector-k8s-tls created mutatingwebhookconfiguration.admissionregistration.k8s.io/linkerd-tap-injector-webhook-config created service/tap-injector created deployment.apps/tap-injector created server.policy.linkerd.io/tap-injector-webhook created serverauthorization.policy.linkerd.io/tap-injector created service/web created deployment.apps/web created serviceprofile.linkerd.io/metrics-api.linkerd-viz.svc.cluster.local created serviceprofile.linkerd.io/prometheus.linkerd-viz.svc.cluster.local created serviceprofile.linkerd.io/grafana.linkerd-viz.svc.cluster.local created
エクステンションをチェックする。
$ linkerd check Linkerd core checks =================== kubernetes-api -------------- √ can initialize the client √ can query the Kubernetes API kubernetes-version ------------------ √ is running the minimum Kubernetes API version √ is running the minimum kubectl version linkerd-existence ----------------- √ 'linkerd-config' config map exists √ heartbeat ServiceAccount exist √ control plane replica sets are ready √ no unschedulable pods √ control plane pods are ready linkerd-config -------------- √ control plane Namespace exists √ control plane ClusterRoles exist √ control plane ClusterRoleBindings exist √ control plane ServiceAccounts exist √ control plane CustomResourceDefinitions exist √ control plane MutatingWebhookConfigurations exist √ control plane ValidatingWebhookConfigurations exist linkerd-identity ---------------- √ certificate config is valid √ trust anchors are using supported crypto algorithm √ trust anchors are within their validity period √ trust anchors are valid for at least 60 days √ issuer cert is using supported crypto algorithm √ issuer cert is within its validity period √ issuer cert is valid for at least 60 days √ issuer cert is issued by the trust anchor linkerd-webhooks-and-apisvc-tls ------------------------------- √ proxy-injector webhook has valid cert √ proxy-injector cert is valid for at least 60 days √ sp-validator webhook has valid cert √ sp-validator cert is valid for at least 60 days √ policy-validator webhook has valid cert √ policy-validator cert is valid for at least 60 days linkerd-version --------------- √ can determine the latest version √ cli is up-to-date control-plane-version --------------------- √ can retrieve the control plane version √ control plane is up-to-date √ control plane and cli versions match linkerd-control-plane-proxy --------------------------- √ control plane proxies are healthy √ control plane proxies are up-to-date √ control plane proxies and cli versions match Status check results are √ Linkerd extensions checks ========================= linkerd-viz ----------- √ linkerd-viz Namespace exists √ linkerd-viz ClusterRoles exist √ linkerd-viz ClusterRoleBindings exist √ tap API server has valid cert √ tap API server cert is valid for at least 60 days √ tap API service is running √ linkerd-viz pods are injected √ viz extension pods are running √ viz extension proxies are healthy √ viz extension proxies are up-to-date √ viz extension proxies and cli versions match √ prometheus is installed and configured correctly √ can initialize the client √ viz extension self-check Status check results are √
Pod を確認する。
$ k -n linkerd-viz get po NAME READY STATUS RESTARTS AGE grafana-8d54d5f6d-25wn6 2/2 Running 0 61s metrics-api-6c59967bf4-zwpxl 2/2 Running 0 61s prometheus-7bbc4d8c5b-r646x 2/2 Running 0 61s tap-86df5fc5b8-94wfv 2/2 Running 0 60s tap-injector-544f99cc4f-s9v52 2/2 Running 0 60s web-db97ff489-c95qb 2/2 Running 0 60s
ダッシュボードを開く。
$ linkerd viz dashboard Linkerd dashboard available at: http://localhost:50750 Grafana dashboard available at: http://localhost:50750/grafana Opening Linkerd dashboard in the default browser
Linkerd の CRD を見てみる。
$ k get crd NAME CREATED AT eniconfigs.crd.k8s.amazonaws.com 2022-01-24T00:51:55Z securitygrouppolicies.vpcresources.k8s.aws 2022-01-24T00:51:58Z serverauthorizations.policy.linkerd.io 2022-01-24T08:24:54Z servers.policy.linkerd.io 2022-01-24T08:24:54Z serviceprofiles.linkerd.io 2022-01-24T08:24:54Z trafficsplits.split.smi-spec.io 2022-01-24T08:24:54Z