Sealed Secretsを試す

Sealed Secretsは数年前に試したことがあるが、もう一度試すメモ。

以前試したときの記事は以下。

• SecretをGitHubに登録したくないのでSealedSecretを使う

コンポーネント	バージョン
EKS	1.21
プラットフォームバージョン	eks.3
eksctl	0.75.0
Sealed Secrets	v0.16.0
Sealed Secrets チャート	1.16.1

クラスターの作成

1.21でクラスターを作成する。ノードなしで作成する。

CLUSTER_NAME="sealedsecrets"
cat << EOF > cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: ${CLUSTER_NAME}
  region: ap-northeast-1
  version: "1.21"
vpc:
  cidr: "10.0.0.0/16"

availabilityZones:
  - ap-northeast-1a
  - ap-northeast-1c

cloudWatch:
  clusterLogging:
    enableTypes: ["*"]

iam:
  withOIDC: true
EOF

eksctl create cluster -f cluster.yaml

ノードを作成する。

CLUSTER_NAME="sealedsecrets"
cat << EOF > managed-ng-1.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: ${CLUSTER_NAME}
  region: ap-northeast-1

managedNodeGroups:
  - name: managed-ng-1
    minSize: 2
    maxSize: 10
    desiredCapacity: 2
    privateNetworking: true
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
EOF

eksctl create nodegroup -f managed-ng-1.yaml

Adminロールにも権限をつけておく。

CLUSTER_NAME="sealedsecrets"
USER_NAME="Admin:{{SessionName}}"
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --output text --query Account)
ROLE_ARN="arn:aws:iam::${AWS_ACCOUNT_ID}:role/Admin"
eksctl create iamidentitymapping --cluster ${CLUSTER_NAME} --arn ${ROLE_ARN} --username ${USER_NAME} --group system:masters

Sealed Secretsのデプロイ

Helmチャートでインストールする。

helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm repo update

$ helm search repo sealed-secrets
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                                  
sealed-secrets/sealed-secrets   1.16.1          v0.16.0         Helm chart for the sealed-secrets controller.
stable/sealed-secrets           1.12.2          0.13.1          DEPRECATED - A Helm chart for Sealed Secrets 

$ helm inspect values sealed-secrets/sealed-secrets
image:
  repository: quay.io/bitnami/sealed-secrets-controller
  tag: v0.16.0
  pullPolicy: IfNotPresent
  pullSecret: ""

resources: {}
nodeSelector: {}
tolerations: []
affinity: {}

controller:
  # controller.create: `true` if Sealed Secrets controller should be created
  create: true
  # controller.labels: Extra labels to be added to controller deployment
  labels: {}
  # controller.service: Configuration options for controller service
  service:
    # controller.service.labels: Extra labels to be added to controller service
    labels: {}

# namespace: Namespace to deploy the controller.
namespace: ""

serviceAccount:
  # serviceAccount.create: Whether to create a service account or not
  create: true
  # serviceAccount.labels: Extra labels to be added to service account
  labels: {}
  # serviceAccount.name: The name of the service account to create or use
  name: ""

rbac:
  # rbac.create: `true` if rbac resources should be created
  create: true
  # rbac.labels: Extra labels to be added to rbac resources
  labels: {}
  pspEnabled: false

# secretName: The name of the TLS secret containing the key used to encrypt secrets
secretName: "sealed-secrets-key"

ingress:
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  path: /v1/cert.pem
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

crd:
  # crd.create: `true` if the crd resources should be created
  create: true
  # crd.keep: `true` if the sealed secret CRD should be kept when the chart is deleted
  keep: true

networkPolicy: false

securityContext:
  # securityContext.runAsUser defines under which user the operator Pod and its containers/processes run.
  runAsUser: 1001
  # securityContext.fsGroup defines the filesystem group
  fsGroup: 65534

podAnnotations: {}

podLabels: {}

priorityClassName: ""

serviceMonitor:
  # Enables ServiceMonitor creation for the Prometheus Operator
  create: false
  # How frequently Prometheus should scrape the ServiceMonitor
  interval:
  # Extra labels to apply to the sealed-secrets ServiceMonitor
  labels:
  # The namespace where the ServiceMonitor is deployed, defaults to the installation namespace
  namespace:
  # The timeout after which the scrape is ended
  scrapeTimeout:

dashboards:
  # If enabled, sealed-secrets will create a configmap with a dashboard in json that's going to be picked up by grafana
  # See https://github.com/helm/charts/tree/master/stable/grafana#configuration - `sidecar.dashboards.enabled`
  create: false
  # Extra labels to apply to the dashboard configmaps
  labels:
  # The namespace where the dashboards are deployed, defaults to the installation namespace
  namespace:

デフォルトでインストールする。

$ helm upgrade -i sealed-secrets sealed-secrets/sealed-secrets -n kube-system
Release "sealed-secrets" does not exist. Installing it now.
NAME: sealed-secrets
LAST DEPLOYED: Wed Dec  1 07:17:09 2021
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You should now be able to create sealed secrets.

1. Install client-side tool into /usr/local/bin/

GOOS=$(go env GOOS)
GOARCH=$(go env GOARCH)
wget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.16.0/kubeseal-$GOOS-$GOARCH
sudo install -m 755 kubeseal-$GOOS-$GOARCH /usr/local/bin/kubeseal

2. Create a sealed secret file

# note the use of `--dry-run` - this does not create a secret in your cluster
kubectl create secret generic secret-name --dry-run --from-literal=foo=bar -o [json|yaml] | \
 kubeseal \
 --controller-name=sealed-secrets \
 --controller-namespace=kube-system \
 --format [json|yaml] > mysealedsecret.[json|yaml]

The file mysealedsecret.[json|yaml] is a commitable file.

If you would rather not need access to the cluster to generate the sealed secret you can run

kubeseal \
 --controller-name=sealed-secrets \
 --controller-namespace=kube-system \
 --fetch-cert > mycert.pem

to retrieve the public cert used for encryption and store it locally. You can then run 'kubeseal --cert mycert.pem' instead to use the local cert e.g.

kubectl create secret generic secret-name --dry-run --from-literal=foo=bar -o [json|yaml] | \
kubeseal \
 --controller-name=sealed-secrets \
 --controller-namespace=kube-system \
 --format [json|yaml] --cert mycert.pem > mysealedsecret.[json|yaml]

3. Apply the sealed secret

kubectl create -f mysealedsecret.[json|yaml]

Running 'kubectl get secret secret-name -o [json|yaml]' will show the decrypted secret that was generated from the sealed secret.

Both the SealedSecret and generated Secret must have the same name and namespace.

Podを確認する。

$ k get po -n kube-system
NAME                              READY   STATUS    RESTARTS   AGE
aws-node-7ksm5                    1/1     Running   0          30m
aws-node-8b76r                    1/1     Running   0          30m
coredns-76f4967988-6jsh9          1/1     Running   0          45m
coredns-76f4967988-dsqcl          1/1     Running   0          45m
kube-proxy-6jv8f                  1/1     Running   0          30m
kube-proxy-zrv6n                  1/1     Running   0          30m
sealed-secrets-7569f57679-jjxwz   1/1     Running   0          3m

Secretを確認する。

$ k get secret -n kube-system --show-labels| grep sealed
sealed-secrets-keyzkf9n                          kubernetes.io/tls                     2      17m   sealedsecrets.bitnami.com/sealed-secrets-key=active
sealed-secrets-token-fflwb                       kubernetes.io/service-account-token   3      17m   <none>
sh.helm.release.v1.sealed-secrets.v1             helm.sh/release.v1                    1      17m   modifiedAt=1638310631,name=sealed-secrets,owner=helm,status=deployed,version=1

sealed-secrets-keyzkf9nがSealing Keyで、30日毎に増えていくと思われる。

Deploymentをeditしてローテーション間隔を1hにする。

$ k -n kube-system edit deploy sealed-secrets

    spec:
      containers:
      - args:
        - --key-prefix
        - sealed-secrets-key
        - --key-renew-period=1h
        command:
        - controller

その後しばらくするとこのように鍵が増えた。

$ k get secret -n kube-system --show-labels| grep sealed
sealed-secrets-keyww7cc                          kubernetes.io/tls                     2      7m58s   sealedsecrets.bitnami.com/sealed-secrets-key=active
sealed-secrets-keyzkf9n                          kubernetes.io/tls                     2      67m     sealedsecrets.bitnami.com/sealed-secrets-key=active
sealed-secrets-token-fflwb                       kubernetes.io/service-account-token   3      68m     <none>
sh.helm.release.v1.sealed-secrets.v1             helm.sh/release.v1                    1      68m     modifiedAt=1638310631,name=sealed-secrets,owner=helm,status=deployed,version=1

Sealed Secretsの作成

ローカルにyamlを作成する。

kubectl create secret generic secret-name --dry-run --from-literal=foo=bar -o yaml | \
  kubeseal \
    --controller-name=sealed-secrets \
    --controller-namespace=kube-system \
    --format yaml > mysealedsecret.yaml

これでこんなyamlができた。

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: secret-name
  namespace: default
spec:
  encryptedData:
    foo: AgBYsvMwOApbU62E51gYMlKuu+l32NYl9BJkNA1mvBaxXBLs4DhKbARGaLQCdatZwOXEfCV62Ud9x3M+QSJ/j1JvQPuKV6EzNDjk5ksmFGVP8UrFZBT2xGE+FZKUlwyR1sxpS9aWOFS4J2tZx8gvHy4zTCZ4ROQDf0WkbPRKULMdyIzzGBGD7+TN9j7L5b7U32aL5J1pDGTGnDsWmDS1T/pNtNiNpdsG8h26XVsMXTAjyLoZ+ZTeeA432qvNKPbcmeYrxRWnQl6O4A5jmXNcw2APtlidzM9jRH5Msd0zfkCsywRxfBH5Eo0Yp6EaIxic8Op6W0ojpt/b4/HNo13+zJL/C+BCU6UUHMSRCSRtY983dI+AWzzx681wmE4AwvJRBC7uhoEhEEWxGY6zBCjBA6R+J3PdN9KNEnOLd0GRZcO0T+bXFTn1EDpPhXqre/uAnFdFsM4LvHhi43K5DbwVTal7gN6VloSU0z2xNk/qXbGUcBa5LeJPobYy+tz+D8pXpfB38/JqhKyZCRaZ6jPJiOkEr+PMqo3tn7oTELs8EkwRmvuAblP5SsnOM3xfRD87vSXJ0VnJ1AqeP2SfpHmX6UV1CYlbX5e+wW+2rzmyJckcm9/A1NaQmo8a5Fw/J90bAw0L+eMDB/Glmd8Mx3yGORjAbp771oYDb2ls5vHkLMRhvMY1xWeA0tdRFf4Zem+MIpnbxT0=
  template:
    data: null
    metadata:
      creationTimestamp: null
      name: secret-name
      namespace: default

これをapplyする。

$ k apply -f mysealedsecret.yaml
sealedsecret.bitnami.com/secret-name created

確認する。

$ k get sealedsecret,secret
NAME                                   AGE
sealedsecret.bitnami.com/secret-name   40s

NAME                         TYPE                                  DATA   AGE
secret/default-token-v5tmb   kubernetes.io/service-account-token   3      55m
secret/secret-name           Opaque                                1      40s

他のNamespaceだと復号されないことを確認する。Namespaceを作成する。

$ k create ns test
namespace/test created

先ほどのyamlを編集してNamespaceだけ変える。

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: secret-name
  namespace: test
spec:
  encryptedData:
    foo: AgBYsvMwOApbU62E51gYMlKuu+l32NYl9BJkNA1mvBaxXBLs4DhKbARGaLQCdatZwOXEfCV62Ud9x3M+QSJ/j1JvQPuKV6EzNDjk5ksmFGVP8UrFZBT2xGE+FZKUlwyR1sxpS9aWOFS4J2tZx8gvHy4zTCZ4ROQDf0WkbPRKULMdyIzzGBGD7+TN9j7L5b7U32aL5J1pDGTGnDsWmDS1T/pNtNiNpdsG8h26XVsMXTAjyLoZ+ZTeeA432qvNKPbcmeYrxRWnQl6O4A5jmXNcw2APtlidzM9jRH5Msd0zfkCsywRxfBH5Eo0Yp6EaIxic8Op6W0ojpt/b4/HNo13+zJL/C+BCU6UUHMSRCSRtY983dI+AWzzx681wmE4AwvJRBC7uhoEhEEWxGY6zBCjBA6R+J3PdN9KNEnOLd0GRZcO0T+bXFTn1EDpPhXqre/uAnFdFsM4LvHhi43K5DbwVTal7gN6VloSU0z2xNk/qXbGUcBa5LeJPobYy+tz+D8pXpfB38/JqhKyZCRaZ6jPJiOkEr+PMqo3tn7oTELs8EkwRmvuAblP5SsnOM3xfRD87vSXJ0VnJ1AqeP2SfpHmX6UV1CYlbX5e+wW+2rzmyJckcm9/A1NaQmo8a5Fw/J90bAw0L+eMDB/Glmd8Mx3yGORjAbp771oYDb2ls5vHkLMRhvMY1xWeA0tdRFf4Zem+MIpnbxT0=
  template:
    data: null
    metadata:
      creationTimestamp: null
      name: secret-name
      namespace: test

applyする。

$ k apply -f mysealedsecret-test.yaml
sealedsecret.bitnami.com/secret-name created

この場合は復号されない。

$ k get sealedsecret,secret -n test
NAME                                   AGE
sealedsecret.bitnami.com/secret-name   39s

NAME                         TYPE                                  DATA   AGE
secret/default-token-5tkzz   kubernetes.io/service-account-token   3      2m53s