CKSコースのOPA部分のメモ。
参考リンク
- https://github.com/killer-sh/cks-course-environment/tree/master/course-content/opa
- https://github.com/killer-sh/cks-course-environment/tree/master/course-content/supply-chain-security/secure-the-supply-chain/whitelist-registries/opa
- https://play.openpolicyagent.org
- https://github.com/BouweCeunen/gatekeeper-policies
前提
GatekeeperのCRDがインストールされていることを確認する。
root@cks-master:~# k get crd NAME CREATED AT configs.config.gatekeeper.sh 2020-12-31T15:50:20Z constraintpodstatuses.status.gatekeeper.sh 2020-12-31T15:50:20Z constrainttemplatepodstatuses.status.gatekeeper.sh 2020-12-31T15:50:20Z constrainttemplates.templates.gatekeeper.sh 2020-12-31T15:50:20Z
全てのPod作成をDenyする例
テンプレートのマニフェストを作成する。
apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8salwaysdeny spec: crd: spec: names: kind: K8sAlwaysDeny validation: # Schema for the `parameters` field openAPIV3Schema: properties: message: type: string targets: - target: admission.k8s.gatekeeper.sh rego: | package k8salwaysdeny violation[{"msg": msg}] { 1 > 0 msg := input.parameters.message }
テンプレートを作成する。
k apply -f template.yaml
spec.crd.spec.names.kindで作成する制約のCRDを定義している- regoのviolationに複数の条件がある場合は、全ての条件がtrueになるとバイオレーションになる
テンプレートを確認する。
root@cks-master:~# k get constrainttemplates NAME AGE k8salwaysdeny 20s
これで既にCRDも作成されている。
root@cks-master:~# k get crd NAME CREATED AT configs.config.gatekeeper.sh 2020-12-31T15:50:20Z constraintpodstatuses.status.gatekeeper.sh 2020-12-31T15:50:20Z constrainttemplatepodstatuses.status.gatekeeper.sh 2020-12-31T15:50:20Z constrainttemplates.templates.gatekeeper.sh 2020-12-31T15:50:20Z k8salwaysdeny.constraints.gatekeeper.sh 2020-12-31T16:02:37Z root@cks-master:~# k api-resources | grep constraint k8salwaysdeny constraints.gatekeeper.sh false K8sAlwaysDeny constraintpodstatuses status.gatekeeper.sh true ConstraintPodStatus constrainttemplatepodstatuses status.gatekeeper.sh true ConstraintTemplatePodStatus constrainttemplates templates.gatekeeper.sh false ConstraintTemplate root@cks-master:~# k get k8salwaysdeny No resources found
制約のマニフェストを作成する。
apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sAlwaysDeny metadata: name: pod-always-deny spec: match: kinds: - apiGroups: [""] kinds: ["Pod"] parameters: message: "ACCESS DENIED!"
制約を作成する。
k apply -f constraint.yaml
制約を確認する。
root@cks-master:~# k get k8salwaysdeny NAME AGE pod-always-deny 6s
動作を確認する。
root@cks-master:~# k run pod1 --image=nginx Error from server ([denied by pod-always-deny] ACCESS DENIED!): admission webhook "validation.gatekeeper.sh" denied the request: [denied by pod-always-deny] ACCESS DENIED!
DenyされるのはPod作成のタイミングだが、describeコマンドのStatus部分を確認すると、現在のバイオレーション状態が確認できる。
root@cks-master:~# k describe k8salwaysdeny pod-always-deny
Name: pod-always-deny
（省略）
Status:
Audit Timestamp: 2020-12-31T16:13:19Z
By Pod:
Constraint UID: 77ed71e4-f613-4520-a781-6580a0e933da
Enforced: true
Id: gatekeeper-audit-65f658df68-tfxlf
Observed Generation: 3
Operations:
audit
status
Constraint UID: 77ed71e4-f613-4520-a781-6580a0e933da
Enforced: true
Id: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Observed Generation: 3
Operations:
webhook
Total Violations: 12
Violations:
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: gatekeeper-audit-65f658df68-tfxlf
Namespace: gatekeeper-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Namespace: gatekeeper-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: coredns-f9fd979d6-4wpg7
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: coredns-f9fd979d6-fkrcz
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: etcd-cks-master
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: kube-apiserver-cks-master
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: kube-controller-manager-cks-master
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: kube-proxy-58g65
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: kube-proxy-rcxcj
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: kube-scheduler-cks-master
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: weave-net-npd7c
Namespace: kube-system
Enforcement Action: deny
Kind: Pod
Message: ACCESS DENIED!
Name: weave-net-rp2b2
Namespace: kube-system
Events: <none>
テンプレートを変えて条件を追加し、一部の条件がfalseになるように変えてみる。
apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8salwaysdeny spec: crd: spec: names: kind: K8sAlwaysDeny validation: # Schema for the `parameters` field openAPIV3Schema: properties: message: type: string targets: - target: admission.k8s.gatekeeper.sh rego: | package k8salwaysdeny violation[{"msg": msg}] { 1 > 0 # true 1 > 2 # false msg := input.parameters.message }
制約のほうをdescribeしてみると、Total Violationsが0になっている。
root@cks-master:~# k describe k8salwaysdeny pod-always-deny
Name: pod-always-deny
（省略）
Status:
Audit Timestamp: 2020-12-31T16:19:36Z
By Pod:
Constraint UID: 77ed71e4-f613-4520-a781-6580a0e933da
Enforced: true
Id: gatekeeper-audit-65f658df68-tfxlf
Observed Generation: 3
Operations:
audit
status
Constraint UID: 77ed71e4-f613-4520-a781-6580a0e933da
Enforced: true
Id: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Observed Generation: 3
Operations:
webhook
Total Violations: 0
Events: <none>
リソースを削除する。
k delete -f template.yaml k delete -f constraint.yaml
Namespaceにラベルを強制する例
テンプレートのマニフェストを作成する。
apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8srequiredlabels spec: crd: spec: names: kind: K8sRequiredLabels validation: # Schema for the `parameters` field openAPIV3Schema: properties: labels: type: array items: string targets: - target: admission.k8s.gatekeeper.sh rego: | package k8srequiredlabels violation[{"msg": msg, "details": {"missing_labels": missing}}] { provided := {label | input.review.object.metadata.labels[label]} required := {label | label := input.parameters.labels[_]} missing := required - provided count(missing) > 0 msg := sprintf("you must provide labels: %v", [missing]) }
以下の部分がよく理解できないが、Comprehension（内包表記？）と思われる。
provided := {label | input.review.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_]}
なぜprovidedとrequiredで書き方が違うかというと、inputで渡されるlabelsはリストでなくオブジェクトで、パラメーターで渡されるlabelsはリストだから。
文法は以下。
- https://www.openpolicyagent.org/docs/latest/policy-language/
- https://www.openpolicyagent.org/docs/latest/policy-reference/
試すには、Playgroundか、ローカルで試すにはopaコマンドを導入する。
brewでも入れられる。
brew install opa
テンプレートを作成する。
k apply -f template.yaml
制約のマニフェストを作成する。
apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sRequiredLabels metadata: name: ns-must-have-cks spec: match: kinds: - apiGroups: [""] kinds: ["Namespace"] parameters: labels: ["cks"]
制約を作成する。
k apply -f constraint.yaml
describeコマンドでバイオレーションを確認する。
root@cks-master:~# k describe k8srequiredlabels ns-must-have-cks
Name: ns-must-have-cks
（省略）
Status:
Audit Timestamp: 2020-12-31T17:30:32Z
By Pod:
Constraint UID: dc4755af-5cc6-4570-9678-178e04245634
Enforced: true
Id: gatekeeper-audit-65f658df68-tfxlf
Observed Generation: 1
Operations:
audit
status
Constraint UID: dc4755af-5cc6-4570-9678-178e04245634
Enforced: true
Id: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Observed Generation: 1
Operations:
webhook
Total Violations: 5
Violations:
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: default
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: gatekeeper-system
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: kube-node-lease
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: kube-public
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: kube-system
Events: <none>
default Namespaceにラベルをつけて、バイオレーションが1つ消えることを確認する。バイオレーションを正しく検出するには少し待つ必要がある。
root@cks-master:~# k label ns default cks=amanzing
namespace/default labeled
root@cks-master:~# k describe k8srequiredlabels ns-must-have-cks
Name: ns-must-have-cks
（省略）
Status:
Audit Timestamp: 2020-12-31T17:33:38Z
By Pod:
Constraint UID: dc4755af-5cc6-4570-9678-178e04245634
Enforced: true
Id: gatekeeper-audit-65f658df68-tfxlf
Observed Generation: 1
Operations:
audit
status
Constraint UID: dc4755af-5cc6-4570-9678-178e04245634
Enforced: true
Id: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Observed Generation: 1
Operations:
webhook
Total Violations: 4
Violations:
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: gatekeeper-system
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: kube-node-lease
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: kube-public
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"cks"}
Name: kube-system
Events: <none>
Namespaceを作成しようとすると拒否されることを確認する。
root@cks-master:~# k create ns test
Error from server ([denied by ns-must-have-cks] you must provide labels: {"cks"}): admission webhook "validation.gatekeeper.sh" denied the request: [denied by ns-must-have-cks] you must provide labels: {"cks"}
必要なラベルを追加する。
apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sRequiredLabels metadata: name: ns-must-have-cks spec: match: kinds: - apiGroups: [""] kinds: ["Namespace"] parameters: labels: ["cks", "team"]
メッセージが少し変わる。
root@cks-master:~# k create ns test
Error from server ([denied by ns-must-have-cks] you must provide labels: {"cks", "team"}): admission webhook "validation.gatekeeper.sh" denied the request: [denied by ns-must-have-cks] you must provide labels: {"cks", "team"}
ラベルを付与したマニフェストを使って成功することを確認する。
apiVersion: v1 kind: Namespace metadata: creationTimestamp: null name: test labels: cks: amazing team: sotoiwa spec: {} status: {}
root@cks-master:~# k apply -f test-ns.yaml namespace/test created
リソースを削除する。
k delete ns test k delete -f constraint.yaml k delete -f template.yaml
Deploymentに最小レプリカ数を強制する例
テンプレートのマニフェストを作成する。
apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8sminreplicacount spec: crd: spec: names: kind: K8sMinReplicaCount validation: # Schema for the `parameters` field openAPIV3Schema: properties: min: type: integer targets: - target: admission.k8s.gatekeeper.sh rego: | package k8sminreplicacount violation[{"msg": msg, "details": {"missing_replicas": missing}}] { provided := input.review.object.spec.replicas required := input.parameters.min missing := required - provided missing > 0 msg := sprintf("you must provide %v more replicas", [missing]) }
テンプレートを作成する。
k apply -f template.yaml
制約のマニフェストを作成する。
apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sMinReplicaCount metadata: name: deployment-must-have-min-replicas spec: match: kinds: - apiGroups: ["apps"] kinds: ["Deployment"] parameters: min: 2
制約を作成する。
k apply -f constraint.yaml
describeコマンドでバイオレーションを確認する。
root@cks-master:~# k describe k8sminreplicacount deployment-must-have-min-replicas
Name: deployment-must-have-min-replicas
（省略）
Status:
Audit Timestamp: 2020-12-31T18:10:45Z
By Pod:
Constraint UID: 908d2fe7-3454-40d0-9ba9-5938f080b782
Enforced: true
Id: gatekeeper-audit-65f658df68-tfxlf
Observed Generation: 1
Operations:
audit
status
Constraint UID: 908d2fe7-3454-40d0-9ba9-5938f080b782
Enforced: true
Id: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Observed Generation: 1
Operations:
webhook
Total Violations: 2
Violations:
Enforcement Action: deny
Kind: Deployment
Message: you must provide 1 more replicas
Name: gatekeeper-audit
Namespace: gatekeeper-system
Enforcement Action: deny
Kind: Deployment
Message: you must provide 1 more replicas
Name: gatekeeper-controller-manager
Namespace: gatekeeper-system
Events: <none>
Deploymentを作成して確認する。
root@cks-master:~# k create deploy test --image=nginx error: failed to create deployment: admission webhook "validation.gatekeeper.sh" denied the request: [denied by deployment-must-have-min-replicas] you must provide 1 more replicas root@cks-master:~# k create deploy test --image=nginx --replicas=2 deployment.apps/test created
リソースを削除する。
k delete deploy test k delete -f constraint.yaml k delete -f template.yaml
レジストリを制限する例
テンプレートのマニフェストを作成する。
apiVersion: templates.gatekeeper.sh/v1beta1 kind: ConstraintTemplate metadata: name: k8strustedimages spec: crd: spec: names: kind: K8sTrustedImages targets: - target: admission.k8s.gatekeeper.sh rego: | package k8strustedimages violation[{"msg": msg}] { image := input.review.object.spec.containers[_].image not startswith(image, "docker.io/") not startswith(image, "k8s.gcr.io/") msg := "not trusted image!" }
テンプレートを作成する。
k apply -f template.yaml
制約のマニフェストを作成する。
apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sTrustedImages metadata: name: pod-trusted-images spec: match: kinds: - apiGroups: [""] kinds: ["Pod"]
制約を作成する。
k apply -f constraint.yaml
describeコマンドでバイオレーションを確認する。
root@cks-master:~# k describe k8strustedimages pod-trusted-images
Name: pod-trusted-images
（省略）
Status:
Audit Timestamp: 2021-01-01T17:58:35Z
By Pod:
Constraint UID: 779e9983-183c-4861-aecc-ec71daabd81c
Enforced: true
Id: gatekeeper-audit-65f658df68-tfxlf
Observed Generation: 1
Operations:
audit
status
Constraint UID: 779e9983-183c-4861-aecc-ec71daabd81c
Enforced: true
Id: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Observed Generation: 1
Operations:
webhook
Total Violations: 3
Violations:
Enforcement Action: deny
Kind: Pod
Message: not trusted image!
Name: flask-sample
Namespace: default
Enforcement Action: deny
Kind: Pod
Message: not trusted image!
Name: gatekeeper-audit-65f658df68-tfxlf
Namespace: gatekeeper-system
Enforcement Action: deny
Kind: Pod
Message: not trusted image!
Name: gatekeeper-controller-manager-5fb6c9ff69-7wk6d
Namespace: gatekeeper-system
Events: <none>
root@cks-master:~#
Podを作成して確認する。
root@cks-master:~# k run nginx --image=nginx Error from server ([denied by pod-trusted-images] not trusted image!): admission webhook "validation.gatekeeper.sh" denied the request: [denied by pod-trusted-images] not trusted image! root@cks-master:~# k run nginx --image=docker.io/nginx pod/nginx created
リソースを削除する。
k delete pod nginx k delete -f constraint.yaml k delete -f template.yaml
