Seccompに入門したメモ。
SeccompはDockerではデフォルトで有効でDocker用の制限されたプロファイルがある。Kubernetesではデフォルトでは有効ではなく、指定する必要がある。ランタイムのデフォルトを参照することは可能。
参考リンク
- Restrict a Container's Syscalls with seccomp | Kubernetes
- Seccomp security profiles for Docker | Docker Documentation
- https://github.com/killer-sh/cks-course-environment/tree/master/course-content/system-hardening/kernel-hardening-tools/seccomp
Docker
この内容をコピーしてdefault.jsonファイルを作成する。
プロファイルを指定しないで起動する。
root@cks-worker:~# docker run --rm nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up
プロファイルを指定して起動する。デフォルトのプロファイルなので特に変わらない。
root@cks-worker:~# docker run --rm --security-opt seccomp=default.json nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up
プロファイルを変更する。default.jsonファイルからwriteシステムコールを探して消す。
再び起動するとエラーになることが確認できる。
root@cks-worker:~# docker run --rm --security-opt seccomp=default.json nginx docker: Error response from daemon: OCI runtime start failed: cannot start an already running container: unknown. ERRO[0000] error waiting for container: context canceled
Kubernetes
KubeletがSeccompプロファイルを読めるようにKubeletの起動引数でディレクトリを指定してあげるか、デフォルトのディレクトリである/var/lib/kubelet/seccompにプロファイルを置いてあげる必要がある。
root@cks-worker:~# mkdir /var/lib/kubelet/seccomp root@cks-worker:~# mv default.json /var/lib/kubelet/seccomp/
先ず存在しないプロファイルを指定してみて試す。
apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: secure name: secure spec: securityContext: seccompProfile: type: Localhost localhostProfile: profiles/audit.json containers: - image: nginx name: secure resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {}
エラーになる。
root@cks-master:~# k apply -f secure.yaml
pod/secure created
root@cks-master:~# k get pod
NAME READY STATUS RESTARTS AGE
secure 0/1 CreateContainerError 0 9s
root@cks-master:~# k describe pod secure
Name: secure
Namespace: default
Priority: 0
Node: cks-worker/10.146.0.7
Start Time: Mon, 04 Jan 2021 15:20:33 +0000
Labels: run=secure
Annotations: seccomp.security.alpha.kubernetes.io/pod: localhost/profiles/audit.json
Status: Pending
IP: 10.44.0.1
IPs:
IP: 10.44.0.1
Containers:
secure:
Container ID:
Image: nginx
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: CreateContainerError
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-r2nz7 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-r2nz7:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-r2nz7
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 18s default-scheduler Successfully assigned default/secure to cks-worker
Normal Pulled 14s kubelet Successfully pulled image "nginx" in 3.215115053s
Normal Pulling 13s (x2 over 17s) kubelet Pulling image "nginx"
Warning Failed 10s (x2 over 14s) kubelet Error: failed to generate security options for container "secure": failed to generate seccomp security options for container: cannot load seccomp profile "/var/lib/kubelet/seccomp/profiles/audit.json": open /var/lib/kubelet/seccomp/profiles/audit.json: no such file or directory
Normal Pulled 10s kubelet Successfully pulled image "nginx" in 3.27904469s
root@cks-master:~# k delete pod secure
pod "secure" deleted
マニフェストを修正して存在するプロファイルを指定する。
apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: secure name: secure spec: securityContext: seccompProfile: type: Localhost localhostProfile: default.json containers: - image: nginx name: secure resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {}
再び試すと先ほどとは違うエラーになる。Dockerで試したときと同じエラー。
root@cks-master:~# k apply -f secure.yaml
pod/secure created
root@cks-master:~# k get pod
NAME READY STATUS RESTARTS AGE
secure 0/1 RunContainerError 0 13s
root@cks-master:~# k describe pod secure
Name: secure
Namespace: default
Priority: 0
Node: cks-worker/10.146.0.7
Start Time: Mon, 04 Jan 2021 15:24:36 +0000
Labels: run=secure
Annotations: seccomp.security.alpha.kubernetes.io/pod: localhost/default.json
Status: Running
IP: 10.44.0.1
IPs:
IP: 10.44.0.1
Containers:
secure:
Container ID: docker://e83c1adb518c82a829376551cdccdc35fc4b95f6406026e70e4681c9b8c55498
Image: nginx
Image ID: docker-pullable://nginx@sha256:4cf620a5c81390ee209398ecc18e5fb9dd0f5155cd82adcbae532fec94006fb9
Port: <none>
Host Port: <none>
State: Waiting
Reason: RunContainerError
Last State: Terminated
Reason: ContainerCannotRun
Message: OCI runtime start failed: cannot start an already running container: unknown
Exit Code: 128
Started: Mon, 04 Jan 2021 15:24:40 +0000
Finished: Mon, 04 Jan 2021 15:24:40 +0000
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-r2nz7 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-r2nz7:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-r2nz7
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 19s default-scheduler Successfully assigned default/secure to cks-worker
Normal Pulled 15s kubelet Successfully pulled image "nginx" in 2.879879032s
Normal Pulling 13s (x2 over 18s) kubelet Pulling image "nginx"
Normal Created 10s (x2 over 15s) kubelet Created container secure
Warning Failed 10s (x2 over 14s) kubelet Error: failed to start container "secure": Error response from daemon: OCI runtime start failed: cannot start an already running container: unknown
Normal Pulled 10s kubelet Successfully pulled image "nginx" in 2.801219695s
root@cks-master:~# k delete pod secure
pod "secure" deleted
Workerノードで先ほど消したwriteを戻す。
再び試すと起動する。
root@cks-master:~# k apply -f secure.yaml pod/secure created root@cks-master:~# k get pod NAME READY STATUS RESTARTS AGE secure 1/1 Running 0 9s root@cks-master:~#
