kubeauditを試す

kubeauditを試してみたメモ。

• https://github.com/Shopify/kubeaudit

導入

brew install kubeaudit

実行方法

3つのモードがある。

1. マニフェストモード
2. ローカルモード
3. クラスターモード

マニフェストモード

適当なマニフェストを作る。

k create deploy nginx --image=nginx --dry-run=client -o yaml > nginx.yaml

検査を実行する。

$ kubeaudit all -f nginx.yaml

DEPRECATION NOTICE: The 'mountds' command is deprecated and will stop working in a future minor release. Please use the 'mounts' command instead. If you use 'all' no change is required.


---------------- Results for ---------------

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx

--------------------------------------------

-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/nginx' should be added.
   Metadata:
      Container: nginx
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/nginx

-- [error] AutomountServiceAccountTokenTrueAndDefaultSA
   Message: Default service account with token mounted. automountServiceAccountToken should be set to 'false' on either the ServiceAccount or on the PodSpec or a non-default service account should be used.

-- [error] CapabilityOrSecurityContextMissing
   Message: Security Context not set. The Security Context should be specified and all Capabilities should be dropped by setting the Drop list to ALL.
   Metadata:
      Container: nginx

-- [warning] ImageTagMissing
   Message: Image tag is missing.
   Metadata:
      Container: nginx

-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: nginx

-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: nginx

-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: nginx

-- [warning] PrivilegedNil
   Message: privileged is not set in container SecurityContext. Privileged defaults to 'false' but it should be explicitly set to 'false'.
   Metadata:
      Container: nginx

-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: nginx

-- [error] SeccompAnnotationMissing
   Message: Seccomp annotation is missing. The annotation seccomp.security.alpha.kubernetes.io/pod: runtime/default should be added.
   Metadata:
      MissingAnnotation: seccomp.security.alpha.kubernetes.io/pod

上記ではallを指定しているが、検査している対象は以下のコマンドのヘルプで確認できし、ドキュメントでも記載されている。

$ kubeaudit
Kubeaudit audits Kubernetes clusters for common security controls.

kubeaudit has three modes:
  1. Manifest mode: If a Kubernetes manifest file is provided using the -f/--manifest flag, kubeaudit will audit the manifest file. Kubeaudit also supports autofixing in manifest mode using the 'autofix' command. This will fix the manifest in-place. The fixed manifest can be written to a different file using the -o/--out flag.
  2. Cluster mode: If kubeaudit detects it is running in a cluster, it will audit the other resources in the cluster.
  3. Local mode: kubeaudit will try to connect to a cluster using the local kubeconfig file ($HOME/.kube/config). A different kubeconfig location can be specified using the -c/--kubeconfig flag

Usage:
  kubeaudit [command]

Available Commands:
  all          Run all audits
  apparmor     Audit containers running without AppArmor
  asat         Audit pods using an automatically mounted default service account
  autofix      Automagically make a manifest secure
  capabilities Audit containers not dropping ALL capabilities
  help         Help about any command
  hostns       Audit pods with hostNetwork, hostIPC or hostPID enabled
  image        Audit containers not using a specified image:tag
  limits       Audit containers exceeding a specified CPU or memory limit
  mountds      Audit containers that mount /var/run/docker.sock
  mounts       Audit containers that mount sensitive paths
  netpols      Audit namespaces that do not have a default deny network policy
  nonroot      Audit containers allowing for root user
  privesc      Audit containers that allow privilege escalation
  privileged   Audit containers running as privileged
  rootfs       Audit containers not using a read only root filesystems
  seccomp      Audit containers running without Seccomp
  version      Prints the current kubeaudit version

Flags:
  -e, --exitcode int         Exit code to use if there are results with severity of "error". Conventionally, 0 is used for success and all non-zero codes for an error. (default 2)
  -p, --format string        The output format to use (one of "pretty", "logrus", "json") (default "pretty")
  -h, --help                 help for kubeaudit
  -c, --kubeconfig string    Path to local Kubernetes config file. Only used in local mode (default is $HOME/.kube/config)
  -f, --manifest string      Path to the yaml configuration to audit. Only used in manifest mode.
  -m, --minseverity string   Set the lowest severity level to report (one of "error", "warning", "info") (default "info")
  -n, --namespace string     Only audit resources in the specified namespace. Not currently supported in manifest mode.

Use "kubeaudit [command] --help" for more information about a command.

自動的に修正する機能もある。

kubeaudit autofix -f nginx.yaml -o nginx-fixed.yaml

修正後のファイルは以下のようになる。limitsとかは設定されていない。

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
      annotations:
        container.apparmor.security.beta.kubernetes.io/nginx: runtime/default
        seccomp.security.alpha.kubernetes.io/pod: runtime/default
    spec:
      containers:
        - image: nginx
          name: nginx
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            privileged: false
            readOnlyRootFilesystem: true
            runAsNonRoot: true
      automountServiceAccountToken: false
status: {}

クラスターモード

クラスターモードといっているのは、クラスター内でコンテナとして実行すること。実行するコマンドはローカルモードと同じ。

ローカルモード

パスを指定する場合は-c "/path/to/config"で指定する。

$ kubeaudit all

DEPRECATION NOTICE: The 'mountds' command is deprecated and will stop working in a future minor release. Please use the 'mounts' command instead. If you use 'all' no change is required.


---------------- Results for ---------------

  apiVersion: apps/v1
  kind: Daemonset
  metadata:
    name: aws-node
    namespace: kube-system

--------------------------------------------

-- [error] AppArmorAnnotationMissing
   Message: AppArmor annotation missing. The annotation 'container.apparmor.security.beta.kubernetes.io/aws-node' should be added.
   Metadata:
      Container: aws-node
      MissingAnnotation: container.apparmor.security.beta.kubernetes.io/aws-node

-- [error] CapabilityShouldDropAll
   Message: Capability Drop list should be set to ALL. Add the specific ones you need to the Add list and set an override label.
   Metadata:
      Container: aws-node

-- [error] CapabilityAdded
   Message: Capability "NET_ADMIN" added. It should be removed from the capability add list. If you need this capability, add an override label such as 'container.audit.kubernetes.io/aws-node.allow-capability-net-admin: SomeReason'.
   Metadata:
      Container: aws-node
      Metadata: NET_ADMIN

-- [error] NamespaceHostNetworkTrue
   Message: hostNetwork is set to 'true' in PodSpec. It should be set to 'false'.

-- [warning] LimitsNotSet
   Message: Resource limits not set.
   Metadata:
      Container: aws-node

-- [error] RunAsNonRootPSCNilCSCNil
   Message: runAsNonRoot should be set to true or runAsUser should be set to a value > 0 either in the container SecurityContext or PodSecurityContext.
   Metadata:
      Container: aws-node

-- [error] AllowPrivilegeEscalationNil
   Message: allowPrivilegeEscalation not set which allows privilege escalation. It should be set to 'false'.
   Metadata:
      Container: aws-node

-- [warning] PrivilegedNil
   Message: privileged is not set in container SecurityContext. Privileged defaults to 'false' but it should be explicitly set to 'false'.
   Metadata:
      Container: aws-node

-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: aws-node

-- [error] SeccompAnnotationMissing
   Message: Seccomp annotation is missing. The annotation seccomp.security.alpha.kubernetes.io/pod: runtime/default should be added.
   Metadata:
      MissingAnnotation: seccomp.security.alpha.kubernetes.io/pod


---------------- Results for ---------------

  apiVersion: apps/v1
  kind: Daemonset
  metadata:
    name: kube-proxy
    namespace: kube-system

--------------------------------------------

（省略）

allとautofixサブコマンドでは、何を検査するかを設定ファイルにして渡すこともできる。

enabledAuditors:
  # Auditors are enabled by default if they are not explicitly set to "false"
  apparmor: false
  asat: false
  capabilities: false
  hostns: false
  image: false
  limits: false
  mounts: false
  netpols: false
  nonroot: false
  privesc: false
  privileged: false
  rootfs: true
  seccomp: true

$ kubeaudit all -k kubeaudit-config.yml -f nginx.yaml

DEPRECATION NOTICE: The 'mountds' command is deprecated and will stop working in a future minor release. Please use the 'mounts' command instead. If you use 'all' no change is required.


---------------- Results for ---------------

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx

--------------------------------------------

-- [error] ReadOnlyRootFilesystemNil
   Message: readOnlyRootFilesystem is not set in container SecurityContext. It should be set to 'true'.
   Metadata:
      Container: nginx

-- [error] SeccompAnnotationMissing
   Message: Seccomp annotation is missing. The annotation seccomp.security.alpha.kubernetes.io/pod: runtime/default should be added.
   Metadata:
      MissingAnnotation: seccomp.security.alpha.kubernetes.io/pod