Creating Policies for Centralized Policy Management

Required access rights: Platform Administrator or Cluster Administrator

In HPE Ezmeral Runtime Enterprise Centralized Policy Management, policies are expressed as a directory of YAML files in a Git repository. Each YAML file contains one or more pairs of OPA Gatekeeper constraint and template objects.

Rego Policy Language

To write OPA Gatekeeper template objects, you need to learn Rego. Rego is the policy language for OPA Gatekeeper. For more information about Rego and working with policies and constraints, see these resources:

Organizing Template and Constraint Objects

You can organize pairs of template and constraint objects in two ways:
  • Combine multiple template and constraint objects into one YAML file. This “one big YAML file” becomes a collection of policies – or one big policy – that includes pairs of templates and constraints. See this example (onebigpolicy.yaml).
  • Create a directory of policies with each policy represented as a single YAML file that contains a pair of constraint and template objects. See this example directory.

Example Policy

The following example policy (allowedrepo-policy.yaml) validates all pods in the cluster and ensures that they come from the openpolicyagent repo.

In this example, the constraint object appears first, followed by the template object. The template object contains logic for how the policy should be validated. In the template object, lines of code in bold face indicate Rego commands. The constraint object contains the values that the template will validate against.

If a pod that is not from the openpolicyagent repo is detected, an error is generated. 
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRepos
metadata:
  name: repo-is-openpolicyagent
  annotations:
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  match:
    kinds:
    - apiGroups: [""]
      kinds: ["Pod"]
    namespaces:
    - "default"
  parameters:
    repos:
    - "openpolicyagent"
---
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: k8sallowedrepos
  annotations:
    description: Requires container images to begin with a repo string from a specified
      list.
spec:
  crd:
    spec:
      names:
        kind: K8sAllowedRepos
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          properties:
            repos:
              type: array
              items:
                type: string
  targets:
  - target: admission.k8s.gatekeeper.sh
    rego: |
      package k8sallowedrepos
      violation[{"msg": msg}] {
        container := input.review.object.spec.containers[_]
        satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
        not any(satisfied)
        msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
      }
      violation[{"msg": msg}] {
        container := input.review.object.spec.initContainers[_]
        satisfied := [good | repo = input.parameters.repos[_] ; good = startswith(container.image, repo)]
        not any(satisfied)
        msg := sprintf("container <%v> has an invalid image repo <%v>, allowed repos are %v", [container.name, container.image, input.parameters.repos])
      }

Required Annotation for All OPA Gatekeeper Constraint Objects

As shown in the constraint section of the example, all OPA Gatekeeper constraint objects must include the following annotation:
argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true

Policy Enforcement Example

After your policies are created and applied to a cluster, you can observe the enforcement of them when operations violate a policy. The following example shows the effect of applying an object that violates multiple policies configured for a cluster:
# kubectl apply -f disallowedcontainerlimit.yaml
Error from server ([denied by container-image-must-have-digest] container <opa> uses an image with a digest <openpolicyagent/opa:0.9.2>
[denied by container-must-have-limits] container <opa> memory limit <2Gi> is higher than the maximum allowed of <1Gi>
[denied by must-have-probes] Container <opa> in your <Pod> <opa-disallowed> has no <readinessProbe>
[denied by must-have-probes] Container ,opa> in your <Pod> <opa-disallowed> has no <livenessProbe>): error when creating "disallowedcontainerlimit.yaml": admission
 webhook "validation.gatekeeper.sh" denied the request: <denied by container-image-must-have-digest] container <opa> uses an image without a digest <openpolicyagent/opa:0.9.2>
[denied by container-must-have-limits] container <opa> memory limit <2Gi> is higher than the maximum allowed of <1Gi>
[denied by must-have-probes] Container <opa> in your <Pod> <opa-disallowed> has no <readinessProbe>
[denied by must-have-probes] Container <opa> in your <Pod> <opa-disallowed> has no <livenessProbe>

Git Repository Containing Example Policies (for HPE-Internal Personnel)

The following HPE Git repository contains policy examples tested with HPE Ezmeral Runtime Enterprise:

https://github.hpe.com/hpe/opa-gatekeeper-policies

Overly Restrictive Policies

As with any security system, it is possible to create policies that interfere with normal system operations and that result in unwanted behavior. For example, when creating policies for HPE Ezmeral Runtime Enterprise, setting the root file system directory to “read only” results in numerous errors, because fsmount daemonset pods must have write access to the /opt/bluedata/share directory on all of the Kubernetes hosts and the /opt/bluedata/share directory inside the pod. One such error is the failure to configure a Kubernetes Web Terminal.

To display a JSON-formated list of policy violations that are occurring in a cluster, enter the following command:

kubectl get constraints -o json