Skip to content

gator doesn't support rego v1 #3907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SuperSandro2000 opened this issue Apr 10, 2025 · 9 comments
Open

gator doesn't support rego v1 #3907

SuperSandro2000 opened this issue Apr 10, 2025 · 9 comments

Comments

@SuperSandro2000
Copy link

SuperSandro2000 commented Apr 10, 2025

What steps did you take and what happened:
I changed my constraint templates with the following diff, migrated the rego file via opa fmt and expected import rego.v1 to work.

diff --git a/system/gatekeeper/templates/constrainttemplate-restrict-monsoon3-namespace.yaml b/system/gatekeeper/templates/constrainttemplate-restrict-monsoon3-namespace.yaml
index 898469d96c..23bf3e1153 100644
--- a/system/gatekeeper/templates/constrainttemplate-restrict-monsoon3-namespace.yaml
+++ b/system/gatekeeper/templates/constrainttemplate-restrict-monsoon3-namespace.yaml
@@ -20,6 +20,10 @@ spec:
                   type: string
   targets:
     - target: admission.k8s.gatekeeper.sh
+      code:
+      - engine: Rego
+        source:
+          version: v1
           rego: |
             package restrictmonsoon3namespace

When running gator against it, I was greeted with adding template: invalid ConstraintTemplate: invalid import: bad import: "rego.v1".

I build a custom gator binary with the following diff:

diff --git a/vendor/github.com/open-policy-agent/frameworks/constraint/pkg/regorewriter/regorewriter.go b/vendor/github.com/open-policy-agent/frameworks/constraint/pkg/regorewriter/regorewriter.go
index 26ab7f4bc..8590f68c9 100644
--- a/vendor/github.com/open-policy-agent/frameworks/constraint/pkg/regorewriter/regorewriter.go
+++ b/vendor/github.com/open-policy-agent/frameworks/constraint/pkg/regorewriter/regorewriter.go
@@ -311,6 +311,10 @@ func (r *RegoRewriter) checkImport(i *ast.Import) error {
        return nil
    }

+   if importRef.String() == "rego.v1" {
+       return nil
+   }
+
    return fmt.Errorf("%w: bad import: %q", ErrInvalidImport, importRef)
 }

and things started to work.

I think that allowedLibPrefixes is missing rego.v1 and only contains data.lib. I am not sure if that would be the right place, to add it though.

In general I would expect gator to forward the rego version from the spec above and inject it into the parser.

The full diff and code can be found at sapcc/helm-charts#8495 and https://github.com/sapcc/helm-charts/tree/master/system/gatekeeper

What did you expect to happen:
To have rego.v1 support

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

 ➜ gator --version
gator version 3.19.0 (Feature State: beta), opa/v1.3.0, frameworks/v0.0.0-20250328190153-08aa5ffa6033

#3880 (comment)

@JaydipGabani
Copy link
Contributor

@SuperSandro2000 For example the following is a ConstraintTemplate/Constraints that uses rego.V1.

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabelsv1
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabelsV1
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            message:
              type: string
            labels:
              type: array
              items:
                type: object
                properties:
                  key:
                    type: string
                  allowedRegex:
                    type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      code:
        - engine: Rego
          source:
            version: "v1"
            rego: |
              package k8srequiredlabelsv1

              violation contains 
                {"msg": msg, "details": {"missing_labels": missing}} 
                if {

                  # dummy code to use rego v1 keywords
                  test := [1, 2, 1]
                  every j in test {
                    j == 1
                  }

                  provided := {label | input.review.object.metadata.labels[label]}
                  required := {label | label := input.parameters.labels[_]}
                  missing := required - provided
                  count(missing) > 0
                  msg := sprintf("you must provide labels: %v", [missing])
                }
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabelsV1
metadata:
  name: all-must-have-label
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
  parameters:
    message: "All namespaces must have an `owner` label that points to your company username"
    labels:
      - key: owner
        allowedRegex: "^[a-zA-Z]+.agilebank.demo$"

Notice that you do not need to explicitly import rego.v1 to use v1 syntax, setting source.version: v1 is required and enough to indicate GK that constrantTemplate is using v1 rego.

You are seeing bad import error becase we only allow imports from libs defined in ConstraintTemplate and not any external imports are allowed.

@SuperSandro2000
Copy link
Author

Would it be possible to make import rego.v1 a noop when the rego version is set to v1 to make the code more portable?

@SuperSandro2000
Copy link
Author

SuperSandro2000 commented Apr 14, 2025

I have migrated all my constrainttemplates to the new code/engine/source construct but I am unable to use libs like done here https://github.com/open-policy-agent/gatekeeper/blob/master/test/bats/tests/templates/k8scontainterlimits_template.yaml#L22-L133 and always received: template:14: rego_type_error: undefined function data.libs.lib.add_support_labels.from_helm_release

Is that still supported? How do I migrate this? I couldn't find any examples the gatekeeper library is not yet updated to rego v1.

I was only be able to make it function by copying my library functions in the rego multiline string.

I tried some vibe based coding and that also got me nowhere.

@SuperSandro2000
Copy link
Author

My ConstraintTemplate template without any helm templating:

---
# Source: gatekeeper/templates/constrainttemplate-deprecated-api-version.yaml
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: gkdeprecatedapiversion
spec:
  crd:
    spec:
      names:
        kind: GkDeprecatedApiVersion
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            helmManifestParserURL:
              type: string
            kubernetesVersion:
              type: string
            apiVersions:
              type: array
              items:
                type: string

  targets:
    - target: admission.k8s.gatekeeper.sh
      libs:
        - |
          
          package lib.add_support_labels
          
          # `obj` must be a full Kubernetes object.
          from_k8s_object(obj, msg) := result if {
          	support_group := object.get(obj, ["metadata", "labels", "ccloud/support-group"], "none")
          	service := object.get(obj, ["metadata", "labels", "ccloud/service"], "none")
          	result := sprintf("{\"support_group\":%s,\"service\":%s} >> %s", [json.marshal(support_group), json.marshal(service), msg])
          }
          
          # `body` must be the response body from helm-manifest-parser
          from_helm_release(body, msg) := result if {
          	support_group := object.get(body, ["owner_info", "support-group"], "none")
          	service := object.get(body, ["owner_info", "service"], "none")
          	result := sprintf("{\"support_group\":%s,\"service\":%s} >> %s", [json.marshal(support_group), json.marshal(service), msg])
          }
          
          # Adds an additional label to a message that already had support labels added with one of the above methods.
          # For example:
          #
          # ```rego
          # msgWithLabels := add_support_labels.extra("severity", "warning", add_support_labels.from_k8s_object(iro, msg))
          # ```
          #
          # Test coverage for this function is obtained in the policies using it.
          extra(key, value, msg) := result if {
          	result := sprintf("{%s:%s,%s", [json.marshal(key), json.marshal(value), trim_prefix(msg, "{")])
          }
          
        - |
          
          package lib.helm_release
          
          # The public interface is the function parse_k8s_object(obj, baseURL). `obj`
          # must be a full Kubernetes object. A Secret containing a Helm release must be
          # given, otherwise an error will be returned. `baseURL` is where
          # helm-manifest-parser is running. This usually comes from the policy's
          # `input.parameters`.
          #
          # If parsing fails, an object with only the field "error" is returned.
          # This error message must be converted into a violation by the calling policy.
          # (This step is something that we cannot do in a library module.)
          #
          # If parsing succeeds, the parsed response body from helm-manifest-parser is
          # returned. Additionally, the returned object will have the field "error" set
          # to the empty string, in order to simplify error checks in the calling policy.
          
          parse_k8s_object(obj, baseURL) := result if {
          	# NOTE: This branch is defense in depth. Constraints using this function
          	# should already be limited to suitable objects via their selectors.
          	not __is_helm_release(obj)
          	result := {"error": "Input to helm_manifest_parser.parse_release() is not a Helm release. This is an error in the policy implementation."}
          }
          
          parse_k8s_object(obj, baseURL) := result if {
          	# This code is structured to ensure that http.send() is never executed more
          	# than once.
          	__is_helm_release(obj)
          	url := sprintf("%s/v3", [baseURL])
          	resp := http.send({"url": url, "method": "POST", "raise_error": false, "raw_body": obj.data.release, "timeout": "15s"})
          	result := __parse_response(resp)
          }
          
          ################################################################################
          # private helper functions
          
          __is_helm_release(obj) if {
          	obj.kind == "Secret"
          	obj.type == "helm.sh/release.v1"
          }
          
          __is_helm_release(obj) := false if {
          	obj.kind != "Secret"
          }
          
          __is_helm_release(obj) := false if {
          	obj.type != "helm.sh/release.v1"
          }
          
          __parse_response(resp) := result if {
          	resp.status_code == 200
          	result := object.union(resp.body, {"error": ""})
          }
          
          __parse_response(resp) := result if {
          	resp.status_code != 200
          	object.get(resp, ["error", "message"], "") == ""
          	result := {"error": sprintf("helm-manifest-parser returned HTTP status %d, but we expected 200. Please retry in ~5 minutes.", [resp.status_code])}
          }
          
          __parse_response(resp) := result if {
          	resp.status_code != 200
          	msg := object.get(resp, ["error", "message"], "")
          	msg != ""
          	result := {"error": sprintf("Could not reach helm-manifest-parser (%q). Please retry in ~5 minutes.", [msg])}
          }
          
      code:
        - engine: Rego
          source:
            version: "v1"
            rego: |
              package deprecatedapiversion

              import data.lib.add_support_labels
              import data.lib.helm_release

              iro := input.review.object

              release := helm_release.parse_k8s_object(iro, input.parameters.helmManifestParserURL)

              violation contains {"msg": release.error} if {
                  release.error != ""
              }

              violation contains {"msg": add_support_labels.from_helm_release(release, msg)} if {
                  release.error == ""

                  # find objects in the manifest that use deprecated API versions
                  obj := release.items[_]
                  input.parameters.apiVersions[_] == obj.apiVersion

                  msg := sprintf(
                      "%s %s declared with deprecated API version: %s (will break in k8s v%s)",
                      [obj.kind, obj.metadata.name, obj.apiVersion, input.parameters.kubernetesVersion],
                  )
              }

When running gator I receive:

=== RUN   deprecated-api-version-k8s1.32
--- FAIL: deprecated-api-version-k8s1.32        (0.005s)
  adding template: unable to compile modules: 2 errors occurred:
template:8: rego_type_error: undefined function data.libs.lib.helm_release.parse_k8s_object
template:14: rego_type_error: undefined function data.libs.lib.add_support_labels.from_helm_release

When I remove libs things work as expected:

=== RUN   deprecated-api-version-k8s1.32
    === RUN   helm-release-deprecated-api-version-1.32
    --- PASS: helm-release-deprecated-api-version-1.32  (0.030s)
    === RUN   helm-release-clean
    --- PASS: helm-release-clean        (0.011s)
---
# Source: gatekeeper/templates/constrainttemplate-deprecated-api-version.yaml
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: gkdeprecatedapiversion
spec:
  crd:
    spec:
      names:
        kind: GkDeprecatedApiVersion
      validation:
        # Schema for the `parameters` field
        openAPIV3Schema:
          type: object
          properties:
            helmManifestParserURL:
              type: string
            kubernetesVersion:
              type: string
            apiVersions:
              type: array
              items:
                type: string

  targets:
    - target: admission.k8s.gatekeeper.sh
      code:
        - engine: Rego
          source:
            version: "v1"
            rego: |
              package deprecatedapiversion

              # `obj` must be a full Kubernetes object.
              from_k8s_object(obj, msg) := result if {
                support_group := object.get(obj, ["metadata", "labels", "ccloud/support-group"], "none")
                service := object.get(obj, ["metadata", "labels", "ccloud/service"], "none")
                result := sprintf("{\"support_group\":%s,\"service\":%s} >> %s", [json.marshal(support_group), json.marshal(service), msg])
              }

              # `body` must be the response body from helm-manifest-parser
              from_helm_release(body, msg) := result if {
                support_group := object.get(body, ["owner_info", "support-group"], "none")
                service := object.get(body, ["owner_info", "service"], "none")
                result := sprintf("{\"support_group\":%s,\"service\":%s} >> %s", [json.marshal(support_group), json.marshal(service), msg])
              }

              # Adds an additional label to a message that already had support labels added with one of the above methods.
              # For example:
              #
              # ```rego
              # msgWithLabels := add_support_labels.extra("severity", "warning", add_support_labels.from_k8s_object(iro, msg))
              # ```
              #
              # Test coverage for this function is obtained in the policies using it.
              extra(key, value, msg) := result if {
                result := sprintf("{%s:%s,%s", [json.marshal(key), json.marshal(value), trim_prefix(msg, "{")])
              }
              # The public interface is the function parse_k8s_object(obj, baseURL). `obj`
              # must be a full Kubernetes object. A Secret containing a Helm release must be
              # given, otherwise an error will be returned. `baseURL` is where
              # helm-manifest-parser is running. This usually comes from the policy's
              # `input.parameters`.
              #
              # If parsing fails, an object with only the field "error" is returned.
              # This error message must be converted into a violation by the calling policy.
              # (This step is something that we cannot do in a library module.)
              #
              # If parsing succeeds, the parsed response body from helm-manifest-parser is
              # returned. Additionally, the returned object will have the field "error" set
              # to the empty string, in order to simplify error checks in the calling policy.

              parse_k8s_object(obj, baseURL) := result if {
                # NOTE: This branch is defense in depth. Constraints using this function
                # should already be limited to suitable objects via their selectors.
                not __is_helm_release(obj)
                result := {"error": "Input to helm_manifest_parser.parse_release() is not a Helm release. This is an error in the policy implementation."}
              }

              parse_k8s_object(obj, baseURL) := result if {
                # This code is structured to ensure that http.send() is never executed more
                # than once.
                __is_helm_release(obj)
                url := sprintf("%s/v3", [baseURL])
                resp := http.send({"url": url, "method": "POST", "raise_error": false, "raw_body": obj.data.release, "timeout": "15s"})
                result := __parse_response(resp)
              }

              ################################################################################
              # private helper functions

              __is_helm_release(obj) if {
                obj.kind == "Secret"
                obj.type == "helm.sh/release.v1"
              }

              __is_helm_release(obj) := false if {
                obj.kind != "Secret"
              }

              __is_helm_release(obj) := false if {
                obj.type != "helm.sh/release.v1"
              }

              __parse_response(resp) := result if {
                resp.status_code == 200
                result := object.union(resp.body, {"error": ""})
              }

              __parse_response(resp) := result if {
                resp.status_code != 200
                object.get(resp, ["error", "message"], "") == ""
                result := {"error": sprintf("helm-manifest-parser returned HTTP status %d, but we expected 200. Please retry in ~5 minutes.", [resp.status_code])}
              }

              __parse_response(resp) := result if {
                resp.status_code != 200
                msg := object.get(resp, ["error", "message"], "")
                msg != ""
                result := {"error": sprintf("Could not reach helm-manifest-parser (%q). Please retry in ~5 minutes.", [msg])}
              }

              iro := input.review.object

              release := parse_k8s_object(iro, input.parameters.helmManifestParserURL)

              violation contains {"msg": release.error} if {
                  release.error != ""
              }

              violation contains {"msg": from_helm_release(release, msg)} if {
                  release.error == ""

                  # find objects in the manifest that use deprecated API versions
                  obj := release.items[_]
                  input.parameters.apiVersions[_] == obj.apiVersion

                  msg := sprintf(
                      "%s %s declared with deprecated API version: %s (will break in k8s v%s)",
                      [obj.kind, obj.metadata.name, obj.apiVersion, input.parameters.kubernetesVersion],
                  )
              }

@JaydipGabani
Copy link
Contributor

@SuperSandro2000 You need to migrate libs to code/engine/source construct as well. It should be a sibling field of rego and that should work.

Would it be possible to make import rego.v1 a noop when the rego version is set to v1 to make the code more portable?

The correct solution would be to allow rego.v1 import rather then keeping it as noop. Can you share how does it make the code more portable?

@SuperSandro2000
Copy link
Author

@SuperSandro2000 You need to migrate libs to code/engine/source construct as well. It should be a sibling field of rego and that should work.

I haven't seen that as an example or in any migration guide, so I just didn't do it 😅 Sadly the yaml has no strict validation, so any field is accepted, even banana.

The correct solution would be to allow rego.v1 import rather then keeping it as noop. Can you share how does it make the code more portable?

Right now if I copy the plain rego code to anywhere else it is not assumed to be rego v1 but rego v0 and that doesn't work.

@JaydipGabani
Copy link
Contributor

JaydipGabani commented Apr 14, 2025

I haven't seen that as an example or in any migration guide, so I just didn't do it 😅 Sadly the yaml has no strict validation, so any field is accepted, even banana.

Do you want to contribute to our docs and lay out a guide?

Right now if I copy the plain rego code to anywhere else it is not assumed to be rego v1 but rego v0 and that doesn't work.

This I think could be resolved with some automation or script. I am not sure if we should allow imports for this reason. We chose not to allow any imports to make sure all rego in CT remains localized. I am curious what others think @ritazh @sozercan @maxsmythe

@JaydipGabani JaydipGabani removed the bug Something isn't working label Apr 15, 2025
@SuperSandro2000
Copy link
Author

My main source of confussion arose from the docs linked at https://github.com/open-policy-agent/gatekeeper/releases/tag/v3.19.0

They mention things like the rego.v1 import which is just wrong for gatekeeper.

Do you want to contribute to our docs and lay out a guide?

I don't have dyslexia per say but the docs I usually write look more like someone with slight dyslexia wrote it. I don't think that is a good use of our time.

@ritazh
Copy link
Member

ritazh commented Apr 28, 2025

Thanks for the feedback @SuperSandro2000 I have updated the v3.19.0's release notes to point to https://open-policy-agent.github.io/gatekeeper/website/docs/constrainttemplates/#enable-opa-rego-v1-syntax-in-constrainttemplates which includes migration steps and examples.

Specifically this:

Rego v1 syntax can only be used under targets[].code[].[engine: Rego].source with version: "v1". No need to add import rego.v1 to use rego v1 syntax.

re: data.ib, we have added more docs for builtin variables https://open-policy-agent.github.io/gatekeeper/website/docs/constrainttemplates/#rego-variables specifically this points to gatekeeper-library examples that are using data.lib

Please try these and let us know if things are still not working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants