Developing Kyverno-Envoy-Plugin: My LFX Mentee Experience

This spring, I had the privilege of serving as an LFX mentee with the Kyverno organization.
Kyverno is a policy engine designed for Kubernetes. Kyverno is a CNCF incubating project. You can read more about Kyverno at https://kyverno.io/.

Acknowledgment

I am deeply thankful to my great mentors, Charles-Edouard Brétéché, Senior Software Engineer at Nirmata, and Anushka Mittal, Software Engineer at Nirmata, for their dedicated guidance and steadfast support during the mentorship program. Their expertise, patience, and encouragement have been instrumental in my professional growth, and their insights have greatly enriched my knowledge and skills in the field.

Let's Get Started

I am Sanskar Gurdasani, a final-year B.Tech student in Information Technology Engineering at Jabalpur Engineering College. As an active contributor and member of Kyverno, I am passionate about open-source development. I have previously worked on various open-source projects, including Kubernetes, Kubescape, and Knative, and I continue to contribute to these initiatives.

What is LFX Mentorship?

The LFX mentorship is a remote learning opportunity designed for open-source contributors to work for 12 weeks under the guidance of experienced mentors, who are maintainers and developers of specific projects. These mentors help mentees contribute effectively to the community and the project. Also, how can I miss the reward. Every mentee gets stipend💲 according to the country region (ft. Purchase Power Parity) which compensates for all the effort and hard work you've put in. So believe in yourself and give your best.

Where to look?

This mentorship program is organized three times a year: Term 1 (March to May), Term 2 (June to August), and Term 3 (September to November). The CNCF maintains a comprehensive repository with all the necessary information, including participating projects and the required skills. For the entire list of projects, visit the LFX official website because it also has Hyperledger, Linux kernel Projects, and a lot more.

My acceptance into the program

The mentorship page provides a comprehensive list of all projects and their associated organizations. I was particularly drawn to Kyverno, as it specializes in Kubernetes Security and Compliance. The technical skills required for this project closely align with my previous experience which was in programming in golang and knowledge of Kubernetes, making it an ideal fit for my background and interests.

The application process is straightforward: first, you need to create an account on the mentorship page. This involves providing information about yourself, detailing your skill set, and optionally, your demographics.

Once the application process began, I drafted a cover letter, acknowledging some of the required questions included How did you find out about our mentorship program? Why are you interested in this program? What experience and knowledge/skills do you have that apply to this program? What do you hope to get out of this mentorship experience?

I submitted two documents to the LFX Mentorship platform: my resume and a cover letter. I ensured to highlight my technical competency with Go and Kubernetes, as well as my extensive experience with these technologies. Additionally, I emphasized my passion for open source development and my previous contributions in this area.

Finally, I was selected for the project CNCF - Kyverno: Kyverno for Envoy Authorization (2024 Term 1)

Let's discuss the project!

The official LFX project link goes below:

https://mentorship.lfx.linuxfoundation.org/project/5da34595-8dd2-4045-a77d-e86d4c64fbc3

Also, you can check out the issue in the project repository.

After going through the above links, you might know what I have worked on! If not, then let me explain you in brief.

Problem Statement

Kyverno and the Kyverno-JSON project can be used to perform authorization, In the same spirit as the envoy opa plugin we should provide an envoy Kyverno plugin. In a certain sense, authorization can be defined using policies. This will allow users to perform authorization with similar concepts as Kyverno and Kyverno-JSON.

The Solution

The project's goal is to allow the envoy to make a decision based on Kyverno policies. To achieve this the plugin leverages Envoy's External Authorization filter feature, the plugin intercepts incoming requests and evaluates them against Kyverno policies.
Envoy is a Layer 7 proxy and communication bus tailored for large-scale, modern service-oriented architectures. Starting for version 1.7.0, Envoy includes an External Authorizaition filter that interfaces with an authorization service to determine the legitimacy of incoming request.

This functionality allows authorization decisions to be offloaded to an external service, which can access the request context. The request context includes details such as the origin and destination of the network activity, as well as specifics of the network request (e.g., HTTP request). This information enables the external service to make a well-informed decision regarding the authorization of the incoming request processed by Envoy.

What is Kyverno-Envoy-Plugin?

Kyverno-envoy plugin extends Kyverno-json with a gRPC server that implements Envoy External Authorization API. This allows you to enforce Kyverno policies on incoming and outgoing traffic in a service mesh environment, providing an additional layer of security and control over your applications. You can use this version of Kyverno to enforce fine-grained, context-aware access control policies with Envoy without modifying your microservice.
Its works basically addition to Envoy sidecar, you application pods will include a Kyverno-Envoy component, either as a sidecar or as a separate pod. This kyverno-envoy will be configured to communicate with the Kyverno-envoy-plugin gRPC server. When Envoy receives an API request intended for your microservice, it consults the Kyverno-envoy-plugin server to determine whether the request should be permitted.

Here is an example of deployment of kyverno-envoy-plugin in your kubernetes Deployments :

- name: kyverno-envoy-plugin
  image: sanskardevops/plugin:0.0.34
  imagePullPolicy: IfNotPresent
  ports:
    - containerPort: 8181
    - containerPort: 9000
  volumeMounts:
    - readOnly: true
  args:
    - "serve"
    - "--policy=/policies/policy.yaml"
    - "--address=:9000"
    - "--healthaddress=:8181"
  livenessProbe:
    httpGet:
      path: /health
      scheme: HTTP
      port: 8181
    initialDelaySeconds: 5
    periodSeconds: 5
  readinessProbe:
    httpGet:
      path: /health
      scheme: HTTP
      port: 8181
    initialDelaySeconds: 5
    periodSeconds: 5

Here is the policy example to check the conditions of the incoming request and denies the request if the user is a guest and the request method is POST at the /book path.

apiVersion: json.kyverno.io/v1alpha1
kind: ValidatingPolicy
metadata:
    name: checkrequest
spec:
    rules:
    - name: deny-guest-request-at-post
        assert:
        any:
        - message: "POST method calls at path /book are not allowed to guests users"
            check:
            request:
                http:
                    method: POST
                    headers:
                        authorization:
                            (split(@, ' ')[1]):
                                (jwt_decode(@ , 'secret').payload.role): admin
                    path: /book                             
        - message: "GET method call is allowed to both guest and admin users"
            check:
            request:
                http:
                    method: GET
                    headers:
                        authorization:
                            (split(@, ' ')[1]):
                                (jwt_decode(@ , 'secret').payload.role): admin
                    path: /book 
        - message: "GET method call is allowed to both guest and admin users"
            check:
            request:
                http:
                    method: GET
                    headers:
                        authorization:
                            (split(@, ' ')[1]):
                                (jwt_decode(@ , 'secret').payload.role): guest
                    path: /book

For the demos please visit the website kyverno-envoy-plugin

From this project, I have learned the following things :

  • Defined the Plugin Architecture: I documented the overall design of the Kyverno-Envoy plugin, including its installation location and workflow.

  • Implemented External Authorization gRPC Server: I built a GRPC server that handlesservice.auth.v3.CheckRequest messages sent by Envoy's Authorization filter. Integrated Kyverno-JSON Engine: I successfully integrated the Kyverno-JSON engine within the GRPC server .

  • Enhanced Policy Functionality: I implemented a jwtdecode function which will be called within the policies.

  • Developed Kyverno-Envoy-Plugin Demo: To showcase the plugin's functionality, I created a demonstration that guides users on installing and using the Kyverno-Envoy-plugin alongside Istio.

  • Admission Controller Integration: I'm currently working on implementing an admission controller that injects the Kyverno-Envoy plugin as a sidecar container during the upstream pod's applying stage.

  • Unit Testing and Scenario Integration: I'll be integrating unit tests and Testing the Kyverno-Envoy plugin in various scenarios. This will ensure the plugin's functionality and reliability across different use cases.

  • Documentation and Policy Writing Guide: I'll write documentation for the Kyverno-Envoy plugin and create a comprehensive guide on writing Kyverno policies specifically for use with the plugin and also expand different demos.

  • Also added Istio and other service mesh demos

Graduation & Concluding it all

Wow!! Eventually, after 12 weeks, the time flies. I didn’t want this program to end. But every good thing comes to an end. I successfully graduated from the program

And that’s a wrap, what an amazing journey it has been, thank you so much for continuing to read this long on my journey as an LFX mentee for CNCF: Kyverno

I would like to express my gratitude to my mentor Anushka Mittal and Charles-Edouard Brétéché being such amazing support throughout the project. Heartfelt thanks to the Kubernetes, LFX, and CNCF community as well without whom neither this project nor this program could have been possible.

The link to the completed project is here and if you have any feedback or questions, feel free to reach out to me on LinkedIn or Twitter and I will be glad to talk and help :)

0
Subscribe to my newsletter

Read articles from Sanskar Gurdasani directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Sanskar Gurdasani
Sanskar Gurdasani