Skip to main content

Command Palette

Search for a command to run...

Day 29 - Kubernetes Custom Resources

Updated
โ€ข3 min read

Kubernetes normally supports built-in resources like:

  • Deployment

  • Service

  • Pod

  • ConfigMap

  • Secret

  • Ingress

These are called native resources.

Sometimes companies (Istio, ArgoCD, Prometheus Operator, Kyverno, etc.) want to add new features that Kubernetes does not support by default.

To do this, Kubernetes allows you to:

๐Ÿ‘‰ Extend the Kubernetes API
This extension is done using:

  1. CRD โ€” Custom Resource Definition

  2. CR โ€” Custom Resource

  3. Custom Controller


๐ŸŸฆ 1. CRD โ€” Custom Resource Definition

CRD = Definition / Schema of a new Kubernetes API.

It tells Kubernetes:

  • What is the name of the new resource?

  • What are the fields allowed?

  • What is the API version and kind?

  • What does the YAML structure look like?

Example: Istio defines a new resource called VirtualService.

Istio provides a CRD so Kubernetes understands:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService

CRD is installed once by DevOps engineers (usually via Helm or operator).

Purpose of CRD:

โœ” Introduces a new type of resource into Kubernetes
โœ” Validates all CR created by users
โœ” Extends Kubernetes API


๐ŸŸฆ 2. CR โ€” Custom Resource

CR = User-created object based on the CRD.

Example: After installing the Istio VirtualService CRD, a user can create:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-app
spec:
  ...

This YAML is the Custom Resource.

Summary:

  • CRD = Template / Model / Schema

  • CR = Actual object created by the user

Same logic as:

  • Deployment resource definition (built-in)

  • Deployment YAML (your object)


๐ŸŸฆ 3. Custom Controller

CRD + CR alone do nothing.
Like Ingress without an Ingress controller โ†’ useless.

A Custom Controller is required to watch the CR and take action.

For example:

In Istio:

  • VirtualService CRDs define the API

  • VirtualService CRs define the config

  • Istio Controller watches those CRs

  • Then configures Envoy proxy accordingly

Controller Responsibilities:

โœ” Watches for create / update / delete of CR
โœ” Performs actions on cluster
โœ” Maintains desired state


โญ Flow Diagram (Short & Clear)

DevOps Engineer:
    1. Install CRD โ†’ Adds new API to Kubernetes
    2. Install Controller โ†’ Logic to handle CRs

User / Developer:
    3. Creates Custom Resource (CR)

Controller:
    4. Watches CR
    5. Performs required actions

โญ Simple Example Using Istio

Step 1: DevOps installs CRDs

Istio CRDs include:

  • VirtualService

  • DestinationRule

  • Gateway

  • PeerAuthentication
    โ€ฆ

Step 2: DevOps installs Istio Controller

This controller will watch all Istio CRs.

Step 3: User creates CR

kind: VirtualService
...

Step 4: Istio Controller sees it and configures Envoy proxies.


โญ How Custom Controllers Are Written

Usually written in Go because:

  • Kubernetes itself is written in Go

  • Official client library client-go

  • Best support + ecosystem

Process (high-level):

  1. Set watchers for CR events (Add/Update/Delete)

  2. Add events to a workqueue

  3. Process each event

  4. Take action โ†’ create/update Kubernetes resources or external systems

Frameworks used:

  • controller-runtime

  • operator-sdk (for building operators)


โญ Key Points to Remember (Interview-Friendly)

  • CRD adds a new resource type to Kubernetes.

  • CR is an instance of that resource.

  • Controller makes the CR actually do something.

  • CRD = schema, CR = object, Controller = brain/logic.

  • Without the controller, CR does nothing.