In this session, we learn how to monitor a Kubernetes cluster using Prometheus and Grafana.
This is not just theory — there is a GitHub repository containing all installation commands and demo steps.
The repo will also be enhanced later with advanced Kubernetes monitoring and custom metric server topics, so you can star it and follow future updates.

---

✅ WHY MONITORING?

If you have only one Kubernetes cluster, monitoring is easy.
But in a real company:

• Multiple teams use the same cluster.

• Teams complain: “My deployment is not receiving requests”, or “Service was down for some time.”

• You may have multiple clusters: dev, staging, prod.

As clusters increase, you need a monitoring solution to understand:

• What is happening inside your clusters?

• Which deployment is down?

• Is API server healthy?

• Are replica counts matching?

• Are nodes running or not?

---

✅ WHY PROMETHEUS?

Prometheus was created by SoundCloud and is now completely open-source.

Prometheus:

• Scrapes metrics from Kubernetes.

• Stores metrics in a Time Series Database.

• Can trigger alerts using Alertmanager.

• Provides a UI for running PromQL queries.

---

✅ WHY GRAFANA?

Prometheus gives output but not visually appealing.

Grafana:

• Connects to Prometheus as a data source.

• Visualizes data using dashboards.

• Makes metrics easy to understand.

---

✅ PROMETHEUS ARCHITECTURE (Simple Explanation)

Prometheus includes:

1️⃣ Prometheus Server

• Scrapes metrics from Kubernetes API Server.

• Stores metrics in time-series format on disk.

2️⃣ Kubernetes API Server

• Exposes built-in metrics at:
  /metrics

• Shows default cluster metrics.

3️⃣ Alertmanager

• Prometheus pushes alerts to it.

• Alertmanager sends notifications (Slack, Email, etc.).

4️⃣ PromQL Interface

• Used in Prometheus UI or Grafana to run queries.

5️⃣ External Access

• Grafana pulls data from Prometheus.

• Tools like curl or Postman can also query Prometheus APIs.

---

✅ GRAFANA USE

Grafana helps visualize Prometheus data through graphs and dashboards.

---

✅ DEMO: INSTALL PROMETHEUS + GRAFANA ON MINIKUBE

We create a Kubernetes cluster using Minikube:

minikube start --memory=4096 --driver=hyperkit

(Use hyperkit on Mac, VirtualBox or Docker on other systems.)

---

🔥 INSTALL PROMETHEUS USING HELM

1. Add the Prometheus Helm repo:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

2. Install Prometheus:

helm install prometheus prometheus-community/prometheus

3. Check pods:

kubectl get pods -A

You will see:

• Prometheus server

• Prometheus alertmanager

• kube-state-metrics

• node-exporter

❗ kube-state-metrics

This component provides extra Kubernetes metrics NOT available in the Kubernetes API server.
It exposes metrics for:

• Deployments

• Daemonsets

• Pods

• ReplicaSets

• Services

• Replica count

• Desired vs Actual state

Without kube-state-metrics you only get basic Kubernetes metrics.

---

🔥 EXPOSE PROMETHEUS SERVER USING NODEPORT

Default service is ClusterIP, so expose it:

kubectl expose service prometheus-server --type=NodePort --name=prometheus-server-ext

Get service:

kubectl get svc

Use Minikube IP:

minikube ip

Open Prometheus:

http://<minikube-ip>:<node-port>

You can now run PromQL queries.

---

🔥 INSTALL GRAFANA USING HELM

1. Add Grafana repo:

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

2. Install Grafana:

helm install grafana grafana/grafana

3. Get admin password:

kubectl get secret -n default grafana -o jsonpath="{.data.admin-password}" | base64 --decode

4. Expose Grafana:

kubectl expose service grafana --type=NodePort --name=grafana-ext

5. Access Grafana:

http://<minikube-ip>:<node-port>

Login using:

• User: admin

• Password: (from command above)

---

🔥 ADD PROMETHEUS AS A DATASOURCE IN GRAFANA

Grafana → Data Sources → Add → Select Prometheus

Enter URL:

http://<minikube-ip>:<prometheus-nodeport>

Save & Test → should show Data source is working.

---

🔥 IMPORT KUBERNETES DASHBOARD (ID: 3662)

Grafana → Dashboards → Import → Enter ID:

3662

Select data source = Prometheus → Import.

A beautiful Kubernetes dashboard appears showing:

• API server health

• Nodes

• CPU & Memory usage

• Cluster info

• Pod counts

• Node uptime

---

🔥 ENABLE kube-state-metrics FOR DEPLOYMENT-LEVEL METRICS

Expose kube-state-metrics:

kubectl expose service prometheus-kube-state-metrics --type=NodePort --name=kube-state-metrics-ext --target-port=8080

Get NodePort:

kubectl get svc

Open:

http://<minikube-ip>:<nodeport>/metrics

Now you will see deployment-level, pod-level, and service-level metrics.

These metrics appear inside your Grafana dashboards.

---

🎉 END RESULT

You successfully installed:

✔️ Kubernetes Cluster (Minikube)
✔️ Prometheus
✔️ Grafana
✔️ kube-state-metrics
✔️ Kubernetes monitoring dashboard (ID: 3662)

Now your Grafana shows:

• Nodes

• API Server

• Pods

• Deployments

• Resource usage

• Replica counts

• Cluster uptime

• Realtime metrics