# Monitoring EC2 Instance using Prometheus and Grafana

> **Prometheus**

Prometheus monitoring solution is a free and open-source solution for monitoring metrics, events, and alerts. It collects and records metrics from servers, containers, and applications. In addition to providing a flexible query language (PromQL), and powerful visualization tools, it also provides an alerting mechanism that sends notifications when needed.

> Node Exporter

Node Exporter is a Prometheus exporter for hardware and OS metrics. It is designed to run on each node (e.g., an EC2 instance, server, or container) to expose metrics about the system’s performance.

> Grafana

Grafana is an open-source platform for monitoring and observability, focused on providing visualization and dashboards for data from various sources, including Prometheus.

> **Add security group for EC2 instances:**

1. Add 9090 port for Prometheus
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722581787436/54c88b37-c6e0-4a42-b044-783e56f44254.png align="center")

2. Add 9100 port for Prometheus Node Exporter
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722581616979/a9af1c49-e0b3-4508-9cb2-6d55d787573d.png align="center")

3. Add 3000 port for Grafana
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722581694612/d6048999-9752-42b2-9117-6d123e16f492.png align="center")

**<mark>Step 1 : Installing Prometheus</mark>**

```plaintext
git clone https://github.com/Phanindra-Sangers/prometheus-monitoring.git
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722582176493/0bf2d7e9-6f46-4dfd-8bac-5e8ce859bd45.png align="center")

```plaintext
cd prometheus-monitoring
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722582243586/9dd2f71b-ac4a-4711-9c99-284255908886.png align="center")

Install prometheus service

```plaintext
./install-prometheus.sh
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722582299125/0347c7dd-1ce8-4036-a20a-057f091c0d9d.png align="center")

Now, Prometheus service is running

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722582320184/0adbdcea-51fc-4965-a2cf-37b377aa7ad3.png align="center")

Creates a new user and add new directories for Prometheus

```plaintext
sudo useradd --no-create-home prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
```

Downloads the Prometheus, extract it and put it in /usr/local/bin folder and finally delete the software

```plaintext
wget https://github.com/prometheus/prometheus/releases/download/v2.54.0-rc.0/prometheus-2.54.0-rc.0.linux-amd64.tar.gz
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722583261899/2bc3d513-2ad5-417e-a5f8-2259c64f4ce4.png align="center")

```plaintext
sudo cp -r prometheus-2.54.0-rc.0.linux-amd64.tar.gz /etc/prometheus
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722583573043/3caa3e6c-508a-4b6a-9035-9a8e5d46b7b8.png align="center")

```plaintext
tar -xvf prometheus-2.54.0-rc.0.linux-amd64.tar.gz
```

```plaintext

sudo cp prometheus-2.54.0-rc.0.linux-amd64/prometheus /usr/local/bin
sudo cp prometheus-2.54.0-rc.0.linux-amd64/promtool /usr/local/bin
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722583857971/391314a4-f618-41bd-99bc-7b254843d12a.png align="center")

```plaintext
rm -rf prometheus-2.23.0.linux-amd64.tar.gz prometheus-2.19.0.linux-amd64
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722583954431/0aadf1c3-9374-4aa0-8d6f-6546fcff389e.png align="center")

**<mark>Configure Prometheus :</mark>**

Create a prometheus.yml file at "**/etc/prometheus/prometheus.yml"** with the below content

```plaintext
cd /etc/prometheus
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722584122335/5e6ebddc-8823-40ff-a0eb-69066ae7d8a9.png align="center")

```plaintext
vi prometheus.yml
```

Copy the following content in yml file.

```plaintext
global:
  scrape_interval: 15s
  external_labels:
    monitor: 'prometheus'

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']
```

Now we want to run the Prometheus as a Service so that in case of server restart service will come automatically

Create a file at "/etc/systemd/system/"

```plaintext
cd /etc/systemd/system/

vi prometheus.service
```

Copy the below content in the service file

```plaintext
[Unit]
Description-Prometheus 
Wants-network-online.target 
After-network-online.target

[Service]
User-prometheus
Group-prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file=/etc/prometheus/prometheus.yml \
    --storage.tsdb.path=/var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy-multi-user.target
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722584740384/e3dd02f0-d2b9-4a92-b2e2-b1fd9851bcff.png align="center")

Save and exit "Press esc :wq"

Change the ownership of all folders and files which we have created to the user which we have created in the first step

```plaintext

sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries
sudo chown -R prometheus:prometheus /var/lib/prometheus
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722584857903/8f9d6067-79fb-4fab-95ad-856147f736b4.png align="center")

Configure the service and start it

```plaintext
sudo systemctl daemon-reload
sudo systemctl enable prometheus
sudo systemctl start prometheus
sudo systemctl status prometheus
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722584960034/f7816fb8-0af6-4f3b-b2b5-5f66388f620b.png align="center")

Now, Prometheus service is running

Now we can access the Prometheus dashboard

```plaintext
http://<public ip>:9090
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585068325/34b8064a-d8e9-4728-8c65-03042758f457.png align="center")

**<mark>Step 2 : Install Node Exporter</mark>**

To monitor your servers you need to install the node exporter on all your target machine which is like a monitoring agent on all the servers.

```plaintext
git clone https://github.com/Phanindra-Sangers/prometheus-monitoring.git
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585614702/cc73138c-3228-484e-9d37-b238dbd6e0ae.png align="center")

```plaintext
cd prometheus-monitoring
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585656944/bfabc290-6dee-47de-882b-bb9aa98319a4.png align="center")

```plaintext
./install-node-exporter.sh
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585726315/fe761ddc-423d-4a18-af16-f5bb2ed8a007.png align="center")

Now, Node exporter service is running

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585750157/c7e39005-153a-400e-9822-8f8a9bda00c5.png align="center")

```plaintext
sudo useradd --no-create-home node_exporter
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585865620/7de02221-18a8-4d83-9578-3d616f6fdb96.png align="center")

```plaintext
wget https://github.com/prometheus/node_exporter/releases/download/v1.0.1/node_exporter-1.0.1.linux-amd64.tar.gz
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722585904128/b5d42372-3613-4958-aa3b-4c5d58f126f9.png align="center")

```plaintext
tar xzf node_exporter-1.0.1.linux-amd64.tar.gz
```

```plaintext
sudo cp node_exporter-1.0.1.linux-amd64/node_exporter /usr/local/bin/node_exporter
rm -rf node_exporter-1.0.1.linux-amd64.tar.gz node_exporter-1.0.1.linux-amd64
sudo cp node-exporter.service /etc/systemd/system/node-exporter.service
sudo systemctl daemon-reload
sudo systemctl enable node-exporter
sudo systemctl start node-exporter
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722586126214/9035f754-8bfe-4be9-a46e-71dcec2cdd16.png align="center")

```plaintext
sudo systemctl status node-exporter
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722586150265/ff26c74e-f282-4590-a967-d02a880d53c5.png align="center")

Now we can access the "**Node Exporter**" dashboard

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722586232839/c36eb7c7-cdb1-4553-8f80-a92818a04d69.png align="center")

**<mark>Step 3: Configure Prometheus for the Nodes</mark>**

Now we will configure the Prometheus for our EC2 instance where we have installed the node-exporter.

Login to the Prometheus server and edit the file file at location in Prometheus instance

```plaintext
vi /etc/prometheus/prometheus.yml
```

Need to add Node Exporter machine ip with port

```plaintext
global:
  scrape_interval: 15s
  external_labels:
    monitor: 'prometheus'

scrape_configs:

  - job_name: 'node_exporter'
    static_configs:
          - targets: ['13.60.63.200:9100']
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722586800600/19b7ef1d-8b81-4438-8d33-5dec2519625f.png align="center")

Restart the Prometheus service & Check the Prometheus service status

```plaintext
sudo systemctl restart prometheus
sudo systemctl status prometheus
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722586885068/9afbac82-4e91-4fe3-855c-65701d6de29b.png align="center")

**Now you can open the Prometheus using below url and can see the new targets added**

```plaintext
http://<ip address>:9090/targets
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722586997180/0f822ca1-19a1-4e1f-b8de-4b5b0890a4d5.png align="center")

**<mark>Step 4: Install Grafana</mark>**

Once Prometheus is installed successfully then we can install the Grafana and configure Prometheus as a datasource.

Grafana is an opensource tool which is used to provide the visualization of your metrics

Install the prerequisite packages:

```plaintext
sudo apt-get install -y apt-transport-https software-properties-common wget
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594403157/e1ed7dca-3889-4a21-b23c-941985efe6af.png align="center")

Import the GPG key:

```plaintext
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594439809/f027f0eb-0411-48b0-b754-8ea320fee64d.png align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594474884/d75baf9d-e851-41b5-a71a-baf1a8c9dc36.png align="center")

To add a repository for stable releases, run the following command:

```plaintext
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594518079/75327080-dc0b-4e3d-b900-24b686917c06.png align="center")

Update the list of available packages:

```plaintext
sudo apt-get update
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594738701/fbd5763c-2092-46ca-a0e4-56a920409db6.png align="center")

To install Grafana, run the following command:

```plaintext
sudo apt-get install grafana
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594764528/699e423b-535d-4d37-8301-c43c02565509.png align="center")

To start the service, run the following commands:

```plaintext
sudo systemctl daemon-reload
sudo systemctl start grafana-server
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594903637/deefc057-1860-46b0-93e4-ddd9deddc6bc.png align="center")

Check the grafana server status

```plaintext
sudo systemctl status grafana-server
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722594925418/13020933-f0f8-4c93-b50b-92d6f9db2812.png align="center")

Now, we can access Grafana dashboard

```plaintext
http://<ip address>:3000
```

Login with default Username : admin & default password : admin

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722595031353/e7a67967-da4b-4e6d-8214-db492e19faa4.png align="center")

Click on "Connections", Click "Add new connection"

Search ( Prometheus ) & Click "Prometheus"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722595997718/b250dd0f-1eea-464e-9e9d-f50137be4169.png align="center")

Click "Add new data source"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722596066639/aeeef3af-11ef-4354-9194-40905b73cab4.png align="center")

Enter the name "Prometheus"

Mention your Prometheus server ip "http://&lt;ip address&gt;:9090"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722596271849/4b7d4b59-02d4-4df5-ad31-40b03a04fd04.png align="center")

Scroll down, Click "Save & test"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722596313581/c58132a2-f6ee-412b-acb6-131e9b5ad5e8.png align="center")

Click on "Data source"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722596394544/03b8b3ed-08d0-4753-9df6-1ee25e9299b1.png align="center")

Choose "prometheus" and Enter "up" query in Metric Browser

Press the "Run query" option

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722598488998/38b510e3-255f-4b95-a603-7fcd3a07100a.png align="center")

You will get dashboard

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722598599998/5cc82923-3f72-4600-a183-b5874c7d756e.png align="center")

Grafana provides lot of dashboards which we can directly import in our Grafana instance and use it.

Import the dashboard

Click on Dashboards

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722598674180/3375ccd9-5194-43df-bb06-e2ca720c4800.png align="center")

Click on new, Click Import

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722600527739/d8dbd169-32c7-4355-9483-49cbd03ffb9c.png align="center")

I have downloaded sample "Dashboard JSON file"

Click on "Upload dashboard JSON file"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722601844092/63f8467f-8598-4d12-b97e-446fbb2f4ea6.png align="center")

Set name for "Dashboard" and Select "Prometheus"

Click "Import"

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722601887383/54ab9fe1-845f-48bc-8449-87959f8158bf.png align="center")

Now, we can see the EC2 instance metrics include CPU, Memory, Network, and more in the Grafana Dashboard.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722601679946/38641305-ad8a-47f1-a7bf-3990629f713b.png align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1722601953355/d4b12ada-2c60-41b6-ac88-bdfab5b7407f.png align="center")