Scribd
Upload
2 views

Monitoring Stack Project

The document outlines a project for monitoring and logging using Grafana, Prometheus, Loki, and Promtail, detailing the tech stack and installation steps on an AWS EC2 instance. It includes instructions for setting up Docker, Docker Compose, cAdvisor, Node Exporter, and configuring Prometheus and Grafana for data visualization. Additionally, it covers the installation of Loki and Promtail for logging purposes, culminating in the creation of dashboards for monitoring application metrics.

Uploaded by

Javier Sepulveda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views

Monitoring Stack Project

The document outlines a project for monitoring and logging using Grafana, Prometheus, Loki, and Promtail, detailing the tech stack and installation steps on an AWS EC2 instance. It includes instructions for setting up Docker, Docker Compose, cAdvisor, Node Exporter, and configuring Prometheus and Grafana for data visualization. Additionally, it covers the installation of Loki and Promtail for logging purposes, culminating in the creation of dashboards for monitoring application metrics.

Uploaded by

Javier Sepulveda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 25

Monitoring using Grafana, Prometheus and Logging using

Loki and Promtail


Ojas Jawale

Project Architecture-

Tech Stack-

 Docker & Docker Compose: Containerization and orchestration.


 Prometheus: Metrics collection and monitoring.
 Grafana: Data visualization and dashboard creation.
 cAdvisor: Container resource monitoring.
 Node Exporter: Hardware and OS metrics exporter.
 Notes App: A custom service to demonstrate monitoring.
Create an AWS EC2 instance for server-

- Name : monitoring-server
- AMI Type : Ubuntu 24.04
- Instance type : t2.medium
- Generate new key pair(.pem)
- Configure Storage : 20Gb
- Launch Instance

- Connect Server using SSH.

- Update the repository


- sudo apt update

- Install Docker on server


- sudo apt install docker.io -y

- Give necessary permission to non-root to use docker.


- sudo usermod -aG docker $USER && newgrp docker

- Install Docker Compose V2


- sudo apt install docker-compose-v2 -y
Create new folder and clone source code,
- mkdir observability && cd observability

- Clone source code


- https://github.com/ojasjawale/django-notes-app.git

- Create docker-compose.yml file for observability stack,

- cd ..
- nano docker-compose.yml

- Above service is for only application and will run in user-defined network called
“monitoring”.
- Run the docker-compose.yml file,

- Open port 8000 in server’s security group.

- Test application on browser,


Installation of cAdvisor and redis-

- cAdvisor (Container Advisor) provides container users an understanding of the


resource usage and performance characteristics of their running containers.
- It is a running daemon that collects, aggregates, processes, and exports information
about running containers. Specifically, for each container it keeps resource isolation
parameters, historical resource usage, histograms of complete historical resource usage
and network statistics. This data is exported by container and machine-wide.
- Basically cAdvisor is used to analyze performance metrix of Docker containers.
- We are capturing all metrices of our containers using cAdvisor.(Refer architecture)

- Install cAdvisor and Redis using Docker containers in compose file,

- Ensure that all containers are running in one networks only(monitoring).

- Run the docker-compose.yml file,


- docker compose up -d
- Verify containers status,
- docker ps

- Open port 8080 of cAdvisor in security group.


- Access it using IP:8080
Installation of Prometheus using docker-

- An open-source monitoring system with a dimensional data model, flexible query


language, efficient time series database and modern alerting approach.
- Prometheus is a Time series database(tsdb) which contains data in time-series format
in tsdb. Data is ingested using cAdvisor and store in tsdb.
- To fetch data of tsdb, there is query language called PromQL.

- Download prometheus configuration file,


- pwd : /home/ubuntu/observability
- wget
https://github.com/prometheus/prometheus/blob/main/documentation/examples/p
rometheus.yml

- Prometheus has its own configuration which need to be configure and copy it
into the prometheus’s container.
- Install prometheus using Docker container in compose file,
- nano docker-compose-yml

- Open port 9090 in security group,


- Access prometheus on browser,

- Adding targets in Prometheus,

- Prometheus has prometheus.yml file which includes scrape_configs.


- Which contains targets from where data metrics will be ingested into tsdb.

So edit, prometheus.yml file to add target of cAdvisor,

- pwd : /home/ubuntu/observability
- nano prometheus.yml
- In this scrape_configs, we need to add target of cAdvisor which already have metrics
of docker containers.

- Restart prometheus container to reflect changes done in configuration file.


- Now go to prometheus -> status -> targets
- cAdvisor should be visible.

- Let’s see metrics of container’s using PromQL in Prometheus,


- rate(container_cpu_usage_seconds_total{name="notes-app"}[5m])

1. Map Format

2. Table Format
Adding Node Exporter service in prometheus as target,

- pwd : /home/ubuntu/observability
- nane docker-compose.yml

- Add target of node exporter in prometheus configuration file as scrape_configs,

- Restart prometheus container to reflect changes and open port 9100.


- Access node exporter on browser

- In Prometheus target is visible now,

- Test metrics of node using PromQL,


- 100 - (node_filesystem_free_bytes{fstype!~"tmpfs|fuse.lxcfs|squashfs"} /
node_filesystem_size_bytes{fstype!~"tmpfs|fuse.lxcfs|squashfs"} * 100)
Installation of Grafana using Docker compose,

- nano docker-compose.yml

- docker compose down


- docker compose up -d

- Open port 3000 in security group for Grafana.

- Access Grafana Interface on browser,


Default credentials of Grafana,
- username : admin
- password : admin

- After that change password.

- Add new data source as Prometheus, so that Grafana can visualize metrics.
- Go to Home -> Connections
- Add Prometheus data source,

- Click on “Add new data source”.


- Add Prometheus server URL.

- Click on “Save & Test”

- Build Dashboard,
- Click on “Add visualization”
- Select “Data Source” as Prometheus

- Add sample query to view memory usage of Notes-app,

- container_memory_usage_bytes{name="notes-app"}
- Select visualization,

- Click on “Apply”.
Import Dashboard in Grafana

- Go to Dashboard -> New -> Import

- Go to Google and search for Grafana, prometheus, node exporter dashaboard


full.
- Get Dashboard ID

- Load dashboard
- Select datasource as “Prometheus”
- Import
- Dashboard should be,
Setup Loki and Promtail using Docker,

- As prometheus has config file, as same Loki and promtail also have config file. Hence
we need to download these files.

- Download both config files,


- Loki

wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/cmd/loki/loki-local-config.yaml -O
loki-config.yaml

- Promtail

wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/clients/cmd/promtail/promtail-
docker-config.yaml -O promtail-config.yaml

- Now, add container for Loki and Promtail in docker compose file.
- docker compose down
- docker compose up -d

- All containers are running,


- Add data-source of Loki in Grafana,

- Build the Dashboard of Loki & Promtail,

Ojas Jawale

You might also like