Promotheus, Grafana and node exporter

Installing Prometheus Monitoring Server with a Grafana Dashboard This tutorial explains how to use a
Prometheus Monitoring server with Grafana Dashboard. Requirements * You have an account and are
logged into cloud.scaleway.com * You have configured your SSH Key Prometheus is a flexible monitoring
solution that is in development since 2012. The software stores all its data in a time series database and
offers a multi-dimensional data-model and a powerful query language to generate reports of the monitored
resources. There are five steps to use Prometheus with Grafana: * Preparing your Environment *
Downloading and Installing Node Exporter * Downloading and Installing Prometheus * Configuring
Prometheus * Downloading and Installing Grafana Preparing your Environment In this tutorial, we use an
instance running on Ubuntu Xenial (16.04). 1 . To run Prometheus safely on our server, we have to create a
user for Prometheus and Node Exporter without the possibility to log in. To achieve this, we use the
parameter --no-create-home which skips the creation of a home directory and disable the shell with --shell
/usr/sbin/nologin. sudo useradd --no-create-home --shell /usr/sbin/nologin prometheus sudo useradd --no-
create-home --shell /bin/false nodeexporter 2 . Create the folders required to store the binaries of
Prometheus and its configuration files: sudo mkdir /etc/prometheus sudo mkdir /var/lib/prometheus 3 . Set
the ownership of these directories to our prometheus user, to make sure that Prometheus can access to
these folders: sudo chown prometheus:prometheus /etc/prometheus sudo chown prometheus:prometheus
/var/lib/prometheus Downloading and Installing Node Exporter As your Prometheus is only capable of
collecting metrics, we want to extend its capabilities by adding Node Exporter, a tool that collects
information about the system including CPU, disk, and memory usage and exposes them for scraping. 1 .
Download the latest version of Node Exporter: wget
https://github.com/prometheus/nodeexporter/releases/download/v0.16.0/nodeexporter-0.16.0.linux-
amd64.tar.gz 2 . Unpack the downloaded archive. This will create a directory nodeexporter-0.16.0.linux-
amd64, containing the executable, a readme and license file: tar xvf nodeexporter-0.16.0.linux-amd64.tar.gz
3 . Copy the binary file into the directory /usr/local/bin and set the ownership to the user you have created in
step previously: sudo cp nodeexporter-0.16.0.linux-amd64/nodeexporter /usr/local/bin sudo chown
nodeexporter:nodeexporter /usr/local/bin/nodeexporter 4 . Remove the leftover files of Node Exporter, as
they are not needed any longer: rm -rf nodeexporter-0.16.0.linux-amd64.tar.gz nodeexporter-0.16.0.linux-
amd64 5 . To run Node Exporter automatically on each boot, a Systemd service file is required. Create the
following file by opening it in Nano: sudo nano /etc/systemd/system/node_exporter.service 6 . Copy the
following information in the service file, save it and exit Nano: [Unit] Description=Node Exporter
Wants=network-online.target After=network-online.target

[Service] User=nodeexporter Group=nodeexporter Type=simple ExecStart=/usr/local/bin/node_exporter

[Install] WantedBy=multi-user.target 7 . Collectors are used to gather information about the system. By
default a set of collectors is activated. You can see the details about the set in the README-file. If you want
to use a specific set of collectors, you can define them in the ExecStart section of the service file by using
the --collectors.enabled flag. Once edited, save the file and close Nano:
ExecStart=/usr/local/bin/nodeexporter --collectors.enabled meminfo,hwmon,entropy 8 . Reload Systemd to
use the newly defined service: sudo systemctl daemon-reload 9 . Run Node Exporter by typing the following
command: sudo systemctl start nodeexporter 10 . Verify that the software has been started successfully:
sudo systemctl status nodeexporter You will see an output like this, showing you the status active
(running) as well as the main PID of the application: ● nodeexporter.service - Node Exporter Loaded:
loaded (/etc/systemd/system/nodeexporter.service; disabled; vendor preset: enabled) Active: active
(running) since Mon 2018-06-25 11:47:06 UTC; 4s ago Main PID: 1719 (nodeexporter) CGroup:
/system.slice/nodeexporter.service └─1719 /usr/local/bin/nodeexporter 11 . If everything is working,
enable Node Exporter to be started on each boot of the server: sudo systemctl enable nodeexporter
Downloading and Installing Prometheus 1 . Download and Unpack Prometheus latest release of
Prometheus. As exemplified, the version is 2.2.1: sudo apt-get update && apt-get upgrade wget
https://github.com/prometheus/prometheus/releases/download/v2.2.1/prometheus-2.2.1.linux-amd64.tar.gz
tar xfz prometheus-.tar.gz cd prometheus- The following two binaries are in the directory: * Prometheus -
Prometheus main binary file * promtool The following two folders (which contain the web interface,
configuration files examples and the license) are in the directory: * consoles * consolelibraries 2 . Copy the
binary files into the /usr/local/bin/directory: sudo cp ./prometheus /usr/local/bin/ sudo cp ./promtool
/usr/local/bin/ 3 . Set the ownership of these files to the prometheus user previously created: sudo chown
prometheus:prometheus /usr/local/bin/prometheus sudo chown prometheus:prometheus
/usr/local/bin/promtool 4 . Copy the consoles and consolelibraries directories to /etc/prometheus: sudo cp -r
./consoles /etc/prometheus sudo cp -r ./consolelibraries /etc/prometheus 5 . Set the ownership of the two
folders, as well as of all files that they contain, to our prometheus user: sudo chown -R
prometheus:prometheus /etc/prometheus/consoles sudo chown -R prometheus:prometheus
/etc/prometheus/consolelibraries 6 . In our home folder, remove the source files that are not needed
anymore: cd .. && rm -rf prometheus-* Configuring Prometheus Prior to using Prometheus, it needs basic
configuring. Thus, we need to create a configuration file named prometheus.yml Note: The configuration file
of Prometheus is written in YAML which strictly forbids to use tabs. If your file is incorrectly formatted,
Prometheus will not start. Be careful when you edit it. 1 . Open the file prometheus.yml in a text editor: sudo
nano /etc/prometheus/prometheus.yml Prometheus’ configuration file is divided into three
parts: global, rulefiles, and scrapeconfigs. In the global part we can find the general configuration of
Prometheus: scrapeinterval defines how often Prometheus scrapes targets, evaluationinterval controls how
often the software will evaluate rules. Rules are used to create new time series and for the generation of
alerts. The rulefiles block contains information of the location of any rules we want the Prometheus server
to load. The last block of the configuration file is named scapeconfigs and contains the information which
resources Prometheus monitors. Our file should look like this example: global: scrapeinterval: 15s
evaluation_interval: 15s
 rule_files:
# - "first.rules"
# - "second.rules"

scrape_configs:
- job_name: 'prometheus'
scrape_interval: 5s
static_configs:
- targets: ['localhost:9090']

The global scrapeinterval is set to 15 seconds which is enough for most use cases. We do not have
any rulefiles yet, so the lines are commented out and start with a #. In the scrapeconfigs part we have
defined our first exporter. It is Prometheus that monitors itself. As we want to have more precise information
about the state of our Prometheus server we reduced the scrapeinterval to 5 seconds for this job. The
parameters staticconfigsand targets determine where the exporters are running. In our case it is the same
server, so we use localhost and the port 9090. As Prometheus scrapes only exporters that are defined in
the scrapeconfigs part of the configuration file, we have to add Node Exporter to the file, as we did for
Prometheus itself. We add the following part below the configuration for scrapping Prometheus: - jobname:
'nodeexporter' scrapeinterval: 5s staticconfigs: - targets: ['localhost:9100'] We overwrite the global scrape
interval again and set it to 5 seconds. As we are scarping the data from the same server as Prometheus is
running on, we can use localhost with the default port of Node Exporter: 9100. If you want to scrape data
from a remote host, you have to replace localhost with the IP address of the remote server. For all
information about the configuration of Prometheus, you may check the configuration documentation. 2 . Set
the ownership of the file to our Prometheus user: sudo chown prometheus:prometheus
/etc/prometheus/prometheus.yml Our Prometheus server is ready to run for the first time. Running
Prometheus 1 . Start Prometheus directly from the command line with the following command, which
executes the binary file as our Prometheus user: sudo -u prometheus /usr/local/bin/prometheus --config.file
/etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --
web.console.templates=/etc/prometheus/consoles --web.console.libraries=/etc/prometheus/consolelibraries
The server starts displaying multiple status messages and the information that the server has started:
level=info ts=2018-04-12T11:56:53.084000977Z caller=main.go:220 msg="Starting Prometheus" version="
(version=2.2.1, branch=HEAD, revision=bc6058c81272a8d938c05e75607371284236aadc)" level=info
ts=2018-04-12T11:56:53.084463975Z caller=main.go:221 buildcontext="(go=go1.10,
user=root@149e5b3f0829, date=20180314-14:15:45)" level=info ts=2018-04-12T11:56:53.084632256Z
caller=main.go:222 hostdetails="(Linux 4.4.127-mainline-rev1 #1 SMP Sun Apr 8 10:38:32 UTC 2018
x8664 scw-041406 (none))" level=info ts=2018-04-12T11:56:53.084797692Z caller=main.go:223 fd_limits="
(soft=1024, hard=65536)" level=info ts=2018-04-12T11:56:53.09190775Z caller=web.go:382
component=web msg="Start listening for connections" address=0.0.0.0:9090 level=info ts=2018-04-
12T11:56:53.091908126Z caller=main.go:504 msg="Starting TSDB ..." level=info ts=2018-04-
12T11:56:53.102833743Z caller=main.go:514 msg="TSDB started" level=info ts=2018-04-
12T11:56:53.103343144Z caller=main.go:588 msg="Loading configuration file"
filename=/etc/prometheus/prometheus.yml level=info ts=2018-04-12T11:56:53.104047346Z
caller=main.go:491 msg="Server is ready to receive web requests." 2 . Open your browser and
type http://IP.OF.YOUR.SERVER:9090 to access the Prometheus interface. If everything is working, we end
the task by pressing on CTRL + C on our keyboard. Note: If you get an error message when you start the
server, double check your configuration file for possible YAML syntax errors. The error message will tell you
what to check. 3 . The server is working now, but it cannot yet be launched automatically at boot. To
achieve this, we have to create a new systemd configuration file that will tell your OS which services should
it launch automatically during the boot process. sudo nano /etc/systemd/system/prometheus.service The
service file tells systemd to run Prometheus as prometheus and specifies the path of the configuration files.
4 . Copy the following information in the file and save it, then exit the editor: [Unit] Description=Prometheus
Monitoring Wants=network-online.target After=network-online.target

[Service] User=prometheus Group=prometheus Type=simple ExecStart=/usr/local/bin/prometheus \ --

[Install] WantedBy=multi-user.target 5 . To use the new service, reload systemd: sudo systemctl daemon-
reload We enable the service so that it will be loaded automatically during boot: sudo systemctl enable
prometheus 6 . Start Prometheus: sudo systemctl start prometheus Your Prometheus server is ready to be
used. We have now installed Prometheus to monitor your instance. Prometheus Web Interface Prometheus
provides a basic web server running on http://your.server.ip:9000 that provide access to the data collected
by the software. We can verify the status of our Prometheus server from the interface: ​ Moreover, do some
queries in the data that has been collected. ​ The interface is very lightweight, and the Prometheus team
recommend to use a tool like Grafana if you want to do more than testing and debugging the installation.
Installing of Grafana 1 . Install Grafana on our instance which queries our Prometheus server. wget
https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana5.0.4amd64.deb sudo apt-get install
-y adduser libfontconfig sudo dpkg -i grafana5.0.4amd64.deb 2 . Enable the automatic start of Grafana
by systemd: sudo systemctl daemon-reload && sudo systemctl enable grafana-server && sudo systemctl
start grafana-server Grafana is running now, and we can connect to it at http://your.server.ip:3000. The
default user and password is admin / admin. ​ Now you have to create a Prometheus data source: * Click on
the Grafana logo to open the sidebar. * Click on “Data Sources” in the sidebar. * Choose “Add New”. *
Select “Prometheus” as the data source. * Set the Prometheus server URL (in our case:
http://localhost:9090/) * Click “Add” to test the connection and to save the new data source. Your settings
should look like this: ​ You are now ready to create your first dashboard from the information collected by
Prometheus. You can also import some dashboards from a collection of shared dashboards Here is an
example of a Dashboard that uses the CPU usage of our node and presents it in Grafana: ​ In this tutorial,
we were able to configure a Prometheus server with two data collectors that are scraped by our
Prometheus server which provides the data to build Dashboards with Grafana. Don’t hesitate to consult the
official documentation of Prometheus and Grafana.

Grafana => admin = rahasia User password