How To Install Prometheus on Ubuntu 16.04

Awesome documentation! Here is some update for Prometheus >2.0 setup with Ubuntu 16.

$ sudo -u prometheus /usr/local/bin/prometheus \ –config.file=/etc/prometheus/prometheus.yml
–storage.tsdb.path=/var/lib/prometheus/

This is a great blog with entire setup + securing Prometheus dashboard. However, it is missing setting up consoles and console_libraries sections.

Due to lack of consoles folder we cannot access url like /consoles/node.html & /consoles/prometheus.html which comes default with Prometheus tar.gz.

Note - I was installing prometheus version 2.0.0 so setting for storage path in version 2.0.0 is storage.tsdb.path but in 1.x it was storage.local.path.

Before running delete command stated below -

rm -rf prometheus-2.0.0.linux-amd64.tar.gz prometheus-2.0.0.linux-amd64

Copy these folders -

1. consoles
2. console_libraries

to location that you want. For my ease i pasted them with my prometheus.yml file i.e /etc/prometheus by following commands -

sudo cp -R ~/prometheus-2.0.0.linux-amd64/consoles /etc/prometheus
sudo cp -R ~/prometheus-2.0.0.linux-amd64/console_libraries /etc/prometheus

After this we need to enhance run prometheus commands both manual and service to support following settings -

1. web.console.templates
2. web.console.libraries

for consoles and console_libraries folder respectively (related gtihub ticket)

Changes needed -

• Manual run Step 4 will change to -

sudo -u prometheus /usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus/ \
--web.console.templates=/etc/prometheus/consoles \
--web.console.libraries=/etc/prometheus/console_libraries

• Service file /etc/systemd/system/prometheus.service will change to -

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus --config.file=/etc/prometheus/prometheus.yml \
 --storage.tsdb.path=/var/lib/prometheus/ \
 --web.console.templates=/etc/prometheus/consoles \
 --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Now we are all set to access both http://<you_ip_address>/consoles/node.html & http://<you_ip_address>/consoles/prometheus.html pages.

Hi, how can I install blackbox exporter? is it the same with the way node exporter was done? a feedback would help a lot

How do I install and configure the alert manager?

Alertmanager is a tool developed by the same prometheus guys and in my vision is quite important. Based on this marvelous tutorial i have developed a set of commands to install alertmanager as well. Since i did not find this anywhere here goes:

#!/bin/bash sudo useradd --no-create-home --shell /bin/false alertmanager wget https://github.com/prometheus/alertmanager/releases/download/v0.12.0/alertmanager-0.12.0.linux-amd64.tar.gz tar xvf alertmanager-0.12.0.linux-amd64.tar.gz cd alertmanager-0.12.0.linux-amd64.tar.gz sudo cp alertmanager /usr/local/bin/ sudo cp amtool /usr/local/bin/ sudo chown alertmanager:alertmanager /usr/local/bin/alertmanager sudo chown alertmanager:alertmanager /usr/local/bin/amtool sudo mkdir /etc/alertmanager sudo mkdir /etc/alertmanager/data sudo mv simple.yml /etc/alertmanager/alertmanager.yml sudo chown alertmanager:alertmanager /etc/alertmanager/alertmanager.yml sudo chown alertmanager:alertmanager /etc/alertmanager/data cd /etc/systemd/system/ SCRIPT=“[Unit]\n Description=Alertmanager\n Wants=network-online.target\n After=network-online.target\n \n [Service]\n User=alertmanager\n Group=alertmanager\n Type=simple\n ExecStart=/usr/local/bin/alertmanager -config.file /etc/alertmanager/alertmanager.yml -storage.path /etc/alertmanager/data\n \n [Install]\n WantedBy=multi-user.target” echo -e $SCRIPT >> alertmanager.service sudo systemctl daemon-reload sudo systemctl enable alertmanager sudo systemctl start alertmanager

Please use this script as you wish. Probably some commands will have to be modified to fit one needs. So have fun ^^

P.S.: OP, can you update your documentation to add alertmanager? Thanks :D

Hello Marko,

Good job on this bootstrapping howto:)

One important note: when configuring Nginx [or any other web server/proxy for that matter] with basic auth to protect the Prometheus I/F, one should also pass along

--web.listen-address=127.0.0.1:9090

to the Prometheus daemon so that it binds on loopback only [default is 0.0.0.0]. Otherwise, while when making a request to the Nginx, you will indeed be prompted for a user and passwd, you could still reach Prometheus directly without auth over port 9090 [of course, 9090 is only the default port, can be anything].

I recommend adding --web.listen-address=127.0.0.1:9090 to your sample /etc/systemd/system/prometheus.service.

Same thing is true for the node_exporter which also uses the same flag.

I have a problem with step 4.

level=info ts=2018-02-05T08:59:41.45936827Z caller=main.go:486 msg="Server is ready to receive web requests."
level=info ts=2018-02-05T08:59:41.460060803Z caller=manager.go:59 component="scrape manager" msg="Starting scrape manager..."

Starting scrape manager complete never. The system hangs in this step. How i can fix it?

Hello,

I am just learning Prometheus, and I actually have one working, However I am confused about the exporters, Where do they get installed? I mean I understand grabbing metrics from the Prometheus server itself. But what if I want metrics from say a Cisco 3750 Switch? I know there is an SNMP Exporter but it can’t be installed on the Cisco. It appears that you install them on the Prometheus server thus far.

Sorry for the stupid question… Thanks, Michael

Hey, I’m trying to configure Prometheus to watch over my Kubernetes cluster on Digital Ocean. I’m playing around with this demo: https://github.com/microservices-demo/microservices-demo which has already configured Prometheus YAML files, and actually prometheus running as a Deployment. But, some of my targets are down and I’m too new to this area in general to figure out why. Could you please help me out, what should I look for? I’m getting errors with kubernetes-dashboard (port 8443/metrics --> malformed HTTP response), kubernetes-nodes (port 10250/metrics --> 403 forbidden), monitoring/ (port 80/metrics --> dial tcp… getsockopt conncetion refused), grafana ( port 3000/metrics --> 401 unauthorized), and also some others… Thanks in advance!

If anyone is interested in saving time, I’ve scripted the installation portion of this tutorial.

You can find my scripts here: https://github.com/icolwell/install_scripts

The following scripts may be of interest:

prometheus_install.bash
prometheus_node_exporter_install.bash
prometheus_apache_exporter_install.bash

You can download and run the scripts using the following:

wget https://raw.githubusercontent.com/icolwell/install_scripts/master/prometheus_install.bash
bash prometheus_install.bash

Note that any existing prometheus configuration will be overridden.