Learn how to – How to Install and Configure Graphite on Ubuntu 18.04

Learn how to – How to Install and Configure Graphite on Ubuntu 18.04.

This guide will cover the installation and configuration of Graphite on Ubuntu 18.04 Bionic Beaver Linux. Graphite is a highly available time-series monitoring and data store with support for data visualization. Ideally, you will install other tools on a system to monitor, which will collect the metrics and ship them to Graphite e.g Statsd.

Graphite Architecture

Graphite consists of 3 software components:

1. carbon – a Twisted daemon that listens for time-series data
2. whisper – a simple database library for storing time-series data (similar in design to RRD)
3. graphite webapp – A Django webapp that renders graphs on-demand using Cairo

Feeding data to graphite is the easy part. Data that are sent to Carbon become immediately available for graphing in the webapp. From webapp, you can create and display graphs including a simple URL API for rendering that makes it easy to embed graphs in other web pages.

Method 1: Installing Graphite on Ubuntu 18.04 using Docker (Recommended)

The first and easy method to install Graphite on Ubuntu 18.04 is using Docker.

Step 1: Update system and installed packages.

sudo apt-get update
sudo apt-get upgrade

Step 2: Install Docker Engine

For installation of Docker Engine on Ubuntu 18.04, use our guide:

How to install Docker CE on Ubuntu / Debian / Fedora / Arch / CentOS

Confirm that the service is running before you can proceed:

$ systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2018-07-11 07:45:15 UTC; 43s ago
     Docs: https://docs.docker.com
 Main PID: 8898 (dockerd)
    Tasks: 16
   CGroup: /system.slice/docker.service
           ├─8898 /usr/bin/dockerd -H fd://
           └─8937 docker-containerd --config /var/run/docker/containerd/containerd.toml

Step 3: Install Graphite and statsd container

Once the docker engine is installed and running, it is time to start docker container for Graphite and Statsd. We will use the official Docker repo for Graphite:

Create directories on the host system for persistent data storage:

sudo mkdir -p /data/graphite/{statsd,configs,data}

Graphite & Statsd can be complex to set up. The docker image provided will have you running & collecting stats in just a few minutes.

sudo docker run -d 
 --name graphite 
 --restart=always 
 -p 80:80 
 -p 2003-2004:2003-2004 
 -p 2023-2024:2023-2024 
 -p 8125:8125/udp 
 -p 8126:8126 
 -v /data/graphite/configs:/opt/graphite/conf 
 -v /data/graphite/data:/opt/graphite/storage 
 -v /data/graphite/statsd:/opt/statsd 
 graphiteapp/graphite-statsd

Wait for docker engine to download and start docker container for you. It may take some time depending on your internet speed. Once it is done, you should be able to see docker container running:

$ docker ps
CONTAINER ID        IMAGE                         COMMAND             CREATED             STATUS              PORTS                                                                                                                                                        NAMES
959fe7ab2abd        graphiteapp/graphite-statsd   "/sbin/my_init"     11 seconds ago      Up 10 seconds       0.0.0.0:80->80/tcp, 0.0.0.0:2003-2004->2003-2004/tcp, 8080/tcp, 0.0.0.0:2023-2024->2023-2024/tcp, 0.0.0.0:8126->8126/tcp, 8125/tcp, 0.0.0.0:8125->8125/udp   graphite

This starts a Docker container named: graphite. Please also note that you can freely remap container port to any host port in case of corresponding port is already occupied on the host.

Includes the following components

• Nginx – reverse proxies the graphite dashboard
• Graphite – front-end dashboard
• Carbon – back-end
• Statsd – UDP based back-end proxy

Mapped Ports

By default, statsd listens on the UDP port 8125. If you want it to listen on the TCP port 8125 instead, you can set the environment variable STATSD_INTERFACE to tcp when running the container.

Step 3: Accessing the Graphite Web interface

When everything is running fine, access Graphite web dashboard by using the server IP address or port number and the port mapped on the host to container’s port 80:

The default logins creds are:

Username: root
Password: root

Change this login credentials after the first login at http://ip-address/admin/auth/user/1/

Provide desired root password and confirm it.

You can now log out and re-login to confirm that the new password is working.

Method 2: Installing Graphite on Ubuntu 18.04 From apt packages

Now that we’ve covered the basics of Graphite, let’s proceed to the installation. Follow the steps below to get Graphite running on your Ubuntu 18.04 server.

Step 1: Update system packages:

The first step in the installation is to ensure that all system packages are updated:

sudo apt update
sudo apt -y upgrade

Consider rebooting the server after an upgrade

sudo reboot

Step 2: Installing Graphite on Ubuntu 18.04

Once the server is up from reboot, proceed to install graphite packages. We will install Graphite web application and carbon, which stores the data in Graphite’s specialized database.

sudo apt-get install graphite-web graphite-carbon

Wait for the packages to finish installing then proceed to the next step.

$ apt policy graphite-web graphite-carbon 
graphite-web:
  Installed: 1.0.2+debian-2
  Candidate: 1.0.2+debian-2
  Version table:
.................................
graphite-carbon:
  Installed: 1.0.2-1
  Candidate: 1.0.2-1
  Version table:
..................................

Step 3: Install and configure MariaDB Database server (Optional)

By default, Graphites uses an SQLite database to store Django models such as saved graphs, dashboards, user preferences, etc. Note that Metric data is not stored here. In this guide, we will use the MariaDB database server. Check our guide on how to Install MariaDB on Ubuntu 18.04.

Once the database server is up and ready, create a database for Graphite. Login to your database server as root user and create a database for graphite:

$ mysql -u root -p

CREATE DATABASE graphite;
GRANT ALL PRIVILEGES ON graphite.* TO graphite IDENTIFIED BY "StrongP@Ssword";
FLUSH PRIVILEGES;
QUIT;

Once the database is ready, continue with step 4.

Step 4: Configure Graphite & Carbon

Once you have installed Graphite packages, proceed to the configuration:

sudo vim /etc/graphite/local_settings.py

Set timezone:

TIME_ZONE = "Africa/Nairobi"

Set a secret key used for salting of hashes used in auth tokens. You can generate one using uuidgen

SECRET_KEY = 'eihi8dee4Otae0i'

Please set this to a long, random unique string to use as a secret key. For the database section, comment below section:

DATABASES = {
    'default': {
        'NAME': '/var/lib/graphite/graphite.db',
        'ENGINE': 'django.db.backends.sqlite3',
        'USER': '',
        'PASSWORD': '',
        'HOST': '',
        'PORT': ''
    }
}

Then add configuration for MySQL / MariaDB

DATABASES = {
    'default': {
        'NAME': 'graphite',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'graphite',
        'PASSWORD': 'StrongP@Ssword',
        'HOST': '127.0.0.1',
        'PORT': ''
    }
}

Install pymysql Python module

sudo apt -y install python3-pymysql python-pymysql

Now sync the database to have proper structure:

$ sudo graphite-manage migrate auth

Operations to perform:
  Apply all migrations: auth
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK

 Enable carbon caching:

$ sudo vim /etc/default/graphite-carbon
CARBON_CACHE_ENABLED=true

Enable Log rotation:

$ sudo vim /etc/carbon/carbon.conf
ENABLE_LOGROTATION = True

start Carbon service/ St to start on boot with the following commands:

sudo systemctl start carbon-cache
sudo systemctl enable carbon-cache

Step 5: Configuring the Webapp

We have our setup ready, the only thing we need to do is configure the web application.

Configure nginx

Install nginx and gunicorn.

sudo apt -y install nginx gunicorn

Create log files for nginx when serving Graphite.

sudo touch /var/log/nginx/{graphite.access.log,graphite.error.log}
sudo chmod 640 /var/log/nginx/graphite.*
sudo chown www-data:www-data /var/log/nginx/graphite.*

Create Nginx site configuration file for Graphite.

sudo nano /etc/nginx/conf.d/graphite.conf

The contents that should be added to file is:

upstream graphite {
    server 127.0.0.1:8080 fail_timeout=0;
}

server {
    listen 80 default_server;

    server_name graphite.example.com;

    root /opt/graphite/webapp;

    access_log /var/log/nginx/graphite.access.log;
    error_log  /var/log/nginx/graphite.error.log;

    location = /favicon.ico {
        return 204;
    }

    # serve static content from the "content" directory
    location /static {
        alias /opt/graphite/webapp/content;
        expires max;
    }

    location / {
        try_files $uri @graphite;
    }

    location @graphite {
        proxy_pass_header Server;
        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_connect_timeout 10;
        proxy_read_timeout 10;
        proxy_pass http://graphite;
    }
}

Change graphite.example.com to match your Graphite server hostname.

Validate nginx configurations.

$ nginx  -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Start Nginx service.

sudo systemctl restart nginx