Learn how to – Deploy Graylog Server using Ansible on Ubuntu/Debian/CentOS

Learn how to – Deploy Graylog Server using Ansible on Ubuntu/Debian/CentOS.

Welcome to this guide on how to deploy the Graylog Server on Ubuntu/Debian/CentOS Linux system with the Ansible role. Graylog is a free and open-source log aggregation and management tool. It is used to collect, analyze, visualize logs and send alerts based on the logs. The Graylog server is made up of 4 components which are:

• Graylog Server– The server that passes logs for visualization on the web Interface.
• MongoDB – This is a database server used to store the data and configurations.
• ElasticSearch– this is the log analysis tool for the Graylog Server.
• Java – provides the runtime environment for ElasticSearch.

All these tools work together to realize the main goal of log aggregation and management. Using Ansible to deploy the Graylog Server makes it easy to automate the recursive task. Ansible just like other orchestration tools needs to be installed on the control node to be able to manage the attached nodes.

This installation with ansible currently works in the following systems:

• CentOS / RHEL: CentOS 7/8, RHEL 7/8
• Debian: Debian 10 / Debian 9
• Ubuntu: Ubuntu 20.04 / Ubuntu 18.04

If you prefer Puppet installation method check our our recent guide in below link:

• Automate Graylog Server installation using Puppet

Let’s dive in and see how we can achieve this.

Step 1. Install and Configure Ansible on Workstation

Ansible can be installed on the control node using several methods. The easiest way to install it on any Linux distribution is using PIP. Before you proceed with this method, you need Python and PIP installed.

##On Ubuntu
sudo apt update
sudo apt install python3 python3-pip -y

##On CentOS
sudo yum install python3 python3-pip -y

Now use the installed PIP to install Ansible.

sudo pip3 install ansible

On macOS you can use brew to install Ansible

brew install ansible

Verify the installation.

$ ansible --version
ansible [core 2.12.4]
  config file = None
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /home/ubuntu/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0]
  jinja version = 2.10.1
  libyaml = True

Ansible can as well be installed from the default package repositories:

##On Ubuntu / Debian
sudo apt install ansible

##On CentOS
sudo yum install epel-release
sudo yum install ansible

Create the Ansible Hosts Inventory file

This file consists of nodes managed by the Ansible control node.

$ sudo vim /etc/ansible/hosts
[graylog]
192.168.205.9 ansible_ssh_user=username

Replace “username” in the command below with the username on the managed node. Generate and copy the SSH keys of the managed node to the control node.

ssh-keygen -t rsa
ssh-copy-id -i ~/.ssh/id_rsa.pub username@192.168.205.9

This will allow you to control the added nodes without a password. Test if this works:

$ ansible -m ping all
192.168.205.9 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}

Step 2. Install Graylog Ansible Role

The Graylog Ansible role allows one to install and configure Graylog. This can be installed using the command:

$ ansible-galaxy install graylog2.graylog
Starting galaxy role install process
- downloading role 'graylog', owned by graylog2
- downloading role from https://github.com/Graylog2/graylog-ansible-role/archive/3.3.7.tar.gz
- extracting graylog2.graylog to /Users/jkmutai/.ansible/roles/graylog2.graylog
- graylog2.graylog (3.3.7) was installed successfully
- adding dependency: lean_delivery.java (7.1.0)
- adding dependency: elastic.elasticsearch (main)
- downloading role 'java', owned by lean_delivery
- downloading role from https://github.com/lean-delivery/ansible-role-java/archive/7.1.0.tar.gz
- extracting lean_delivery.java to /Users/jkmutai/.ansible/roles/lean_delivery.java
- lean_delivery.java (7.1.0) was installed successfully
- extracting elastic.elasticsearch to /Users/jkmutai/.ansible/roles/elastic.elasticsearch
- elastic.elasticsearch (main) was installed successfully

From the above output, you will notice that the below dependencies have been installed.

• Java
• Elasticsearch

Verify if the Graylog Ansible role dependencies have been installed using the command:

ansible-galaxy install -r ~/.ansible/roles/graylog2.graylog/requirements.yml

Remember to replace ~/.ansible/roles/graylog2.graylog/ with the correct path of your Graylog Ansible role.

Step 3. Deploy Graylog Server using Ansible Roles

Create a playbook YAML for a single-instance Graylog server installation.

vim graylog-playbook.yaml

The file will contain the below lines:

- hosts: "graylog"
  remote_user: "username"
  become: True
  vars:
    #Elasticsearch vars
    es_major_version: "7.x"
    es_version: "7.10.2"
    es_enable_xpack: False
    es_instance_name: "graylog"
    es_heap_size: "1g"
    es_config:
      node.name: "graylog"
      cluster.name: "graylog"
      http.port: 9200
      transport.tcp.port: 9300
      network.host: "127.0.0.1"
      discovery.seed_hosts: "localhost:9300"
      cluster.initial_master_nodes: "graylog"
    oss_version: True
    es_action_auto_create_index: False

    #Graylog vars
    graylog_version: 4.2
    graylog_install_java: True
    graylog_password_secret: "ncc4jque0VvGImadZ7jzX26NrESt30dY4U4nNfZWAXubcvUGDKnMjbC4eEAU0KcfWX6CDk4ME80CrYPP9ErpvyFPXc2H2xKf" # Insert your own here. Generate with: pwgen -s 96 1
    graylog_root_password_sha2: "434e27fac24a15cbf8b160b7b28c143a67d9e6939cbb388874e066e16cb32d75" # Insert your own root_password_sha2 here.
    graylog_http_bind_address: "{{ ansible_default_ipv4.address }}:9000"
    graylog_http_publish_uri: "http://{{ ansible_default_ipv4.address }}:9000/"
    graylog_http_external_uri: "http://{{ ansible_default_ipv4.address }}:9000/"

  roles:
    - role: "graylog2.graylog"
      tags:
        - "graylog"

Remember to replace the graylog_password_secret generated with the command:

$ pwgen -N 1 -s 96
ncc4jque0VvGImadZ7jzX26NrESt30dY4U4nNfZWAXubcvUGDKnMjbC4eEAU0KcfWX6CDk4ME80CrYPP9ErpvyFPXc2H2xKf

Also, replace the graylog_root_password_sha2 generated using the command:

$ echo -n "Enter Password: " && head -1 </dev/stdin | tr -d 'n' | sha256sum | cut -d" " -f1
Enter Password: Str0ngPassw0rd
434e27fac24a15cbf8b160b7b28c143a67d9e6939cbb388874e066e16cb32d75

Now deploy the Graylog server.

ansible-playbook graylog-playbook.yaml

## With custom inventory file ###
ansible-playbook graylog-playbook.yaml -i myinventory

Sample Output:

If the above command fails with “Missing sudo password“}, you need to edit the /etc/sudoers file on the managed host and allow the remote user to execute sudo commands without a password.

After the command, all the 3 services (MongoDB, Elasticsearch, and Graylog) should be running on the managed node:

Verify if Elasticsearch is running:

$ curl -X GET localhost:9200
{
  "name" : "graylog",
  "cluster_name" : "graylog",
  "cluster_uuid" : "O6qVFbgjQvmTDZ3j-cAVSg",
  "version" : {
    "number" : "7.10.2",
    "build_flavor" : "oss",
    "build_type" : "rpm",
    "build_hash" : "747e1cc71def077253878a59143c1f785afa92b9",
    "build_date" : "2021-01-13T00:42:12.435326Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Deploying a Graylog cluster using Ansible (Optional for HA)

It is also possible to deploy a Graylog cluster with more Elasticsearch and Graylog instances. The below example includes 3 Elasticsearch and 3 Graylog instances.

Begin by deploying the Elasticsearch cluster:

- hosts: "elasticsearch"
  vars:
    es_major_version: "7.x"
    es_version: "7.10.2"
    es_enable_xpack: False
    es_instance_name: "graylog"
    es_heap_size: "1g"
    es_config:
      node.name: "{{ ansible_hostname }}"
      cluster.name: "graylog"
      http.port: 9200
      transport.port: 9300
      network.host: "0.0.0.0"
      discovery.seed_hosts: "elasticsearch01:9300, elasticsearch02:9300, elasticsearch03:9300"
      cluster.initial_master_nodes: "elasticsearch01, elasticsearch02, elasticsearch03"
    oss_version: True
    es_action_auto_create_index: False

  roles:
    - role: "elastic.elasticsearch"

Then proceed and deploy the MongoDB instances:

- hosts: "graylog"
  vars:
    mongodb_version: "4.4"
    bind_ip: "0.0.0.0"
    repl_set_name: "rs0"
    authorization: "disabled"
  roles:
    - community.mongodb.mongodb_repository
    - community.mongodb.mongodb_mongod
  tasks:
    - name: "Start MongoDB"
      service:
        name: "mongod"
        state: "started"
        enabled: "yes"

- hosts: "graylog01"
  tasks:
    - name: "Install PyMongo"
      apt:
        update_cache: yes
        name: "python3-pymongo"
        state: "latest"
    - name: Configure replicaset
      community.mongodb.mongodb_replicaset:
        login_host: "localhost"
        replica_set: "rs0"
        members:
        - graylog01
        - graylog02
        - graylog03

Finally, deploy the Graylog instance:

- hosts: "graylog"
  vars:
    graylog_is_master: "{{ True if ansible_hostname == 'graylog01' else False }}"
    graylog_version: 4.2
    graylog_install_java: False
    graylog_install_elasticsearch: False
    graylog_install_mongodb: False
    graylog_password_secret: "" # Insert your own here. Generate with: pwgen -s 96 1
    graylog_root_password_sha2: "" # Insert your own root_password_sha2 here.
    graylog_http_bind_address: "{{ ansible_default_ipv4.address }}:9000"
    graylog_http_publish_uri: "http://{{ ansible_default_ipv4.address }}:9000/"
    graylog_http_external_uri: "http://{{ ansible_default_ipv4.address }}:9000/"
    graylog_elasticsearch_hosts: "http://elasticsearch01:9200,http://elasticsearch02:9200,http://elasticsearch03:9200"
    graylog_mongodb_uri: "mongodb://graylog01:27017,graylog02:27017,graylog03:27017/graylog"

  roles:
    - role: "graylog2.graylog"

With that, you will have a Graylog cluster with 3 Elasticsearch and 3 Graylog instances.

Step 4. Access Graylog Web interface

Now allow port 9000 through the firewall:

##For Firewalld
sudo firewall-cmd --add-port=9000/tcp --permanent
sudo firewall-cmd --reload

##For UFW
sudo ufw allow 9000/tcp

Proceed and access the Graylog Web interface using the URL http://IP_adrress:9000

Login using the default user admin and password set with the graylog_root_password_sha2. On successful authentication, you should be able to access the below dashboard.

Now proceed and configure the inputs, create dashboards visualize logs on the Graylog web interface.

Verdict

We have successfully deployed the Graylog Server on Ubuntu / CentOS with the Ansible role. We can all agree that ansible makes it easy to run repetitive tasks on multiple servers. I hope this was significant to you.

Related posts:

• How To Install Ansible AWX on Ubuntu
• Install Ansible AWX on CentOS 8 / Rocky Linux 8
• Manage Logs with Graylog server on Ubuntu