Learn how to – Install Sensu Monitoring on Rocky Linux 9|AlmaLinux 9

Learn how to – Install Sensu Monitoring on Rocky Linux 9|AlmaLinux 9.

System monitoring provides observations and reports that help manage applications and servers in an infrastructure. It involves continuous analysis of the CPU, memory, bandwidth, application performance, switches, and other network components. Monitoring is done almost everywhere nowadays since most businesses rely on applications to run. It mainly helps identify errors on time and act accordingly. There are several monitoring stacks in the market today, some of these stacks are Elastic Stack, Dynatrace, Zenoss, Prometheus, New Relic, N-able RMM, Datadog Infrastructure Monitoring, PRTG Network Monitor, Nagios, Zabbix e.t.c.

Sensu is an open-source solution that provides observability at a scale. It is capable of monitoring private, public, hybrid cloud, and container-based infrastructure, therefore, eliminating the challenges associated with the mix of these infrastructures. It has a simple, scalable, and malleable design for easy customization and monitoring.

Sensu works using the client-server model. The server schedules and publishes the checks to be performed by the client subscriptions. Subscriptions are a list of tasks to be executed on the sense client. RabbitMQ is used in Sensu as a message bus to send check requests and results. The obtained data is then stored in a persistent storage, the Redis database. The client machine has the Sensu client software which is responsible for client registration, subscriptions, and checking schedules from the server.

Below is an illustration of the Sensu Monitoring architecture.

Features of Sensu Monitoring Stack

The Sensu Monitoring Stack offers a lot of features, some ot them are:

• Plugins support: with its open architecture, Sensu integrates with several tools and services that help extend its functionality
• Intuitive API: The Sensu API with web interfaces and the sensuctl command-line tool allows one to create checks, register entities, and manage configs. The WebUI provides a view of utilities and checks
• Commercial support: Sensu is a commercial product, based on an open-source core that is freely available under the MIT License.
• Built-in support for industry-standard tools: Other tools like Graphite plaintext protocol, InfluxDB line protocol, Nagios performance data, StatsD metrics, Prometheus Exposition Text Format, and OpenTSDB data specification are supported on Sensu
• Alerting and Notifications: this provides filtered, context-rich alerts that improve incident response
• Automatic agent registration-deregistration: this helps reduce overloaded alerts. Check subscriptions can also be automated.

This guide provides a deep demonstration of how to install Sensu Monitoring on Rocky Linux 9|AlmaLinux 9. The Sensu Monitoring Stack installation follows through 3 main sections. These 3 main segments are illustrated in the diagram below.

So we will begin!

#1. Install Sensu Backend on Rocky Linux 9|AlmaLinux 9

To install the Sensu backend, begin by adding the Sensu repositories on the Rocky Linux 9|AlmaLinux 9 system.

curl -s https://packagecloud.io/install/repositories/sensu/stable/script.rpm.sh | sudo bash

Confirmed the repository was added using the commands:

$ cat /etc/yum.repos.d/sensu_stable.repo
[sensu_stable]
name=sensu_stable
baseurl=https://packagecloud.io/sensu/stable/el/9/$basearch
repo_gpgcheck=0
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/sensu/stable/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

[sensu_stable-source]
name=sensu_stable-source
baseurl=https://packagecloud.io/sensu/stable/el/9/SRPMS
repo_gpgcheck=0
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/sensu/stable/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300

Once the repository has been added, install the Sensu Go backend.

sudo dnf install vim sensu-go-backend

Dependency Tree;

Transaction Summary
================================================================================
Install  1 Package

Total download size: 14 M
Installed size: 40 M
Is this ok [y/N]: y

#2. Configure and Start the Sensu Backend

We will download the main configuration file to /etc/sensu/backend.yml with the command:

sudo curl -L https://docs.sensu.io/sensu-go/latest/files/backend.yml -o /etc/sensu/backend.yml

You can open the file for editing using the commands below:

sudo vim /etc/sensu/backend.yml

Now start and enable the Sensu backend service:

sudo systemctl start sensu-backend
sudo systemctl enable sensu-backend

Confirm if the service is running;

$ systemctl status sensu-backend
● sensu-backend.service - The Sensu Backend service.
     Loaded: loaded (/usr/lib/systemd/system/sensu-backend.service; enabled; vendor preset: disabled)
     Active: active (running) since Wed 2022-08-31 07:58:03 CEST; 7s ago
   Main PID: 146533 (sensu-backend)
      Tasks: 7 (limit: 23441)
     Memory: 32.5M
        CPU: 142ms
     CGroup: /system.slice/sensu-backend.service
             └─146533 /usr/sbin/sensu-backend start -c /etc/sensu/backend.yml

We will then create the admin user and password by exporting the credentials as shown below:

export SENSU_BACKEND_CLUSTER_ADMIN_USERNAME=admin
export SENSU_BACKEND_CLUSTER_ADMIN_PASSWORD=StrongPassword

In the above command, replace the username and password as preferred. For the changes to apply, initialize the backend;

sensu-backend init

#3. Access the Sensu web UI

The Sensu web UI runs on port 3000 by default, It allows users to view events and alerts on Sensu. So we will allow the required ports through the firewall;

sudo firewall-cmd --add-port=3000/tcp --permanent
sudo firewall-cmd --add-port=8080/tcp --permanent
sudo firewall-cmd --add-port=8081/tcp --permanent
sudo firewall-cmd --reload

To verify if the backend is running as desired, use the cult command below to check its health.

$ curl http://127.0.0.1:8080/health
{"Alarms":null,"ClusterHealth":[{"MemberID":13195394291058371180,"MemberIDHex":"b71f75320dc06a6c","Name":"default","Err":"","Healthy":true}],"Header":{"cluster_id":2037210783374497686,"member_id":13195394291058371180,"raft_term":2}}

Access the Sensu web UI using the URL http://IP_address:3000

Login using the created credentials to access the Sensu dashboard below.

Now you can view the dashboards and events in the preferred namespace.

Currently, we do not have any events in the default namespace. So we will configure them as shown below.

#4. Install sensuctl on Rocky Linux 9|AlmaLinux 9

The sensuctlis a command line utility that allows users to manage the Sensu resources. It calls the HTTP API when creating, reading, updating, and deleting events, resources, entities e.t.c

This package resides in the Sensu repositories added to the system earlier. To install it, we will simply execute the command:

sudo yum install sensu-go-cli

Once the installation is done, we need to configure the tool to call the HTTP API. This can be done with the command bearing the syntax below:

sensuctl configure -n 
--username 'admin' 
--password 'StrongPassword' 
--namespace default 
--url 'http://127.0.0.1:8080'

In the command, replace the username, password and HTTP URL to connect to the Sensu Backend.

Now use the tool to perform the desired operations. For example, changing the password for the admin user, use the command:

sensuctl user change-password --interactive

Configure Bash and Zsh completion:

# Bash
echo 'source <(sensuctl completion bash)' | tee -a ~/.bashrc
source  ~/.bashrc

# Zsh
echo 'source <(sensuctl completion zsh)' | tee -a ~/.zshrc
source  ~/.zshrc

#5. Install and Configure Sensu Go agents

The Sensu Go agent is the software installed on the client machines. The package is available for Debian-based, RHEL/CentOS and Windows systems.

The Sensu Go agent can be installed with the commands below

• On Debian/Ubuntu

# Add the Sensu repository
curl -s https://packagecloud.io/install/repositories/sensu/stable/script.deb.sh | sudo bash

# Install the sensu-go-agent package
sudo apt-get install sensu-go-agent

• On Rhel/CentOS/Rocky Linux/Alma Linux

# Add the Sensu repository
curl -s https://packagecloud.io/install/repositories/sensu/stable/script.rpm.sh | sudo bash

# Install the sensu-go-agent package
sudo yum -y install sensu-go-agent

• On Windows

#via Chocolatey - https://chocolatey.org/install
choco install sensu-agent

Before we start and enable the service, we need to download the configuration file.

• On Linux systems

# Copy the config template from the docs
sudo curl -L https://docs.sensu.io/sensu-go/latest/files/agent.yml -o /etc/sensu/agent.yml

# Start sensu-agent using a service manager
sudo systemctl enable --now sensu-agent

• On Windows systems

# Copy the example agent config file from %ALLUSERSPROFILE%sensuconfigagent.yml.example
# (default: C:ProgramDatasensuconfigagent.yml.example) to C:ProgramDatasensuconfigagent.yml
cp C:ProgramDatasensuconfigagent.yml.example C:ProgramDatasensuconfigagent.yml

# Change to the sensusensu-agentbin directory where you installed Sensu
cd 'C:Program Filessensusensu-agentbin'

# Run the sensu-agent executable
./sensu-agent.exe

# Install and start the agent
./sensu-agent service install

If you have the Sensu agent running on a remote host, provide the Sensu backend URL in the configuration file.

sudo vim /etc/sensu/agent.yml

Provide the Sensu backend URL:

name: "node1.ownyourlife.com.ng"
namespace: "default"
backend-url:
  - "ws://192.168.205.13:8081"

Save the file and restart the service:

sudo systemctl restart sensu-agent

#6. Verify keepalive events

The keepalives can be defined as the mechanisms provided to ensure the added agents reach the Sensu backend.

To verify the added agents, use the Sensu CLI

$ sensuctl entity list
              ID                Class    OS               Subscriptions                        Last Seen             
────────────────────────────── ─────── ─────── ──────────────────────────────────── ─────────────────────────────────
  localhost.localdomain         agent   linux   entity:localhost.localdomain         2022-08-31 08:32:40 +0200 CEST  
  node1.ownyourlife.com.ng   agent   linux   entity:node1.ownyourlife.com.ng   2022-08-31 08:32:30 +0200 CEST 

It is also possible to view the added agents from the WebUI by navigating to the Entities tab in the default namespace.

Now we will send a sample event from the Sensu agent to the Sensu backend.

curl -X POST 
-H 'Content-Type: application/json' 
-d '{
  "check": {
    "metadata": {
      "name": "check-mysql-status"
    },
    "status": 1,
    "output": "could not connect to mysql"
  }
}' 
http://127.0.0.1:3031/events

The above request should create a warning in the events tab. To verify this, check the Events tab

Get detailed information about the event by clicking on it.

Now the Sensu dashboard will appear as shown below.

Closing Thoughts

This guide has provided the required knowledge on how to install Sensu Monitoring on Rocky Linux 9|AlmaLinux 9. We have also learned how to add entities and configure events on Sensu. I hope this was helpful.

Related:

• Install Grafana with Prometheus on Rocky Linux 9|AlmaLinux 9
• Install Cacti Monitoring Server on Debian
• Install Grafana on Kubernetes for Cluster Monitoring
• How To Install Zabbix on RHEL 8 | CentOS Stream 8