Learn how to - Monitor Your Applications Health and Alerts using Gatus

Learn how to – Monitor Your Applications Health and Alerts using Gatus.

Monitoring can be defined as collecting and analyzing data about the performance, availability, and usage of an application or system. It forms an integral part of maintaining the health and reliability of any application, as it provides insights into how the application is performing, allowing proactive identification and mitigation of issues before they become critical. It can also be used to provide valuable insights for future application development and architecture decisions.

There are several application monitoring tools. The most popular ones are Splunk, Prometheus, Nagios, AppDynamics, Dynatrace, New Relic, Datadog etc. Today, we will learn about a tool known as Gatus.

What is Gatus?

Gatus is an open-source, cloud-native application monitoring tool designed to help teams monitor their microservices-based architectures. This provides developer-oriented tool provides real-time visibility into the health and performance of microservices and their dependencies using HTTP, ICMP, TCP, and even DNS queries as well as evaluates the result of said queries by using a list of conditions on values like the status code, the response time, the certificate expiration, the body and many others. To add on that, each of the health checks can be paired with alerting via Teams, Slack, PagerDuty, Discord, Twilio etc allowing teams to quickly detect and resolve issues before they impact end-users.

It comes with a unified dashboard, making it easy to understand the overall health of the system. It is also built with modern technologies that include Kubernetes, Prometheus, and Grafana.

The amazing features provided by Gatus are:

Very easy to configure: The configuration is designed to be as readable as possible, and it is also easy to add a new service or a new endpoint to monitor.
Low resource consumption: As with most Go applications, the resource footprint that this application requires is negligibly small.
Highly flexible health check conditions: checking the response status may be enough for some scenarios, Gatus goes much further and allows you to add conditions on the response time, the response body and even the IP address.
Ability to use Gatus for user acceptance tests: you can leverage Gatus to create automated user acceptance tests.
Alerting: Although Gatus has a pretty visual dashboard that is useful to keep track of the state of your application(s), you probably don’t want to stare at it all day. Therefore, notifications via Slack, Mattermost, Messagebird, PagerDuty, Twilio, Google chat and Teams are supported out of the box with the ability to configure a custom alerting provider for any needs you might have, whether it be a different provider or a custom application that manages automated rollbacks.

Join me in this guide as we learn how to monitor Applications’ Health and Alerts using Gatus.

Getting Started

The easiest way to spin Gatus is by running it in Docker containers. This will require you to have the Docker Engine installed. To achieve this, get help from the below guide:

How To Install Docker CE on Linux Systems

Ensure that your system user is added to the Docker group:

sudo usermod -aG docker $USER
newgrp docker

You might also need to have Docker Compose installed. This can be done by following the below guide:

How To Install Docker Compose on Linux

#1. Create Gatus configuration file

To spin Gatus in a container, we need a configuration file that explains what Gatus will be monitoring and where alerts can be sent. For that reason, we will create a dedicated directory:

mkdir gatus && cd gatus/

Now in this directory, we will create a simple configuration file with the desired variables.

vim config.yaml

In the file, add the below lines and replace them where required:

alerting:
  email:
    from: "user@example.com"
    username: "user@example.com"
    password: "my_email_password"
    host: "mail.example.com"
    port: 587
    to: "recipient1@example.com,recipient2@example.com"
    # You can also add group-specific to keys, which will
    # override the to key above for the specified groups
    overrides:
      - group: "core"
        to: "recipient3@example.com,recipient4@example.com"

endpoints:
  - name: Example Site  # Name of your endpoint, can be anything
    url: "https://example.org/"
    interval: 60s  # Duration to wait between every status check (default: 60s)
    conditions:
      - "[STATUS] == 200"  # Status must be 200
      - "[BODY] == pat(*<h1>Example Domain</h1>*)" # Body must contain the specified header
    alerts:
    - type: email
      failure-threshold: 3
      success-threshold: 2

  - name: Gatus site
    url: "http://gatus.learnhowto.org/health"
    interval: 5m
    conditions:
      - "[STATUS] == 200"
      - "[BODY].status == UP"     # The json path "$.status" must be equal to UP
      - "[RESPONSE_TIME] < 300"   # Response time must be under 300ms
    client:
     dns-resolver: "tcp://192.168.205.20:53"
     insecure: true
    alerts:
    - type: email
      failure-threshold: 3
      success-threshold: 2

storage:
  type: sqlite
  path: data.db

In the above configuration, we have used email for alerting, you need to provide:

alerting.email.from: Email used to send the alert
alerting.email.username: Username of the SMTP server used to send the alert. If empty, it uses alerting.email.from
alerting.email.password: Password of the SMTP server used to send the alert. There is an illustration of how to obtain an application password form Google.
alerting.email.host: Host of the mail server (e.g. smtp.gmail.com)
alerting.email.port: Port the mail server is listening to (e.g. 587)
alerting.email.to: Email(s) to send the alerts to
alerting.email.overrides: List of overrides that may be prioritized over the default configuration

We have also configured endpoints/clients to be monitored. Here, there are variables that include:

client.insecure: Whether to skip verifying the server’s certificate chain and host name. Default value False
client.ignore-redirect: Whether to ignore redirects (true) or follow them (false, default).
client.dns-resolver: Override the DNS resolver using the format {proto}://{host}:{port}.
client.oauth2: OAuth2 client configuration.
client.oauth2.token-url: The token endpoint URL
client.oauth2.client-id: The client id which should be used for the Client credentials flow
client.oauth2.client-secret: The client secret which should be used for the Client credentials flow
client.oauth2.scopes[]: A list of scopes which should be used for the Client credentials flow.

You can also use a custom DNS for the endpoint. For example, I have a DNS server set up on 192.168.205.20, SEO my config will be:

endpoints:
  - name: with-custom-dns-resolver
    url: "https://your.health.api/health"
    client:
      dns-resolver: "tcp://192.168.205.20:53"
    conditions:
      - "[STATUS] == 200"

To use client.oauth2 configuration to query a backend API with a Bearer token, use the below syntax:

endpoints:
  - name: with-custom-oauth2
    url: "https://your.health.api/health"
    client:
      oauth2:
        token-url: https://your-token-server/token
        client-id: 00000000-0000-0000-0000-000000000000
        client-secret: your-client-secret
        scopes: ['https://your.health.api/.default']
    conditions:
      - "[STATUS] == 200"

We also have a Storage configuration. The variables that can be used here are:

storage.path: Path to persist the data in. Only supported for types sqlite and postgres.
storage.type: Type of storage. Valid types: memory, sqlite, postgres.
storage.caching: Whether to use write-through caching. Improves loading time for large dashboards. Only supported if storage.type is sqlite or postgres

The default storage type is memory. For this guide, we have used sqlite. But you can also use postgres with the below syntax:

storage:
  type: postgres
  path: "postgres://user:password@127.0.0.1:5432/gatus?sslmode=disable"

See more on how to configure Postgres by following this link examples/docker-compose-postgres-storage. More and more configurations can be made for Gatus. See more on how to configure Gatus.

Configure Gmail Receiver for Gatus Alerts

For this guide, I will use Gmail for the alerts. For that reason, you need to have an App password for AlertManager. This can be obtained from Gmail account under Security > App passwords

Monitor Your Applications Health and Alerts using Gatus 1

While here, click on App passwords, and select the applications as Mail from the drop-down menu.

Monitor Your Applications Health and Alerts using Gatus 2

On the device drop-down menu, select Other (Custom name) and provide your preferred name:

Monitor Your Applications Health and Alerts using Gatus 3

Finally, generate the app password to be used for Gatus alerting:

Monitor Your Applications Health and Alerts using Gatus 4

Now provide the password in your Gatus config file at my_email_password.

Monitor Your Applications Health and Alerts using Gatus 5

Once the desired configurations have been made, start the container.

#2. Running Gatus in Docker Containers

Now after replacing all the required values in the configuration for alerting, endpoint and storage, we can spin the container using the configuration YAML in two ways:

Using Docker CLI
Using Docker Compose

Option 1 – Running Gatus in Docker Containers using Docker CLI

With the config file, you can run the Gatus container with the command:

docker run  -d --name gatus -p 8080:8080 
    --mount type=bind,source="$(pwd)"/config.yaml,target=/config/config.yaml 
    twinproduction/gatus:latest

This will run Gatus and expose port 8080. Verify with the command:

$ docker ps
CONTAINER ID   IMAGE                         COMMAND    CREATED          STATUS          PORTS                                       NAMES
3a45d3972fbf   twinproduction/gatus:latest   "/gatus"   11 seconds ago   Up 10 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp   gatus

Option 2 – Running Gatus in Docker Containers using Docker Compose

Docker Compose is the recommended way of spinning containers in Production environments. This requires you to create a docker-compose file:

vim docker-compose.yaml

In the file, add the below lines:

version: '3.9'
services:
  app:
    container_name: gatus
    image: twinproduction/gatus:latest
    restart: always
    ports:
      - 8080:8080
    volumes:
      - type: bind
        source: ./config.yaml
        target: /config/config.yaml
      - type: bind
        source: ./db
        target: /db

Create the specified path:

mkdir db
sudo chmod 775 -R db
sudo chown -R $USER:docker db

On Rhel-based systems, configure SELinux as shown:

sudo setenforce 0
sudo sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config

Now save the file and start the container with the command:

docker-compose up -d

Verify if the container is running:

$ docker compose ps
NAME                IMAGE                         COMMAND             SERVICE             CREATED             STATUS              PORTS
gatus               twinproduction/gatus:latest   "/gatus"            app                 17 seconds ago      Up 16 seconds       0.0.0.0:8080->8080/tcp, :::8080->8080/tcp

You can now access Gatus with the URL http://IP_Address:8080

Monitor Your Applications Health and Alerts using Gatus 6

You can also view the container logs using the below command:

Monitor Your Applications Health and Alerts using Gatus 7

As configured, after failing 3 times in a row, an alert is triggered.

Monitor Your Applications Health and Alerts using Gatus 8

The Gatus site is unavailable on the specified URL because we have not configured the Nginx proxy as below.

#3. (Optional) Configure Nginx Proxy for Gatus

It is always easier to access a webpage with a domain name than the server IP. For this guide, we will configure Nginx reverse proxy for Gatus.

First, ensure that Nginx has been installed on your host:

##For Debian/Ubuntu
sudo apt update && sudo apt install nginx

##For CentOS/Rocky Linux/Alma Linux/RedHat
sudo yum install nginx -y

Once installed, create a virtual host file:

sudo vim /etc/nginx/conf.d/gatus.conf

In the file, add the below lines

server {
    listen 80;
    server_tokens off;
    server_name gatus.learnhowto.org;

    if ($host !~* ^(gatus.learnhowto.org)$) {
        return 444;
    }

    location / {
        proxy_redirect     off;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Host $server_name;
        proxy_pass http://127.0.0.1:8080;
    }
}

Save the file and restart the Nginx:

sudo systemctl restart nginx
sudo systemctl enable nginx

#4. Access Gatus Web UI

Now access the Gatus Web dashboard using the URL http://domain_name

Monitor Your Applications Health and Alerts using Gatus 9

Now after a few minutes, we should have the Gatus site available. You can view more details by clicking on it:

Monitor Your Applications Health and Alerts using Gatus 10

Conclusion

That marks the end of this guide on how to monitor your applications’ Health and Alerts using Gatus. We have seen how vital this tool can be. Aside from email, Gatus can be configured to provide alerts via Slack, Mattermost, Messagebird, PagerDuty, Twilio, Github, Google Chat, Teams etc. I hope this was informative.

See more:

Cart

Learn how to – Monitor Your Applications Health and Alerts using Gatus