Learn how to – How To Use Multi-Branch Pipeline in Jenkins.

In many tech organizations that have embraced DevOps culture and tooling, one of the tools that you are probably going to get is a version control system (VCS) and a DevOps engine such as Jenkins, GitLab CI/CD, Travis or any other implementation that uniquely works for them. For the VCS, it may be GitHub, BitBucket or an on-prem solution such as GitLab.

The idea is to automatically pull, build and deploy code once pushes, merges or any other trigger is sensed by the DevOps engine. Recently, we had a challenge that required a solution which would keep checking for any changes in the various branches in a VCS solution which was GitLab in this example. We figured that we needed a multi-branch pipeline so that each branch can be served equally and deployed to the right environment accordingly.

For that, we took advantage of a multi-branch pipeline plugin and is the basis of this guide today so that anyone who would wish to implement the same can use it or add more insight on how the same can be improved. Here we go with the solution we had for our use-case. In case you do not have Jenkins installed, you can use this how to install Jenkins on CentOS 8 guide to get it running as soon as possible.

Step 1: Install Multi-Branch Pipeline

In case your Jenkins instance does not have a Multibranch Pipeline Plugin, then you should consider installing it though it usually comes by default in your installation. There is a Multibranch Pipeline Plugin you will need to install so that you can be able to accomplish this goal. Simply login to Jenkins and proceed to install tools that will allow us to connect and communicate with GitLab’s branches for the repositories created. Luckily, there is an amazing plugin ready for you to install and configure. Head over to your Jenkins Server Web portal, click on “Manage Jenkins” > “Manage Plugins” > Click on the “Available tab” then search for “Pipeline: Multibranch“. The screenshots for the above steps are shared below.

Manage plugins

Available

For this example the Pipeline: Multibranch is already installed by default so we will not find it in the available tab.

Step 2: Install Multibranch Scan Webhook Trigger

In order to add triggers or webhook triggers to our Multi-branch Pipeline, we need another plugin. Yeah, I know, but this will be quick. The plugin is known as Multibranch Scan Webhook Trigger. To get this done, simply follow the steps like we did in our first step as follows. Login to Jenkins, click on “Manage Jenkins” > “Manage Plugins” > Click on the “Available tab” then search for “Multibranch Scan Webhook Trigger“. It should show up as shown below.

Manage Plugins

Available

Click on “install without restart“

It should successfully installed as follows:

Step 3: Add a new Multi-Branch Pipeline Item

When the required plugins are installed as shared above, you will now be able to access the multi-branch feature shown below.

With this, we can be able to connect to your VCS solution such as GitLab in this example and Jenkins can now be able to “see” all of the branches in a particular repository therein. To create a new multi-branch pipeline, login to Jenkins, then on the left menu pane click on “New Item“.

Once that is clicked you will be ushered to the normal area where you have to choose an item name and select the right item (Freestyle project, Maven Project, Pipeplie, Multibranch Pipeline etc).

In this example, give your item a name then scroll down and choose Multibranch Pipeline item. You will be ushered into your Multibranch Pipeline area where you will be expected to add all the details required to make the Multibranch Pipeline complete. Give it a “Display Name“, a “Description” if you want then choose the source control type which is Git in this example.

We shall proceed to add the URL of your GitLab repository which I presume can be reached by your Jenkins instance. Once that is done, add credentials that can log in to GitLab as follows:

Click on the “Add” drop down menu then choose “Jenkins“.

That will open the “Jenkins Credentials Provider: Jenkins” where you can add the right username and password that can connect to your repository successfully.

Once you have populated the fields, click “Add” and you will be back to the “New-Multi-Branch” item area. Under credentials, click on the down arrow and choose the new credential we just added.

After that is done, scroll down to the “Build Configuration” space where we will be specifying the file we want our Jenkins to read once it has cloned the sources from GitLab. You can specify any other file if you do not like the default “Jenkinsfile“. For this example, we will retain the default though.

Once that is done, scroll to the next tab, that is “Scan Multibranch Pipeline Triggers“. Here is where the plugin we installed in Step 2 comes to make our work beautiful. Here, you will see two options: “Periodically if not otherwise run” and “Scan by webhook“.

As you can guess, we will be going for the “Scan by webhook” option as shown above. Click on the check box which will reveal a text area where you will input a “Trigger token” as below.

This token will be added to the GitLab webhook side as well. Input a good token and we should be good to go. Click on “Apply” and then “Save“.

Step 4: Create a webhook in Gitlab and Test

Once we are done adding all of the details on our Multibranch Pipeline, we are ready to add a webhook so that git pushes, merges, tags or any other desirable action by the developer will trigger an automatic build and deployment courtesy of Jenkins. Head over to your GitLab, Login and drive into your project where repository we added in Jenkins resides. On the left pane, hover over “Settings” menu which will reveal other items right beside it. If you see “Integrations” as part of the items, click on it.

And you will be ushered to the space where we will add the webhook as illustrated below.

In the URL field, this is where you have to be very attentive. Go to he multi-branch pipeline we have just configured and copy its url available in the browser. For this example, it is “http://172.17.64.128:8080/job/New-Multi-Branch/“. On this url, change the word “job” to “project” as follows: “http://172.17.64.128:8080/project/New-Multi-Branch/“. As you can see, “job” has been replaced with “project“. Copy the modified url and paste it on the URL field in GitLab as shown below.

Under Token, as you have already guessed, this is the same token we added in Jenkins’ side. In our example, it was “seel0sou7bohT%o” as verified by the screenshot taken earlier below.

After that is added, you can add various “Triggers” depending on your environment. Choose the ones that you prefer then click “Add Webhook“. And we should be done with that.

Step 5: Update your Jenkinsfile

In our Jenkinsfile, we are going to add conditions that will check which branch has been updated and then take relevant action depending on the environment. For example, the “test” branch can be deploying to test cluster which the “production” branch can be deploying to the production cluster. You can add simple conditions like the one shared in the snippet below.

        stage('Deploy in Test env') {
            when {
                branch 'test'
            }            
        steps {
                sh "kubectl apply -f deployment.yaml"
            }
        }

        stage('Deploy in Prod env') {
            when {
                branch 'prod'
            }
            steps {
                sh "kubectl apply -f deployment.yaml"
            }
        }

Above, the various deployments can have the right namespaces or anything that will match with the right environment. It is just an example of how the multibranch can be utilised. There must be better ways this can be done which is what we are looking for so that others can benefit as well. Let us know if there is a better way to handle this.

Step 6: Scan Webhook Trigger

Once the above steps have been successfully configured, our pipeline should be ready. You can do a dry run to see if your pipeline works by clicking on “Scan Multibranch Pipeline now” then check on the “”Scan Multibranch Pipeline Log” just below it to check what went right or wrong.

It will check if all criteria is met such as the availability of Jenkinsfile or the file you mentioned in the respective branches. Sample log is shared below.

If everything goes well, the multi-branch pipeline should start running and display all of the branches in your repository. An example is illustrated below.

And our pipeline should be good to go. Once you push, merge or do other actions matching your triggers in GitLab, Jenkins should start building and deploying your application.

Concluding Remarks

The guide was all about using multi-branch pipeline in your project instead of using the single pipeline feature that is normally used. With a multi-branch pipeline, you get the freedom to deploy code in whichever branch to the right environment without much hassle. You do the configuration once and the rest is handled by Jenkins. We hope the guide was as informative as possible. In case of any improvements or better ways to handle a step above, kindly let us know as we continue learning from one another.

Video Courses For Learning Jenkins:

• Jenkins, From Zero To Hero: Become a DevOps Jenkins Master
• Learn DevOps: CI/CD with Jenkins using Pipelines and Docker
• Jenkins 2 Bootcamp: Fully Automate Builds to Deployment
• DevOps Project: CI/CD with Jenkins Ansible Docker Kubernetes

Otherwise, we continue to appreciate your readership, your feedback and relentless support. Thank you guys! Other guides you might enjoy include the following:

How To Integrate SonarQube with Jenkins

How to Configure Jenkins FreeIPA LDAP Authentication

How to Install Jenkins on CentOS 8 / RHEL 8

Install and Configure GitLab CE on Debian 10 (Buster)

Install and Configure GitLab on CentOS 8 / RHEL 8

How to Configure GitLab FreeIPA LDAP Authentication