How to Create Dynamically Scalable Jenkins Slaves with Docker and Kubernetes

Jenkins becomes a popular and unavoidable CICD tool. Also, Jenkins is used Because of its huge availability of plugins and features. Such a tool which is in high demand will have to run N number of job N number of time with the high resource. So scaling is needed to face this resource demand. So, Vertical scaling is used by most of the IT specialist. Vertical scaling is having some issues like cost per server and complex configuration of servers. So, Scaling Jenkins by adding Jenkins slaves Statically and Dynamically. In this article, we will discuss how to create dynamically Scalable Jenkins slaves with Docker and Kubernetes.

Read How to Configure Static Jenkins Master Slave Setup.

Plan of Action

We are going to create Kubernetes template to run the Jenkins Pod as master and dynamically creatable slaves which is created by Jenkins itself.

So, first, we are going to create Jenkins master from Kubernetes itself. Then we are going to create the Jenkins template which can be created from Jenkins master’s UI.

To implement the above idea, we are going to create the following steps to implement the dynamically Scalable Jenkins slaves.

1. Jenkins Instance service
2. Jenkins Service Discovery Service
3. Creating Docker Image with required plugins
4. Jenkins Deployment
5. Configuring Jenkins Slaves
6. Test The configuration – Create Jenkins jobs and run all together

1. Jenkins Instance Service

As discussed, we are going to use Kubernetes to create Jenkins instance dynamically and provision it to The Jenkins server. In order to provision Jenkins instance, we need Jenkins Kubernetes service to create the Jenkins instance in Kubernetes pods. I have the snippet of code to create the Jenkins instance service in NodePort to route external traffic to Jenkins server. You can use ingress or external Load balancer to achieve the same.

apiVersion: v1
kind: Service
metadata:
  name: jenkins-instance
  namespace: jenkins-instance
spec:
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      name: master
  selector:
    app: master
  type: NodePort

Create a .yml file for Jenkins instance service (In our case jenkins-instance-service.yml) and create the service. In this tutorial, I am using kubectl.

kubectl create -f jenkins-instance-service.yml

This will create the Jenkins instance service and throw the below output.

service “Jenkins-instance” created 

2. Jenkins Service Discovery Service

Now, we are going to create another service for service discovery which will help us to route the external traffic to Jenkins-instance clusters. Just like Jenkins-instance service, we must create .yml file for Kubernetes to create the service discovery service. Copy the code and create jenkins-service-discovery-service.yml

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service-discovery
  namespace: jenkins
spec:
  selector:
    app: master
  ports:
    - protocol: TCP
      port: 3838
      targetPort: 3838
      name: slaves

In this, we are defining port 3838 for service discovery ports. Make your network configuration according to that.

Just like Jenkins-instance service, pass the kubectl command to create the service discovery service.

kubectl create -f jenkins-service-discovery-service.yml

This will create the service and should throw the following output.

service “jenkins-service-discovery” created 

3. Create Docker Image with required plugins

To create Kubernetes clusters, Docker image is very important. So, we are going to create the docker image and push it to your docker registry so that we can use it when we deploy the application in Kubernetes. Use the below Dockerfile to create the docker image.

from jenkins/jenkins:lts

COPY install-plugin.sh /usr/local/bin/install-plugin.sh
RUN /usr/local/bin/install-plugin.sh ssh-slaves
RUN /usr/local/bin/install-plugins.sh kubernetes

Then, from the directory of this Dockerfile, run below docker build command to create the docker image

docker build -t digitalvarys/Jenkins

Then, push the docker image created docker image by passing below command

docker push <Image ID> digitalvarys/Jenkins

Understand docker and its basic Concepts from here.

4. Jenkins-instance Deployment

Now we need to deploy Jenkins-instance so that it will be available as pods. Then, use the below .yml code to create Kubernetes deployment script (jenkins-instance-deploy.yml).

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins-instance
  namespace: jenkins-instance
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: master
    spec:
      containers:
        - image: digitalvarys/jenkins
          name: jenkins-instance
          ports:
            - containerPort: 8080
              name: jenkins-port
            - containerPort: 3838
              name: jenkins-sd-port
          env:
            - name: JAVA_OPTS
              value: -Djenkins.install.runSetupWizard=false

To deploy the jenkins-instance, run the below kubectl command.

kubectl create -f jenkins-instance-deploy.yml

Then, this will create the pods and check whether all the pods are up and running by passing following command

kubectl get pods –namespace jenkins-instance

this will throw following output

NAME READY STATUS RESTARTS AGE
jenkins-instance-5ea2b32ace 1/1 Running 0 3m

If this is running, your Jenkins instance through Kubernetes is running and all other configurations are working. Then, Make sure the two plugins mentioned in the Dockerfile is available or not.

5. Configuring Jenkins Slaves

Now, We need to configure the Jenkins slaves by configuring in the Jenkins UI itself. Follow the below steps to configure.

Go to Configure System from Manage Jenkins option and select Cloud section

From here, “Add New Cloud” and select “Kubernetes” the enter the required details of Kubernetes.

The, to get the details that need to be filled in Kubernetes form, get it by passing following command

kubectl describe pod –namespace jenkins-slave

Fill the details as mentioned in the below screenshot

Make sure you are filling Kubernetes URL, Jenkins URL, “Kubernetes Pod Template” by selecting “Add Pod Template” and finally by adding “Container Template” by selecting “Add Container”

That’s it. Configuration of dynamic Scalable Jenkins slaves with Docker and Kubernetes is done.

6. Test The configuration – Create Jenkins jobs and run all together

To test the above configuration, create two jobs and keep it running for a while. Which mean, create Jenkins jobs such a way that both jobs should run at the same time. Then, once the jobs are started running, check the pod status by running the following command.

kubectl get pods --namespace jenkins-instance

This should return output something like this

NAME READY STATUS RESTARTS AGE
jenkins-instance-5ea2b32ace 1/1 Running 0 3m
jenkins-slave-245da2c165 2/2 Running 0 1m

This means, Both jobs are running in two different the Jenkins pods and this will be deleted as soon as the jobs are finished running.

Then, All the codes used in this tutorial are available in below Github repository.

Conclusion.

Dynamically creating slaves especially as Kubernetes clusters are very effective as those are available as containers and closed as soon as the jobs are finished running. So, This will save a lot of cost and effort towards configuring the slave Jenkins nodes. Hence, In this article, we have discussed how to create dynamically Scalable Jenkins slaves with Docker and Kubernetes. In our upcoming articles, we will discuss mode usage of Kubernetes and Jenkins. Stay tuned and subscribe DigitalVarys for more articles and study materials on DevOps, Agile, DevSecOps and App Development.