How to Setup Jenkins CICD Pipeline for AWS Lambda with GitHub and SAM Template

AWS Lambda is a serverless computing service from AWS. Application logic can be written as functions and the actions can be triggered. For this cloud service, no platform-level services are exposed to end-user. We can also call this as Functions as a Service (FaaS). Since Serverless Computing is different than other traditional computing, Serverless application Delivery pipeline and Continuous Integration and Continuous Deployments are handled differently. In this article, we will discuss How to Setup Jenkins CICD Pipeline for AWS Lambda with GitHub and SAM Template.

The intention of this article is to clarify the process of the Delivery pipeline of AWS Lambda with Jenkins and get the idea of it.

Requirements

Before getting into this article, let us assume you have the following things handy.

• Knowledge on
  • AWS Lambda
  • AWS IAM
  • AWS S3
  • AWS API Gateway
  • AWS SAM or CloudFormation
• Jenkins Server
• GitHub and WebHooks

In this article, we will discuss the typical architecture of AWS Lambda based application. So, We will be running this setup walkthrough with AWS Lambda and AWS API Gateway with some resources and HTTP methods which will point respective lambda function. Let us see How to Setup Jenkins CICD Pipeline for AWS Lambda with GitHub and SAM Template.

Step by Step CICD Pipeline setup

Lambda functions are written by the developer in Python or NodeJS or Java. Then they will push the code to the version control system. Here in our case, we are taking GitHub as our version control system.

Along with this, here we are adding SAM (Serverless Application Model) Templates to define the architecture of the serverless application. Basically, These SAM Templates are the extensions of CloudFormation Templates.

So, Our Jenkins Job will be triggered by GitHub whenever we push the code into GitHub Repo. Jenkins job is configured by starting with cloning the code from SCM. Then, building it with the specified build information of the applications from SAM Template files.

By running the Unit test of the code, the Application codebase can be tested and validated before it gets deployed into AWS Lambda. Then, Jenkins will deploy the Lambda application in AWS Platform by Jenkins AWS SAM plugin. So, whenever we push any update on the Lambda function, this pipeline will automatically get triggered and the entire workflow will be executed and finally it will automatically be deployed again to the AWS platform.

Let’s Prepare for the Setup. So, To set up the complete workflow, we need to do the following actions

1. Creating AWS Service Roles
2. Application Setup
3. Jenkins Pipeline setup

Creating AWS Service Roles

Since we are going to have multiple AWS service and we need to create permissions to make them access each other, we need to create Service Roles for following services.

1. Lambda.
2. API Gateway
3. S3 Bucket
4. CloudFormation
5. IAM

To create a Service role, Follow the following steps

Step 1: Sign in to the AWS console

Step 2: Then, Go to the IAM Console

Step 3: Click Roles (from the left menu)

Step 4: Then, Click Create Role button -> select AWS Service (1), CloudFormation services (2), Permission (AWSLambdaExecute) -> Click Review.

Step 5: Give Name and Description->then click Create Role button.

Step 6: Click the created Role

Step 7: Then, Click Add inline policy

Step 8: Click the JSON tab and add all three services the click Review Policy. Example,

Sample Policy JSON file

{
	"Version": "2012-10-17",
	"Statement": [
			{ 
				“Action”: [“Lambda”],
				“Resource”: [“arn:aws:lambda:XX-XXXXXXX-XXXX”]
			},
			{
				“Action”: [“apigateway”],
				“Resource”: [“arn:aws: apigateway:XX-XXXXXXX-XXXX”]
			}
		]
}

Step 9: Name the Policy the click Create Policy

That’s it. All our AWS setup is done.

Application Setup in GitHub

So, we are going to set up the Codebase with the following content in it.

1. Application Codebase
2. SAM Template.yml

Application codebase is basically the application logic which is going to be our lambda functions. The SAM Template.yml is the CloudFormation template to create Lambda application with API Gateway. Then, All the files need to be pushed into GitHub Repo which is later used in Jenkins job to get triggered whenever we get the update in the repo.

Sample SAM Template.yml

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: A starter AWS Lambda function.
Parameters: 
    IdentityNameParameter: 
      Type: String
Resources:
  helloworld:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: index.handler
      Runtime: nodejs8.10
      CodeUri: .
      Description: A starter AWS Lambda function.
      MemorySize: 128
      Timeout: 3
      Policies:
        - SESSendBouncePolicy:
            IdentityName: !Ref IdentityNameParameter

This SAM template is like AWS CloudFormation template except for the Transform tag. See here for a lot more example applications from AWSLab. Then, We will discuss more about SAM Template and AWS Lambda in another article.

Importantly, Create the Hook to Github to trigger Jenkins job from Jenkins by selecting Admin button of your Repository -> Service Hooks -> Post-Receive URLs and add the Jenkins webhook Credentials. Which looks like

“https://USERNAME:PASSWORD@your-jenkins-server.com/github-webhook/”

Before that, Make sure you have given Job/Read and Job/Build access in ‘Project-based Matrix Authorization Strategy’ for the GitHub user (USERNAME).

Jenkins CICD Setup

Most importantly, we are going to create the Jenkins job to set up the CICD pipeline. For that, we are going to implement the following stages

1. SCM block setup
2. Building the Application code according to the platform of our choice.
3. Deploying into AWS Platform

Before proceeding with the above stage implementations, make sure you have installed AWS SAM Plugin for Jenkins (AWS SAM Plugin)

So, Let’s see the step by step process.

Step 1: Create Global credentials in AWS Credentials kind by entering AWS Access Key and Secret Access Key.

Step 2: Create ‘New Item’ in a freestyle job.

Step 3: Add the URL of the GitHub Repo which is our codebase with SAM template.

Step 4: Then, Select “Build when a change is pushed to GitHub” in the ‘Build Trigger’. (Then, Make sure you have given Job/Read and Job/Build access in ‘Project-based Matrix Authorization Strategy’ for the GitHub user.)

Step 5: Run the Build and Unit test the Application Codebase (Python/NodeJS/Java)

Step 6: Add the “AWS SAM deploy application” build step.

Step 7:  Then, Select the Global AWS credentials which are created in Step 1, Select Region, Select S3 Bucket name, Select SAM Template Filename.

Step 8: Then, Save and Exit

So, This Jenkins CICD Pipeline will automatically trigger the deployment if there is a change in the respective branch of the repository. And then, you can immediately see the change in the AWS Lambda function.

Conclusion

In this article, we have seen how to Setup Jenkins CICD Pipeline for AWS Lambda with GitHub and SAM Template. Then, In our future articles, we will see the complete tutorial of each and every step in detail.  Later, In our upcoming articles, we will discuss more every services and tutorial of Azure DevOps. Stay tuned and subscribe DigitalVarys for more articles and study materials on DevOps, Agile, DevSecOps and App Development.