Header Background Image

AWS CodePipeline: How to Create a Custom Action

Those who have used CodePipeline, know the benefits of the service and how easy it can be to setup and run a pipeline. For those new to CodePipeline, it is AWS’s take on a continuous delivery service. It provides the tools to model, visualize, and automate the software release process.

codepipeline_what_is_pipeline1

CodePipeline, by default, comes with six action “categories” that can be added to any stage in the pipeline.  These categories include 15 providers, comprised of both AWS and third party services:

  • Approval: Manual
  • Source: AWS CodeCommit, AWS S3, GitHub
  • Build: Solano CI, Jenkins
  • Test: BlazeMeter, RunScope, Ghost Inspector, HPE StormRunner Load, Apica Loadtest, Jenkins
  • Deploy: AWS OpsWorks, AWS Elastic BeanStalk, AWS CodeDeploy
  • Invoke: AWS Lambda

This list is no slouch! Many of the heavy hitters are here, and these default options should be enough for most individuals or organizations.

However, what if your software requires a special action provider? Something that had been tailored specifically to build or test your unique application? Perusing CodePipeline in the AWS Console cannot provide you with any insight into how (or even if) you can create, add, and implement your own custom action that uses your custom provider. Only if you dive deep into the AWS CodePipeline documentation, do you learn that you can create your own actions using a custom provider, but it can only be created via the AWS Command Line interface (CLI). And once you get the custom action created, you still need to configure your custom provider (the “job worker”) to communicate with AWS.

Scenario:  Add a Custom Actions to Call Upon a Custom Build Application

Let’s say that you have a custom-built application for your software — something like Jenkins — that was built in-house specifically to build the software you want to release.

Step 1: Create a Custom Action

  1. In a text editor download, and copy and paste in the AWS Custom Action sample template. (Custom Action Template)
  2. Edit/Configure the template: The image below shows the fields in the template that need to be edited, what the fields are, and how they are reflected once the action has been created.CodePipeline Blog2
  3. Save the template as a .json file.
  4. Upload the file to AWS using the command:

aws codepipeline create-custom-action-type --cli-input-json file://<FILE_NAME>.json

The custom action will then be available for your to add in the AWS Console (you may have to exit our of your pipeline, refresh the page, then go back into the pipeline for the custom action to appear in the drop down menus).

Step 2: Configure Your Custom Build Application for use with CodePipeline

Moving forward, the custom build application will be known as a “job worker.”

The job worker must be configured to continuously poll CodePipeline for a “job request.”  When a “job worker” detects a “job request” it must be set up to complete the following:

  1. Acknowledge (to CodePipeline) that it has detected the job request
  2. Pull the target (input) artifacts from the pipeline’s S3 bucket
  3. Execute the custom job (perform actions on the artifacts)
  4. Push the (output) artifacts back to the pipeline’s S3 bucket
  5. Return the job results to CodePipeline (succeeded or failed)

codepipeline_blog_post1

There are three main tasks that you need to accomplish:

  1. Give the job worker permissions to access CodePipeline:
    • The easiest way to do this is to create an instance role, and attached the “AWSCodePipelineFullAccess” to it.
    • Launch an EC2, and assign the instance role to it (at launch).
    • Install your custom build application on the EC2 instance
  2. Setup and configure the API calls:
    • For polling: PollForJobs
    • For job acknowledgement: AcknowledgeJob
    • For failure: PutJobFailureResult
    • For updates & succeeded status: PutJobSuccessResult
      • The continuation token allows you to update the action in the AWS console to include the ExternalExecutionID that will populate the link in “executionUrlTemplate
      • Setup API calls so that your “job worker” can communicate with CodePipeline & accomplish the needed tasks (AWS Codepipeline API Documentation)
  3. Configure an Amazon S3 Client to retrieve and put artifacts:
    • To download pipeline artifacts from S3, you must create a S3 Client that uses “signature version 4 signing” (Sig V4).
    • To upload pipeline artifacts to S3, you must configure the S3 PutObject request to use encryption.
    • Currently only SSE-KMS is supported for encryption. (AWS S3 API Documentation, AWS SSE-KMS Documentation)

Step 3: Add the Custom action to your pipeline & select the job worker as your custom provider

Step 4: Run the pipeline!

For a detailed, video walkthrough of this scenario – head on over to LinuxAcademy.com and checkout the “Manage & Deploy Code with AWS Developer Tools” course.

Thomas Haslett

Thomas Haslett is a enthusiast of cutting edge technology. Cloud computing, virtual/augmented reality, and the Internet of Things (IoT) are what he considers "fun & cool". Tom's background includes years of experience creating and designing proprietary business applications, virtual reality game development, and android app development. Tom currently holds three AWS certifications: AWS Solutions Architect (associate), AWS SysOps Admin (associate), & AWS Developer (associate).

4 thoughts on “AWS CodePipeline: How to Create a Custom Action

  1. Hi Thomas,
    Thank you for this blog. It was extremely valuable in understanding Custom Action. Actually currently I am
    involved in a proof of concept where I am trying to set up a custom action which will invoke a Jenkins Job and
    the jenkins job will pull the code from the codecommit repository and run a maven test on it and give back the
    results to the codepipeline. The Jenkins is installed on an EC2 instance. Is this possible and if yes how?
    Jenkins is somehow not able to see the codecommit repository although i provide it the correct https clone url for the repository. Is there any additional role settings for this assuming that is the problem?.
    Thanks & Regards
    Joseph

    1. Hi, Joseph.

      Based on your scenario, I do not think you need a custom action. CodePipeline already has Jenkins integration built in. If your source is CodeCommit, all you need to do is set the Pipelines “source” provider as CodePipeline, then in the Build-stage set the “provider” to Jenkins and point it to your Jenkins server (which can be installed on an EC2 instance). You do not have to link the Jenkins server directly to CodeCommit (Pipeline does the artifact hand-off for you).

      There is a video in our “Manage and Deploy Code with AWS Developer Tools” course that specifically addresses how to setup Jenkins and use it in a pipeline. I would suggest starting there to get a better understanding of how you can accomplish what you are trying to do.

      1. Hi Thomas,

        Sorry for the delayed response. I was expecting the response to come on my email address.Thank you
        for elaborating in detail the working of the process.
        Actually I have already achieved success in everything that you have mentioned. The polling works
        and I also have AWS Codedeploy installing the software on the ec2 Tomcat instance.
        Let look at a scenario where a Jenkins job is executing a script for me or actually integrated with another
        third party solution. I am looking at calling this jenkins job from my pipeline and then accepting the
        results from Jenkins and stopping or proceeding with the pipeline stage.
        Is there any way to do this invocation without the pull..?
        Looking forware to your reply.

        Regards,
        Joseph

  2. Hi Thomas, First Thanks for this blog… This is helpfull configuring the jenkins.
    But I need to configure the shell script to run instead of jenkins url.
    I created API server with endpoint and used it instead of Jenkins URL in the template.
    Running job worker for waiting hit from codepipeline.
    But it is not working… Is there any suggestion?

Leave a Reply

Your email address will not be published. Required fields are marked *