Skip to main content
Introduction

In this guide, we’ll configure Jenkins and CodeCommit to allow users to easily create and destroy infrastructure using Terraform code.

For this guide, let’s suppose our deployment has the following requirements:

  • Users don’t need to install anything from their own, especially on their laptop.
  • Users don’t need to use the command line.
  • Users need to have a simple input experience with select boxes, etc. Where there are open input fields, the value inserted needs to be evaluated and verified.
  • Users can check the output results for errors and all users can see the previous executions from other users
TERRAFORM

Terraform is used to define and build infrastructures using code, and is becoming more popular day by day. One of its advantages is that you can build infrastructure using many providers, including AWS, Azure, Google Cloud, and VMware (a full listing can be found here).

Another important advantage is that writing code using the Terraform language is comfortable and much simpler than other similar technologies.

You can download and use Terraform for free. It is an open source project, but the community version is command line based and does not include a web console.

When you run Terraform code, you will have the `terraform.tfstate` file and `terraform.tfstate.backup` after the second run. You need these files to delete or update an existing configuration. If you want update the infrastructure, you need also the code that you use during the builds so you’ll need to save these text files to reuse in the future, or at minimum, for your records.

 

JENKINS

We’ll use Jenkins to provide a simple way for users to create and destroy architecture using jobs. Everything is region dependent and when you change regions, you have different parameters to pass. To have this dynamicity with parameters, we can install the Uno Choice plugin . This allows us to change a menu depending on the region selected:

user_6212_595782c93a098.png

In this example, only the networks for the selected region are shown. We’ll explain how to configure the plugin to behave this way in a later section.

 

Install and Configure Jenkins

We’ll use a Linux AMI machine for our Jenkins installation, but you can easily adapt these instructions to other Linux distributions. These commands should be run by a user or role with administrator access:

 

yum update -y
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
rpm --import http://pkg.jenkins-ci.org/redhat-stable/jenkins-ci.org.key
yum install jenkins -y
chkconfig jenkins on

 

Change the `jenkins` user before starting Jenkins for the first time. For our purposes, we’ll use one with full access like the `ec2-user` in Linux AMI or the `ubuntu` user for an Ubuntu LTS distribution.

 

sed -i 's/JENKINS_USER="jenkins"/JENKINS_USER="ec2-user"/g' /etc/sysconfig/jenkins
chown ec2-user:ec2-user /var/lib/jenkins/
chown ec2-user:ec2-user /var/log/jenkins
chown ec2-user:ec2-user /var/cache/jenkins/
service jenkins start

 

Optionally, install Apache or put the machine behind a load balancer if you want to use SSL or you can’t access the default Jenkins port 8080.

To complete the setup, copy the following into a script and execute it:

 

#!/bin/bash

###### install programs
sudo yum update -y
sudo yum install git python27-pip.noarch -y
sudo /usr/local/bin/pip install –upgrade pip
sudo /usr/local/bin/pip install requests
sudo /usr/local/bin/pip install requests –upgrade

##### configure git #############
git config –global credential.helper ‘!aws –profile codecommit credial-helper $@’
git config –global credential.UseHttpPath true

git config –global user.name “Jenkins automation tool”
git config –global user.email myemail@mycompany.com

REPO=”/home/ec2-user/git-repos”

if [ ! -d “$REPO” ]; then
mkdir /home/ec2-user/git-repos ;
fi

if [ ! -f “/usr/bin/terraform” ]; then
cd /tmp
wget https://releases.hashicorp.com/terraform/0.8.7/terraform_0.8.7_linux_amd64.zip unzip terraform_0.8.7_linux_amd64.zip
sudo mv terraform /usr/bin/
fi

 

CodeCommit

In this section, we’ll configure CodeCommit quickly and without much explanation. For more detail, check out the course on Manage & Deploy Code with AWS Developer Tools

Create an IAM user without password access and with only permission to have CodeCommit access:

user_6212_595784ae66984.png

On the Jenkins machine, create a set of SSH keys. Do not use a passphrase:

 

ssh-keygen

 

Upload the generated .pub file using the AWS web console and configure the security credentials as follows:

user_6212_59578531a2e13.pnguser_6212_5957854485971.png

Configure Git as follows:

 

cd /home/ec2=user/.ssh
~/.ssh> cat config
Host git-codecommit.*.amazonaws.com
 User REPLACE-YOUR-SSH-KEY-ID
 IdentityFile ~/.ssh/codecommit

 

Add the keys to the administrative user’s `.ssh` directory:

 

~/.ssh> ll

-rw——- 1 ec2-user ec2-user 1.7K Sep 26 09:05 codecommit
-rw-r–r– 1 ec2-user ec2-user 405 Sep 26 09:06 codecommit.pub
-rw——- 1 ec2-user ec2-user 110 Sep 26 09:07 config

 

Test the CodeCommit connection. Here’s example output from a successful connection message:

 

ssh git-codecommit.us-east-1.amazonaws.com
The authenticity of host 'git-codecommit.us-east-1.amazonaws.com (54.239.20.155)' can't be established.
RSA key fingerprint is a6:9c:7d:bc:35:f5:d4:5f:8b:ba:6f:c8:bc:d4:83:84.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'git-codecommit.us-east-1.amazonaws.com,54.239.20.155' (RSA) to the list of known hosts.
You have successfully authenticated over SSH. You can use Git to interact with AWS CodeCommit. Interactive shells are not supported.
Connection to git-codecommit.us-east-1.amazonaws.com closed by remote host.
Connection to git-codecommit.us-east-1.amazonaws.com closed.

 

Dynamic Uno Parameter Plugin

Next, we’ll setup the advanced dynamic input type parameters, which allow us to select different options for each AWS region. For example, if we select one region, we’ll see a list of databases available in that region, but if we select another region, we’ll have a different set of database options.

user_6212_5957861b91817.png

Copy the following script into the “Script” text area:

 

import static groovy.io.FileType.FILES
if (binding.variables.get('aws_region') != null) {
 region = binding.variables.get('aws_region')
 def dir = new File("/home/ec2-user/git-repos/databases-resources/"+region+"/cassandra-resources/"); 
 def files = [];
 dir.traverse(type: FILES, maxDepth: 0) { def values = (it.toString()).split('/'); files.add(values[values.size()-1]) };
 return files
}

 

The other parameter, `aws_region` can be a simple “Choice Parameter” with a list of regions available.

To use this parameter:

user_6212_59578667977f6.png

To add this option:

  1. Install the Update Sites Manager plugin, available here: https://wiki.jenkins-ci.org/display/JENKINS/UpdateSites+Manager+plugin
  2. Add the biouno site following the instraution in this link http://biouno.org/jenkins-update-site.html

Install the uno-choice-plugin. Instructions can be found here: https://github.com/biouno/uno-choice-plugin/wiki/Uno-Choice-Cascade-Dynamic-Choice-ParameterFor the people that doesn’t have so much experience with Jenkins, the update site manager is like use apt/yum with additional sources respect the original. The point 2 means add the source biouno package repository, only after this action you can install all the plugin available from biouno. I installed the package uno-choice-cascade

 

Save the “State” with AWS CodeCommit

Now that AWS CodeCommit is working from your Jenkins machine, you can work inside a versioned directory and to save all the files of your Terraform runs using Git commands.

After each Terraform run, save the state of the Terraform code to CodeCommit by committing and pushing the files in the job’s directory by adding the following commands to your job:

 

git add -A
git commit -m ‘any comments that you like’
git push origin master 

 

Your `terraform.tfstate` and `terraform.tfstate.backup` will be versioned and stored for future reference.

Jenkins as a Web Console

You are the Jenkins Administrator, you the access to every job with full rights, but in many cases this is not what you want. In my case I want give access only to run the job but not to modify that. To do this you need to setup this special settings with role.

 

It is important for users to be able to run the jobs that you created for them, but not to be able to modify them. To configure this, install the following plugin: https://wiki.jenkins-ci.org/display/JENKINS/Role+Strategy+Plugin

After the installation, enable the role access control:

user_6212_595787253c648.png

Click on Manage Jenkins, and a new option will be shown:

user_6212_5957872ee8ecf.png

user_6212_5957876adc44c.png

Click on Manage Roles:

user_6212_595787992059c.png

Create a role called, for example, `run_job_only` and assign the options shown here:

user_6212_595787cc0b26b.png

user_6212_595787d5d5104.png

From here, assign your user to the `run_job_only` role we just created.

user_6212_595787de4d82c.png

 

Conclusion

That’s it! Your users can now use the Jenkins web console to create and destroy architecture using Terraform.

Future Developments

These have not been tested yet but here are some possible improvements:

  • Integrate Terraform in Jenkins with the plugin https://wiki.jenkins.io/display/JENKINS/Terraform+Plugin
  • Integrate the CodeCommit inside the Jenkins using a git plugin https://wiki.jenkins.io/display/JENKINS/Git+Plugin
  • Save the Terraform state files in S3 to have a durable repository https://www.terraform.io/docs/backends/types/s3.html. The disadvantage is not to have the state and the creation code in the same place.
Comments are disabled for this guide.