Build and Deploy Pipelines with Microsoft Azure
DevOps Training Architect II in Content
This course uses Azure DevOps to demonstrate many different build and deployment scenarios using Pipeline as code. Familiarity with Azure cloud is expected prior to taking this course, as well as basic knowledge about application development. We'll get hands-on with building Web Apps, Azure Functions, and AKS clusters, all using the Azure Pipelines workflow.
Topics that we'll cover include:Deploying an ASP.NET Application and Database to Azure App ServicesDeploying to AKS, Azure Functions, and Azure Container RegistryUsing Jenkins and Azure DevOps togetherAzure DevOps CLIPipeline Build TemplatesSelf-Hosted Build AgentsDeployment GroupsUsing Terraform in the Pipeline to create infrastructure in AzureBuild Artifacts
About the Training Architect
Introducing your course instructor, Chad Crowell.
About the Interactive Diagram
The interactive diagram is going to be your study guide for this course. We will go over how to use it so you can follow along throughout the course!
DevOps and Azure DevOps
DevOps Practices and Methodologies
Integrating your code is important for many reasons — the most important being that you can test if your application works as it should. The use of tools like Jenkins, Travis CI, Circle CI, or Bamboo CI has been growing rapidly in the last 10 years. The key is to identify problems with your code early on, so you can solve many small problems along the way versus waiting for an accumulation of problems that take weeks to fix. Helpful Resources Linux Academy Course: Implementing a Full CI/CD Pipeline
Continuous Delivery is the act of getting the application in a deployable state, whereas Continuous Deployment is the automated process of publishing your application, reaching the end user. This process is accomplished by using a Deployment Pipeline, which we'll explore with Azure Pipelines later in this course. Helpful Resources Continuous DeliveryDeployment Pipeline
Source Control Management
SCM is fundamental to working in teams, where many people are changing the same code. SCM can also be referred to as a version control system (VCS) and comes with many features. Some of those features include being able to manage permissions of files, add version to files, resolve merge conflicts, keep a history to revert back to, and automate which version or branch we'd like to test. This is important to Azure DevOps, as you can work these features into your Deployment Pipeline.
In this lesson, we talk about what branches are, and we demonstrate how they are useful in a pipeline scenario. You may want to perform many different actions before or during your build pipeline, and knowing this is key to utilizing a tool like Azure Pipelines properly.
Artifacts allow us to do testing within our deployment pipeline, because the artifact (in its executable form) is necessary to carry out the testing plan. This is called execution-based testing. We can use artifacts to perform testing to the application in its finished state, or we can conduct testing (which we'll talk about in the next lesson).
Whether it's testing the finite detail of a function within a system, or testing the broad black box, you can work code testing into your pipeline as a part of deploying your application. This comes in handy when you run into errors and need to identify, track, and fix that error as quickly as possible. You can bake this into your DevOps pipeline, and perform the tests with an additional build step, or you can wait for manual intervention before proceeding and publishing the application.
Distributing your builds is important to the build process becuase you don't want the CI server to reach its physical limitations and potentially delay your build process. Having different servers run different jobs in a distributed fashion is not only a fault-tolerant way of building your pipelines, but increases the efficiency of builds over time. As you increase the number of pipelines in your organization, this will become more important.
Introduction to Azure DevOps
What is Azure DevOps?
In this lesson, we talk about how the DevOps terms we talked about in the last section relate to Azure services in Azure DevOps. The way in which you group DevOps projects together is called an organization, and we go through the process of creating an organization.
DevOps Services and DevOps Server
Azure DevOps server comes in two main configurations: hosted and self-maintained. There are some positives and negatives to each. The main pro with DevOps Services is the ability to stay up to date with the latest features and automatically scale. The pros of DevOps Server include the control over your own security and integration with on-site Active Directory.
Azure Pipelines as Code
Pipelines as code refers to the build steps being saved in a YAML file within the version control system. In Azure DevOps, this YAML file can easily be created by the built-in templates or by building from scratch. If you are unfamiliar with YAML, Azure offers a way in which you can drag and drop your build steps into a graphical interface, and search for plugins in order to build your pipelines.
Azure Repos and GitHub Integration
Azure Repos is not your only option when choosing a version control system for Azure DevOps. There are many integrations with GitHub, which allow you to entirely replace Azure Repos if you wish. In this lesson, we go through the considerations for choosing Azure Repos or GitHub. We also cover how to connect your GitHub account to Azure Boards and Azure Pipelines.
Agile Tools in Azure DevOps
If you work for a company that embraces the DevOps culture, you are most likely using agile methods to accomplish your work. It has been proven to get more work done faster and is observable, so everyone gets an even distribution of work. Nonetheless, Azure Boards has incorporated all of these methods (Kanban, Scrum, sprints, etc.) into the CI/CD process — meaning you can now use Azure Boards as your one stop for work item tracking.
Build and Pipeline Steps
CI/CD Pipelines in Azure
In this lesson, we immediately dive right into creating a web app, which is built and deployed by Azure Pipelines.
pools section in the YAML:
Separate the build and deploy steps:
name: Hosted VS2017 demands: - msbuild - visualstudio - vstest
Add the deploy step to the bottom of the YAML:
stages: - stage: Build jobs: - job: Build
- stage: Deploy jobs: - job: Deploy pool: name: Hosted VS2017 steps:
Types of Pipelines
In the last lesson, we built an ASP.NET pipeline, but there are many other types of pipelines, such as container, .NET Core, Node.js, Java, and Python, to name a few. We can also use pipelines to build infrastructure with Terraform or Ansible. In this lesson, we go through a number of pipeline examples, so you can see the similarities and differences. Helpful Resources .NET Core pipeline repositoryContainers pipeline repositoryNode.js pipeline repositoryJava pipeline repositoryPython pipeline repository
Create a Pipeline from the CLI
The Azure DevOps CLI Extension can be extremely useful for building pipelines programatically. In this lesson, we build a pipeline using the Azure DevOps CLI (prompt-based) and also learn how to script the creation so you can use it with existing automation tools. Helpful Resources DevOps Extension GitHub pageFork this Java app for pipeline creation To create a new service connection:Project Settings > Service Connections > New Service Connectioin > GitHub
Templates in Azure DevOps can mean two different things. In previous lessons, we've been using templates to initially build our pipelines. Templates can also be a reference to a build file from within the
azure-pipelines.yml. You can store these de-coupled YAML files within a separate directory, and then call them however many times you wish within the main
azure-pipelines.yml build file.
The repository used in this lessonTemplates documentationParameters documentation
Stages and Tasks
Stages represent major operations between tasks in your pipeline YAML. Even though the stages keyword is not a requirement, there are two major benefits. The first benefit is the ability to stop the pipeline after the first stage if necessary (due to build failure). The second benefit is to create order and logical separation between the elements of the YAML for human readability. In this lesson, we look at how stages are used and how tasks are used within stages to perform commands (commands you may be familiar with in the scripts you are already using). Helpful Resources Azure Pipelines task to CLI command referenceStages documentationTasks documentation
Conditions are basically if statements that allow the pipeline to continue or fail based on certain parameters. You can use the
failed() condition, or you can create your own custom conditions. In this lesson, we look at an example of a custom condition within a valid pipeline YAML.
Conditions documentationYAML used in this lesson
From Build to Release
You may have noticed the releases feature in Azure DevOps. This allows you to deploy to various environments in an aesthetically pleasing way. The GUI walks you through choosing your artifact to deploy from, and then you can choose your dev, staging, uat, and prod environments to deploy to. Helpful Resources Release pipelinesReleases
Build agent and CI server are synonomous for the server you use to build and test your application. In this lesson, we go through what types of build servers you can use for free with Microsoft, and how you can opt for using your own CI server on-prem or with Jenkins. Helpful Resources Microsoft-hosted build agentsFree-tier services
You can control which build agents to choose from within your YAML. This is not just limited to Windows servers — you can also choose from Linux, MacOS, or containers and additionally choosing demands for those pooled resources. In this lesson, we go through the many different kinds of Microsfot-hosted agents you can use, and how to insert them into your build pipeline. Helpful Resources Microsoft-hosted agentsDefault agent poolsContainer pipeline jobsUsing demands
Build agents can be Microsoft-hosted or self-hosted. Furthermore, self-hosted agents can be on cloud servers or on-prem. You can even use your existing Jenkins CI server with Azure Pipelines. In this lesson, we go through installing the build agent on all three types of operating systems (Windows, Linux, and MacOS). We also use one of the self-hosted agents in our builds. Helpful Resources Self-hosted Microsoft agentsSelf-hosted Linux agentsSelf-hosted MacOS agentsUsing demands in pipeline YAMLSelf-hosted parallel jobs YAML Used in This Lesson
trigger: - master pool: name: Default demands: - Agent.OS -equals Linux variables: buildConfiguration: 'Release' steps: - script: dotnet build --configuration $(buildConfiguration) displayName: 'dotnet build $(buildConfiguration)'
After your build is complete, if you want to deploy to specific servers, Azure DevOps allows you to have deployment groups. Deployment groups can be Linux or Windows machines behind a firewall. Within your deployment group, you can control whether to deploy to a specific server by specifying tags. If you have many servers within the same deployment group, tags allow you to choose which specific ones to deploy to. Helpful Resources Repo used for this lessonBuild pipeline YAMLDeployment groupsHow to provision agents for deployment groups
With all the build agents and deployment groups we've discussed, we can also use our own Jenkins CI server. There is a Jenkins plugin for Azure DevOps that allows you to forward build artifacts to the release pipeline, so you can deploy to Azure or any other environment you choose. In this lesson, we go through the process of installing a Jenkins server, starting a build, and adding a post-build step to copy the artifact to our Azure DevOps project. Helpful Resources The repo used for this lesson
Build and Release Variables
Variables (whether predefined or custom) can be valuable in a pipeline, in order to service deployments to different environments. Built-in variables are global across Azure DevOps and standardize on the location of artifacts and other build files needed, without having to explicitly locate those files or reference them in an isolated build. Those variables can be used across many deployment types and allow you the freedom to use similar pipeline templates across many environments. Helpful Resources Complete list of predefined variablesUsing arguments such as BuildConfiguration in your pipeline YAMLUsing release variables such as AgentReleaseDirectory
Library section of your pipelines, you can set custom variables to use across many projects and pipelines within your organization. These can also be linked to secrets stored in Azure Key Vault. In this lesson, we go through how to add a variable group and how to use it within a YAML pipeline.
Variable groupsLink secrets from Azure key vaultLibrary security modelImportant notes about secrets
The authorization we get to perform deployments to various environments comes from service connections. In this lesson, we discover two examples of projects with different service connection types and how they are included in our pipeline YAML. Helpful Resources Service connectionsCustom service connections
There is a way in Azure Pipelines that we can push our packages from our pipeline to a custom package feed. We can use this custom feed to maintain a list of packages our team can use for our applications. This is useful to eliminate the dependency of downstream packages that could potentially break your application if managed by someone else. In this lesson, we take our existing pipeline for an ASP.NET application and push a package to a custom package feed, to be used by another solution in Visual Studio. Helpful Resources Build pipeline YAMLRestore NuGet packages from pipelineUsing feeds with NuGet packages
Types of Deployments
Deploy to Azure Container Registry
In this lesson we'll go through a deployment of a container image to Azure Container Registry. Github Repository used in this lesson
Deploy to Azure Kubernetes Service (AKS)
In this lesson we'll go through deploying to an AKS cluster using Azure DevOps Github Repository used in this lesson
Deploy to Azure Web App
In this lesson we'll deploy a python flask app to Azure App Services Web App Github Repository used in this lesson
Deploy to Azure Function
In this lesson we'll deploy a Azure Functions app using Azure DevOps Github Repository used in this lesson
Deploy to Azure Web App for Containers
In this lesson we'll deploy a container web app using Azure Pipelines Github Repository used in this lesson
Build Cloud Infrastructure with Terraform
Using Azure DevOps, you can create Azure resources (i.e. Web Apps and VMs) and then deploy to them using Azure Pipelines. This is done by incorporating Terraform into your pipeline. Azure DevOps Demo Generator
Code Project Management
Creating a Board and Adding a Team
Azure Boards is a useful tool for working in teams. When you are creating builds and releases, you can track your progress using a work item. This work item can be automatically updated with the current release's success. In this lesson, we go over how to add team members to your organization and assign them a work item. Helpful Resources Azure Boards
Define a Sprint and Set Tasks
Sprints are used in many agile organizations to define an amount of work to complete in a set amount of time. Your existing sprint workflow can be incorporated into Azure Boards, and you can relate the work items to sprints in your overall DevOps process. Helpful Resources Demo-Tailwind ProjectSprints
Establishing Git Flow
Git flow pertains to the management of the version control through processes such as creating branches, making a pull request, and merging into the master branch. In this lesson, we go through a sample workflow, to show what that process might look like in Azure DevOps and how any changes can be tracked in the work items on your board. Helpful Resources Azure Boards and GitHub integration
Track Build History
You can track build history using the dashboard in Azure DevOps. This gives you a quick view for your build success and/or failures. The dashboard also allows widgets from Visual Studio to be incorporated into the summary of events in Azure DevOps. Helpful Resources Azure DevOps dashboard
Add a Build Badge to Repo
A build badge represents the success or failure of your last build right within your GitHub repository. Azure Pipelines gives you the badge markdown that you can simply copy and paste into your markdown file in the base of your repository. Helpful Resources Add a status badge to your repo
CWE, CVE, and CVSS
In this lesson, we will discuss three assessment tools you can use with your .NET application, to integrate into your pipelines in Azure DevOps. These tools should be used for every one of your applications, to mitigate your code's security vulnerabilities. Helpful Resources DevOps Security ToolsSonarCloudSonarAnalyzerFxCop
Automated and Manual Tests
To test the validity of your application, you must run tests so you can catch the bugs before your application is deployed to production users. In Azure DevOps, you can run automated or manual tests to ensure both unit tests and UI tests are covered. Helpful Resources GitHub repository used in this lessonYAML used for this lessonManual testsAutomated tests
Tests from the Kanban Board
In this lesson, we go over how to run tests directly from the Kanban board. Those tests are automatically updated with your existing test cases. Helpful Resources Inline tests from Azure Boards
You Did It! What's Next?
Great job finishing this course! I hope you learned a lot! Check out some of these other courses on Linux Academy to continue your learning: Other DevOps courses on Linux Academy
Take this course and learn a new skill today.
Transform your learning with our all access plan.Start 7-Day Free Trial