Using Salt for Configuration Management and Orchestration
November 8th, 2018
DevOps Training Architect I in Content
Using Salt for Configuration Management and Orchestration teaches how to use Salt (SaltStack) to work with Linux-based infrastructure. The course begins by exploring the fundamentals and components of Salt, then demonstrates how to install, configure, and manage Salt using a master-minion setup. Next, we learn about Salt execution modules, which allows us to perform actions on multiple servers, and Salt states, which allow us to define end-state recipes for configuring our infrastructure. We also deep dive into creating formulas and learn how to use Jinja and Pillar to produce modular formulas that can be reused.
About the Course
Using Salt for Configuration Management and Orchestration teaches the student a functional knowledge of Salt. In this video, find out the specifical goals and objectives of each portion of the syllabus, and learn a little more about the goals of this course.
About the Author
Linux Academy and Cloud Assessments Course Author Elle Krout is a SaltStack Certified Engineer with a background in technical writing for Linux and DevOp systems.
What Is Salt?
Salt, or SaltStack, is a configuration management platform that allows users to perform a vast array of remote execution, orchestration, and other infrastructure and overall DevOps-related tasks. In this lesson, we learn a little about Salt's background, how it works, and some basic concepts we'll put to use later in this course.
In the scenario we are working on in this course, we take on the role as the intern at a small publication that is looking to expand their internet presence by creating a simple WordPress website in which they can post articles and insights. This lesson covers our infrastructure needs, the goals we hope to achieve by the end of this course, and provides an architectural diagram for our end state.
Remote Execution and Configuration Management
In this lesson, we take a look at the difference between remote execution, and configuration management within Salt. These two systems within Salt allow us to both manage our minions in real-time, as well as provide end-state blueprints for configuring our server.
This lesson provides us with a basic overview of grains and how they fit into the Salt ecosystem. Grains are static data about the minion(s) and include facts such as operating system family, kernel architecture, or even user-defined static information, such as data center cabinet.
In this lesson, we take a conceptional look at Pillar and how it fits into the Salt ecosystem. Pillar allows us to store user-defined variables that may vary by minion based primarily on server purpose, not on static data such as operating system.
In this lesson, we learn how to format YAML, then see how YAML looks when used with Salt. YAML is a human-readable markup language for data serialization and is the default end-state configuration language for Salt.
QUIZ: Salt Concepts
Installing and Configuring Salt
Single Master, Multi-Master, and Masterless Salt
In this lesson, we take a brief look at the differences between the various setup options for our Salt environment, and how to determine what Salt environment is best for your use-case scenario.
Bootstrapping the Master
In this lesson, we set up our Salt Master using the SaltStack bootstrap script. The bootstrap script is a simple way to install various Salt components with minimal configuration for most common use-cases.
Bootstrapping the Minions
In our last lesson, we used the bootstrap script to set up our Master server; in this lesson, we complete our environment by adding our two minion servers, also using the bootstrap command, with a few alterations. Once finished, we will have our complete Salt environment from which we will work for the rest of this course.
Installing Salt via the SaltStack Repositories (Ubuntu)
Learn how to install Salt and Salt components using the SaltStack repository on Ubuntu machines. This is a supplemental lesson intended to demonstrate an alternative set-up process for installing Salt on Ubuntu machines. Please note that while the process for this remains the same on Debian-based distributions, it does use a different key and repository, which can be found here.
Installing Salt via the SaltStack Repository (CentOS/Red Hat)
In this lesson, we go over how to install Salt on CentOS and Red Hat servers using the SaltStack repository. This is a supplemental lesson intended to demonstrate an alternative set-up process for installing Salt on CentOS-based machines. Note that, unlike with the previous lesson, this can be used on both Red Hat and CentOS machines with no changes.
In this lesson, we learn how to create a masterless minion that can run Salt States and execution modules. Masterless minions are excellent for small projects and testing Salt formulas and custom modules. Masterless minions can also Salt themselves through the use of the salt-call command, addressed later in this course.
In this lesson, we use the Salt environment we provisioned earlier to set up our minions to work with our master server; we also use the salt-key command to manage and accept our minion's keys. Key management within Salt is what determines whether our minions can talk to our master, and any servers with unaccepted keys cannot be used.
Occasionally, when working with certain providers, Salt will read the hostname from provider metadata and set this as the minion ID. Learn how to update a minion's ID by changing the /etc/salt/minion_id file and regenerating our Salt keys. This lesson also gets us hands-on experience with deleting keys and lets us work more extensively with the salt-key command.
QUIZ: Installing and Configuring Salt
Introducing Execution Modules
In this lesson, we learn the basics of execution modules, break down how to run them, then go over how adding arguments to a function works with a few hands-on examples by using test.ping and cmd.run. Execution modules allow us to run a command on multiple servers from the Salt master, letting us manage our infrastructure efficiently.
In this lesson, we learn how to define which minions we want to run our execution modules against. We demonstrate this via the user.add function by completing some basic goals for our end-state project. Targeting allows for more fine-grained orchestration between our minions, by ensuring only the minions that need the change receive the change.
Salt has too many modules for us to talk about them all, but let's take a look at some common modules, including the sys, pkg, and cmd modules. We'll also take this time to ensure our Salt master has some basic configurations to make our life easier. View a full list of Salt modules here: https://docs.saltstack.com/en/latest/ref/modules/all/index.html?
Running Modules Locally with salt-call
In this lesson, we learn how to run execution modules locally on a minion with the salt-call command. This is especially useful when working with a masterless minion, although it can be used on any minion alongside the --local parameter.
QUIZ: Execution Modules in Salt
Salt States and Formulas
Introducing Salt States by Building a PHP Formula, Part 1
In this lesson, we learn some Salt state concepts and begin building out our PHP formula. We review the goals for this formula, learn how to use the sys module to view state-related module documentation and gain an understanding of how Salt states work before we begin to write our first formula.
Introducing Salt States by Building a PHP Formula, Part 2
Continuing directly from our last lesson, we use what we've just learned to build our PHP formula. We get our first taste of Salt states and state modules, as we install PHP and related packages with the pkg module. We also learn how to use include, and how to test our states to ensure our YAML is functional.
The top.sls File
In this lesson, we set up a top.sls file, which allows us to map our formula and states to defined minions. After creating this file, we add our newly-created PHP formula to the file so that it is installed on our web* servers, including the web1 server in our demo environment.
Starting an Apache Formula
Using what we just learned by creating our PHP formula, we build out the first part of the Apache formula we'll be using for the next portion of this course. To do this, we create an initialization state that installs Apache and ensures the Apache service is running on our server. Resources: Apache Formula GitHub Lesson Branch on Github
In this lesson, we learn how to manage files on our minion from within a Salt state using our Apache formula. We utilize the file.managed state function to use a static file provided on the Salt master as a reference for the Apache configuration file on our Ubuntu 16.04 web minion. Apache Formula GitHub Lesson Branch on Github
In this lesson, we learn how to create relationships between our Salt state by using Salt's requisite system by adding a restart state to our Apache formula. By using the module.wait state function, we can add this state to our minions but also ensure it only runs when the provided parameters are met. In this instance, we use the onchanges option to ensure that this state is only run when there are changes to the configuration file. Apache Formula GitHub Lesson Branch on Github
QUIZ: Salt States and Formulas
In this lesson, we learn how to make our states usable across multiple distributions by introducing Jinja templating. We use our Apache formula and begin to expand our init.sls so it can be used to download Apache on both Ubuntu and CentOS while using the appropriate package names. This allows us to use a single formula for multiple minions, despite that minion's architecture. Apache Formula GitHub Lesson Branch on GitHub
Using a map.jinja File
In our previous video, we expanded our Apache formula by adding variables to our init.sls state. By using a map.jinja file, we can set operating system family-level variables for our entire formula, without having to map out each variable in each state file. Apache Formula GitHub Lesson Branch on GitHub
In this lesson, we learn how to use Jinja within our files managed by Salt to create dynamic file templates. In this final step for our Apache formula, we allow our configuration files to work across multiple distros and grant our managed files access to the map.jinja file created earlier in this section. Debian envvars file Apache Formula GitHub Lesson Branch on GitHub
Best Practices for Salt States
In this lesson, we review some best practices to think about while creating Salt states and formulas in the future. This ensures we will maintain cleanly-written Salt states that can be used within the confines of any one infrastructure.
QUIZ: Templating in Salt
In this lesson, we take a look at a MySQL formula, then set up some basic pillar data to configure MySQL via Salt so we can bind MySQL to the private IP address of our db1 minion. We also learn how to call basic pillar data from within our Salt states by adding the bind variable to our MySQL configuration. Debian envvars file MySQL Formula GitHub Lesson Branch on GitHub
In this lesson, we learn how to enhance our security within our Salt states by encrypting sensitive data, such as passwords. Although pillar is already considered secure (or as secure as your Salt master), Salt's GPG renderer extends us an extra layer of protection, especially in cases where we may merge password data to GitHub. Debian envvars file MySQL Formula GitHub Lesson Branch on GitHub
Using Pillar with Jinja, Part 1
In this lesson, we learn how to use deeper pillar data to work alongside Jinja for more powerful and extensible states. In part one of this lesson, we first create and pillar data to use with our database state, then use a for loop in Jinja, along with the iteritems function, to popular our state with this data. Debian envvars file MySQL Formula GitHub Lesson Branch on GitHub
Using PIllar with Jinja, Part 2
Continuing where our previous lesson left off, we start this video off by going through our current users state and restructing its data in our mysql.sls pillar file. After encrypting the necessary passwords for our database users, we then craft another for loop within our state, iterating through both users provided in our pillar. Debian envvars file MySQL Formula GitHub Lesson Branch on GitHub
Pillar Best Practices
In this short lesson, we spell out some best practices for using pillar and structuring our pillar files, with concern to infrastructure size and end goals.
QUIZ: Pillar in Salt
Putting It All Together
Putting It All Together
Having created all the formulas we need for our underlying LAMP stack, we finish filling out our top.sls file. We then review what's left of our end state, take one last look at our architectural diagram, and learn what we need to consider when creating our final WordPress formula.
Building a WordPress Formula: wpcli.sls
In this lesson, we start by creating our wpcli.sls file to pull in the WordPress command line tool via the file.managed state module. This gives us the basis to create and test the rest of our formula. We also begin to craft our map.jinja file, concurrent to the creation of the formula – no going back to fix things here! By the time we're done, we'll be able to use the wp command alongside Salt to configure and install WordPress on our existing Salt-managed LAMP stack.
Building a WordPress Formula: init.sls
Now that we have access to the wp command, we're going to work with both the file.directory function and the cmd.run function to ensure that our website's root directory is present and that we have WordPress's initial installation files downloaded to that directory. We'll also use our first instance of the unless parameter, which will let us test our cmd.run functions against certain minion states to make sure the state only runs when needed and not with every highstate.
Building a WordPress Formula: wpconfig.sls
The next step in our WordPress formula is creating the wpconfig.sls file, which will let us set our database information – taken from the very database we created earlier in this course. We'll also begin to craft our wordpress.sls pillar file, which will contain any website-specific data, as well as any passwords we may have to use.
Building a WordPress Formula: wpinstall.sls
In this lesson, we complete our WordPress formula by crafting another cmd.run state that allows us to install WordPress into a choosen directory with the parameters we give it via pillar. We'll then put our formula through some final tests, and add it all to our top.sls file, completeing the end state infrastructure for this course.
Now that we've finished our course on Salt, explore what other options you have at Linux Academy! Salt uses Python: Python 2.7 Scripting for System Administrators Python 3 Scripting for System Administrators ?Gain some deeper systems knowledge and write more functional formulas Linux Academy Red Hat Certified Systems Administrator/Engineer Prep Course LPIC-2 Get committed to formula writing: Git Quick Start Git and GitLab – Start to Finish Explore more DevOps: Docker Certified Associate Prep Course Using Ansible for Configuration Management and Deployments