Skip to main content

Sed, the Stream Editor

Along the same line as my last blog,  I would like to reiterate that in some cases, it is important to have what I would consider to be basic skills on the command line. Bash block logoIn this blog, I will be talking about the sed command. Sed, or the “Stream EDitor”, is another utility that is in almost all distributions and is something that is invaluable when it is used correctly.

To start off, let’s talk about what a stream is. Bash receives streams of characters and each one is independent of the one that was before it and the one that is after it. This means that when you ‘cat’ a file you are getting the contents of the file as a stream of characters. A newline is just a control character that is used to display the file in a way that is easy for you to read. In this example, stdout is just a stream, so if I say that sed is a stream editor and it is used to modify the stream of characters, you would understand what I am trying to convey, right?

How do I use sed?

The sed command is invoked from the command line and it is used as:

 sed (command) /(script)  (file)(file)....

Sed has many commands, but the most common use of the sed command is for substitution. This is done using this format:

‘s/characters I do not want/characters I do want/’

If we create a file called example, that contains some characters:

 echo “EiEiEjejeieiO” > example

We can change the first ‘i’ to an ‘X’ using this command:

sed 's/i/X/' example

Output:

 EXEiEjejeieiO

Case matters and so the first lowercase i has been changed to an X.

It is important to note that we have not actually changed the file on disk, we are only manipulating the stdout stream. In order to change the file on disk, we would need to run the command with the -i “in place” flag, like so:

 sed -i 's/i/X/' example

This command will not return an output but we can then use:

cat example

To see the updated contents of the example file.

If I want to change all of the lowercase “i” to capitol “X”, I can add a modifier at the end of the expression, such as:

  sed 's/i/X/g' example

Output:

 EXEXEjejeXeXO

The same rules apply as before. We have not changed the file on disk, only the stdout stream. This is handy as it gives us a “dry-run” feature so that we can test our commands.

Sed and regular expressions

In its simplest form, sed can take the regex “.” and use it to match all the characters passed. In this case, we can use the operator L to convert the match to lowercase, and if we add an ampersand, this will use the entire match.

  sed ‘s/./L&/g’ example

Output:

 eieiejejeieio

I do not know why you would ever do this, but as an example of the regular expressions capability of sed, we can use it to print all of the IPv4 matches in our ifconfig using the -n flag and the p command (this is effectively the same as using grep):

 ifconfig | sed -E -n  "/(([0-9]{1,3}[.]){3}[0-9]{1,3})/ p"

Here, the -E flag is for Extended regular expressions, and the -n flag suppresses printing, unless an explicit print is found. The ending p is the print command that is inside of the match script.

Actual daily use of sed

Learn and get hands-on with modifying config files or YAML files in this Kubernetes Quick Start. If you are not familiar with YAML files, dive into YAML Essentials.

In the following example, here’s how you can change the service config from LoadBalancer to NodePort for a Kubernetes manifest:

type: LoadBalancer
selector:
app: someapplication
ports:
- name: http
nodePort: 31380
port: 80
targetPort: 80

You can change this to NodePort and provide the port number using this expression:

sed  ‘s/LoadBalancer/NodePort/;s/31380/30080/’

Sed is case sensitive, so make sure that you have your case correct. You can also separate the commands using the semi-colon ‘;’.

Here’s one last command example that I will leave here and let you figure out. I am not much of an internet troll but this should be good for some discussion:

   sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

If you want to know more about bash scripting, we have training for that. We also have courses covering regular expressions as well as the previously mentioned course about YAML. These are things that everyone should know as they will set you apart and get you noticed when it comes time to shine in front of the decision makers!

Until next time.. remember to feed your inner penguin!

0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *