The AWK Command | BASH Basics

Posted on March 13, 2019 by MichaelMcClarenMichaelMcClaren

Bash block logo

 

When I was first starting out as a sysadmin and I wanted to know what it took to get promoted, one of the things I was told was that I needed to know the BASH utilities, such as AWK. I’m not sure AWK is actually a utility — it was originally intended to be a data-driven programming language when Alfred Aho, Peter Weinberger, and Brian Kernighan wrote it at Bell Labs. The name comes from the initials of their last names, but the program we use in BASH on Linux is (G)AWK or Gnu AWK, and there are many derivatives.

What Can You Do with AWK?

Now that we have the trivia out of the way, what can we do with AWK? AWK can be invoked from the command prompt by simply typing awk. On the command line, it is all lower case. AWK is most often used for processing files. AWK processes a condition if one is provided and then takes an action. The default action is to print whatever meets the criteria of the condition.

To search a file for a regular expression pattern match:

   awk ‘/regex/’  /etc/passwd 

This would result in the matching lines from the /etc/passwd file being printed to the command line. This is fairly basic, and we are using the default behavior of printing the matches.

AWK can also be used to search columns of data and cleanly output information.

To know if a specific service is running and display the name if located:

   ps -ax | awk ‘/systemd/ {print $5}’

One of the nice things about awk is that it tokenizes the lines that are fed into it. This means for items that have a separator, you can select a column. You can specify a field separator on the command line. One example would be to print a list of users from /etc/passwd, the entries are separated by a colon (:) and the username is in the first column. Remember that $0 is the entire line.

  awk -F: ‘{print $1}’ /etc/passwd

Putting AWK to Work

This is great, but we want to do some work with AWK, right? So if you have a directory (we’ll use /etc for this one) and want to know the size of that directory:

  ls -l /etc | awk ‘{x += $5} END {print “total bytes:” x}'

Here, we take the output of the ls -l command and add each line to the variable x, since variables in AWK are initialized to 0 and the fifth column is the file size we are adding each file’s size to x as we parse the output. The END keyword indicates an action we want to take once the input parsing has completed. In this case, we print the value of x.

In kilobytes, we divide the result x by 1024:

  ls -l /etc | awk ‘{x += $5} END {print “total Kilobytes:” (x/1024)}

AWK can also be used as a full scripting language, and you can pass script files to it on the command line using the -f flag. That might be something we get into in a future blog post.

Where to Get Started

As I said in the beginning, command line usage is something that really separates people from their contemporaries when it comes to job interviews. Companies are into automation and everything as code. The place this starts is with knowledge of the Bash shell and all of its features. If you want to set yourself apart and show you are comfortable in what you are doing, you might consider taking a look at the Admin’s Guide to Bash ScriptingThis will take you to a place where you can really show your comfort level with the command line.

0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *