Skip to main content

Mapping CSV Rows to Dictionaries

Hands-On Lab

 

Photo of Larry Fritts

Larry Fritts

Python Development Training Architect II

Length

00:30:00

Difficulty

Beginner

CSV files are an efficient and very common way to move data around. The can be imported into Excel very easily for review and editing, and then exported to CSV for other operations. They can also easily be imported into pandas dataframes for work.

So in this lab we will use Python's csv module to read CSV into dictionaries, allow the intended user to make necessary operations on the data in the dictionaries, and then return them to CSV for further operations.

You will need to know basic Python programming and have knowledge of Python's csv module:

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Mapping CSV Rows to Dictionaries

Introduction

CSV files are an efficient and very common way to move data around. The can be imported into Excel very easily for review and editing, and then exported to CSV for other operations. They can also easily be imported into pandas dataframes for work.

So in this lab we will use Python's csv module to read CSV into dictionaries, allow the intended user to make necessary operations on the data in the dictionaries, and then return them to CSV for further operations.

You will need to know basic Python programming and have knowledge of Python's csv module:

The Scenario

You work in the IT department of a school district. The district has switched to a new Learning Management System (LMS) to track grades. The new system only takes grade data in CSV format, and outputs information as CSV. Teachers have an app that can read a list of dictionaries and will aid them in entering the grades from the students.

Your job is to test that you can create a list of dictionaries out of the CSV file and then test that you can convert the dictionaries back into an updated CSV file.

To this end, you have created a test sample class and retrieved the CSV file for that class.

Logging In

There are a couple of ways to get in and work with the code. One is to use the credentials provided in the hands-on lab overview page, log in with SSH, and use a text editor in the terminal.

The other is using VS Code in a web browser. If you'd like to go this route, then you will need to navigate to the public IP address of the workstation server (provided in the hands-on lab page) on port 8080 (example: http://PUBLIC_IP:8080).

Whether you use a command line text editor or VS Code, the files we'll be working on are all sitting in /home/cloud_user.

Convert a CSV File to a List of Dictionaries

Open up csv_dict_conversion.py, and let's complete the def csv2dict(filename) function in there. What we need to add is after the three comment lines here:

def csv2dict(filename):

    # open file and create DictReader object
    # use the DictReader to get data and
    # add data to a variable named data_list
    with open(filename, newline='') as infile:
        reader = csv.DictReader(infile)

        data = []
        for row in reader:
            data_dict.append(row)
        return data

Run the script:

python csv_dict_conversion.py

We'll get an error, but it's not related to what we're doing at the moment. We've successfully read CSV data into dictionaries, so we can move on.

Convert a List of Dictionaries to CSV

To go from dictionaries back to CSV, we've got to finish the def dict2csv(filename, fieldnames, data) function. Again, what we need to add is everything here after the comment lines:

def dict2csv(filename, fieldnames, data):
    # open file and create DictWriter object
    # use the DictWriter to write csv data
    # to file

    with open(filename, 'w', newline='') as outfile:
        writer = csv.DictWriter(outfile, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(data)

If we run python csv_dict_conversion.py again, we'll see that it wrote the data to the CSV file, and even printed it out for us to see on the screen.

Conclusion

Congratulations! You've mastered moving data from CSV to dictionaries and dictionaries to CSV, and completed this hands-on lab.