Automatically grading students’ Python assignments using pytest unit tests

Python continues to be one of the most widely used languages on the planet and is one of the most commonly graded languages in CodeGrade too. CodeGrade’s Input/Output (I/O) test step makes grading simple Python scripts a breeze, although it can be limiting when students start developing more complicated programs.

In this guide, I’ll discuss why unit testing could be useful for your python assignment. I will walk you through how you can set up an autograded Python assignment in CodeGrade using the pre-installed pytest unit testing framework.

Why Unit Test?

As students begin to develop complex programs, it is often accompanied by a segue into the concept of using functions. This encourages students to break down complex programs into manageable steps that each perform one role. In other words, they encapsulate their code using functions.

It is possible to test python functions in CodeGrade using I/O tests by importing the code and calling the functions as you would from any other module. However, by creating a unit test script instead, changing and personalizing your tests becomes much easier. What’s more, many instructors may already have unit test scripts available for their assignments that you can use in CodeGrade to give instant feedback to your students!

CodeGrade has almost all of the industry-standard unit testing tools pre-installed and available in the drop-down menu of the Unit Test step, including pytest. CodeGrade's pytest wrapper, `cg-pytest`, parses the xml produced by running the tests and displays each test separately as its own test step, providing some flexibility in the way you design your unit test setup.

Pytest for unit testing Python code

Pytest is an open-source python module that enables unit testing of python code in a simple and easy to understand format. The unit testing framework uses simple assertion statements to compare actual outcomes to predicted outcomes. It's designed for writing simple tests but can be scaled for complex functional testing. Read more about pytest and how it works. CodeGrade simply uses pytest under the hood, so all features and documentation of pytest apply to CodeGrade’s implementation too.

Creating an example Python assignment for unit testing

To demonstrate how we can set up a pytest unit test in CodeGrade, we first need the assignment to be tested. For this situation I have created a simple assignment in which students have been asked to define four basic functions accounting for four mathematical operations: addition, subtraction, multiplication and division. As a small bonus task, we ask the students to handle division by zero by raising an exception. The file they hand in should be called `calculator.py`. Following this prescription we find ourselves with the following code:

-!- CODE language-py -!-def add(x, y):
    ans = x + y
    return ans

def subtract(x, y):
    ans = x + y
    return ans

def multiply(x, y):
    ans = x * y
    return ans

def divide(x, y):

    if y ==  0:
    raise ValueError("Can not divide by zero!")
    ans = x / y
    return ans