Grading Python code and Jupyter Notebooks automatically
February 4, 2021

Webinar: How to automatically grade Python and Jupyter Notebook assignments

In 30 seconds...

Watch our webinar on Python and Jupyter Notebook autograding on demand now! This webinar was part of our monthly focus group series and took place on January 29th of 2021.

Creating an automatically graded Python assignment

In the first part of this webinar, we guide you through all the steps needed to set up an autograded Python assignment in CodeGrade. First, we create some simple I/O (Input / Output) tests to check the functionality of the student submission. Afterwards, we add a Code Quality test to automatically check the style of the student code for the PEP8 styleguide, for this we use the linter Flake8. Finally, we finish our AutoTest with adding some unit tests to autograde the functions our students create, for this we use Pytest, which CodeGrade slightly transformed for education and ease of use.

In our example, we grade Python code that uses Numpy. Of course, you can also grade Python code that uses other libraries like matplotlib, pandas or SciPy, these can be easily installed in the CodeGrade autograder.

Some other best practices shared in our focus group are:

Save time by autograding your Python and Jupyter Notebook assignments now!

Autograding Jupyter Notebooks

CodeGrade can also be used to automatically grade and review Jupyter Notebooks (or IPython Notebooks). Firstly, we will go over how to automatically run .ipynb files in CodeGrade, for manual code grading. After that, we will go over the easy steps required to autograde Jupyter Notebooks in CodeGrade, we do this by first converting the code to a regular Python script, which we then import in our I/O Test to grade and check variables or functions.

Finally, we briefly mentioned our in-house Jupyter Notebook unit testing framework called cg-jupyter-unit. This has similar functionalities to common tools like NBGrader but is made especially for code grade and has an easier user experience. This framework is now in closed beta; feel free to email us at if you want to participate.

Using Semgrep to autograde Python or Jupyter code structure

As per very common request by computer science instructors, we have researched and developed a way to very easily automatically grade code structure for Python code (but also for Java, JavaScript, C and many more). This done using a custom Semgrep CodeGrade script, which is made specifically for computer science educational purposes. You can read more about this in our latest blog post: Autograding code structure using CodeGrade and Semgrep.

Importing Python code without printing (for Jupyter I/O Tests)

We finish our webinar with a quick trick that will help you better grade Jupyter Notebooks automatically. As we import the distilled Python script in the Python interpreter, we get superfluous output if students manually print things in their Notebook. To prevent this, the following script can be used:

-!- CODE language-python -!-from contextlib import redirect_stdout
from os import devnull

with redirect_stdout(open(devnull, 'w')):
    import jupyter #<-- The name of your script

This webinar and article point you in the right direction to set up your own automatic tests for your programming assignments. Of course, the possibilities with CodeGrade's AutoTest are endless. Would you like to learn more about setting up Python or Jupyter Notebook automatic grading for your code exercise? Or would you like to participate in our closed betas? Feel free to email me at and I’d be more than happy to help out!

Devin Hillenius

Devin Hillenius

Co-founder, Product Expert
Devin is co-founder and Product Expert at CodeGrade. During his studies Computer Science and work as a TA at the University of Amsterdam, he developed CodeGrade together with his co-founders to make their life easier. Devin supports instructors with their programming courses, focusing on both their pedagogical needs and innovative technical possibilities. He also hosts CodeGrade's monthly webinar.

Continue reading

How to adopt automatic grading in your coding course

Autograding is becoming more and more important in coding courses. In this article we discuss how to successfully adopt autograding in your computer science classroom.

Pedagogical Benefits of Automatic Grading in Computer Science education

Computer Science teachers use autograding to scale up their coding classrooms, but also to improve student leaning outcomes and satisfaction. Learn the pedagogical benefits of autograding in this article.

CodeGrade Success Story at the University of Nevada, Las Vegas

CodeGrade increases student performance and helps teachers at UNLV. Professors and e-learning experts from the University of Nevada, Las Vegas share how they adopted CodeGrade and how it made their lives easier.

New CodeGrade features now delivered continuously to you!

CodeGrade now uses Continuous Delivery (CD) to deploy new bug fixes, tweaks and features even quicker and smoother to our customers. We chat with CodeGrade CTO Thomas Schaper to learn more about this improvement!

Sign up to our newsletter

Book a quick 30-minute demo and see how it works for you!