Grading Python code and Jupyter Notebooks automatically
February 4, 2021

Webinar: How to automatically grade Python and Jupyter Notebook assignments

In 30 seconds...

Watch our webinar on Python and Jupyter Notebook autograding on demand now! This webinar was part of our monthly focus group series and took place on January 29th of 2021.

Creating an automatically graded Python assignment

In the first part of this webinar, we guide you through all the steps needed to set up an autograded Python assignment in CodeGrade. First, we create some simple I/O (Input / Output) tests to check the functionality of the student submission. Afterwards, we add a Code Quality test to automatically check the style of the student code for the PEP8 styleguide, for this we use the linter Flake8. Finally, we finish our AutoTest with adding some unit tests to autograde the functions our students create, for this we use Pytest, which CodeGrade slightly transformed for education and ease of use.

In our example, we grade Python code that uses Numpy. Of course, you can also grade Python code that uses other libraries like matplotlib, pandas or SciPy, these can be easily installed in the CodeGrade autograder.

Some other best practices shared in our focus group are:

Save time by autograding your Python and Jupyter Notebook assignments now!

Autograding Jupyter Notebooks

CodeGrade can also be used to automatically grade and review Jupyter Notebooks (or IPython Notebooks). Firstly, we will go over how to automatically run .ipynb files in CodeGrade, for manual code grading. After that, we will go over the easy steps required to autograde Jupyter Notebooks in CodeGrade, we do this by first converting the code to a regular Python script, which we then import in our I/O Test to grade and check variables or functions.

Finally, we briefly mentioned our in-house Jupyter Notebook unit testing framework called cg-jupyter-unit. This has similar functionalities to common tools like NBGrader but is made especially for code grade and has an easier user experience. This framework is now in closed beta; feel free to email us at if you want to participate.

Using Semgrep to autograde Python or Jupyter code structure

As per very common request by computer science instructors, we have researched and developed a way to very easily automatically grade code structure for Python code (but also for Java, JavaScript, C and many more). This done using a custom Semgrep CodeGrade script, which is made specifically for computer science educational purposes. You can read more about this in our latest blog post: Autograding code structure using CodeGrade and Semgrep.

Importing Python code without printing (for Jupyter I/O Tests)

We finish our webinar with a quick trick that will help you better grade Jupyter Notebooks automatically. As we import the distilled Python script in the Python interpreter, we get superfluous output if students manually print things in their Notebook. To prevent this, the following script can be used:

-!- CODE language-python -!-from contextlib import redirect_stdout
from os import devnull

with redirect_stdout(open(devnull, 'w')):
    import jupyter #<-- The name of your script

This webinar and article point you in the right direction to set up your own automatic tests for your programming assignments. Of course, the possibilities with CodeGrade's AutoTest are endless. Would you like to learn more about setting up Python or Jupyter Notebook automatic grading for your code exercise? Or would you like to participate in our closed betas? Feel free to email me at and I’d be more than happy to help out!

Devin Hillenius

Devin Hillenius

Co-founder, Product Expert
Devin is co-founder and Product Expert at CodeGrade. During his studies Computer Science and work as a TA at the University of Amsterdam, he developed CodeGrade together with his co-founders to make their life easier. Devin supports instructors with their programming courses, focusing on both their pedagogical needs and innovative technical possibilities. He also hosts CodeGrade's monthly webinar.

Continue reading

Teaching Intro to Python with CodeGrade

The CodeGrade Introduction to Python course is an 8-week basic Python course. Students are not required to have any prior knowledge on programming or Python. This course will cover the basic concepts of programming up to Python specific modules and OOP design. It is available for all instructors now.

New release CodeGrade QuietStorm.1!

Find out about all the features and updates coming with our latest release, QuietStorm.1!

New release CodeGrade QuietStorm.1!

Automatically grading Haskell code assignments

Learn about autograding Haskell coding assignments for Computer Science education courses. CodeGrade can help you use tools like input and output checking, Quickcheck, Tasty and HUnit unit test autograding, HLint code quality checking and code structure autograding using semgrep.

Top tips for teaching programming

The most efficient ways to teach students how to code, by defining your target audience, designing your assignments in a meaningful way and eliminating distractions in the classroom.

Top tips for teaching programming

Learn more about CodeGrade!

Grow your coding classroom
without compromise.