Grading Python code and Jupyter Notebooks automatically
February 4, 2021

Webinar: How to automatically grade Python and Jupyter Notebook assignments

Watch our webinar on Python and Jupyter Notebook autograding on demand now! This webinar was part of our monthly focus group series and took place on January 29th of 2021.

Creating an automatically graded Python assignment

In the first part of this webinar, we guide you through all the steps needed to set up an autograded Python assignment in CodeGrade. First, we create some simple I/O (Input / Output) tests to check the functionality of the student submission. Afterwards, we add a Code Quality test to automatically check the style of the student code for the PEP8 styleguide, for this we use the linter Flake8. Finally, we finish our AutoTest with adding some unit tests to autograde the functions our students create, for this we use Pytest, which CodeGrade slightly transformed for education and ease of use.

In our example, we grade Python code that uses Numpy. Of course, you can also grade Python code that uses other libraries like matplotlib, pandas or SciPy, these can be easily installed in the CodeGrade autograder.

Some other best practices shared in our focus group are:

Save time by autograding your Python and Jupyter Notebook assignments now!

Autograding Jupyter Notebooks

CodeGrade can also be used to automatically grade and review Jupyter Notebooks (or IPython Notebooks). Firstly, we will go over how to automatically run .ipynb files in CodeGrade, for manual code grading. After that, we will go over the easy steps required to autograde Jupyter Notebooks in CodeGrade, we do this by first converting the code to a regular Python script, which we then import in our I/O Test to grade and check variables or functions.

Finally, we briefly mentioned our in-house Jupyter Notebook unit testing framework called cg-jupyter-unit. This has similar functionalities to common tools like NBGrader but is made especially for code grade and has an easier user experience. This framework is now in closed beta; feel free to email us at if you want to participate.

Using Semgrep to autograde Python or Jupyter code structure

As per very common request by computer science instructors, we have researched and developed a way to very easily automatically grade code structure for Python code (but also for Java, JavaScript, C and many more). This done using a custom Semgrep CodeGrade script, which is made specifically for computer science educational purposes. You can read more about this in our latest blog post: Autograding code structure using CodeGrade and Semgrep.

Importing Python code without printing (for Jupyter I/O Tests)

We finish our webinar with a quick trick that will help you better grade Jupyter Notebooks automatically. As we import the distilled Python script in the Python interpreter, we get superfluous output if students manually print things in their Notebook. To prevent this, the following script can be used:

-!- CODE language-python -!-from contextlib import redirect_stdout
from os import devnull

with redirect_stdout(open(devnull, 'w')):
    import jupyter #<-- The name of your script

This webinar and article point you in the right direction to set up your own automatic tests for your programming assignments. Of course, the possibilities with CodeGrade's AutoTest are endless. Would you like to learn more about setting up Python or Jupyter Notebook automatic grading for your code exercise? Or would you like to participate in our closed betas? Feel free to email me at and I’d be more than happy to help out!

Devin Hillenius

Devin Hillenius


Continue reading

How to Stop Caring about Grades

Learn to save time with grading with alternative methods for teaching. Make your course accessible and inclusive with methods backed by research.

How to Stop Caring about Grades

Embracing a New Era: Harnessing Generative AI for Computer Science Education

Discover the transformative impact of Generative AI (Gen AI) on computer science education. Explore innovative tools and methodologies reshaping learning experiences and preparing learners for an AI-centric future. Uncover benefits, challenges, and evolving practices at the intersection of technology and pedagogy.

Coding Quizzes are here!

Discover the latest addition to CodeGrade: coding quizzes! Elevate your learning experience with dynamic assessments seamlessly integrated with AutoTest V2. Engaging, clear, and user-friendly. Learn more now!

Coding Quizzes are here!

Transforming CS Education: Innovative Strategies for Curriculum Enhancement

Discover how Experiential Learning Theory transforms traditional teaching methods and improves computer science curriculum for optimal student engagement and success.

Sign up to our newsletter

See how CodeGrade can transform your courses today!