Grading Python code and Jupyter Notebooks automatically
February 4, 2021

Webinar: How to automatically grade Python and Jupyter Notebook assignments

In 30 seconds...

Watch our webinar on Python and Jupyter Notebook autograding on demand now! This webinar was part of our monthly focus group series and took place on January 29th of 2021.

Creating an automatically graded Python assignment

In the first part of this webinar, we guide you through all the steps needed to set up an autograded Python assignment in CodeGrade. First, we create some simple I/O (Input / Output) tests to check the functionality of the student submission. Afterwards, we add a Code Quality test to automatically check the style of the student code for the PEP8 styleguide, for this we use the linter Flake8. Finally, we finish our AutoTest with adding some unit tests to autograde the functions our students create, for this we use Pytest, which CodeGrade slightly transformed for education and ease of use.

In our example, we grade Python code that uses Numpy. Of course, you can also grade Python code that uses other libraries like matplotlib, pandas or SciPy, these can be easily installed in the CodeGrade autograder.

Some other best practices shared in our focus group are:

Save time by autograding your Python and Jupyter Notebook assignments now!

Autograding Jupyter Notebooks

CodeGrade can also be used to automatically grade and review Jupyter Notebooks (or IPython Notebooks). Firstly, we will go over how to automatically run .ipynb files in CodeGrade, for manual code grading. After that, we will go over the easy steps required to autograde Jupyter Notebooks in CodeGrade, we do this by first converting the code to a regular Python script, which we then import in our I/O Test to grade and check variables or functions.

Finally, we briefly mentioned our in-house Jupyter Notebook unit testing framework called cg-jupyter-unit. This has similar functionalities to common tools like NBGrader but is made especially for code grade and has an easier user experience. This framework is now in closed beta; feel free to email us at if you want to participate.

Using Semgrep to autograde Python or Jupyter code structure

As per very common request by computer science instructors, we have researched and developed a way to very easily automatically grade code structure for Python code (but also for Java, JavaScript, C and many more). This done using a custom Semgrep CodeGrade script, which is made specifically for computer science educational purposes. You can read more about this in our latest blog post: Autograding code structure using CodeGrade and Semgrep.

Importing Python code without printing (for Jupyter I/O Tests)

We finish our webinar with a quick trick that will help you better grade Jupyter Notebooks automatically. As we import the distilled Python script in the Python interpreter, we get superfluous output if students manually print things in their Notebook. To prevent this, the following script can be used:

-!- CODE language-python -!-from contextlib import redirect_stdout
from os import devnull

with redirect_stdout(open(devnull, 'w')):
    import jupyter #<-- The name of your script

This webinar and article point you in the right direction to set up your own automatic tests for your programming assignments. Of course, the possibilities with CodeGrade's AutoTest are endless. Would you like to learn more about setting up Python or Jupyter Notebook automatic grading for your code exercise? Or would you like to participate in our closed betas? Feel free to email me at and I’d be more than happy to help out!

Devin Hillenius

Devin Hillenius

Co-founder, Product Expert
Devin is co-founder and Product Expert at CodeGrade. During his studies Computer Science and work as a TA at the University of Amsterdam, he developed CodeGrade together with his co-founders to make their life easier. Devin supports instructors with their programming courses, focusing on both their pedagogical needs and innovative technical possibilities. He also hosts CodeGrade's monthly webinar.

Continue reading

Could Natural Language Programming change the future of coding for the better?

We can all agree, coding is hard. But, imagine if one tool could simplify the process? In this blog, Sam discusses OpenAI Codex, a platform that translates natural language into code, using AI. Could this be the start of a new era of coding?

Do you know the 3 types of coders and their needs?

Learn how to identify the three types of coders: students coding to understand, students coding as a skill and those learning coding as a career. We also explain the needs of these groups and how you can best tailor your code classroom to them.

New release CodeGrade PerfectlyNormal.2 and Happy Holidays from CodeGrade!

Happy Holidays from Team CodeGrade! We recap 2021 and would love to tell you about CodeGrade PerfectlyNormal.2, improving the Peer Feedback feature and making CodeGrade even more efficient!

Using NBGrader for Python Jupyter notebooks in CodeGrade

Learn how to autograde Python Jupyter Notebooks using the NBGrader tool in CodeGrade to give your students instant feedback.

Learn more about CodeGrade!

Grow your coding classroom
without compromise.