In our latest webinar, we tell you everything you need to know to get started with Jupyter Notebooks (also known as IPython Notebooks) in CodeGrade. We explain how Jupyter Notebooks are structured and how you can best manually grade and give feedback to them using CodeGrade's inline feedback, general feedback and render viewer. Furthermore, we teach you how you can effectively and easily autograde the Python code in Jupyter Notebooks. This webinar was part of our monthly CodeGrade Webinars series and was recorded live on December 3rd 2021, it is available on demand now.
Jupyter Notebook theory
Jupyter Notebooks, formerly known as IPython Notebooks, are files with the `.ipynb` extension. Even though Jupyter Notebooks support multiple programming languages, we most often see them used for Python courses. They combine these "code cells" with text cells (supporting Markdown and LaTeX). This combination makes for a very powerful educational tool that is used not only for courses like Introduction to Python, but also more advanced courses like Machine Learning, Data Science and Computer Vision.
On the inside, a Jupyter Notebook is simply a JSON document. All cells, metadata and output are stored in JSON. It is very important to understand that the output is also statically saved here and can thus be altered manually or be outdated if a Jupyter Notebook has not been run.
Automatically running Jupyter Notebooks for manual grading
As mentioned before: the output cells are also saved in the Jupyter Notebook JSON document. This means that when we are grading a Jupyter Notebook handed in by the student in CodeGrade, we are not guaranteed to have the latest version of the output (it can be manually altered by the student or the notebook could not be run). To solve this, we can use CodeGrade AutoTest to automatically run each IPython Notebook immediately after they are handed in. An up-to-date version of the Jupyter Notebook will then show up in the Code Viewer under the AutoTest output:
Formatting a Jupyter Notebook for easy and effective autograding
The easiest and most common way to autograde Jupyter Notebooks is to convert them to code, i.e. appending all code cells to form a regular Python script. In CodeGrade, we can then assess this Python script like we would normally do.
There are a couple of ways to easily assess multiple assets of a Jupyter Notebook assignment in CodeGrade:
We can assess the content of variables for correctness
We can assess the correct working of functions
We can run the whole sequence of code cells and check the output as a whole
In the webinar, we show you step by step how you can set up an AutoTest in CodeGrade to autograde these parts of a Jupyter Notebook assignment. What is important to note is that you need to know the names of the variables and functions to be able to assess them. A good practice is to provide your students with a template Jupyter Notebook file with predefined variables and functions that they can fill in themselves.
Autograding Jupyter Notebooks in CodeGrade
Autograding Jupyter Notebooks in CodeGrade becomes easy when formatting your assignment as explained above. The steps to do so are shown in the webinar, to summarize:
Convert the `.ipynb` file to a Python script in a Run Program step using `jupyter nbconvert --to script jupyter.ipynb`. (change the name `jupyter.ipynb` to the name of your Jupyter Notebook assignment, you can enforce this using Hand In Requirements)
Create an I/O Test in the same that runs `python -i jupyter` (change the name `jupyter` to the name of your assignment, this is the file you generated in the step above).
In this I/O test, you can now use regular Python as input and assess individual parts of the assignment. For instance, printing a variable or running a function and printing its result. You can write the expected output in the "Expected Output" field.
Ignoring superfluous print statements
One thing to be aware of is that we are checking the stdout of the scripts, which are run completely when importing. As a result, students can clutter the output with additional print statements outside of functions. There’s two ways to prevent this:
Importing the script with the stdout redirected. This can be done using this little code snippet, which you run via `python3 -i import_without_print.py`:
-!- CODE language-python -!-from contextlib import redirect_stdout from os import devnull
with redirect_stdout(open(devnull, 'w')): import jupyter #<-- The name of your script
If you are providing Jupyter Notebook skeleton code to your students, you can make sure to add a `if __name__ == "__main__":` statement every time you print solutions. This way, the students can see their solutions while interacting with the notebook, but these solutions will not be printed when importing the code (using the script above or with a regular import).
The above just lists a brief overview of what is discussed in the webinar. In the webinar, we went into detail on all things grading Jupyter Notebooks and went over all the steps on how to reproduce this for your assignment. We are here to make your course with CodeGrade a success, so please do not hesitate to reach out to us on Discord or via email@example.com with any question, feature request or if you would like some technical or educational consultancy. We are always more than happy to help you out!
Co-founder, Product Expert
Devin is co-founder and Product Expert at CodeGrade. During his studies Computer Science and work as a TA at the University of Amsterdam, he developed CodeGrade together with his co-founders to make their life easier. Devin supports instructors with their programming courses, focusing on both their pedagogical needs and innovative technical possibilities. He also hosts CodeGrade's monthly webinar.
The CodeGrade Introduction to Python course is an 8-week basic Python course. Students are not required to have any prior knowledge on programming or Python. This course will cover the basic concepts of programming up to Python specific modules and OOP design. It is available for all instructors now.
Learn about autograding Haskell coding assignments for Computer Science education courses. CodeGrade can help you use tools like input and output checking, Quickcheck, Tasty and HUnit unit test autograding, HLint code quality checking and code structure autograding using semgrep.