Learn how to use ChatGPT or AI-assistants in your coding classroom without plagiarism, 5 code assignment design principles for ChatGPT.
Articles
April 20, 2023

5 assignment design ideas for ChatGPT in Computer Science Education

In 30 seconds...

  • OpenAI's ChatGPT lets students easily generate code and solve coding problems;
  • We discuss 5 assignment design ideas to make ChatGPT-proof assignments and familiarize students with AI;
  • The first two design ideas work with an instant autograder. With these methods, students will explore new requirements and edge-cases on their own while interacting with the autograder;
  • Design idea 3 discusses on-site exams using CodeGrade's IDE and a lockdown browser. We believe that final exams should always be in combination with formative assessments throughout the course;
  • Finally, we discuss how you can set up assignments to teach students AI ethics and AI literacy and also help them develop a healthy relationship with the AI tools that are here to stay;
  • This is the second article in our ChatGPT series, you can read our previous article here.

ChatGPT has been causing educators headaches since its release in late 2022. ChatGPT allows users to chat with an AI language model (”GPT-3.5” to be exact) in an easy-to-use environment for free. Naturally, this has disrupted education and caused great challenges for academic integrity. One of the highly affected fields is that of Computer Science, in which students can simply use ChatGPT or AI-assistants to generate code.

In our previous article, we discussed how ChatGPT can already help students generate code and solve coding problems (ChatGPT can easily pass the 2022 AP Computer Science A exam). Detecting AI-generated code is inherently hard, and as the language model keeps evolving, ChatGPT classifiers and plagiarism scanners will still remain very unreliable. Instead of banning ChatGPT, many teachers rather consider ChatGPT as the “next calculator” or “next Wikipedia”, in other words: design education that works with ChatGPT instead of against it.

In today’s article, we will discuss 5 code assignment design ideas that will make your computer science courses more ChatGPT-proof. These ideas “exploit” some of the known limitations of ChatGPT, which essentially is just a “bullshit generator” , a term coined by Princeton CS professor Arvind Narayanan. We also discuss how you can better prepare students for a future with AI. In an effort to help towards education that works with ChatGPT instead of against it.

Want to learn more about ChatGPT in Coding Education? Join our live webinar on June 1st 2023, sign up below:

Design idea 1: Inexplicit Edge Cases

From our internal research on using ChatGPT to generate code, we see that it does a very good job at solving (basic) programming assignments. As you can read in our previous article, ChatGPT has no problem coming up with the correct code if we feed it any of these:

  • The entire assignment description, including context and background information;
  • Just the example (terminal) output and ask it to reverse engineer that;
  • Partial code and ask it to complete that.

What we did notice, however, is that the solution ChatGPT comes up with is often the most bare-bones solution. It fails to include edge cases and error handling, both often an important part of understanding a solution. See this example when tasked to create a Fibonacci generator:

ChatGPT generating a solution to a code assignment

The basics are correct: the code indeed does print the Fibonacci sequence. However, it misses essential edge-cases and input validation:

  • Missed edge case: for n=0 and n=1 the output is incorrect, it will always print at least the first two elements of the fibonacci sequence.
  • Missed input validation: it does not check if the user input is indeed an integer and it does not check if the input is indeed non-negative.

This is a relatively small code task, but the larger the task, the more potential edge cases ChatGPT will miss. Use this to your advantage: don’t explicitly mention required edge-cases in your descriptions, but instead only grade them using your autograder or unit tests. With an instant code autograder like CodeGrade, students will directly see the tests that passed and failed after submitting / running their code. This workflow also encourages students to continuously hand in intermediate submissions too. This will prevent students from using ChatGPT to come up with a full correct version instantly but instead discover these edge-cases on their own and come up with a solution themselves. These edge-cases simply show up as autograder tests and are not explicitly mentioned in the assignment description.

Using CodeGrade’s autograder? Let students interactively discover these edge-cases using our instant feedback (and online IDE). Want to challenge your students more? Simply make the edge cases more vague, so that your students know they missed “an edge-case” but have to discover which one this is themselves.

Design idea 2: Progressive Assignments

Traditionally, many computer science assignments look very similar: students get a document with one larger code task, sometimes including additional example output or code snippets. They then tackle this problem and hand in their code once before the deadline. This approach is very vulnerable to AI code generators: teachers have no way to inspect students progress and students lack feedback before the deadline. If they have only one assignment and one hand-in attempt, it is very easy to exploit other resources in coming up with that one solution.

With new interactive grading tools like CodeGrade, students can interact with an autograder (and optionally work in an online IDE). This has two large benefits against ChatGPT:

  • Students can interact with the autograder and are encouraged to hand in subsequently before the deadline;
  • Naturally, this allows teachers to get an overview of the progress made by the student.

We can use these benefits to improve our assignment design, going from a one-shot assignment to progressive assignments. In short:

  1. Give students a very brief and minimal assignment description to start with.
  2. Let students discover further assignment requirements and further sub-tasks using your autograder.

The benefit of this method over a traditional one-shot assignment is that students still need computational thinking and problem solving skills to solve an incremental assignment. With each additional sub-task, they have to first understand the additional requirements and then interpret how this should affect and change their code project. It is impossible to experience these problems if students simply get the entire code for a one-shot assignment generated through ChatGPT.

In the example of Fibonacci, we can have a very basic initial assignment description telling students to implement Fibonacci using the for-loop. After which, students are prompted with new subsequent sub-tasks using automatic tests and rubric categories in CodeGrade that only show up after they start handing in, see the AutoTest example from CodeGrade below:

Progressive code assignment set-up in CodeGrade's automatic code grader

Design idea 3: On-site exams with lockdown browser and IDE

We believe coding is a skill best learned by doing. Students need constant formative assessments throughout their academic career to acquire the coding skills they need. However, a great addition to these weekly or bi-weekly coding assessments can be an on-site final exam.

It is no surprise that it is easier to prevent students from using AI assisted coding tools in the controlled setting of an on-site final exam. What is hard, however, is to make sure students get the right coding environment during such an exam and that the logistics work for you as a course coordinator. This often required manually installing the right tools on hundreds of lab computers and afterwards manually collecting all submissions: a lot of administrative and logistic work for the teacher.

This does not mean we should go back to letting students code on paper (as former computer science students ourselves, we have been there), but instead make use of some novel online tools to reduce the administrative burden and let students code in a familiar environment. By combining CodeGrade’s IDE with a lockdown browser, you can control exactly which sites your students are and are not allowed to use. For instance: disabling ChatGPT but enabling useful resources like W3Schools or the official documentation of a used library. With CodeGrade’s IDE, students have a bare-bones but familiar coding experience, and with easy to set up instant autograding, they can validate and run (parts of) their code instantly during the exam.

We have great experience helping teachers set up on-site exams with a lockdown browser and CodeGrade's IDE. Contact us to learn more about our integrations!

Design idea 4: Debug-ChatGPT assignments

Just like many CS teachers, we believe that it is important to consider incorporating ChatGPT in programming courses. In this way students become familiar with the strengths and weaknesses of AI assistants during coding and can create a healthy relationship with these tools continuing their (academic) careers.

As Orit Hazzan discusses in her recent ACM article, we should explore ways to use ChatGPT in the computer science classroom and even make it part of code tasks. Some suggestions she makes are letting students explain code output from ChatGPT, explain differences between student code and ChatGPT code and analyzing ChatGPT code output.

The latter is especially interesting and lets us come up with a new type of assignment: a “Debug-ChatGPT Code” assignment. For a programming assignment, students are given a piece of code generated by ChatGPT and are tasked to debug / improve that code on aspects like readability, documentation, code style, correctness, efficiency, edge-cases and input handling. Also having to check if the logic works as expected. As most code is already supplied to the students, the difficulty level of the code can be a little bit higher than the students’ skill level.

By including a “Debug-ChatGPT Code” assignment to your code curriculum, students quickly get familiarized with the common weaknesses of ChatGPT. What’s more, it is a good way to show students that their skills have far from become obsolete. This will help students develop a healthier relationship to AI assisted coding tools and better understand when to use them and when not to.

Design idea 5: More focus on AI ethics and AI literacy

If anything has become clear from the launch of ChatGPT, it is that we are quickly entering a new era of AI in our lives. Over the past months, we have already seen rapid developments in the field and more is still to come. Just like calculators and Wikipedia shook up the educational world but have stayed ever since, AI assisted writing and coding tools are here to stay. (It was surprising how closely the concerns on Wikipedia in the early 2000s resemble the current concerns with ChatGPT)

With great AI tools comes great responsibility. But also, an opportunity to develop new and improved coding workflows. Just a quick Google Search already finds hundreds of articles discussing how ChatGPT can reduce workloads in many fields. It is important for teachers and curriculum designers to quickly step in and make sure to aid their students deal with the effects of AI in their work and life.

AI literacy will become more important, even as early as in K-12 education. A great open source K-12 AI literacy course is the DAILy curriculum designed by MIT educators. As discussed in this recent MIT Technology Review article, an added benefit is how ChatGPT forced many schools and universities to quickly come up with AI 101 courses. It also suggests a great AI literacy initiative for adults named Elements of AI, developed by MinnaLearn and the University of Helsinki.

Wrapping up…

More and more tools to generate code with AI are popping up, from ChatGPT to GitHub Copilot or Amazon’s CodeWhisperer, and if the past months of rapid developments have taught us anything is that these tools are here to stay and are not just a temporary hype. Many teachers have already drawn the conclusion that banning ChatGPT is not the best solution and instead we should focus on incorporating ChatGPT in our assignments. We hope that the assessment design principles above can help you come up with not just “AI-proof” code assignments, but also with ways to prepare students for a life and work with AI.

With that being said, in this blog post and our previous one, we have focussed mostly on the negative implications of AI on education and help instructors battle that. We are also very much excited by these new innovations in the field of AI. It feels too one-sided to not focus on the positive implications tools like ChatGPT can have on education. That’s why in our next article in our ChatGPT series, we have focussed on the benefits AI has for teachers, how you can use it to automate parts of your work, generate parts of your assignments and help set up your CodeGrade autograders. Click here to read our article on using ChatGPT as a CS instructor.

Already want to learn more about CodeGrade? Or would you like to chat with us to discuss the implications that ChatGPT has on your specific CS course? Shoot us an email at hello@codegrade.com and we would love to call!

Continue reading

Teaching Engineers to Program with AI

Explore how AI and tools like CodeGrade are transforming coding education, with insights from Ferris State University's innovative approach to teaching Python and microcontrollers

Watch here! AI Code Assistant Webinar with Brian Brady!

Learn how CodeGrade’s AI Code Assistant enhances programming education with real-world examples, practical tips, and a live demo from Ferris State University.

Join our next webinar: Teaching Introductory Python!

Discover updated Python content, assignment strategies, and grading tools in our Dec 12th webinar!

How to automatically grade R assignments

Streamline R Assignments with Automatic Grading in CodeGrade

Sign up to our newsletter

See how CodeGrade can transform your courses today!