Pedagogical benefits of automatic code grading in coding education
Articles
October 25, 2022

Pedagogical Benefits of Automatic Grading in Computer Science education

In 30 seconds...

  • Computer Science is the fastest growing program in the US and UK, causing teachers and students to face unprecedented challenges.
  • Adopting autograding assists teachers to scale their classroom effectively and also comes with pedagogical benefits.
  • With autograding, the feedback loop is optimized: resulting in better learning outcomes and higher student satisfaction.
  • Autograders should not and will not replace teachers, they rather are "virtual assistants" that save teachers time grading and allow them to spend more time teaching.
  • Many institutions have seen an increase in student satisfaction and performance thanks to CodeGrade, read their stories here.

Challenges in Computer Science education in 2022

Computer Science programs all over the globe are facing unique challenges due to the high demand for skilled computer scientists and programmers. Student numbers are through the roof: the Computing Research Association reported that the number of CS undergraduates in the US more than doubled between 2013 to 2017 and Computer Science is the largest growing A Level subject in the UK according to BCS. At the same time, hiring and retaining faculty to meet this demand has been more challenging than ever. Universities simply cannot compete with tech giants and other big companies.

A 2019 article from The New York Times illustrates these challenges very well. As a result, computer science students from some campuses said they faced overcrowded classes with overworked professors. Coding electives have long waiting lists and university leaders are concerned they will have to limit access to Computer Science programs soon. Experts warn that having such acceptance criteria to CS majors may disadvantage those who are already unrepresented in computer science education.

More and more professors are adopting tools to assist them with this higher demand, in an effort to decrease the workload and manage unprecedented class sizes. Furthermore, with the popularity of coding MOOCs and electives rising, universities need to find ways to provide high quality coding education to a much larger and diverse audience. 

But, seeing the adoption of automatic code grading tools just as a way to manage larger class sizes does not do justice to the effects they have on professors and students. Effectively using autograding tools can greatly increase the quality of education, student performance and satisfaction. Those benefits will be discussed in this article.

Scale your computer science courses with CodeGrade!

Benefits from autograding in Computer Science courses

Chris Wilcox from Colorado State University paints an ideal coding course [1]: “ In an ideal world, instructors carefully review and grade all program submissions to provide feedback to students and identify coding problems. [...] Allowing multiple submissions per assignment to encourage students to fix defects and improve their code further increases this workload.” In his research he mentions: “To meet these requirements within our resource limitations we have adopted automated grading.”

Additionally, timely feedback is essential in coding education. Learning to code is a hands-on process, with a lot of practical assignments necessary. As students progress through these practicals, the only way for them to learn is to get near-immediate feedback on their iterative attempts. 

This so-called feedback loop is what accelerates learning:

  1. Student hands in code;
  2. Student waits for feedback;
  3. Student learns from their feedback and improves code;
  4. The above is repeated.

In traditional Computer Science classes, this feedback loop is often broken in two ways:

  • Students only get one attempt for their practicals (deadline), and cannot incorporate their feedback anymore.
  • The delay between handing in and feedback is too long.

The above two are a near constant factor if you look at the situations at institutions from before they adopted CodeGrade in our case studies. Professors traditionally have too few practical coding assignments (or none), do not allow multiple attempts with intermediate feedback or have no resources to give timely feedback (if they have resources to give feedback at all). The professors always know this is not an optimal learning environment, but lack resources (due to the challenges mentioned before) to do something about it.

With an autograding solution like CodeGrade, this feedback loop is optimized. Students get unlimited attempts before the deadline and get instant feedback on their code, giving them the option to learn and improve their code continuously. Using CodeGrade’s autograder has gotten the fastest growing OOP coding course at The University of Edinburgh a nomination for Best Course, based on students’ votes.

In line with our experience, Chris Wilcox concludes his research on autograding [1] with: “The benefits of automation are both tangible, such as higher exam scores, and intangible, such as increased student engagement and interest.”

Using an autograder to assist teachers

An often heard concern about autograders is that they would replace a teachers’ role and decrease the quality of education. This is a fair argument: automatically generated feedback can not compete with that of an experienced professor, nor can it replace valuable teacher-student interactions. But, in practice, effectively adopting an autograder in your coding course can improve personal feedback and give a professor more time for teacher-student interactions.

At CodeGrade we have recently changed our branding from “autograding” to “virtual assistant”, as we believe that is what CodeGrade does: we are a virtual assistant for CS professors. The University of Nevada, Las Vegas summarizes this very well in their recent article on CodeGrade: “Because CodeGrade is automated”, Jorgensen said that he can dedicate more time to focusing on students’ unique needs. “CodeGrade allows us to engage and give feedback to students too, going line by line and making comments,” he said. “With personalized feedback, students are much more likely to learn the concepts and do better on assignments.”. They also mentioned that: “CodeGrade has really made our lives easier. The lives of the TAs, the students, the teachers. We can focus more on helping the students and making sure that they’re understanding those concepts and focusing on reinforcing those concepts.”

By using an autograder to do all coding checks automatically, teachers can use that information to give personalized feedback and interact with students. Teachers shouldn’t be put in a position to decide between giving their students quality feedback and getting grades out on time. A sophisticated autograder can provide checks on code quality, structure, and functionality, and give the teacher time to actually teach.

The future of coding courses uses autograding

The increase in Computer Science students is not likely to slow down in the future. More and more students want to learn to code, whether it’s through a 1000 students Python MOOC from UNIR, already in middle school at Harvard-Westlake or in a brand new Data Science masters at Eastern University. Autograding can play a crucial role in assisting teachers to scale their course and improve the quality of their education. The bottom line is that teachers cannot and should not be replaced, but they should be supported.

As UNLV writes: “As the Computer Science industry continues to boom, the need for skilled coders is rapidly growing. With the help of CodeGrade, Jorgensen and his fellow professors can help the next generation of codemasters hone their skills and prepare them for the workforce.”

Bibliography

[1] Wilcox, C. (2015, February). The role of automation in undergraduate computer science education. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (pp. 90-95).

[2] Natasha Singer (2019). The Hard Part of Computer Science? Getting into class. In The New York Times (https://www.nytimes.com/2019/01/24/technology/computer-science-courses-college.html).

[3] Nicole Johnson (2022). Uncoding Grades: Autograding Tool Improves Student Performance. In UNLV IT News Center (https://www.it.unlv.edu/news/uncoding-grades-autograding-tool-improves-student-performance-0).

Devin Hillenius

Devin Hillenius

Co-founder, Product Expert
Devin is co-founder and Product Expert at CodeGrade. During his studies Computer Science and work as a TA at the University of Amsterdam, he developed CodeGrade together with his co-founders to make their life easier. Devin supports instructors with their programming courses, focusing on both their pedagogical needs and innovative technical possibilities. He also hosts CodeGrade's monthly webinar.

Continue reading

How to adopt automatic grading in your coding course

Autograding is becoming more and more important in coding courses. In this article we discuss how to successfully adopt autograding in your computer science classroom.

CodeGrade Success Story at the University of Nevada, Las Vegas

CodeGrade increases student performance and helps teachers at UNLV. Professors and e-learning experts from the University of Nevada, Las Vegas share how they adopted CodeGrade and how it made their lives easier.

New CodeGrade features now delivered continuously to you!

CodeGrade now uses Continuous Delivery (CD) to deploy new bug fixes, tweaks and features even quicker and smoother to our customers. We chat with CodeGrade CTO Thomas Schaper to learn more about this improvement!

5 reasons to use CodeGrade in your high school coding classroom

Coding education is growing, especially in high schools. Find out how CodeGrade can support K12 students.

Sign up to our newsletter

Book a quick 30-minute demo and see how it works for you!