Benefits from autograding in Computer Science courses
Chris Wilcox from Colorado State University paints an ideal coding course [1]: “ In an ideal world, instructors carefully review and grade all program submissions to provide feedback to students and identify coding problems. [...] Allowing multiple submissions per assignment to encourage students to fix defects and improve their code further increases this workload.” In his research he mentions: “To meet these requirements within our resource limitations we have adopted automated grading.”
Additionally, timely feedback is essential in coding education. Learning to code is a hands-on process, with a lot of practical assignments necessary. As students progress through these practicals, the only way for them to learn is to get near-immediate feedback on their iterative attempts.
This so-called feedback loop is what accelerates learning:
- Student hands in code;
- Student waits for feedback;
- Student learns from their feedback and improves code;
- The above is repeated.
In traditional Computer Science classes, this feedback loop is often broken in two ways:
- Students only get one attempt for their practicals (deadline), and cannot incorporate their feedback anymore.
- The delay between handing in and feedback is too long.
The above two are a near constant factor if you look at the situations at institutions from before they adopted CodeGrade in our case studies. Professors traditionally have too few practical coding assignments (or none), do not allow multiple attempts with intermediate feedback or have no resources to give timely feedback (if they have resources to give feedback at all). The professors always know this is not an optimal learning environment, but lack resources (due to the challenges mentioned before) to do something about it.
With an autograding solution like CodeGrade, this feedback loop is optimized. Students get unlimited attempts before the deadline and get instant feedback on their code, giving them the option to learn and improve their code continuously. Using CodeGrade’s autograder has gotten the fastest growing OOP coding course at The University of Edinburgh a nomination for Best Course, based on students’ votes.
In line with our experience, Chris Wilcox concludes his research on autograding [1] with: “The benefits of automation are both tangible, such as higher exam scores, and intangible, such as increased student engagement and interest.”
Using an autograder to assist teachers
An often heard concern about autograders is that they would replace a teachers’ role and decrease the quality of education. This is a fair argument: automatically generated feedback can not compete with that of an experienced professor, nor can it replace valuable teacher-student interactions. But, in practice, effectively adopting an autograder in your coding course can improve personal feedback and give a professor more time for teacher-student interactions.
At CodeGrade we have recently changed our branding from “autograding” to “virtual assistant”, as we believe that is what CodeGrade does: we are a virtual assistant for CS professors. The University of Nevada, Las Vegas summarizes this very well in their recent article on CodeGrade: “Because CodeGrade is automated”, Jorgensen said that he can dedicate more time to focusing on students’ unique needs. “CodeGrade allows us to engage and give feedback to students too, going line by line and making comments,” he said. “With personalized feedback, students are much more likely to learn the concepts and do better on assignments.”. They also mentioned that: “CodeGrade has really made our lives easier. The lives of the TAs, the students, the teachers. We can focus more on helping the students and making sure that they’re understanding those concepts and focusing on reinforcing those concepts.”
By using an autograder to do all coding checks automatically, teachers can use that information to give personalized feedback and interact with students. Teachers shouldn’t be put in a position to decide between giving their students quality feedback and getting grades out on time. A sophisticated autograder can provide checks on code quality, structure, and functionality, and give the teacher time to actually teach.
The future of coding courses uses autograding
The increase in Computer Science students is not likely to slow down in the future. More and more students want to learn to code, whether it’s through a 1000 students Python MOOC from UNIR, already in middle school at Harvard-Westlake or in a brand new Data Science masters at Eastern University. Autograding can play a crucial role in assisting teachers to scale their course and improve the quality of their education. The bottom line is that teachers cannot and should not be replaced, but they should be supported.
As UNLV writes: “As the Computer Science industry continues to boom, the need for skilled coders is rapidly growing. With the help of CodeGrade, Jorgensen and his fellow professors can help the next generation of codemasters hone their skills and prepare them for the workforce.”
Bibliography
[1] Wilcox, C. (2015, February). The role of automation in undergraduate computer science education. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (pp. 90-95).
[2] Natasha Singer (2019). The Hard Part of Computer Science? Getting into class. In The New York Times (https://www.nytimes.com/2019/01/24/technology/computer-science-courses-college.html).
[3] Nicole Johnson (2022). Uncoding Grades: Autograding Tool Improves Student Performance. In UNLV IT News Center (https://www.it.unlv.edu/news/uncoding-grades-autograding-tool-improves-student-performance-0).
