Until recently, grading an intro programming assignment was fairly simple. If the program ran and passed the tests, that was decent evidence the student had learned something. Today a working program can come from a single prompt, so passing the tests says a lot less about what the student actually understands.
In the 2026 Student Generative AI Survey from the Higher Education Policy Institute (HEPI), 94% of full-time UK undergraduates said they use generative AI to help with assessed work. That's UK data, but the direction is hard to ignore.
What the research says about understanding AI output
The picture for novices is mixed. In The Widening Gap, a 2024 study by Prather and colleagues based on 21 lab sessions with novice programmers, students who already knew what they wanted to build used AI to get there faster. Students who struggled kept their old difficulties, and AI could make them worse. Many of them, in the authors' words, "finished with an illusion of competence."
A randomized trial by Anthropic points the same way. It involved 52 mostly junior software engineers, and the group using AI averaged 50% on a follow-up quiz against 67% for those who coded by hand, a gap of nearly two letter grades. The accompanying paper by Shen and Tamkin found that AI use "impairs conceptual understanding, code reading, and debugging abilities", although the ways of using AI that kept people thinking preserved learning.
The code itself deserves some suspicion too. In the 2025 Stack Overflow Developer Survey, more developers said they distrust the accuracy of AI tools (46%) than trust it (33%), and their biggest frustration, at 66%, was "AI solutions that are almost right, but not quite." For its 2025 GenAI Code Security Report, the security firm Veracode tested code from more than 100 language models and found security vulnerabilities in 45% of cases.
Experience doesn't fully protect against this either. A 2025 study by METR, a research nonprofit, followed 16 experienced open-source developers working on real issues. They took 19% longer when they could use AI, yet believed it had sped them up by 20%. Those were early-2025 tools and METR has since published newer data. Even so, it's a good reminder that feeling fast with AI and being right are easy to confuse, and students have far less experience to check against.
Why bans and detection fall short
Banning AI is hard to enforce, and it skips the skill students will need most. Detection answers the question of who typed the code, which tells you little about whether the student understood it.
Secure exams and proctoring tools have their place, because they show what a student can do with no AI in the room, and you can run secure, proctored coding exams right inside CodeGrade. But if all the weight moves to the exams, open assignments stop counting, and the grade ends up measuring a way of working that looks less and less like the job. A better goal, we think, is students who can question AI code and explain it, since that's what they'll be doing at work.
Thinking in loops
Working well with AI usually takes several rounds. The student asks, reads what comes back and decides what's worth keeping, then asks again with a better question.
Each round involves a decision, and those decisions are what a teacher should be looking at. A student who writes "I kept this function because it handles the empty list, but I rewrote the sort because it changed the original data" clearly understands the code. A student who accepted everything, or rejected everything, hasn't shown that yet.
So we'd ask students to show the loop along with the result. If students work inside CodeGrade’s AI assistant, every conversation is visible to the instructor, so the loop is already on record. In practice that can be a short note with the prompts that mattered and a sentence or two on what they changed and why.
What the teacher's role becomes
Teachers have always judged learning as well as output, and with AI in the room that part of the job matters more. So more of the work goes into designing assignments that show how the student thought, such as how they steered the AI and whether they can stand behind what they handed in.
That changes what a teacher looks at when grading, because the process behind the work becomes part of the evidence. It gives you something better to go on than a hunch about who wrote what, and the grade stays with you.
Four things to try next term
1. Ask students to explain code in plain English. As Denny and colleagues note in a 2024 paper, reading, understanding and explaining code "have traditionally been important skills for novices learning programming." They propose pairing these with prompting practice.
2. Hold short conversations about their own submission. In a 2024 study, Kannam and colleagues converted most weekly TA sections into code interviews on five take-home assignments. The interviews pushed students to discuss their own work, with mixed effects on stress and less room for TA feedback. If you try this, start small and sample a few students per assignment. Pairing this with inline code feedback lets you leave the follow-up comments right on the lines you discussed.
3. Write rubrics that reward verification and correction. If points only go to passing tests, students will optimize for passing tests. Give credit for tests the student wrote and for fixes they can justify, including bugs they found in AI output.
4. State your AI policy up front, per assignment. Say where AI is allowed and where it isn't, and what students should be able to explain afterward. More about this in our guide to teaching intro programming in 2026.
None of these needs a new university-wide policy, and each can be written into an existing syllabus.
What happens when students can explain their work
If students can explain their own work, open assignments and projects can count again, and grades start to mean something. The teacher still makes the call. How to get there is the question at the center of our work at CodeGrade right now.
Rethinking how you assess coursework this term? We’d like to hear what’s working in your courses, and what isn’t. Book a 30-minute call with our team and we’ll walk through how other departments are handling AI in open assignments.
If you’d rather try things first, CodeGrade Free lets you set up an assignment with a rubric, an AI assistant with guardrails and a secure exam in an afternoon.
.png)
